19:01:12 #startmeeting infra 19:01:13 Meeting started Tue Dec 2 19:01:12 2014 UTC and is due to finish in 60 minutes. The chair is jeblair. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:14 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:01:17 The meeting name has been set to 'infra' 19:01:19 #link agenda https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting 19:01:22 o/ 19:01:26 #link last meeting http://eavesdrop.openstack.org/meetings/infra/2014/infra.2014-11-25-19.01.html 19:01:33 hello 19:01:39 Morning 19:01:50 o/ 19:02:09 #topic Actions from last meeting 19:02:14 * fungi dons the hat of shame again in advance 19:02:27 good point 19:02:35 actually, i think we should leave the hat of shame on the shelf this meeting 19:02:43 since for many of us, we have had 2 working days since the last one 19:02:49 ++ 19:02:58 and also, we have the infra-manual sprint going on. 19:03:00 right now in fact 19:03:00 which have been spent sprinting hopefully 19:03:03 that 19:03:08 o/ 19:03:10 o/ 19:03:16 ++ 19:03:31 so let's not spend too much time making excuses for ourselves, and instead catch up on what we think might be useful to talk about now, then get back to sprinting 19:03:50 o/ 19:04:10 i worked out what's needed for the openstack-ci elastic-recheck bit with mtreinish and jogo at least 19:04:11 helo helo 19:04:24 now i just need to find time after the sprint to do it 19:04:24 o/ 19:04:31 hi 19:04:42 fungi: yay 19:04:49 fungi: cool beans; need anyone else to do anything for that? 19:05:01 fungi: i note that infra-manual will need a patch :) 19:05:10 (it says to file bugs in openstack-ci) 19:05:14 (for gate failures) 19:05:33 jeblair: post sprint we should stab at nodepool on trusty again 19:05:35 yeah, and we can crowdsource adding the openstack-gate project to existing bugs mentioned in the elastic-recheck quwry set i guess 19:05:38 jeblair: see the etherpad - we need to change for storyboard in several places 19:05:40 I am just not sure what next steps should be 19:05:56 the other thing I plan to do is make third party test accounts self service this week 19:06:06 AJaeger_: which etherpad? link? 19:06:10 clarkb: next steps are add that project to relevant bugs, and then i can do a final import and close bug tracking for openstack-ci 19:06:12 fungi: would it help if I drafted up the email announcement for that? 19:06:23 clarkb: yay 19:06:39 jeblair: the sprint etherpad 19:06:41 clarkb: i was going to, but only because i didn't want you stuck with all the work for that 19:06:42 clarkb: I was going to but you go ahead, and I'll review 19:06:42 you volunteered but you keep volunteering for things 19:06:55 jeblair: https://etherpad.openstack.org/p/infra-manual-sprint-December-2014 19:06:56 anteaya: oh maybe you should write it. you understand the audience best 19:06:57 we all want to do it 19:07:03 oh, right, and then anteaya volunteered after i did 19:07:04 anteaya: assuming you are ok with doing it 19:07:08 I will take on creating the ehterpad 19:07:09 AJaeger_: ah, okay this isn't actually a change to storyboard 19:07:16 and invite you to help me fix it 19:07:20 AJaeger_: this is a new lp project to collect elastic-recheck gate bugs 19:07:21 anteaya: sounds good 19:07:23 #action draft messaging to communicate the new third-party account process 19:07:27 er 19:07:30 #undo 19:07:33 #undo 19:07:34 Removing item from minutes: 19:07:36 i think the chair has to 19:07:39 yep 19:07:39 jeblair: ah, ok 19:07:45 #action anteaya draft messaging to communicate the new third-party account process 19:07:49 thanks 19:07:56 #action fungi nibalizer get pip and github modules split out 19:08:22 #action clarkb script new gerrit group creation for self service third party accounts 19:08:22 clarkb: did you propose the ci docs change? 19:08:27 jeblair: I did 19:08:29 #action fungi close openstack-ci and add openstack-gate to e-r bugs 19:08:30 cool 19:08:39 #link https://review.openstack.org/#/c/137240/ 19:08:57 okay, so i think that's it for this topic? 19:09:04 didn't we split pip and github? 19:09:04 yep 19:09:17 okay woot 19:09:18 nibalizer: did we finish that? 19:09:24 i was yep'ing to jeblair 19:09:37 * jeblair is holding so we can resolve this 19:09:47 yep we did 19:09:58 okay, cool beans 19:10:07 on with the show 19:10:24 #roll 2 pop 19:10:41 oh well. next meetbot i write is totally going to support that. :) 19:10:44 hah 19:10:53 #topic Priority Specs 19:11:14 I tried to review a bunch of the specs last weke as things got quiet 19:11:25 so you may have comments from me 19:11:28 i fell down on the job 19:11:35 last week we highlighted a storyboard spec that got so much useful feedback it's in wip. it seems that was really useful! 19:11:49 i want to bring this one to our attention though: 19:11:56 #link Storyboard streaming API: https://review.openstack.org/#/c/105252/ 19:11:57 jeblair, what's status of nodepool remote building spec? 19:12:37 that storyboard one is not on the critical path, but i do think that general infra folks should pay attention to it, as we may end up writing/maintaining some tools that use it 19:12:51 small nit, on that spec: any idea what the topic means? 19:13:04 yolanda: looks like it's got positive reviews but needs more core reviewers 19:13:20 also remote building is honestly very low on the nodepool priority list for me 19:13:26 there are a ton of issues with nodepool right now 19:13:30 yolanda: unknown, but it's not a priority; i'll swing back around to it soon. 19:13:32 clarkb: yeah 19:13:35 and I think fixing them is >> new features 19:13:42 jeblair, /me going to review specs tomorrow 19:13:58 clarkb: What are the issues with nodepool? We might be experiencing them too and would love to help. 19:14:04 clarkb: i think it's related to the dib work, so i'd consider it related to our ongoing nodepool-dib priority, but not yet. 19:14:08 bug fixes are more important than features. no argument there 19:14:22 cody-somerville: gear times out on our new nodepool host 19:14:34 cody-somerville: this causes the allocation algorithm to default to min ready only 19:14:45 might be ubuntu trusty specific, but not positive yet 19:14:52 cody-somerville: nodepool cannot build dib and snapshot images for teh same label 19:14:58 might be network related 19:15:00 might be a gear bug 19:15:14 cody-somerville: nodepool cannot build dib images for rackspace 19:15:24 there's a veritable grab-bag of potential causes 19:15:26 hoping to dig more into that later in the week 19:15:37 we see this issue where instances get put in delete state in nodepool but apparently nothing has happened on cloud side then all of sudden boom they all finally get deleted 19:15:38 the big one is the gear thing and jeblair has been driving most of that debugging 19:15:41 back to this topic though 19:15:45 Migrate to Zanata: https://review.openstack.org/#/c/133222/ 19:15:50 and it results in nothing moving. no new nodes getting created. no jobs getting ran 19:16:00 i believe this is a priority we agreed on at the summit 19:16:11 so we should probably add it to our list 19:16:19 so AJaeger_ and I are pretty much on the same page with this spec, made some updates based on clarkb's comments, just need some more eyeballs 19:16:21 i agree the zanata migration is a top priority. i'll try to make time for the spec and any related help needed 19:16:29 yup the spec looked good to me 19:16:34 I need to rereview after the update 19:16:45 thanks 19:16:57 pleia2: is that another nick for AJaeger_ in the spec under assignees? 19:17:17 so let's try to get that reviewed this week and maybe merged by next meeting 19:17:18 anteaya: jaegerandi? That's my launchpad username 19:17:21 anteaya: launchpad 19:17:22 jeblair: +1 19:17:24 AJaeger_: ah 19:18:03 ooh, gerrit shames your trailing whitespace with red coloring 19:18:21 tsk, I'll fix in next revision 19:18:38 don't want to disappoint the whitespace gods 19:18:44 lulz 19:18:55 does anyone have anything in the priority efforts section of the agenda that would be useful to discuss now, or should we punt that until next week? 19:19:05 Yep 19:19:17 #topic Priority Efforts (Swift logs) 19:19:26 i don't have anything new since last week 19:19:29 So the next step for swift logs is to approve https://review.openstack.org/#/c/133172/ which I'm blocking on. Then it's some more testing and changing existing jobs over 19:19:39 yeah, my stuff is pretty badly stagnant there for the week 19:19:42 #link https://review.openstack.org/#/c/133172/ 19:20:11 It shouldn't be anything controversial, just more iterating 19:20:12 well we have to restart all of the jenkinses first 19:20:29 Ah, there is that then :-( 19:20:41 if we don't, we'll probably break jjb updates, yeah? 19:20:48 jhesketh: ping me tomorrow late PST which should be midday ish for you and I can do it 19:20:51 jeblair: ya 19:20:59 sdague: did you want to discuss that change? 19:21:09 clarkb: will do, thanks 19:21:12 oh, right, need the plugin loaded 19:21:47 I'm not sure what it breaks without the update to be honest. Better safe though 19:21:50 jhesketh: i'll +2 that change but you might want to wip it so nobody accidentally approves 19:22:26 fungi: noted, thanks 19:22:31 it has many +2s now. :) 19:22:35 since it sounds like you plan to be around for it anyway 19:22:39 heh, already 4 ;) 19:22:47 indeed it does 19:22:50 hah 19:22:52 #topic Schedule next project renames 19:23:09 there are several of these pending 19:23:32 * SergeyLukjanov volunteering to prepare and hopefully make the renaming 19:23:51 i'm available all weekend or friday if we want to do a friday maintenance 19:24:18 i'm similarly around 19:24:25 we have two projects moving from stackforge into openstack namespace and an infra project being renamed 19:24:46 o/ 19:25:00 zaro, hi 19:25:02 SergeyLukjanov: istr you wanted to do more of these to learn more... 19:25:24 SergeyLukjanov: should we schedule it around you to make sure you can do a lot of the work? 19:25:49 this weekend is good for me 19:26:00 jeblair, this weekend is ok 19:26:27 SergeyLukjanov: what time saturday would be good for you? 19:26:36 US morning 19:26:57 before 21:00 UTC 19:27:07 how is 1600 utc? 19:27:19 that works fine for me 19:27:20 it's works great for me 19:27:46 SergeyLukjanov: want to send the announcement email too? :) 19:27:48 i'm happy to take point with SergeyLukjanov for the benefit od pst'ers for whom that's 8am local 19:28:02 jeblair, + 19:28:04 1600 is actually fine for me 19:28:15 see everyone then sounds like 19:28:15 #action SergeyLukjanov send email announcing project moves for saturday 1600utc 19:28:22 clarkb: ya :) 19:28:47 * fungi notifies his personal activities planner 19:29:09 #topic Docker(Hub) Spec (nibalizer) 19:29:27 * SergeyLukjanov trying to make more stuff on infra, starting with review, hopefully will find some non-critical area to work on in background 19:29:32 so sdague pointed out in a review that we're thowing a lot of docker stuff at the wall 19:29:44 * mordred just threw some at the wall this weekend 19:29:54 we are? 19:30:06 and maybe specing our docker efforts would make it clearer what we're doing and where to collaborate 19:30:15 jeblair: I think that's a broader we 19:30:33 at the least some people want to test in it, mordred wants a docker hub, and i want to run livegrep in it 19:31:00 so maybe this is just an announcement that if you want to do stuff with docker, add it to the spec or leave a comment and I will 19:31:06 i clearly am not paying attention to reviews 19:31:14 nibalizer: this sounds like 3 very different things 19:31:37 and yeah, a general "all things docker kthx" spec seems counter-productive 19:31:56 yah - you can also add the kolla folks who want to deploy openstack in it to the mix 19:32:00 so let's take those 3 one at a time real quick: 19:32:06 ya thats a good point 19:32:19 I am already -1.9 for deployinglivegrep in it 19:32:23 for the most part, one can imagine that we're going to need the ability to build and publish docker images in order for any of those things to be workable in our systems 19:32:31 we have a way of deploying stuff. it doesn't involve an extra set of machinery 19:32:32 here is the review that started me thinking 19:32:33 https://review.openstack.org/#/c/132222 19:32:33 mordred: i'm going to take the floor 19:32:38 jeblair: go for it 19:32:52 first topic: testing openstack in docker 19:33:00 nibalizer: did you mean sdake ? I don't think I said that 19:33:10 ya i might have confused you 19:33:23 this is a big project. people have tried to do parts of it. it hasn't really gotten off the ground yet. see dox, etc. 19:33:26 * nibalizer yelds floor to jeblair 19:33:47 mordred: i think probably if people want to work on that, they should talk to you, see dox, maybe jd too? 19:34:04 this sounds like something which needs a working poc before it needs an implementation spec 19:34:13 fungi: yep. 19:34:30 jeblair: yes 19:34:31 also nothing prevents testing openstack in docker today aiui 19:34:36 anyone can just do it 19:35:01 well, modulo figuring out what needs figuring out. it isn't blocked on infra work at any rate 19:35:08 okay, so i think that's the way forward on that -- collaborate on working poc, then start infra-spec and possibly even openstack-spec 19:35:14 fungi: right, you can have jobs that run docker. jayf and jroll do this 19:35:25 next subtopic: docker hub 19:35:41 :) 19:35:44 this seems straightforward to me -- i think that we should help people publish stuff there like the other places we publish stuff 19:35:51 ++ 19:35:59 * nibalizer nods 19:36:00 i'm not even sure we really need a spec for it, but it wouldn't hurt. it should be short. 19:36:04 open question - should we also run our own dockerhub? 19:36:05 this is the docker community's index/clearing house for making docker images discoverable? 19:36:05 are they open source? 19:36:14 fungi: yes. it's their pypi 19:36:15 anteaya: yes 19:36:18 thanks 19:36:23 ++, me likes an idea to have a docker hub 19:36:24 mordred: where are on getting access to the openstack account? 19:36:31 where are we 19:36:32 I believe we have it 19:36:37 mordred: I would love something like docker hub with a better story for e.g. signed images 19:36:39 neato, is it in passwords.gpg? 19:36:39 I also have an infra account, fwiw 19:36:51 unsure. let me go verify all of that 19:37:06 #action mordred ensure dockerhub credentials are recorded in appropriate infra places 19:37:10 it _may_ be a group that my user was added to the owner status of 19:37:18 which means we may need an infra _account_ 19:37:23 that we add asa user, same as on pypi 19:37:30 but I'll learn that info 19:37:43 mordred: okay, regardless, it sounds like that's probably something that's blocked on you 19:37:47 yup 19:37:57 but it seems like it should be straightforward to resolve 19:38:19 is Sergey Skripnick on irc? 19:38:39 it looks like this is the relevant review: 19:38:43 #link https://review.openstack.org/#/c/132222/ 19:40:19 is someone interested in eithr writing a spec or just doing the work to put dockerhub creds on the pypi slave? 19:40:32 i can do that 19:40:47 same as the puppetforge stuff I just did so should be easy 19:41:13 #action nibalizer make it so we can upload to dockerhub 19:41:27 okay, last of the trio: using docker to run stuff in infra (eg, livegrep) 19:41:43 fair warning, this may be a hard sell for some of us :) 19:41:50 sure 19:41:58 and i didn't mean to imply that the decision had been made 19:42:03 nibalizer: what's the advantage? 19:42:08 i've spent some free time exploring what that would look like 19:42:22 not saying its the right or best way to do it 19:42:31 the advantage with a tool like livegrep is that there is a compliation phase 19:42:37 nibalizer: thank you for doing that 19:42:38 so I feel pretty strongly about this. We have our abstraction layer in VMs. It has worked for a variety of deployments including deploying things from source. I am unsure why something like livegrep would be special enough to require a completely new deployment mechanism 19:42:50 and i think that sets it apart from much of the python stuff 19:42:59 since you have these artificats afterwords which are anoying 19:43:02 one that would require touching almost all of our tooling used to deploy things 19:43:10 so you could build rpms/debs from the artificats 19:43:16 but we dont have a pipleline for that right now 19:43:28 so its not docker for contanierization, its docker for leightweight packaging 19:43:31 i think this is probably the point at which we traditionally say we should build debs, but yeah, no one has gotten around to building that pipeline. 19:43:33 right. to me docker is becoming like distro packaging for apps except without the pain of packaging for distros or maintaining repos 19:43:47 and the last person that volunteered to that is off building docker containers right now. ;) 19:43:52 :) 19:43:56 * jeblair waves at mordred 19:44:03 turns out they do what I want without the insane amounts of pain 19:44:06 mordred: nibalizer does that mean you will go and deploy gerrit, jenkins, etherpad, et al this way? 19:44:13 because we have trouble with their packaging too 19:44:23 clarkb: I'm not 100% sold on the idea - still beating it with a stick 19:44:29 (trying to understand why livegrep prompts this and not say gerrit which is probably a million times harder to deal with) 19:44:31 so etherpad, im a big fan of putting the dumb node app in the docker 19:44:34 but I'd say that if we get to the point where it feels better 19:44:37 its worked twice for me, and very well 19:44:45 then I see no reason not to do it for the other things that meet the pattern too 19:45:03 but like I said, I'm still beating things with sticks 19:45:04 clarkb: well aren't there gerrit packages? 19:45:07 no one is packaging livegrep 19:45:07 nibalizer: no 19:45:13 its the same problem 19:45:15 i think docker could be a very good way for simple test such as python, pep8 ... without spinning up a vm 19:45:15 oh ew 19:45:21 but no one has said "run gerrit in docker" 19:45:27 clarkb: it's been in my brain 19:45:30 which is why I am so skeptical 19:45:32 clarkb: I just haven't gotten there yet 19:45:36 is gerrit java? 19:45:40 nibalizer: yes 19:45:43 i did some approach with lxc containers for that, and was flying 19:45:44 becuase a fatjar gives you somewhat the same thing 19:46:07 i'm obviously missing something significant, but i don't see why it's any easier to deploy this with docker than without docker 19:46:13 fungi: it isn't 19:46:16 and thats what I am trying to get at 19:46:18 clarkb: so I'm much more positive on the topic - but not far enough along that I think I recommend anything 19:46:27 I think the reasn no one has said gerrit in docker is it doesn't help with the pain 19:46:29 ya i think the rest of the meeting will continue 19:46:30 it just moves it 19:46:34 er can continue 19:46:39 i can explore this livegrep/docker thing 19:46:46 and if its something i think is mature enought to try to sell you all on it 19:46:50 i will try 19:46:56 nibalizer: even after the recent cve? 19:47:09 i haven't seen this cve 19:47:11 clarkb: even after all the upcoming cves ;) 19:47:22 well this one was particularly bad from the "mature" standpoint 19:47:28 anyone could craft a dockerhub image that pwed you 19:47:42 linux containers are still not fully-baked for secure separation 19:47:49 well i meant the maturity of my approach to using docker w/ livegrep 19:47:51 neither are debian packages 19:47:54 fungi: i think part of the idea is to keep 'building apps' out of puppet. instead of doing that by setting up an apt repo and building our own debs, we build docker containers instead 19:48:01 you can craft a debian package that will pwn yuo too 19:48:06 jeblair: yes 19:48:12 jeblair: thank you for saying that succinctly :) 19:48:14 agreed. i wonder if dockerhub is as picky about who gets to upload packages as debian is 19:48:14 mordred: ya the differnce is who can upload to the offical source 19:48:15 jeblair: that is the idea 19:48:38 fungi: definitely not, it's more like pypi than debian. 19:48:39 clarkb: if you install "emonty/ubuntu" - the onus is on you that you installed something from my repo 19:49:07 and yes, we can just as easily pip install something which takes over the system 19:49:10 yup 19:49:16 this is _not_ about adding security 19:49:23 anyways I think the conversation should be framed as "should we do gerrit in docker" not livegrep in docker 19:49:26 because livegrep is easy 19:49:27 it _may_ be about a packaging format 19:49:37 and if gerrit in docker isn't a win then maybe we shouldn't use it 19:49:47 clarkb: I agree 19:50:16 okay, so essentially moving our abstraction layer for all the vcsrepo and pip install and npm install and whatever else we do for various services 19:50:31 fungi: sort of 19:50:36 fungi: we still need to run all of that somewhere 19:50:41 shift those things into some image-building system and then deploy just images 19:50:44 fungi: yah. essentially, it's like if we decided to make debs of all of those things 19:50:46 it just happens to be in a docker image build then at deploy time you deploy image 19:50:53 yup 19:50:54 okay, i think we can move on now. i think this has been surprisingly productive. :) 19:50:55 fungi: except that instead of bulding debs, we build dockers 19:51:11 nibalizer, mordred: thanks :) 19:51:12 i can understand the angle anyway 19:51:18 woot! 19:51:20 #topic Python 2.6 deprecation 19:51:22 productive meeting ftw! 19:51:26 jeblair: you too 19:51:36 this took a leap forward on monday 19:51:53 we are basically done until juno is killed 19:51:57 yeah, this looked timely, so i agenda-jumped to it :) 19:52:04 huge thanks to AJaeger_ for making and keeping up with rebases for the changes to do that step 19:52:41 and we're at the phase where if stackforge projects want python26, they can add it back 19:52:42 i've seen no real complaints so far, though it's only been a bit over a day 19:52:51 yep 19:52:51 but when juno is eol, it goes away globally 19:53:01 correct 19:53:23 very few really want it when asked - they just copied without thinking 19:53:24 i'll be interested to see how many stackforge projects we remove it from at that point 19:53:44 me too 19:54:04 ++ 19:54:30 #topic Propose an alternate meeting time more suitable for EMEA engineers (rcarrillocruz) 19:54:37 so 19:54:45 i wanted to get a pulse on people's opinion 19:54:53 this meeting is hard to attend for EMEA folks 19:55:02 would be good to have an alternate meeting like some other projects do 19:55:14 it works great for me from home ;) 19:55:25 this time is complicated for me as well 19:55:29 the current time was selected to make it possible for both europe and america to attend 19:55:31 Works for me but then I have meetings before and after that one. 19:55:45 seems like it's a lot harder for apac attendees than emea 19:55:49 it is great to have jhesketh here 19:55:53 it's actually one of the most sought-after meeting times for that reason :) 19:55:55 fungi: yes 19:56:00 and he has to get up at what, 5am? 19:56:13 but yes, it's hardest for apac 19:56:22 jhesketh: what time is it for you? 19:56:46 anteaya: it's 6am meeting time in eastern australia 19:56:58 okay 6am 19:57:12 anteaya: it's 5am when daylight savings swizzles 19:57:12 some projects do a/b meeting times, not sure how effective that is for them though 19:57:18 obv, it's going to be hard a time that is cool for everyone 19:57:19 so here's my opinion on the subject -- if we have core contributors who can't attend the meeting, i think we should try to accomodate them (with an alternate time, etc) 19:57:24 that's why i suggest a/b meeting times 19:57:49 I can never remember which week is a and which is b 19:58:02 but otherwise, there are meeting logs, and most of us are around irc quite a lot anyway and are happy to do impromptu meetings or even schedule a time to have a conversation on a particular subject 19:58:06 I've been missing recent neutron meetings since they switched 19:58:13 fwiw as an .au person i'm happy with this time 19:58:23 ianw: thanks 19:58:47 last time we asked jhesketh about this, he was also okay with this time (at least, for the next few months while dst is in effect :) 19:58:49 if it becomes necessary, i'm more of a fan of the two-meetings-each-week method and let people who want to attend both convey relevant information or skim and discuss the other meeting's most recent minutes 19:58:52 ok, since i'm not core i'll stick to meeting logs :-) 19:59:29 fungi: thanks, glad you have your input on that one :D 19:59:34 #topic Potential Zuul mascot/logo feedback (pleia2) 19:59:35 fungi: a I prefer that to alternating 19:59:43 I can just take this to the mailing list for feedback 19:59:52 #link comments and feedback on Zuul proposal: http://princessleia.com/temp/Zuul-sketch.jpg 19:59:52 so, see you there :) 19:59:58 pleia2: where would the mascot be used? 20:00:08 anteaya: slides, documentation page 20:00:12