19:03:46 #startmeeting infra 19:03:47 Meeting started Tue Dec 23 19:03:46 2014 UTC and is due to finish in 60 minutes. The chair is jeblair. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:03:48 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:03:51 The meeting name has been set to 'infra' 19:03:55 #topic Meeting schedule 19:04:39 so.... normally we follow the lead of the other general project meetings (eg, tc and cross-project) and cancel on those dates... 19:04:57 but that kind of snuck up on me and i forgot to bring it up last meeting 19:05:11 heh 19:05:20 I second the motion ot cancel next weeks meeting 19:05:34 so to avoid having two meetings where we know lots of people won't be here.... 19:05:40 is tc meeting also cancelled today then? i saw the announcement for cross-project 19:06:07 i'd suggest we cancel next weeks, and be sort of informal about this one 19:06:14 yes it went to the tc list 19:06:15 jeblair: wfm 19:06:20 wfm 19:06:23 i expect to be around next tuesday, but will likely be holding down the fort in irc so i'm fine with no meeting 19:06:34 #agreed cancel dec 30 infra meeting 19:06:38 http://lists.openstack.org/pipermail/openstack-tc/2014-December/000908.html 19:06:41 #topic Open Discussion 19:06:45 sounds good 19:06:58 heh 19:07:07 ha ha ha 19:07:19 I did add an agenda item this week about where we land 3rd party CI documentation 19:07:23 my only thing is I will be rebuilding slave images shortly so that we fix this pbr freeze bug 19:07:23 fwiw in Russia Jan 1-11 are holidays, so, /me will be mostly unavailable 19:07:25 how 'bout that pep 440? 19:07:27 I would like to hear from jhesketh as he did get up for this 19:07:34 fungi: ha ha ha 19:07:44 jaypipes has volunteered to write the documentation in .rst format, but he needs to know where to land it 19:07:45 fungi: I think you deserve a few days off now :) 19:08:10 clarkb: maybe i'll take thursday off 19:08:13 anteaya: for what sorry? 19:08:14 clarkb: ++ 19:08:20 jhesketh: swift logs 19:08:25 jhesketh: are logs logging? 19:08:37 o/ 19:08:43 SergeyLukjanov: thanks for letting us know 19:08:43 fungi: how about it! :) 19:09:03 Ah, seems like we're discussing 3rd party docs first 19:09:09 fungi: pep440/pip/pbr/etc is a good thing to talk about, actually... 19:09:18 let's talk about that for a minute 19:09:29 because it's something some of us may need to continue dealing with over the holidays 19:09:34 it's mostly (entirely? knock on wood) resolved at this point 19:10:05 clarkb: is the pbr freeze thing causing actual errors, or just not working as intended? 19:10:05 I am personally interested in a recap of yesterday as I was mostly caught up until then 19:10:11 i think the only lingering issues are we haven't built new nodepool images without the setuptools <8 cap in install_puppet.sh 19:10:13 jeblair: its actually causing errors 19:10:20 I was here and would appreciate a recap 19:10:25 jeblair: jobs are failing due to the check >0 tests run check 19:10:33 ahh, right, pbr freeze got implemented in the slave scripts incorrectly 19:10:43 i approved clarkb's fix just now 19:11:06 i'm guessing the change adding it ended up merging after yesterday's images built 19:11:17 so we didn't see the issue until today after we got new images 19:11:36 mostly pip 6.0 release was uneventful 19:11:37 yes I do believe I approved it yesterday midday sometime 19:12:08 it pointed out one pep 440 issue we missed in devstack, which required an icehouse->juno->master series to fix 19:12:12 so since we're going to be in and out and flying and driving around, etc, perhaps if anything further comes up, we should make an etherpad and stick the link in the channel topic so we can keep up to date on what's broken and needs reviewing, etc. 19:12:34 but aside from that it was mostly finding regressions in pip 6.0 and pointing them out so that they would get fixed in 6.0.1 19:12:41 which they did 19:12:44 +1 19:13:32 oh, other lingering item is that some projects may not have yet merged their reqs sync patches to fix the setuptools entry 19:13:48 one of the tripleo jobs was struggling with ceilometer this morning for that reason 19:14:06 s/setuptools/sqlalchemy/? 19:14:06 s/setuptools/sqlalchemy/ 19:14:12 yeah 19:15:18 so anyway, i think things are likely to be quiet on the pep 440 front from this point forward 19:15:31 pip 6 was the big unknown until yesterday 19:15:44 and all the pbr fixes outstanding have merged in the past few hours 19:15:48 fungi: did pip 6 have the single regression? 19:16:05 clarkb: there were a couple regressions with it which bit us 19:16:06 whcih was don't copy VCS dirs because speed 19:16:12 clarkb: yeah, that was one 19:16:39 another was failing to make scripts executable when building wheels 19:17:27 we had quite a few jobs failing on "permission denied" in various places because of that 19:17:41 oh, also we had to work around the fact that pip uses ~/.cache now 19:18:01 so sudo pip whatever was making a root-owned .cache in the calling user's homedir 19:18:11 sudo -H pip is preferred to avoid that 19:19:02 anyway, it was way fewer issues than we had to work around for setuptools 8 19:20:13 also, i'd like to see us finish whatever we need to be able to release pbr master so we can leave the feature/0.10 branch behind us 19:21:12 there are a few random improvements hanging out in review for pbr master that have been there a while (since before setuptools 8) 19:21:20 fungi: its the semver series which was mostly lifeless 19:21:32 so we probably need someone to adopt that code if we want it to move forward in the near future 19:21:48 clarkb: has lifeless abandoned that? 19:22:08 i think he just has other stuff going on, so temporarily abandoned maybe 19:22:22 i know he wants to finish it when he gets time 19:22:34 jeblair: no he is just afk 19:22:37 it's basically usable now, but there may be one or two reported bugs to triage 19:22:51 related to the semver-esque implementation details 19:24:10 anyway, that's all the updates for that topic i think 19:24:47 takeaway is that we're fragile around python packaging, and i'm not sure there's a good answer other than try to help the python packaging devs with quick feedback 19:25:35 dstufft has been awesomely responsive, and has given us a heads up on releases too 19:25:40 would anyone like to continue to explore idea of upgrading gerrit? 19:26:04 zaro: you were going to look into upgrading review-dev next, right? 19:26:10 dstufft has been fantastic 19:26:53 fungi: greghaynes and I were talking about it over brewing beer yesterday and we could run a periodic job that runs devstack/tempest with latest pip master 19:26:56 zaro: and based on your email from last night it sounds like the only way to utilize the its-storyboard plugin you are working on upstream is to upgrade 19:27:10 anteaya: yes, that is correct. 19:27:19 fungi: maybe dstufft would "subscribe" to that? 19:27:32 fungi: i kinda wanted to get an idea which version we were gonna target before testing. 19:27:52 zaro: I thought we decided 2.9 and 2.10 isn't released 19:27:56 i think there was the idea of going to 2.10 ? 19:27:57 clarkb: yeah, that would be a good step. though even better would be if someone released a github or bitbucket trigger for zuul so we could trigger and report on pull requests 19:28:06 zaro: 2.10 is an option? 19:28:25 clarkb: perhaps. assuming that they merge pull requests fairly continuously rather than right before releasing 19:28:47 2.10 is what google uses, jeblair mentioned that. don't remember if he meant we should do same 19:28:49 and yeah, zuul trigger is the answer there 19:29:18 zaro: i think we got as far as thinking that we should at least work on 2.9 19:29:37 zaro: i think we may want to defer the conversation about whether to track upstream in a CD fashion until more people are around 19:29:38 also jeblair wanted to review the ssh problem 19:29:47 fungi: yup long term I think we want a proper trigger but we should get a lot of value out of "pip stopped working today" 19:30:02 fungi: espeically if someone like dstufft is able to watch that 19:30:12 jeblair: I have a bunch of zuul refactoring that needs reviewing but will hopefully make a github trigger easier 19:30:22 yeah 19:30:26 jeblair: did you get a chance to read about the stream events issue? 19:30:32 btw, does anyone want to review jhesketh's pep8 change to zuul? 19:31:01 i kind of feel like if people think it's important, they ought to review it. i do not think it is important, and yet i reviewed the first patchset and left 40 (not exaggerating) comments 19:31:04 i believe they have fixed in 2.9.3 19:31:41 https://review.openstack.org/#/c/115187/ 19:34:18 zaro: not yet 19:35:02 jhesketh: isn't there a hacking guideline for that change? 19:35:11 ya if we go to latest stable gerrit that would be good 19:35:23 then we can worry about CD gerrit when not time sunk into holidays 19:35:35 zaro: not sure? 19:35:39 latest stable +1 19:35:51 jeblair: for zuul you mean that if other reviewers are happy to review and approve the tox changes yo uare on board? 19:35:53 zaro: i have not had a chance to read up on that, but i think prepping for 2.9 doesn't have to wait 19:35:59 jeblair: or you reviewed it and feel even more strongly it shouldn't merge? 19:36:00 zaro: that's just to re-enable the broken pep8 job 19:36:01 jeblair: here's a similar change https://review.openstack.org/#/c/133465 19:36:13 opps jhesketh ^ 19:36:41 zaro: yeah, i'm pretty sure that jjb change durned of pep8 checks 19:36:45 grr 19:36:47 "turned off" 19:37:03 or rather, would turn off, since it hasn't merged 19:37:31 I do have to say that hacking is one of the most frustrating things ever 19:37:44 and am a huge vote in favor of ignoring H* in infra projects 19:38:13 also, what the heck? 19:38:25 i mean there is a comment that says "do not do what that jjb change does" 19:38:32 and that jjb change does it and removes the comment 19:38:33 zaro: if you could leave a comment that'd be great 19:38:53 ok. i'll start testing review-dev.o.o with Gerrit 2.9.3. I remember talking to someone in channel that got their system working with 2.9 19:38:54 fwiw all of those broken hacking import jobs which can't work are being removed in the next release 19:39:05 s/jobs/checks/ 19:39:06 zaro: i have left a -2 on https://review.openstack.org/#/c/133465/ 19:39:23 mtreinish: its not just import ordering though its docstrings must end with punctuation. commit title must end in period. and so on 19:39:57 sigh, yeah a lot of those are annoying. Yell at jogo 19:40:02 period in docstring titles, and periodless in commit message titles 19:40:04 I was just looking at the review link 19:40:15 clarkb: and what i meant about zuul and pep8 is that i have already spent more time than i would like reviewing whitespace changes. in doing so i found that the change was _much_ larger than needed. i think it would be great if someone else who is actually interested in whitespace enforcement review that change and ensure it is minimal. 19:40:42 jeblair: gotcha 19:41:07 I can give it a go. In general normal pep8/pyflakes doesn't bother me 19:42:53 clarkb: most of it doesn't bother me either. it's going _beyond_ pep8 that bothers me. 19:43:04 ya E125 and friends 19:44:01 zaro: so yeah, we don't use hacking at all in infra projects. 19:44:20 zaro: there was an attempt to use it to check one thing in zuul. it massively backfired and reminded us why we don't use hacking in infra projects 19:44:35 zaro: that's why we're having this conversation about getting zuul back into compliance with pep8 19:45:03 so i thought it was a bad idea before, and now i think it's a really, really, really bad idea. 19:45:07 why is it still on our wiki? 19:45:14 zaro: what? 19:45:21 hacking guidlines 19:45:25 zaro: link? 19:45:52 http://docs.openstack.org/developer/hacking/ 19:46:32 zaro: that's for the openstack project itself, not for openstack-infra 19:46:46 zaro: there are quite a lot of guidelines for openstack that we do not adhere to 19:46:57 ahh, gotcha. 19:47:20 notably, we work on code that isn't in python, have different CLA requirements, etc. 19:48:06 should probably include this disclaimer in the openstack-infra manual? 19:48:13 with respect to hacking and the like, what works for 2000 developers working together isn't necessarily desirable for 10 19:48:32 zaro: maybe in the ci docs, but the infra-manual is for openstack developers, so that might be confusing 19:48:40 s/isn't necessarily desirable/is sometimes actively harmful/ 19:48:57 zaro: though tbh, we should minimize that kind of content in the manual anyway 19:49:49 can we talk about docs for a few minutes? 19:49:50 zaro: it should focus more how to use the tools to get things done 19:50:27 pleia2: are we excluding folks who might want to participate? 19:50:34 pleia2: ahh, yes, the raging debate about where third-party testing documentation lives long-term 19:50:39 fungi: that's the one 19:50:52 I have to leave a few minutes early, sorry guys. Re swift logs it is working for infra jobs and I have fixed in review to make it work for devstack gate and improve the log index from feedback. (I have plenty of open changes if anybody feels like reviews) 19:50:53 jeblair: I don't think so 19:51:10 jhesketh: thank you 19:51:24 s/fixed/fixes 19:51:26 jhesketh: thanks! 19:51:48 Okay catch you later :-) 19:51:50 I think it's important, it's really difficult to support these folks via jaypipes' blog posts and I think it would bring value to our project to have it documented, since it supports CI as an open source project too 19:51:52 o/ 19:51:56 jhesketh: enjoy holidays 19:52:10 pleia2: http://ci.openstack.org/third_party.html 19:52:29 pleia2: so is the question just ci.openstack.org vs docs.openstack.org/infra/manual vs something else new? 19:52:37 so there's that... 19:52:38 fungi: that's it 19:53:10 the infra manual, as i see it, is about teaching people how to interact with the infrastructure we're running 19:53:24 whereas third-party testing is only tangentially that 19:53:35 * pleia2 nods 19:53:44 fungi: right. infra manual is "the manual for users of the infrastructure" 19:53:53 we also have http://ci.openstack.org/running-your-own.html which has a fair amount of overlap 19:53:58 (the consuming from our gerrit event stream and automated commenting in gerrit is the extent of their interaction with our infrastructure) 19:54:18 so what I think makes the most sense is bringing jaypipes' blog posts in to http://ci.openstack.org/third_party.html and updating /running-your-own.html where there is overlap 19:54:27 pleia2: i think that asselin's proposal to make a puppet module for third-party folks that lives in infra is really useful 19:54:41 pleia2: and i think maybe the best way forward is to proceed with that, and then document that thing appropriately 19:54:43 jeblair: yeah, that will help a lot 19:54:55 ok 19:55:04 still not completely clear where to put the docs though 19:55:14 third_party.rst 19:55:25 yeah, in system-config 19:55:32 running-your-own was i believe initially intended as a walkthrough for someone who wanted to set up a copy of most of our infrastructure to test it out and develop improvements for it, so slightly different use case from third-party testing 19:55:46 yep, that way the docs live with the code 19:55:58 fungi: yeah, it's what lifeless wrote when he went through it, but there are similarities 19:56:22 and running-your-own is already bitrotting because we don't "run our own" or test things in the way it descibes, and no one is keeping it up to date 19:56:25 agreed, there is common content in several documents which we might be able to link up/de-dup more effectively 19:56:37 fungi: ++ lots of good stuff to pull from 19:57:00 anteaya, third_party.rst == http://docs.openstack.org/infra/manual/third-party.html? 19:57:01 i think running-your-own and the readme in devstack-gate suffer for similar reasons 19:57:05 so that's why i'm hoping we end up with a real puppet module for some of this, and real testing of it, and docs that go along with that, so we can actually keep everything up to date and working :) 19:57:13 asselin: no, http://ci.openstack.org/third_party.html 19:57:19 they're not written for the people who are managing the codebase, so we aren't the ones who notice when they get stale 19:57:23 asselin: no, we just outlined that third party docs do not belong in the infra manual 19:57:39 asselin: these are the docs generated from our system-config doc repo 19:57:40 asselin: as pleia2 said, the third_party.rst file in system-config 19:58:05 ok I'm +1 with third_party.rst 19:58:12 jeblair: ok, thanks, we have a plan then 19:58:13 not to say that no third-party stuff belongs in infra-manual, just not a complete set of documentation about how to run a system 19:58:28 * jeblair is all out of negatives 19:58:31 we should probably remove this empty link then: http://docs.openstack.org/infra/manual/third-party.html 19:58:59 * clarkb hands jeblair a never 19:59:37 clarkb: i'm going to hold on to that... 19:59:42 i have a nor you can borrow, like new condition 19:59:53 asselin: probably so 20:01:16 well, thanks everyone, and see you next year! enjoy the holidays! 20:01:18 i guess we're at time, if we want to free up the channel for the dearth of meetings scheduled the rest of the day 20:01:27 #endmeeting