19:00:34 #startmeeting infra 19:00:35 Meeting started Tue Oct 7 19:00:34 2014 UTC and is due to finish in 60 minutes. The chair is fungi. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:36 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:00:36 o/ 19:00:39 The meeting name has been set to 'infra' 19:00:41 o/ 19:00:54 #link https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting 19:01:03 #topic Actions from last meeting 19:01:05 o/ 19:01:11 anteaya take a stab at writing the openstack-infra/config rename changes 19:01:17 how'd that go? 19:01:25 Morning 19:01:26 I've been reviewing them, they're looking good 19:01:35 care to #link? 19:01:37 all set to WIP right now 19:01:52 ah ok I was ging to say I haven't seen them but gertty doesn't show me WIP by default 19:01:55 there are a lot of them, let's see how I can do this 19:02:11 if they have the same topic you can link that url 19:02:13 they all have "Rename config => system-config" in the commit message topic thingy 19:02:26 hi 19:02:34 o/ 19:02:44 o/ 19:03:01 #link https://review.openstack.org/#/q/status:open+topic:config-rename,n,z 19:03:02 https://review.openstack.org/#/q/project:openstack-infra/config+topic:config-rename,n,z 19:03:06 ah, thanks anteaya 19:03:07 I need to rebase 19:03:12 sorry I'm late, thanks pleia2 19:03:15 excellent 19:03:18 o/ 19:03:43 so yes, do review, I hope to rebase this afternoon 19:03:44 so anyway, that's underway, no longer an action item. we can move on unless there are urgent comments about the reviews 19:03:51 election business has kept me occupied 19:04:03 i understand entirely 19:04:05 fungi delete old ci-puppetmaster.openstack.org 19:04:05 well shall we set a time for the rename? 19:04:07 done 19:04:12 thanks fungi \o/ 19:04:26 did we mark the associated bug to rebuild the puppet master as fixed? 19:04:36 for those who have shell accounts on the new puppetmaster, i backed up the contents of the old ci-puppetmaster root homedir and stuck it in the new root homedir 19:04:39 he did 19:04:39 clarkb: yep 19:04:49 perfect 19:05:07 so lets also puppet node deactivate ci-puppetmaster.o.o 19:05:11 on the new master 19:05:18 so it doesn't show up in puppetboard 19:05:22 glad i did too. i missed getting jeblair's gencert.sh convenience script off of it during the migration, and who knows what else is lurking in that tarball ;) 19:05:37 nibalizer: oh, great input. will do 19:06:00 i'll knock it out while topics are being discussed so i don't forget 19:06:12 #topic Kilo summit topic brainstorming pad 19:06:16 just a reminder 19:06:22 oh I was supposed to look at that /me opens it now 19:06:24 #link https://etherpad.openstack.org/p/kilo-infrastructure-summit-topics 19:06:43 i added some background info and a distillation of our likely session schedule there 19:06:45 I added infra-manual as a summit topic 19:06:54 since I didn't know where else to mention it 19:07:09 dev there has essentially stagnated due to lack of reviews 19:07:15 and once jeblair's back he can work his ptl magic wand and bless the sessions which float to the top 19:07:26 if we don't want it as a summit topic that is fine 19:07:36 #topic Priority Efforts (fungi) 19:07:37 as long as we can kick it back into play somehow 19:07:45 Puppet 3 Migration 19:07:57 anteaya: infra-manual could be a topic for a future meeting as well 19:07:57 we can mark that as done done once you deactivate the old master ya? 19:08:00 this can be struck off the list once i disable the cert 19:08:02 yep 19:08:06 AJaeger_: we can try 19:08:13 i'll make sure to clear it off the agenda after the meeting too 19:08:20 awesome 19:08:22 Swift logs 19:08:32 #link https://etherpad.openstack.org/p/swift_logs_next_steps 19:08:42 I just merged a change of jheskeths to handle timestamps in logs 19:08:50 there's more hacking going on for tim estamps in console logs, i saw 19:08:54 yeah, that 19:08:56 * in console logs. and will be rebuilding our images asap to get that chagne on the slaves 19:09:59 nibalizer: i've deactivated the old puppetmaster on the new one now, btw 19:10:06 I will do a semi coordindate merge of the change that uses the new script output in the jobs while the images are building 19:10:11 \o/ fungi++ 19:10:13 this will reduce the time where test will break for infra 19:10:31 good idea 19:10:44 clarkb: Don't rebuild the images just yet though 19:10:46 clean enough for our purposes anyway 19:10:58 jhesketh: ok 19:11:00 clarkb: just catching up with stuff but we aren't fetching the text version of the logs 19:11:08 so we've broken the job anyway 19:11:10 I'll shoot a fix 19:11:15 jhesketh: we haven't 19:11:20 jhesketh: we split your job in two 19:11:24 er your change 19:11:35 though there may be additional changes that need to be done 19:11:40 so I will hld off on image builds 19:11:50 clarkb: yes, but https://review.openstack.org/#/c/126471/3/jenkins/scripts/grab_console_log.sh has the text version fetch commented out 19:12:18 oh I see ya, if we uncomment that then this will go more smoothly. lets do that 19:12:29 sounds good 19:12:38 anyway, once we're fetching the html verson of console log we can start comparing results 19:12:50 I'm not sure what metrics we should use though if anybody has some suggestions? 19:13:06 jhesketh: mordred and I noticed some 404ing yesterday 19:13:18 jhesketh: so we will probably want to fix that then look at mean time to fetch from disk vs swift? 19:13:26 jhesketh: hrm... probably wouldn't hurt to analyze the upload delay as well 19:13:32 clarkb: 404ing on which paths? 19:13:32 (for a specific non trivially sized file probably) 19:13:42 jhesketh: the console.txt?source=swift path 19:13:49 jhesketh: it would work most of the time but if you refreshed enough 404 19:14:11 I wonder if we're hitting issues with the swiftclient not being threadsafe 19:15:08 https://bugs.launchpad.net/python-swiftclient/+bug/1372429 19:15:09 Launchpad bug 1372429 in python-swiftclient "swiftclient is not thread safe" [Undecided,New] 19:15:39 jhesketh: possibly. I can try to dig into server logs and provide more info 19:15:44 #link https://launchpad.net/bugs/1372429 19:16:42 clarkb: cool, that'd be good 19:16:46 anything else we need to cover on current status of this effort? 19:17:23 nope, just general help on comparing logs would be good but I'm not sure I'll be around much to progress it this week unfortunately 19:17:35 noted, thanks! 19:17:38 Config repo split 19:17:52 this is basically already covered during the action items topic 19:18:02 anteaya's wip reviews are the next phase 19:18:18 so no need to dwell further unless there are other concerns on this one? 19:18:30 just timing 19:18:39 do we want to select a time for the rename? 19:18:46 or punt until jim returns 19:19:12 renaming soon would be nice 19:19:26 i suggest we wait until the reviews are no longer wip and jeblair's back to discuss scheduling, but soon would definitely be nice 19:19:44 the reviews are only wip so we don't accidentally merge them and break all the things 19:19:47 I'm planning on wiping them until the rename 19:19:51 just don't want to prematurely set a date and then have it slip after being announced 19:19:53 what pleia2 said 19:20:20 all I need to do is rebase to make jenkins happy and respond to any reveiw concerns 19:20:27 I can be ready by friday 19:20:29 oh, got it. so they're not actually a work in progress. we need a better convention for this state. sometimes we use wip, somtimes a core -2 19:20:38 right 19:20:48 whatever the convention is, that is all this is 19:20:50 but generally neither of those get visibility to reviewers so sub-optimal 19:20:57 agreed 19:21:00 okay, moving on 19:21:14 so no set time then? 19:21:36 oh, we can vote on whether we want to pick a time and do it while jeblair's away i suppose 19:21:44 I'm fine with Friday 19:21:45 are we holding off on splitting out modules until after the rename? 19:22:05 if we can do the rename soonish, that would make sense to me jesusaurus 19:22:06 module break-out could probably happen in parallel i would think? 19:22:09 jesusaurus: we shouldn't need to, but I haven't thought about that for very long 19:22:10 jesusaurus: you make it sound like we have a plan :) 19:22:23 * nibalizer has been going forward with module split out 19:22:41 any objections to renaming config to system-config on friday later afternoon eastern time? 19:22:55 like 2000 utc or 2100 utc? 19:23:06 yeah, whatever reviews/changes you have posted against openstack-infra/config for that work will move seamlessly to the new project name, so should be non-impacting 19:23:06 anyone hate that idea? 19:23:25 * nibalizer no objection 19:23:28 * krtaylor runs and hides from third-party impacts 19:23:44 impact with be the same friday or any other day 19:23:52 true 19:24:04 fungi: I move for 2100 utc on Friday for the rename 19:24:23 anyone want to second? 19:24:30 +1 19:24:34 and fold in the stackforge ones while we are at it 19:24:43 clarkb: can you be around? 19:24:50 ya I can be around 19:25:11 2100 utc lands around my dinner time, but i can try to plan to not do stuff friday evening 19:25:11 fungi: how does that time work for you? 19:25:23 fungi: do you want a different time? 19:25:30 fungi: pick a better time 19:25:32 nah, i'll make it work 19:25:41 so the other thing to consider is when do we release juno? 19:25:44 actually my wife's in class that night, so it'll be fine now that i think about it 19:25:58 and is there any worry that we will wedge ourselves and make that hard? 19:26:02 #link https://wiki.openstack.org/wiki/Juno_Release_Schedule 19:26:04 fungi: you can post the announcement to dev? 19:26:05 I think the 16th so we should actually be ifne 19:26:15 ya so 10th is probably fine 19:26:40 anteaya, fungi: I can draft something up 19:26:45 pleia2: thanks 19:26:47 pleia2: appreciated! 19:27:20 okay thanks, I just didnt' see any point in waiting other week 19:28:21 sure, i see the logic in that 19:28:34 thanks 19:28:43 so anyway, this is impinging on the release schedule since that's next week 19:29:23 ttx: if you're around, objections to us having an infra configuration shake-up and potentially prolonged system impact after you go to sleep on friday? 19:30:22 if so, holding off to week after next (so that it's post-release) may be more prudent 19:30:37 ttx's call then 19:30:52 anyway, we can get feedback from him out of band before sending the announcement 19:30:59 cool 19:30:59 ok 19:31:01 Nodepool DIB 19:31:10 o/ 19:31:12 that's also humming along, seems like 19:31:23 (sorry I'm late) 19:31:43 mordred: glad you're here 19:32:04 nodepool dib is sort of moving along hit a roadblock in that rax expects a different image format than hpcloud 19:32:14 so I wrote a change to dib to spit out multiple formats of the same image 19:32:23 did they release yet? 19:32:26 #link https://review.openstack.org/#/c/125475/ 19:32:30 it hasn't merged yet :/ 19:32:44 greghaynes is concerned about perceived backward incompat that I don't think is an issue 19:33:11 so I am working on getting devstack-precise into dib with hpcloud too 19:33:20 okie-dokie 19:33:21 #link https://review.openstack.org/#/c/126609/ I need to fix whatever issue the tests pointed out 19:33:26 * mordred is also working on a beefier dib patch 19:34:15 but this way I don't stop moving so I am reasonably happy with the situation 19:34:22 yep 19:34:27 ++ 19:34:32 then once dib change merges (if it does) I will have to update nodepool to set the right flags to get both formats 19:34:47 as an alternative nodepool could do the conversion too 19:34:56 but I feel pretty strongly my image building tool should handle that 19:35:04 agree 19:36:03 okay, anything else? 19:36:09 I was also going to start fiddlign with hosting those images off of nodepool.o.o 19:36:19 hopefully have a change up for that in the not too distant future 19:36:21 oh, right, that too needs doing 19:36:25 okay, great 19:36:46 Jobs on trusty 19:36:49 #link https://etherpad.openstack.org/p/py34-transition 19:36:55 no substandial movement this week 19:36:59 er, substantial 19:37:10 subsundial 19:37:16 indeed 19:37:22 for sundials below the earth 19:37:25 so we can skip this for the same of teh schedule 19:37:33 moving on to meeting topics 19:37:39 #topic Kilo cycle Infra liaisons... should we have them? 19:37:54 I feel I kinda already do in some sense 19:38:02 with Neutron at the very least 19:38:15 just a quick question to put a bug in your brains, though jeblair should probably send out a proper request for liaisons when he returns if that's something we want to do as a agroup 19:38:22 not sure how my work with third party is categorized 19:38:25 can we define infra liason? 19:38:33 yeah, let's have a definition 19:38:40 because I think the term has grown organically over in oslo land 19:38:41 and where did the idea come from? 19:38:45 and may not map 1:1 with what we do 19:38:47 ah oslo 19:39:14 designated people in the server programs, generally core reviewers or other heavily-involved contributors, who help coordinate changes needed to support our efforts 19:39:30 in which direction is the liason? Is this a infra member that joins e.g. Neutron - or a neutron dev that helps infra? 19:39:30 where did that come from? 19:39:39 is that the oslo definition? 19:39:47 AJaeger_: person in neutron that helps infra aiui 19:39:55 all the cool kids are doing it (qa, oslo, et cetera) 19:40:00 ah well not me then 19:40:12 that leaves me out, never been one of the cool kids 19:40:35 all that said I don't think I have any objection to various project volunteers stepping up to learn how bits of project-config work 19:40:41 right, anyway, just something to think about as you see the requests for people to sign up as liaisons to the other so-called "horizontal" efforts 19:40:42 (I assume most of the related happenings will be in that repo) 19:40:55 makes sense 19:40:58 effectively points of contact we can count on in the various projects 19:41:06 that is where the jobs and acls action is 19:41:22 ah well if anyone signs up, they are welcome 19:41:32 didn't want to take up a ton of time on this one, since it's not something we'll decide today 19:41:46 #topic Fedora/Centos testing updates (ianw) 19:41:57 ianw: are you back from your travels i take it? 19:42:25 hi 19:42:56 one was a dib update, but i saw that above 19:43:16 okay 19:43:31 hp cloud now has centos7 images, so i hope to bring up nodepool instances on there so we can get the centos7 job active for devstack changes 19:44:08 i will keep working on the centos7 changes for d-i-b and help out where i can 19:44:22 I'd rather land the centos dib stuff tbh 19:44:24 ianw: we did switch hpcloud devstack-trusty to dib while you were out 19:44:38 so ya we can probably go straight to dib in hpcloud for centos7 19:44:44 ++ 19:45:27 yeah, at this point the train has mostly sailed 19:45:51 clarkb / modred : ok, changes are out there for that, i'm not sure if there are blockers? 19:45:54 #link https://review.openstack.org/#/c/124256/ 19:46:04 #link https://review.openstack.org/123639 19:46:13 #link https://review.openstack.org/124255 19:46:40 ianw: I don't think so. Just review time. I will add that to my dibification list 19:47:00 we can probably just review those normally rather than going over them in the meeting (we've got 13 minutes left) 19:47:08 yep, fine 19:47:19 #topic Log Download (ianw) 19:47:26 #link https://review.openstack.org/122615 19:47:32 not got much feedback on that change 19:47:39 was this a contentious change, or just a request for reviews? 19:47:46 not sure if i'm the only one who wants log bundles 19:47:57 I still don't understand why wget doesn't work 19:47:59 no, i've definitely wished we had that ability in the past 19:48:17 wget should work, though it ends up doing a bunch of downloads 19:48:25 clarkb: one issue with wget is that it sends uncompressed logs down 19:48:28 fungi: for what is roughyl the same number of bits though 19:48:39 ianw: if you set the right header ou get it all gzipped 19:48:46 accept-encoding: gzip iirc 19:48:50 true, modulo the log viewer compression header situation 19:48:58 er, that, right 19:49:30 clarkb: ok, unless you set flags, you get uncompressed logs :) it's pretty unintuitive and i don't think helps people dig into gate failures quickly 19:49:35 anyway, this is another we probably don't need to spend meeting time on. we can follow up in #openstack-infra 19:49:52 ok, yeah, just a request for reviews on that 19:49:57 cool 19:49:59 #topic Publish devstack.org content under infra 19:50:08 (anteaya) 19:50:24 so fungi got control of the domain on the servers 19:50:38 so the next task is to write some jobs to publish the content 19:50:41 we have control over dns for that domain now, yes 19:50:45 where shall we publish? 19:50:57 docs.openstack.org 19:51:03 new server or space on an existing server 19:51:07 seemed like we had discussed publishing to the same place as other developer docs? 19:51:16 sure, where is that? 19:51:20 ++ to existing developer docs 19:51:23 the docs.openstack.org server? 19:51:25 docs.openstack.org/developer 19:51:40 #link http://docs.openstack.org/developer/ 19:51:50 we are just going to have devstack.org as a redirect then? 19:51:52 and link it from http://docs.openstack.org/developer/openstack-projects.html 19:52:16 or maybe have devstack.org also be a ServerName 19:52:26 anteaya: yeah, that was the thought is that devstack documentation ultimately lives there and we deprecate the devstack.org domain using it simply to host a 301 moved redirect 19:52:34 okay 19:52:36 or that 19:52:37 mordred: ya I think CNAME and vhost servername solves it well 19:52:41 clarkb: ++ 19:52:43 like we did with e.g. nova.openstack.org 19:52:45 AJaeger_: you feel like creating the patch? 19:52:51 AJaeger_: and I will review? 19:53:01 anteaya: ok 19:53:02 the current docs for devstack are writting in what? 19:53:05 AJaeger_: my guess is you have more experience than I 19:53:07 AJaeger_: thanks 19:53:20 fungi: feel like making it official with an action item? 19:53:24 mordred: combination of hand-coded html and shocco output 19:53:36 and that should close out this agenda item I do believe 19:53:39 #action AJaeger_ draft documentation publishing job for devstack 19:53:45 dtroyer: ah - so they're just in docs/source in the devstack repo? 19:53:51 yup 19:53:55 should be easy-peasy 19:54:02 tools/build_docs.sh creates docs/html 19:54:06 kk 19:54:06 anything else to cover on this topic? 19:54:08 cool 19:54:12 thanks dtroyer 19:54:47 i'm reordering the last few topics for the sake of time 19:54:56 fungi defer mine to next week please 19:55:02 dtroyer: thanks 19:55:03 The Storyboard migration can be deferred until jeblair is back 19:55:04 hogepodge: okay, can do 19:55:11 Actually, just pull it altogether 19:55:13 krotscheck: thanks, that makes this easy 19:55:19 krotscheck: oh? 19:55:42 fungi: We can bring that up at the storyboard meeting, which he attends, to ask whether he feels ready to propose a migration. 19:55:43 krotscheck: i guess elaborate in #openstack-infra after this 19:55:52 and that's a great venue for it. thanks 19:55:56 so that leaves... 19:56:04 #topic Agenda mgmt 19:56:08 (ianw) 19:56:22 just a quick one, i'm often confused as to when updates happened on the agenda page 19:56:32 can we have a standard of tagging with name and date? 19:56:34 the meeting agenda? 19:56:35 yeah, it's a free-for-all right now 19:56:53 tag with the date of the meeting you want to discuss it at? 19:56:57 and also copy current week to an archived page, that will help too i think 19:56:59 or the date you added it? 19:57:13 fungi: well, either, i just put the date i want to talk about it next to my name 19:57:21 i think we tried keeping the previous week's agenda around for a bit, don't remember how that worked out 19:57:30 it gets long 19:57:33 I thought you were already required to put your name by it? 19:57:41 third party has it, so does qa 19:57:47 * krtaylor needs to archive that 19:57:58 it does get too long 19:58:02 i'll admit i got sloppy and didn't put my name next to a couple since i knew i was going to chair anyway 19:58:06 the page gets really long, and the meeting logs have the agenda items that are called 19:58:08 bad me 19:58:17 * anteaya shakes her finger at fungi 19:58:22 anteaya: yes it gets very long... 19:58:35 I like having a tidy page 19:58:42 so anyway, i don't mind adding a date for the meeting which the topic is targeting 19:58:43 but I can see ianw's concern 19:58:50 might make for a reasonable compromise 19:58:57 wfm 19:58:58 we can try that yeah 19:59:01 anyone else mind doing that? 19:59:11 anteaya : i think bits are cheap, and i'd say move it to a separate archive page 19:59:12 still tider than what thirdparty or qa does 19:59:12 wfm 19:59:19 also we need to get better about removing our topics soon after the meeting if they're suitably addressed 19:59:24 fungi: yes 19:59:35 ianw: then who maintains the archieve page 19:59:40 it's almost like we should hae automation for that ... 19:59:48 mordred: ++ 19:59:53 the challenge with archiving them is the question of whether you have any carry-over or start with a blank slate every time 19:59:58 I've chased people down about expired agenda items, it gets old 19:59:58 we've tended to carry over some 20:00:02 also, we're at time 20:00:03 like #topic++ to pop to the next topic, etc 20:00:16 thanks fungi 20:00:22 thank you everyone! 20:00:31 #endmeeting