19:03:52 #startmeeting infra 19:03:53 Meeting started Tue Mar 21 19:03:52 2017 UTC and is due to finish in 60 minutes. The chair is fungi. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:03:54 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:03:56 The meeting name has been set to 'infra' 19:04:04 #link https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting 19:04:15 #topic Announcements 19:04:30 something something forum something 19:04:47 haha 19:05:09 #link http://lists.openstack.org/pipermail/openstack-dev/2017-March/114352.html if you think we should have specific forum sessions for infra. please tell me 19:05:53 oh, slightly wrong link (but mostly right) 19:05:57 #undo 19:05:58 Removing item from minutes: #link http://lists.openstack.org/pipermail/openstack-dev/2017-March/114352.html 19:06:13 #link http://lists.openstack.org/pipermail/openstack-dev/2017-March/114352.html if you think we should have specific forum sessions for infra, please tell me 19:06:33 my perspective of infra at the openstack forum is that we fight for the users (like tron) 19:06:52 how are those links different? 19:06:58 they look the same to me 19:07:03 anteaya: hidden white space? ;) 19:07:06 yeah, they're not :/ one more 19:07:08 ah 19:07:12 #undo 19:07:13 Removing item from minutes: #link http://lists.openstack.org/pipermail/openstack-dev/2017-March/114352.html 19:07:25 and I was believing AJaeger 19:07:29 #link http://lists.openstack.org/pipermail/openstack-dev/2017-March/114399.html if you think we should have specific forum sessions for infra, please tell me 19:07:31 THAT one 19:07:48 as always, feel free to hit me up with announcements you want included in future meetings 19:07:49 I'm going to propose a sessions to talk about https://etherpad.openstack.org/p/openstack-user-api-improvements but thats not infra specific and is instead fighting on light cycles 19:07:51 thanks 19:08:20 clarkb: i'm in favor of anything involving lightcycles. go for it 19:08:36 #topic Actions from last meeting 19:09:12 #link http://eavesdrop.openstack.org/meetings/infra/2017/infra.2017-03-14-19.03.html Minutes from last meeting 19:09:22 ianw try booting a Xenial-based replacement for planet.openstack.org 19:09:36 not yet sorry ... will do 19:09:45 eh, no worries 19:09:50 #action ianw try booting a Xenial-based replacement for planet.openstack.org 19:10:03 #topic Specs approval 19:10:20 #info APPROVED: Zuul v3: remove references to swift 19:10:43 #link http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html#jobs Zuul v3: remove references to swift 19:10:53 #info APPROVED: Zuul v3: update job trees to graphs 19:11:11 #link http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html#projects Zuul v3: update job trees to graphs 19:11:58 doesn't look like there are any new proposed for council vote this week, but keep an eye on open specs changes for any which are ready to be brought forward 19:12:14 #topic Priority Efforts 19:12:28 nothing called out specifically here for this week 19:12:47 #topic puppet 4? (cmurphy) 19:12:54 hi 19:12:58 exercising chair privilege to reorder my topic to teh end 19:13:09 first of all I apologize for being out of the loop for a while 19:13:19 I wanted to gage current temperature on some old ideas 19:13:38 is it worthwhile to try to move to puppet 4? 19:14:01 we use puppet 4 on fedora? so things should mostly work? 19:14:09 i assume as long as we're using puppet we should try to keep current on it, but i grant that not everyone necessarily shares my ideas on that matter 19:14:17 cmurphy: welcome back! 19:14:21 I think its trickier in some of our more involved manifests (the test image builds are relatively simple and work on 4) 19:14:33 ianw: that is my thinking, most of it should work already 19:15:00 ianw: i expect most of our puppet modules are not exercised on the f25 jobs 19:15:03 how soon will puppet 5 be a thing and should we just jump if its soon? 19:15:24 ahh, yes, for general deployment 19:15:30 * fungi also wonders about puppet 6 and 23 19:15:30 I think we we enhance some of our beaker testing we can do a better job of validating whether our modules work on puppet 4 19:15:47 i got puppet-openstackci to work on puppet 4 https://review.openstack.org/#/c/447243/ 19:15:57 fungi: well I think 5 is in dev and coming soon? I may have confused myself though and am thinking of something else like ansible 19:16:39 puppet 3 is not supported by puppet anymore AFAIK 19:16:51 AJaeger: yes that is correct 19:16:53 so, no more security release if something comes up 19:17:03 o/ 19:17:10 clarkb: sure, i'm mostly wondering if we in general should stay on the puppet train wherever it leads, until we decide otherwise 19:17:15 also the upstream modules are moving away from supporting puppet 3 19:17:27 AJaeger: ya which is a really small surface for us because we don't run the puppet master anymore. The upstream modules leaving us behind is the bigger issue I think 19:17:52 yeah, the upstream puppetlabs support of puppet is less of a concern as we have distro packages we could use 19:18:00 modules on the other hand 19:18:19 relatedly I was wondering if it would be a good idea to try to move openstack_project::single_use_slave into dib elements so that the nodepool images no longer need to depend on puppet 19:18:51 Ya, we talked about that for zuulv3 things 19:19:06 that should be pretty close now, yeah? 19:19:34 it is? 19:20:12 * fungi feels like zuulv3 is close for that matter 19:20:34 i thought we made significant progress a while back. what's left? iptables, unbound, exim? did the ssh key thing get worked out? 19:20:47 ssh is a non issue imo 19:20:52 we use devuser element and its done 19:20:59 cool 19:21:02 it should be a fairly hollow class at this point, and i don't think we have any objections to cleaning out the rest of it 19:21:09 ah I wasn't part of that 19:21:14 ya, we have a base zuul-worker element now, we should start iterating more on that 19:21:22 bindep was a big step for that 19:21:38 but yes iptables, unbound, exim would be the big ones 19:21:54 deploying admin users too 19:21:56 iptables is likely straightforward as well and we could/should use a generic element for that if one doesn't arleady exist 19:22:01 the bindep part is done right, or do we still depend on puppet for fallback? 19:22:09 ianw: we had talked about just installing zuul/jenkins 19:22:15 ianw: we discussed about stopping deploying admin users, in favor of using zuul 19:22:19 clarkb: ++ 19:22:25 ianw: and force admin users to get on the hosts the same as anyone else using the images locally 19:22:26 ianw: i'm fine with not having my account on the workers and just logging in as jenkins^Wzuul 19:22:40 I can try to dedicate some time to finishing that up 19:22:54 and I can propose a spec to finish the puppet 4 work 19:22:59 cmurphy: that would be highly appreciated 19:23:14 fwiw I can't find a release date for puoppet 5 so I am probably mistaken on that being soon 19:23:20 both the cleanup and any puppet 4 spec you want to kick around 19:24:01 cool 19:24:17 I would not object to createing ubuntu-xenail-ng in nodepool.yaml, and iterating on that in parallel for zuulv3 things 19:24:30 if we don't think that complicates things 19:24:36 i doubt we even need a new image for that 19:24:46 unless there's something obvious i'm overlooking 19:25:01 ya I think just go piece by piece and test local builds should be good 19:25:11 we already integration test with the nodepool devstack job right? 19:25:17 yes 19:25:22 but no ready-script things any more 19:25:38 but, that should be okay 19:25:41 That seems simpler 19:25:48 we erady script on master 19:25:55 yeah, i'd like to assume our current testing is sufficient, or improve it 19:25:56 true 19:26:14 however, we don't use install_puppet.sh for our nodepool dsvm jobs 19:26:18 granted, it's possible i live in some sort of fairly land 19:26:23 fairy land too 19:26:28 :) 19:26:29 ya its a much simpler ready script 19:26:30 either way, I am happy to help 19:26:32 but the machinery is there still 19:26:39 would be great if you lived in fairly land, I'd go there 19:26:50 it's a fairly satisfying place 19:26:56 sounds like it 19:27:05 if you like compromises, i recommend it 19:28:09 cmurphy: anything else on this, or do you have what you need? 19:28:25 fungi: nothing else, just wanted to float the idea 19:28:48 idea floated, reinforced, and currently planned for condo units 19:29:12 #topic Can we ever get rid of openstack-infra and openstack-dev Git namespaces? (fungi) 19:29:22 please? 19:29:32 the shade split-out discussion has brought this back to the forefront 19:29:36 * fungi gets link 19:29:42 the whatnow? 19:30:13 #link https://review.openstack.org/446426 Move shade into its own top-level team 19:30:23 i feel like someone buried the lede on this 19:30:48 the idea that shade, requestsexceptions and oaktree belong in a team unto themselves (probably with some/lots of infra people) 19:31:19 in part beacuse apparently openstack wants to be able to consume these tools and can't do that unless they are part of openstack themselves? 19:31:21 neat? did i miss something? 19:31:23 avoiding making infra deliverables direct deps on openstack[tm] services 19:31:49 maybe that should have been an infra meeting topic instead 19:32:03 i'm sorry, i don't want to distract from the topic at hand. but i am curious what newsletter i should sign up for so i get to hear about this sort of thing. 19:32:11 mordred is not here to defend himself so feel free to have at him ;) 19:32:25 I'm still not sure I agree with that premise (I also don't think that openstack should dep on shade and friends on principle/principal? english hard) 19:32:33 umm, does that mean our recent discussions about dib mean it's not appropriate for infra? (do not wish to derail, but seems similar) 19:32:37 jeblair: I only caught it bceause I saw the governance patch go by in #opensatck-dev 19:33:04 jeblair: i think this came from a discussion in #openstack-shade between mordred and dhellmann 19:33:17 based on scrollback i read 19:33:22 ianw: IMO it should be fine for openstack to consume libs from not openstack as long as the licensing is compatible and the software is not dead 19:33:28 ianw: that's a good point if dib is a dep of official openstack[tm] services (the subsequent discussion there brought up that dib-utils is a dependency of tripleo) 19:33:35 ianw: which means it should be fine from that perspective to consume dib from infra and shade from infra 19:33:56 (I separately think that shade being the hack around openstack's problems means that openstack itself shouldn't dep on it and should instead fix its problems) 19:34:16 clarkb: such yes 19:34:27 shade as a dep of openstackclient and oaktree is what i think brings this discussion to a head 19:34:42 clarkb: oxford says 'principle' http://blog.oxforddictionaries.com/2011/08/principle-or-principal/ 19:34:56 fungi: osc depends on shade? that's news to me 19:35:09 occ, yes, but not shade 19:35:21 Shrews: oh, occ, not osc. you're right 19:35:35 the dependency thing doesn't make sense to me. however, if folks who work on shade more than i do think that it should become a first-class deliverable of the openstack project because openstack needs this kind of interface to actually make it work, that sounds more compelling. that's not what the commit message says though. 19:36:24 if folks who work on openstack are okay with the fact that openstack needs this kind of interface to actually make it work, i'm pretty sad 19:36:40 but that seems like where things are headed 19:36:40 is the issue more that we shade doesn't have stable branches? 19:36:48 which makes it hard to track against 19:36:55 and isn't under openstack release management 19:37:04 i'd prefer realism to idealism either way. :) 19:37:10 jeblair: Can't say it was a group decision. I was just informed about it yesterday myself and haven't put much thought into whether or not it is a good idea or not. 19:38:21 fwiw, i haven't voted on it in either a ptl or tc capacity (yet). mordred did briefly run the commit message by me 19:38:26 yesterday in #openstack-dev mordred said "I think the actual problems go away with restification" 19:38:33 which is already the plan aiui 19:38:35 (also, minor footnote for history, requestsexceptions was pulled from gertty) 19:38:57 restification? 19:39:02 clarkb: ya, that also makes packaging easier too 19:39:10 jeblair: replacing shade's deps on python-fooclient and just talking openstack api with rest directly 19:39:15 i think requestsexceptions was rolled into the proposal because shade depends on it 19:39:29 again i highlight the term "proposal" 19:39:30 oh, the packaging problems, yeah 19:40:00 restification is great for that. it makes nodepool packagable. :) 19:40:12 my only real concern here is that it feels like we are proxying the real issues with this proposal rather than directly tackling the problems that shade users face 19:40:26 its possible that this is actually a good fix for those problems but its hard to know without calling them out more explicitly 19:40:29 s/shade/openstack/ 19:41:14 shade is one (fairly successful) attempt by openstack users to make sense of the openstack api landscape 19:41:56 clarkb: yeah, i'd love it if we can have a conversation about what's actually going on. hopefully mordred will drop by and chat. 19:42:36 fungi: i dunno if you want to get back to the actual topic, or if this one is more fun -- but yeah, i'd love to flatten the namespace. :) 19:42:49 he indicated to me that he's unable to attend this week. i'm happy to table this discussion until he's around. it's not like that governance proposal is getting approved without my vote 19:42:51 +1 to further flattening 19:43:05 I think the biggest pain there is going to be devstack 19:43:11 honestly, the stackforge transition was doable. the big thing is.. 19:43:13 yeah devstack :) 19:43:38 but maybe we just set up a window, do it, accept things will be broken and fix them as quickly as possible 19:43:40 so anyway, to un-derail (my fault really) this topic, the suggestion of moving shade to its own official team was met with the objection "but it's in the openstack-infra git namespace" 19:44:10 which is irrelevant, but i'm happy to take the opportunity to clean it up (to further show it's irrelevant) :) 19:44:19 "users will be confused if openstack-infra namespace repos are dependencies of openstack proper namespace services" 19:44:25 but also we could use redirects more aggressively in places 19:44:33 fungi: that's a quote said by a person? 19:44:37 to alleviate the pain of transitioning 19:44:42 fungi: I am more confused my the statement :) 19:44:42 by* 19:44:46 it would be relatively simple even now to redirect all namespaces on git.o.o 19:45:25 a related question is: should we drop openstack/ from gerrit and git.o.o? 19:45:45 pabelanger: so the supposition is that the current semi-random use of different git namespaces is non-random, because people want to assume patterns when they look at things, and so will assume misleading meanings 19:45:58 jeblair: if we can, absolutely in my opinion 19:46:11 i think that would be a nice benefit to help justify the pain 19:46:15 jeblair: yes 19:46:24 That would remove assumptions altogether 19:46:28 i thought it was the most compelling reason, honestly 19:46:33 +1 to dropping namespaces instead of merging. 19:46:34 jeblair: ++ 19:46:45 imagine how much "money" we will save by not making developers type so much! 19:46:53 ha 19:46:53 hah 19:46:57 hahaha, I was thinking how much it'd be nicer as a user though! 19:47:06 user of gerrit 19:47:09 end result being that github has openstack/\(.*\) and we serve git.o.o/\1 19:47:28 which also makes sense, and clearly establishes github as a mirror 19:47:38 and project shortnames in gerrit (and storyboard for that matter) 19:47:53 oh, yes, that! 19:47:56 don't forget tox.ini 128 char limit too :) 19:48:02 * ttx always wanted to get rid of the prefix in storyboard 19:48:03 so maybe we should just do all of those things at once? pick a nice time in a cycle to do it (is there a nice time?). lots of warnings, etc. prepare project-config changes ahead of time. use codesearch to help prep related changes. 19:48:03 yeah, it would make shortname support in storyboard much easier 19:48:15 jeblair: summit week? 19:48:25 at least it's not around release time 19:48:30 or very early queens 19:48:32 and we expect lower activity 19:48:57 ttx: tempting -- though would it annoy people that we will "break" things while they are summiting? 19:49:09 this is something which would probably need wide advance notice (much like the stackforge migration, but without the sliding migration opportunity probably) 19:49:34 jeblair: only drawback I think is that we might have less bandwidth to fix it for people who are trying to do work during that week 19:49:43 i expect to need to fix quite a lot 19:49:48 fungi: how much time in advance was the stackforge migration again? 19:49:55 fungi: yeah -- if we start now, we can probably do it in conjunction with the boston summit 19:50:00 I'll have to check how much would need fixing 19:50:08 i don't remember, but my recollection is at least 4 months 19:50:42 people will complain, but at least we won't hear about it anymore after that 19:50:44 ttx: i expect we'd end up manually bypassing gating for more than a few circular dep changes 19:50:48 i'm not sure that much more than a month is very useful. 19:50:55 but obviously as much as possible. 19:51:35 the people who will listen will listen if you give them a month, the people that won't listen won't listen regardless of how much time you give them 19:51:56 well, there's a good point. with new redirects on git.o.o (and as usual on github) we really only need the dev community aware beforehand 19:51:58 we're at ~7 weeks before summit 19:52:38 mainly because there will be LOTS of .gitreview changes needed 19:52:45 * EmilienM is hoping to have time for quick open discussion 19:53:08 i'm mostly convinced we should exercise admin oversight and just manually approve .gitreview patches in bulk even 19:53:25 that might work 19:53:28 maybe someone can write up a quick proposal and we can iterate on the ml? 19:53:40 cause it seems like we're generally in favor and have folks willing to do the work. 19:53:47 yeah, i'm happy to put this one forth, since it's my meeting topic 19:54:03 sold 19:54:19 I'll look up the release machinery for openstack/ or / stickyness 19:54:31 infra people are generally in favor, the wider community may raise objections but as long as we have good answers that's probably tractable 19:55:01 "takes less space on disk" 19:55:03 #action fungi put forth proposal to flatten git namespaces 19:55:12 ttx: now you're getting it 19:55:29 #topic Open discussion 19:55:35 EmilienM: what's up? 19:55:44 hey infra! I want your feedback & vote on moving DIB to infra project: https://review.openstack.org/#/c/445617/ 19:55:58 IIRC at one point we proposed replacing all git repo names with UUIDs to avoid renaming altogether 19:56:04 I think I resolved all concerns in my patch 19:56:09 EmilienM: funny, that just came up in the context of the reason shade was proposed to move out of infra ;) 19:56:25 ttx: I want to see that patch 19:56:28 it's okay, it's already in openstack/ :) 19:56:42 ttx: sold, as long as we can spell fun things in hexidecimal 19:57:10 are we doing any precise migrations this week? 19:57:18 EmilienM: seems good to me -- it's important enough i'm happy to contribute to emergency maintenance if needed. 19:58:14 pabelanger: ianw is still planning to test planet.o.o on xenial, jeblair and clarkb snapshotted lists.o.o to test in-place upgrading, and puppetdb can just be deleted for now until someone has time to fix the service entirely. that's all our remaining precise servers afaik 19:58:32 okay, I can dive into puppetdb 19:59:02 pabelanger: if you want to difure out puppetdb and puppetboard, then awesome. otherwise feel free to remove the server from site.pp and delete it in the meantime 19:59:15 s/difure/figure/ 19:59:23 fungi: sure, I'll propose a patch for removal 19:59:34 maybe get a spec up for ARA? 19:59:50 seems like a reasonable alternative now that we ansibkle 19:59:53 ansible 19:59:56 also we're at time 20:00:06 thanks everyone! productive meeting 20:00:10 #endmeeting