22:03:03 #startmeeting zuul 22:03:04 Meeting started Mon Apr 17 22:03:03 2017 UTC and is due to finish in 60 minutes. The chair is jeblair. Information about MeetBot at http://wiki.debian.org/MeetBot. 22:03:05 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 22:03:07 The meeting name has been set to 'zuul' 22:03:14 #link agenda https://wiki.openstack.org/wiki/Meetings/Zuul#Agenda_for_next_meeting 22:03:42 #link previous meetig http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-04-03-22.03.html 22:04:03 #topic Actions from last meeting 22:04:05 mordred add custom path-blocking lookup plugin 22:04:08 that happened! 22:04:14 tests for that are in-progress 22:05:06 twas a good bit of usefulness. 22:05:34 ++ 22:05:35 #topic Status updates (Zuul test enablement) 22:05:43 jeblair: (fwiw, I have not made any progress in the car on more tests - but I may stil lbefore you wake up in the morning) 22:05:51 * mordred is not really here 22:06:00 mordred: cool, thanks. i'll sleep in to increase your chances. :) 22:06:59 i'm not fully caught up yet; are there any new test fixes that are waiting on reviews or anything? 22:07:21 I believe we're good on the ones done 22:07:47 I have one that I don't really understand regarding timer+sshkey things... 22:07:49 but that's ongoing 22:08:00 and then there are a few less-trivial ones left to pick up 22:08:06 overall I think we're in the home stretch. :) 22:10:54 SpamapS: cool, maybe we can poke at the timer+sshkey stuff later this week 22:11:23 #topic Status updates (Zuul sample jobs) 22:13:36 pabelanger: what's the status here? 22:14:13 I think we might be ready to split out some of the playbooks in zuul playbooks folder into another git repo, then start on some stdlib things 22:14:34 right now, it is somewhat confusing what is stdlib and what is openstack-infra 22:14:46 we also discussed bringing nodepool project online too 22:15:22 when you say stdlib, do you mean things that would live in zuul proper? 22:15:26 pabelanger: i'm fairly interested in what we define as stdlib here if you want help/discussion on that 22:15:58 clarkb: right 22:16:06 https://review.openstack.org/#/c/441617/ is the context 22:16:20 ditto what jamielennox said 22:16:24 al la python's stdlib. which batteries we're including with zuul basically 22:16:26 right now, to me, everything in our playbooks folder today, is very openstack-infra specific 22:16:42 which, is okay for now, because it allows faster iteration 22:18:46 yeah, i think there may be three levels here: 1) something so fundamental it shuld go inside of the zuul repo itself and be automatically available to every zuul installation (eg, copy git repos). 2) generally useful things that should maybe iterate faster than the dev cycle for zuul itself (eg, tox). 3) purely local things that should be in openstack-infra/project-config (eg, infra mirror setup). 22:20:41 if we want to draw a distinction at 2 for faster iteration, i think putting the zuul stdlib in a separate repo (covering 1 and 2) which is considered tightly coupled with zuul makes some sense 22:21:21 there's not much about zuul's design which necessitates the stdlib being included _in_ the same git repo, i don't think? 22:21:49 so can sort of have our cake and eat it to 22:22:07 fungi: no, i just wanted to broach the possibility that if it takes extra effort to install the stdlib, we can consider putting things that are very fundamental in zuul itself. 22:22:30 but if we solve "make stdlib easily available", then we don't have to. 22:22:54 could potentially just be an install dep 22:23:03 whcih gets updated more often than zuul (potentially) 22:23:21 thpigh i do think whatever we consider "stdlib" needs some fairly tight api/behavior contracts 22:23:38 s/thpigh/though/ 22:23:41 I don't think we need a new git repo for now, we could start prefixing roles with zuul.foobar or stdlib.foobar, until we decide that step. That allows things to live zuul/playbooks for a while longer 22:24:24 starting in the zuul repo now and splitting them later seems fine to me too 22:24:35 (splitting them later if we decide it's warranted, i mean) 22:25:31 but i do think any playbooks we consider distributed "with" zuul as a stdlib, we need to be fairly certain we can keep their behaviors consistent or provide backward-compatibility when we need to alter them 22:26:13 pabelanger: yeah. i know in review we've discussed things which make sense for stdlib and things which are local. so maybe we should take the step you suggest and start namespacing them so that's clear. then we can defer making the new repo for just a little bit longer. 22:26:41 sure, I'm happy to do that 22:26:51 it will also help make it concrete with low effort so we can paint the bikeshed more easily :) 22:27:16 oh, I imagine stdlib will be many colors 22:27:53 plaid 22:29:51 #agreed start namespacing roles in zuul to help with transition to stdlib 22:30:06 #topic Progress summary 22:30:25 SpamapS: how's the board look? 22:30:35 To be honest I did not look at it the entire last week. 22:30:54 neither did i. but i did see a bear. 22:31:00 #link https://storyboard.openstack.org/#!/board/41 22:31:16 I saw bear _tracks_ in the snow in Mammoth. ;) 22:31:52 SpamapS: oh, we should drop the swift task 22:31:55 jeblair: we had talked about splitting up tags 22:32:10 jeblair: right that's a post playbook now, yes? 22:32:16 so we should change that one to delete the test 22:32:22 SpamapS: or will be, yes. 22:33:47 SpamapS: do we have enough zuulv3.1 stories to warrant a new tag/board? 22:33:57 I think it would help with visualizing where we're at if we start pushing things that aren't critical to infra's use of zuulv3 off to another tag. 22:34:12 jeblair: I'm not sure if it's a lot 22:34:27 seems worth it to just go through them, so I can take that as an action 22:34:59 SpamapS: okay. though it seems there's some overlap with 'backlog' there. 22:35:33 it's a grey area. something to consider at least. :) 22:36:08 well my current thinking had been that all of these things had to be doen before we call v3 done 22:36:22 includin backlog (backlog is just "we know we have to , we don't know when yet") 22:36:41 SpamapS: yeah, though many of the things in backlog aren't actually critical to openstack-infra v3. 22:37:05 so i think there may be overlapping subsets. :) 22:37:10 we could avoid a new tag, and just make the backlog column mean "stuff we know we need to do because we're changing things" and todo can mean "all the things we need to do to use zuul" 22:37:43 yeah, or maybe even add another column or two? 22:37:55 yeah that's another option 22:38:07 the whole point of the board is to allow people to see what to do next 22:38:29 SpamapS: i'll defer to you on how to structure and stand ready to help sort. 22:38:35 (and to see what is thought of, but not next) 22:38:59 jeblair: I think we should just make todo "stuff we have to do to use zuulv3 in infra" 22:39:11 if it gets too big 22:39:20 we can retag some things out of backlog 22:40:04 SpamapS: okay, let's work on that in channel this week 22:40:10 yeah 22:40:24 #topic Where should image username go? (jamielennox) 22:41:12 so yea, simple enough problem that we are building images that use a different username 22:41:31 previously this was controlled in zuul.conf, but it was brought up by jhesketh that maybe this should be controlled by nodepool 22:41:52 i think that's probably right, but we just removed the username option from nodepool because it doesn't ssh any more 22:42:19 would it be terrible to just require a zuul user? 22:42:27 (I'm not sure if this needs to be configurable end of day?) 22:42:35 both options have reviews up but it just requires consensus on an approach 22:43:00 Unless we want images to have different usernames in a provider, I think zuul.conf is fine, right? 22:43:50 it's easy enough to make it configurable in zuul.conf, so even if we wanted to "hard code" it, i'd say we could *at least* do that. however, there's a pretty good argument to be made that when you tell nodepool to build an image, that's a good place to put the information "this is the username to use to log into the image". 22:45:01 i'm also ok with completely punting on this until we have a hard case we can't get around, it was an option we used in 2.5 that is no longer available in v3 that i tried to add back 22:45:36 but in general i think its useful and might help the nodepool outside zuul case 22:45:43 i think 'add to zuul.conf' is also fairly future-proof. in that if we add something either to nodepool or zuul's own node definitions later, that just becomes "default username" instead of "username". 22:46:10 ssh username in nodepool might also makes sense if we every added snapshot images back to nodepool-launcher? 22:46:16 nodepool-builder* 22:47:04 pabelanger: well, the snapshot image build process may not use the same user as the final image, so i don't think that's a given. 22:47:06 my concern with putting it in nodepool is we'd maybe want to have nodepool config affect dib elements somehow to set the user that way you don't set username: foo then wonder why you can't log in as foo? 22:47:11 jeblair: agreed, at the moment the default of zuul comes from the user zuul is running as, not actually in code anywhere so it is a useful default 22:47:17 clarkb: good point 22:47:25 where as if its just "make sure there is a zuul user" that seems to be a straighforward requirement for setup 22:47:41 if we choose nodepool, do we REALLY have to store the username in the ImageBuild, ImageUpload, AND Node objects? 22:47:48 jamielennox: oops, i think 'default to current running user' is a mistake. :) 22:47:59 that seems excessive 22:48:27 Shrews: it was the only way i could see to pass it from build through to ready 22:48:36 clarkb: I know off-topic, but the removal of ssh from nodepool makes that a little more difficult to troubleshoot, since zuul needs to now be involved 22:48:41 could maybe skip imageupload since it references imagebuild 22:49:09 yeah, i think we're unintentionally getting the ssh client implementation's default, so being explicit even about what our default is seems preferable 22:49:10 (you could also skip node, but then that would require zuul to look up the imagebuild, which it doesn't have to now) 22:49:44 pabelanger: i'd like to follow up on how that's more complex after the meeting 22:50:32 so i'm reading lukewarm support for addition to nodepool -- for now at least; and more support for zuul.conf. 22:50:54 think so 22:51:00 anyone disagree with that ^? 22:51:19 i think it's good to have a default there anyway and we can deal with nodepool if/when required 22:51:52 I agree, I do not disagree 22:52:36 i agree with jlk's not disagreeing 22:53:47 #agreed configure zuul ansible ssh user in zuul.conf for now; revisit adding to nodepool/zk as needed later. 22:54:14 jamielennox: thanks for pushing that through! 22:54:27 #topic Executor security spec: https://review.openstack.org/444495 22:54:39 i put this on the agenda at fungi's suggestion earlier 22:54:50 we chatted about it at length in channel 22:55:13 i don't know whether any further discussion is warranted after we talked in here a few hours ago, but it's coming up on last call for dissent 22:55:39 I'm personally comfortable running with it for BonnyCI 22:55:44 so just wanted to be sure since clarkb felt pretty strongly about some of it 22:55:54 which is an attempt to make a Zuul-aaS for all github projects 22:56:05 which is likely to have a really nasty set of horrible bad actors 22:56:14 so.. 22:56:16 I'm deferring to SpamapS on the wrapper around the thing 22:56:30 I've focused more on the thing, and the bad things the thing can be coaxed into doing 22:56:31 bubblewrap is some of the simplest, easiest to read C I've looked at 22:56:43 I highly recommend taking the tour 22:56:56 "the least painful open would i've ever had" 22:57:02 wound 22:57:23 I have tried treating the wound with liberal application of Rust... https://github.com/SpamapS/rubble ;-) 22:57:25 i also think this is reasonable for openstack-infra. 22:57:35 i'm confident at this point that if particularly concerned zuul admins wanted to limit things to only running trusted playbooks on the executor and chaining to running untrusted ansible playbooks on the test nodes, that they can basically ignore the sandboxing (but it does help protect the system from their trusted playbooks, even then) 22:58:42 we could also make the wrapper libvirt ... nested VMs are so much fun 22:58:54 fungi: I think you'd need to modify zuul a bit for that, but agreed it would be simple to do if necessary 22:59:08 fungi: strictly speaking, "trusted" playbooks in zuulv3-speak would run outside of the sandbox, but your scenario is adding another layer, so maybe that's what you meant (the playbooks they trust will run as untrusted zuul playbooks and would then run something else on a test node) 22:59:10 weighing the features this allows against the risks, i'm in favor of supporting it, personally 22:59:39 oh, yep, s/trusted/tightly-controlled/ 22:59:49 fungi: then yes, agreed. :) 22:59:51 sorry, overloading terminology there 23:00:20 i only brought up the nitpick in case it exposed some misunderstanding, but i don't think so. 23:00:21 overloading terminology that's actively being changed :) 23:00:37 fun for the whole family 23:00:44 jlk: "this statement may contain future-looking terminology" 23:00:50 thanks for following up on this! 23:00:51 hah! 23:01:03 i have what we need to move forward on the spec review 23:01:18 fungi: thanks! 23:01:30 looks like this should end up on tomorrow's infra meeting agenda 23:01:32 thanks all! 23:01:34 #endmeeting