22:00:48 <jeblair> #startmeeting zuul
22:00:48 <openstack> Meeting started Mon Jan 30 22:00:48 2017 UTC and is due to finish in 60 minutes.  The chair is jeblair. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:00:49 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:00:50 <GeraldK> bye
22:00:52 <openstack> The meeting name has been set to 'zuul'
22:01:05 <rcarrillocruz> o/
22:01:13 <adam_g> hiya
22:01:14 <jeblair> #link agenda https://wiki.openstack.org/wiki/Meetings/Zuul
22:01:25 <pabelanger> o/
22:01:36 <jeblair> #link previous meeting: http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-01-23-22.05.html
22:01:49 <jeblair> no announcements or actions so...
22:01:57 <jeblair> #topic Status updates: Nodepool Zookeeper work
22:02:06 <jlk> o/
22:02:07 * mordred waves
22:02:15 <jhesketh> o/
22:02:28 <jeblair> the zuul-nodepool integration job is running/working now
22:02:41 <jamielennox> o/
22:02:45 <jeblair> you can say 'check experimental' in a gerrit comment on either the zuul or nodepool repo to run it
22:02:51 <rattboi> o/
22:03:05 <jeblair> it currently fails, correctly, because it's testing that it gets a node from nodepool
22:03:26 <pabelanger> neat
22:03:32 <pabelanger> will check that out
22:03:40 <Shrews> new integration test (that actually succeeds) needs merged (two +2's): https://review.openstack.org/425686
22:03:47 <jeblair> Shrews proposed this patch to add a test for a failed allocation:
22:03:49 <jeblair> yep that one :)
22:03:54 * phschwartz following along, somewhat.
22:04:21 <jeblair> so now it will have a test that passes as well
22:04:31 <jhesketh> neat :-)
22:04:38 <SpamapS> o/
22:04:45 <jeblair> (also -- cool!  nodepool can correctly *fail* a request!)
22:05:02 <SpamapS> You should have asked me, I'm an expert at failing.
22:05:08 <jlk> Lol
22:05:12 <jeblair> SpamapS: plenty of opportunities remain!
22:05:48 <SpamapS> so you're saying I _failed_ to recognize the opportunities...
22:05:51 * SpamapS scores again
22:05:56 <jeblair> SpamapS: success
22:06:29 <jeblair> here's the result of that job run on Shrews's change: http://logs.openstack.org/86/425686/2/experimental/gate-zuul-nodepool/a83475e/
22:06:40 <jeblair> nodepool logs are in the logs/logs/ directory
22:06:59 <jeblair> sorry about the double logs there.  it works and i got tired of futzing with the config.
22:07:20 <Shrews> it's log! it's log! it's big, it's heavy, it's wood
22:07:36 <pabelanger> Still pretty neat
22:08:03 <jeblair> nodepool builder and launchers run as real daemons.  zuul runs inside of tox as a functional test driver, not as an actual daemon.  so its logs are just in the testr subunit artifacts as usual for tests.
22:09:02 <jeblair> any questions or other nodepool related topics?
22:10:02 <jeblair> #topic Status updates: Devstack-gate roles refactoring
22:10:20 <jeblair> rcarrillocruz: is back!
22:10:31 <rcarrillocruz> o/
22:10:34 <rcarrillocruz> https://review.openstack.org/#/c/403732/13
22:10:50 <rcarrillocruz> i just pushed a revision cos i realized the  change did not do what i intended
22:10:59 <rcarrillocruz> with that, we can set CI_USER envvar
22:11:07 <rcarrillocruz> and will be replaced if set
22:11:13 <rcarrillocruz> otherwise, defaults to ansible_user_id
22:11:24 <rcarrillocruz> which is a fact that holds the user running ansible-playbooks
22:11:30 <rcarrillocruz> which in the gate is 'jenkins'
22:11:36 <rcarrillocruz> clarkb ^
22:11:49 <jeblair> and should automatically switch to zuul when we start using that user
22:11:55 <rcarrillocruz> yep
22:12:23 <rcarrillocruz> after this, i'll rebase the network sanity check off this commit and we can flag the setup_host as completed
22:12:28 * fungi breaks radio silence to cheer the return of rcarrillocruz
22:12:44 <rcarrillocruz> :-P
22:12:47 <rcarrillocruz> which raises
22:13:00 <rcarrillocruz> what else do folks think we should refactor as playbook/roles?
22:13:04 * jeblair triangulates fungi
22:13:17 <rcarrillocruz> i held from doing other refactors till i completed this
22:13:17 <jeblair> rcarrillocruz: is that the end of the list we came up with in germany?
22:13:25 <fungi> moshing on a plane over kentucky right now, i think
22:13:41 <rcarrillocruz> pretty much, but i can see a lot of ther things in functions that could be ansiblized
22:14:15 <rcarrillocruz> anyway, i guess we can get back to this when we merge the current patches
22:15:03 <jeblair> rcarrillocruz: personally, i'd say maybe this is a good stopping point and we can focus on other things for a bit -- at least until we're ready to start making a 'standard library' of roles....
22:15:11 <rcarrillocruz> good
22:15:45 <rcarrillocruz> cos that way i can slot my free time for infra for the zuul secrets stuff
22:15:48 <rcarrillocruz> ++
22:15:50 <jeblair> ++
22:16:44 <jeblair> we have enough to demonstrate how we can structure some of our existing jobs in v3, and when v3 is ready to run jobs, this will be ready to plug in and we can demonstrate it.
22:16:56 <jeblair> that was slightly redundant, but you get the idea :)
22:17:20 <SpamapS> rcarrillocruz: zuul secrets stuff?
22:17:45 <SpamapS> I just learned about something really cool 2 days ago that is for automated decryption and I have been wondering about whether it can be used for zuul secrets..
22:17:53 <SpamapS> so perhaps we can loop back to that in open disc
22:17:54 <rcarrillocruz> SpamapS: https://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html#secrets
22:17:59 <SpamapS> ty
22:18:03 <rcarrillocruz> i started it
22:18:10 <rcarrillocruz> then was forced to take vaca as i was leaving hp
22:18:39 <rcarrillocruz> then started at ansible and got sidetracked with new house, new city and new job
22:18:57 <SpamapS> right now I remember
22:19:00 * SpamapS has too many plates
22:19:05 <rcarrillocruz> ++
22:19:23 <jeblair> anything else roles-related?
22:19:30 <rcarrillocruz> jeblair: i'm good for now
22:19:44 <jeblair> rcarrillocruz: thanks!
22:19:47 <jeblair> #topic Status updates: Zuul test enablement
22:19:58 <rcarrillocruz> sure thing!
22:20:52 <adam_g> i updated 409376 to remove one test around merge conflicts that uncovered an issue that had me down a rabbit hole. ill look into it again when i have more cycles to dig, and will probably start picking off lower hanging fruit in the meantime
22:21:20 <jeblair> adam_g: ok, should i be thinking about picking up that work?
22:21:21 <adam_g> where is the best place to record the issues i hit when reenabling some of the tests? storyb oard?
22:21:39 <jeblair> adam_g: yeah, i think SpamapS was developing a 'blocked' story procedure
22:21:47 <SpamapS> We were early on
22:21:57 <SpamapS> but we never hit it so I figured we'd make it ad-hoc until we see 2 or 3 blocks
22:22:17 <SpamapS> The main idea was just to make sure we know _how_ blocked jeblair is. ;)
22:22:21 <jeblair> there's one blocked story (for a test)
22:22:28 <jeblair> from SpamapS :)
22:22:47 <adam_g> ok, i can put it there
22:22:51 <SpamapS> Right and that's blocked on rcarrillocruz's secrets work I think
22:23:00 <adam_g> tho it smells more like a bug to me but ill record what ive found and let others decide
22:23:12 <rcarrillocruz> ah yeah, the swift test no?
22:23:13 <jeblair> adam_g: that sounds good
22:23:26 <SpamapS> adam_g: yeah I'd say something that smells like a bug is story-worthy
22:23:28 <jeblair> rcarrillocruz: yep
22:23:41 <SpamapS> with a zuulv3 tag to help automation
22:23:48 <adam_g> SpamapS: k
22:24:06 <SpamapS> I rebased https://review.openstack.org/413768 and fixed the configs it is based on per jeblair's suggestions, so it is now ready for a second +2
22:24:29 <jeblair> SpamapS: thanks!
22:25:09 <SpamapS> np, I started looking for another test to re-enable now
22:25:23 <jeblair> note for anyone working on tests: https://review.openstack.org/425810 and https://review.openstack.org/425450 may cause conflicts or new failures.  the launcher now asserts that a playbook actually exists for a job, so all the tests now need (noop) playbooks.  i'm thinking of adding an option to specify a playbook for a job (so you can override the implied playbook), which should reduce but not eliminate the need for a bunch of ...
22:25:23 <jeblair> ... noop playbooks in tests.
22:25:57 <jeblair> but look on the bright side:  the launcher now actually cares about and runs playbooks.  :)
22:26:40 <SpamapS> Guard rails on this windy road are appreciated :)
22:26:45 <pabelanger> Yay
22:26:50 <pabelanger> that is exciting
22:27:05 <jeblair> so short actionable version of the above notice: your fake test job may need a fake test playbook from now on.  :)
22:27:09 <SpamapS> I was looking at how many more tests we have to do the other day..
22:27:38 <SpamapS> I believe there are 42 still disabled in feature/zuulv3 ..
22:27:54 <SpamapS> but I can't find where I had looked at how many are in review.. I think it's 10 or so
22:29:18 <SpamapS> we may also have found a point of scale failure in storyboard with https://storyboard.openstack.org/#!/story/2000773
22:29:22 <SpamapS> since it won't load for me right now
22:29:24 <jeblair> SpamapS: i wonder if some stuck in review should go in blocked, or if we need to ping people (possibly me? though i think i've been keeping up)
22:29:45 <SpamapS> jeblair: Indeed, I think that's a worthwhile thing to do.. review them and see if there are some that need kicking out
22:30:00 <SpamapS> I haven't looked closely
22:30:14 <pabelanger> I can go back and look at the stories I created, to make sure I properly closed things
22:30:35 <jeblair> i know the layoutvalidator tests are another area that's different enough in v3 to need a rethink.
22:31:03 <SpamapS> jamielennox: this one looks like it just needs work https://review.openstack.org/406699
22:31:27 <jamielennox> SpamapS: yep, i was just thinking about that one, i'll try and re-propose today
22:31:38 <SpamapS> ^5
22:31:47 <jeblair> i'll +0 that so i stop masking jhesketh's -1
22:31:50 <SpamapS> other than that there's just Adam and my tests waiting
22:32:28 <SpamapS> pabelanger: oh also this one https://review.openstack.org/396684
22:32:45 <SpamapS> test_time_database
22:32:50 <pabelanger> right, I can look at that again
22:33:13 <jeblair> i know there was a lot of travel recently so i refrained from +3ing those last week, but i'll do so tomorrow if no one else has yet.
22:33:27 <SpamapS> feels like most of the ones left are the "hard ones"
22:33:59 <SpamapS> though there might still be a few low hanging fruit growing
22:34:55 <jeblair> so likely good opportunities to do more substantial v3 dev work.  i remain available to help guide anyone who wants to pick one of those.  :)
22:35:41 <jeblair> and of course, happy to help with pre-filtering if you want to know "what would fixing this test entail?"
22:36:06 <jeblair> is that about it for tests?
22:36:26 <SpamapS> jeblair: I like the idea of pre-filtering
22:36:40 <SpamapS> I wonder if we could break the remaining ones out into a few classes.
22:36:51 <SpamapS> and retire the current story.
22:38:06 <jeblair> SpamapS: possibly, but i think i'd prefer to continue to do it on demand.
22:39:01 <SpamapS> jeblair: ACK, that's all I have then.
22:39:11 <jeblair> #topic Status updates: Zuul Ansible running
22:39:39 <jeblair> i added this to the agenda because i suspect there's going to be opportunity for more people to jump in on things soon
22:40:18 <jeblair> my current line of development is mostly around actually running playbooks (as mentioned earlier)
22:41:00 <jeblair> i'll be adding pre- and post- playbooks soon (which will need mordred's security context stuff from the v2 ansible launcher), and then i think we might actually write some ansible :)
22:41:50 <mordred> yay ansible!
22:41:57 <pabelanger> I heard it was a thing
22:41:59 <jeblair> jhesketh: i pulled https://review.openstack.org/385964 into that stack -- your commit message was spot on -- it is very useful for those things.  :)
22:42:18 <jhesketh> jeblair: yeah, when I was looking at that I was looking at adding the pre/post playbooks as you are now
22:42:29 <jhesketh> but I got distracted by other commitments to follow up
22:42:45 <jhesketh> I'm hoping to return to some of it now, but will chat to you later so not to step on your toes
22:42:51 <SpamapS> I'm pretty sure it's a thing, because my socks say "Ansible" on them.
22:43:23 <jeblair> jhesketh: cool, thanks :)
22:43:33 <jeblair> jhesketh: it was very helpful :)
22:44:13 <jeblair> #topic Progress summary
22:44:42 <SpamapS> do I get to link?
22:44:47 <jeblair> SpamapS: all yours
22:44:51 <SpamapS> #link https://storyboard.openstack.org/#!/board/41
22:45:08 <SpamapS> Apologies for my absence the last few meetings, I haven't been able to attend much to this board or talk about it.
22:45:28 <SpamapS> What I see there is mostly that we have _a ton_ of work still.
22:45:34 <SpamapS> But progress has definitely been made.
22:46:47 <jeblair> rcarrillocruz: some ansible tasks are in backlog -- is that correct?
22:46:54 <SpamapS> If you're working on v3, and you don't see your name on that item above, please make sure there's a story or a task on a story, and that the story is tagged 'zuulv3'
22:47:44 <SpamapS> yeah looks like rcarrillocruz tasks are all over in backlog. Probably shoudl be switched to in progress right?
22:48:01 <rcarrillocruz> the installer is def. backlog
22:48:07 <rcarrillocruz> the write localrc too
22:48:12 <rcarrillocruz> but log collector
22:48:16 <rcarrillocruz> i don't remember
22:48:23 <rcarrillocruz> i'll check and update SB accordingly
22:48:39 <jeblair> rcarrillocruz: sounds good
22:49:06 <SpamapS> AWESOME
22:49:08 <SpamapS> oops
22:49:10 <SpamapS> capslockfail
22:49:18 <SpamapS> anyway that's all I have.
22:49:25 <jeblair> SpamapS: i think you had it right
22:49:31 <SpamapS> true
22:49:37 <jeblair> #topic Fedmsg reporter (pabelanger)
22:50:36 <pabelanger> ohai, so i start hacking on this today, because of some discussions at devconf.cz.  I wanted to simply highlight the fact some code is online, but opens the topic of how to report data into fedmsg.
22:51:05 <pabelanger> I was going to propose a simple json format, of the string we have today, but wasn't sure if we should talk more about it
22:51:33 <pabelanger> patch is working BTW, I am hoping to add something to our integration testing too
22:51:40 <jeblair> pabelanger: you might want to consult the sqlalchemy reporter too, that formalizes some data structures
22:51:52 <pabelanger> okay, I will poke into that
22:52:03 <jeblair> pabelanger: tbh, i'd prefer some good fakes for this.  :)
22:52:33 <pabelanger> yes, I could use some help with it.  I have something going today, on how we do smtp reports. But feedback is most welcome
22:52:57 <pabelanger> that is about it, just getting it on the radar of people
22:53:31 <jeblair> pabelanger: (if it's easy to spin up a fedmsg system with a fake broker, that could be done in unit tests)
22:53:58 <pabelanger> fedmsg-tail is what we would use. threebean suggested that
22:55:09 <jeblair> pabelanger: but does that require a zeromq running?
22:55:19 <pabelanger> nope, it works in passive mode
22:55:30 <jeblair> ok cool.  that sounds plausible then.  :)
22:55:39 <pabelanger> ya
22:55:49 <jeblair> #topic  JJB to ansible-playbook (onetime) converter (pabelanger)
22:56:17 <SpamapS> I'm very curious about that one.
22:56:34 <pabelanger> So, I'd like to start work on out 1 time converter, i have no code started yet. I think it is something we might be ready to get going in parallel
22:56:41 <jeblair> i know mordred expressed interest in this topic too
22:56:46 <pabelanger> this would be for our openstack-infra JJB
22:57:01 <mordred> yah - my initial thoughts were that we are _currently_ converting on the fly at runtime
22:57:02 <pabelanger> however, other JJB people are curious on this converter too
22:57:30 <mordred> so I was going to start with the code in zuul ansiblelauncher and use what it's doing to run the conversion on _everything_
22:57:32 <mordred> then see the carnage
22:57:53 <jeblair> pabelanger, mordred: it may be worth designing it so that it can handle our limited use of jjb, but expandible to handle more jjb so that other folks can enhance it?
22:57:54 <SpamapS> Yeah the real question is, does Zuul want to maintain that code long term, or would the world be better if v3 just did a one-time converter and left it behind.
22:57:55 <mordred> then iterate on special cases
22:58:08 <pabelanger> jeblair: yes, my thought too
22:58:53 <mordred> yah - I think other people will need a migraiton tool, so maintaining it to some definition of maintain is likely friendly of us
22:59:00 <jeblair> SpamapS: i think that we don't want to maintain a general jjb -> ansible tool in perpituity.  but considering that there are other v2 zuuls out there that may benefit from it, continuing to support a one-time tool for a while may be useful
22:59:09 <mordred> yah. what jeblair said
22:59:30 <pabelanger> Or even people just running Jenkins with no zuul, they would be interested to see how JJB -> ansible looks like
22:59:43 <SpamapS> ack
22:59:47 <mordred> yup. and what of their jjb would break things :)
22:59:55 <SpamapS> time
22:59:57 <SpamapS> 5s
23:00:21 <jeblair> we can continue in #zuul if needed
23:00:23 <jeblair> thanks everyone!
23:00:25 <jeblair> #endmeeting