19:00:24 <dmsimard> #startmeeting infra
19:00:25 <openstack> Meeting started Tue May 15 19:00:24 2018 UTC and is due to finish in 60 minutes.  The chair is dmsimard. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:25 <fungi> howdy
19:00:26 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:00:28 <openstack> The meeting name has been set to 'infra'
19:00:38 <dmsimard> #link https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting
19:00:53 <dmsimard> #topic Announcements
19:01:21 <dmsimard> OpenStack Summit and OpenDev are next week ! Let us share our excitement. Yay
19:01:33 <corvus> and there was much rejoicing!
19:01:40 * fungi rejoices
19:01:42 <corvus> also panic
19:01:51 <fungi> wait, what, next week?!?
19:01:57 <dmsimard> I certainly hope you have your slides and stuff ready cause I don't
19:02:05 * fungi checks his todo list in bewilderment
19:02:22 <dmsimard> Also clarkb will be back friday
19:02:58 <dmsimard> and there won't be a meeting next week due to the majority of bodies being in Vancouver
19:03:27 <fungi> that's sort of a meeting
19:03:38 <dmsimard> did we have other announcements worth of a mention ?
19:04:02 <fungi> there will be a zuul v3 for gating session in room 301 at this time next week
19:04:15 <fungi> and a git and gerrit lunch&learn right after
19:04:38 <dmsimard> Oh, I suppose there will be something social happening next week. Hopefully you can be with us on monday evening - details on the mailing list
19:04:41 <dmsimard> #link http://lists.openstack.org/pipermail/openstack-infra/2018-May/005943.html
19:05:05 <dmsimard> fungi: do you have links for those sessions ?
19:05:20 <fungi> dmsimard: nope, this is from my ~/.reminders file
19:05:47 <fungi> probably better to just remind people to search the summit schedule for keywords like "zuul" and "opendev" and "forum"
19:05:56 <dmsimard> #link https://www.openstack.org/summit/vancouver-2018/summit-schedule/events/21663/zuul-v3-for-gating
19:06:16 <dmsimard> #link https://www.openstack.org/summit/vancouver-2018/summit-schedule/events/20783/git-and-gerrit-lunch-and-learn
19:06:22 <dmsimard> there you go
19:06:32 <dmsimard> okay, let's move on
19:06:33 <fungi> much appreciated
19:06:49 <dmsimard> #topic Actions from last meeting
19:07:36 <dmsimard> #link http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-05-08-19.01.txt minutes from last meeting
19:08:25 <dmsimard> pabelanger: we were looking for scheduling the xenial upgrade sprint
19:09:12 <dmsimard> #link https://ethercalc.openstack.org/osiuhjzjq336 ethercalc for sprint availability
19:09:23 <dmsimard> #link https://etherpad.openstack.org/p/infra-sprint-xenial-upgrades-part2 servers to upgrade
19:10:25 <fungi> i guess that wasn't listed as an action item, but could have been
19:10:41 <dmsimard> aye, that was my takeaway :)
19:11:23 <dmsimard> not sure if pabelanger is around but it looks like people are generally available so it should hopefully only be a matter of scheduling it and reserving the sprint channel ?
19:12:15 <dmsimard> let's formalize it as an action item especially considering there's no meeting next week ?
19:14:04 <fungi> sounds great
19:14:41 <fungi> i'm guessing pabelanger isn't around today or something
19:15:30 <dmsimard> #action schedule the sprint for xenial upgrades and reserve the virtual sprint channel
19:16:02 <dmsimard> moving on
19:16:12 <dmsimard> #topic Specs approval
19:16:32 <dmsimard> The only spec that was up for approval last week was the survey tool spec which was merged: https://review.openstack.org/#/c/349831/
19:16:38 <fungi> yay!
19:16:47 <dmsimard> Was there any other spec that should be up for approval ?
19:16:57 <fungi> and the initial puppet change for that one is probably close to approval
19:17:09 <dmsimard> I guess we've made progress on the modern configuration management specs
19:17:12 <fungi> #link https://review.openstack.org/557979 This patch impliments the survey spec
19:17:19 <dmsimard> fungi++
19:17:39 <fungi> so if we can get some eyes on that, i'm working to put together a follow-on documentation addition for system-config too
19:18:00 <fungi> oh, and tehre's a demo up built from the current state of that change
19:18:08 <fungi> #link https://survey01.openstack.org/
19:18:19 <dmsimard> I was pleasantly surprised at limesurvey, I didn't know about it until I saw the spec about it
19:18:24 <fungi> self-signed cert for now, and the real deal will have a survey.openstack.org cname to it
19:18:57 <anteaya> I'd be interested in feedback if anyone has the time to try it
19:19:16 <anteaya> create a survey if you can and let me know how you felt about the experience
19:19:32 <anteaya> the limesurvey docs should be helpful for survey creation
19:19:35 <dmsimard> anteaya: on survey01 ?
19:19:40 <anteaya> correct
19:19:55 <anteaya> openstackid auth should be working
19:20:04 <anteaya> survey01.openstack.org/admin
19:20:08 <dmsimard> I'll give it a try :)
19:20:14 <corvus> anteaya: you could set up a survey to ask folks for their experience creating a survey
19:20:16 <anteaya> should get you the openstackid page
19:20:25 <dmsimard> wouldn't that be amazing corvus
19:20:26 <anteaya> corvus, great idea
19:20:39 <fungi> yeah, we know of a couple warts which can probably be solved (elevating an initial openid-autocreated account to superadmin perms requires a manual db insert, autocreated users get static initial name and e-mail address they need to adjust in their profile which can likely be solved by writing a very tiny php hook to get that info from mod_auth_openid)
19:21:22 <fungi> also it's at the default theming now, but is highly themable
19:21:39 <anteaya> highly themable
19:21:52 <fungi> themeable?
19:21:58 <fungi> something like that ;)
19:22:00 <dmsimard> ¯\_(ツ)_/¯
19:22:08 <anteaya> I think your first go at it looked good to me
19:22:21 <dmsimard> ok, let's move on to specs discussion
19:22:34 <dmsimard> unless there was something else about survey
19:22:41 <anteaya> I'm done, thank you
19:22:59 <fungi> not from me
19:22:59 <dmsimard> #topic Priority efforts
19:23:23 <dmsimard> Any storyboard things ?
19:24:16 <diablo_rojo> LOCI migrated last week
19:24:27 <dmsimard> sweet !
19:24:32 <diablo_rojo> We have one forum session at the Summit as well
19:24:37 <diablo_rojo> I think its Monday?
19:24:45 <dmsimard> I've been seeing some amount of progress from the different TripleO projects as well
19:24:58 <diablo_rojo> #link https://etherpad.openstack.org/p/continuing-the-migration-lp-sb Storyboard Forum Session Etherpad
19:25:12 <fungi> yeah, i should add some things to that etherpad
19:25:15 <dmsimard> diablo_rojo: https://www.openstack.org/summit/vancouver-2018/summit-schedule/events/21736/continuing-the-migration-launchpad-storyboard ?
19:25:17 <fungi> might happen while i'm on an airplane
19:25:21 <diablo_rojo> dmsimard, thats the one
19:25:25 <dmsimard> #link https://www.openstack.org/summit/vancouver-2018/summit-schedule/events/21736/continuing-the-migration-launchpad-storyboard
19:25:32 <diablo_rojo> fungi, that works :)
19:25:41 <diablo_rojo> can do it on our sb meeting tomorrow too
19:26:44 <fungi> ahh, yes
19:26:48 <fungi> i'll be around
19:26:56 <diablo_rojo> After the summit we will get some more migrations scheduled.
19:27:06 <diablo_rojo> I saw masakari talking about it in their meeting last night
19:27:09 <fungi> keep in mind, if you're helping with the speed mentoring lunch, you need to duck out early to catch the start of the sb session
19:27:14 <diablo_rojo> Keystone wants me to do a test migration too.
19:27:30 <diablo_rojo> fungi, thanks for the reminder :)
19:27:53 <fungi> like, maybe skip the group photo at the end (luckily it's just down the hall on the same floor)
19:28:15 <diablo_rojo> There's a group photo at the end?
19:28:19 <dmsimard> mordred, cmurphy: are you around for the next topic ? specs for configuration management
19:28:38 <fungi> diablo_rojo: i guess you skipped the speed mentoring conference call today ;)
19:29:31 <mordred> dmsimard: heya!
19:29:40 <mordred> (sorry, was late returning from lunch)
19:30:36 <dmsimard> diablo_rojo, fungi: did we cover our updates for storyboard ?
19:31:05 <diablo_rojo> dmsimard, yep I think so
19:31:14 <fungi> yeah
19:31:17 <diablo_rojo> fungi, oops yeah I missed that entirely
19:31:18 <fungi> all good
19:31:33 <fungi> diablo_rojo: i'll sync up with you after this meeting
19:31:50 <dmsimard> alright, moving on
19:31:59 <dmsimard> #topic #topic Modern Config Management
19:32:05 <dmsimard> let's try that again
19:32:06 <dmsimard> #topic Modern Config Management
19:32:16 <dmsimard> #link https://review.openstack.org/449933 Puppet 4 Infra
19:32:24 <dmsimard> #link https://review.openstack.org/469983 Ansible Infra
19:32:32 <dmsimard> #link https://review.openstack.org/565550 Containerized infra
19:33:04 <mordred> so ... I owe everyone a combined/wrap-up spec
19:33:11 <mordred> which I have not written because I'm a bad person
19:33:12 <dmsimard> My take away is that we're relatively certain the way forward is not with Puppet
19:34:11 <mordred> dmsimard: yah - the tl;dr of the combined spec is - short-term puppet 4, start shifting things to ansible and start deploying things with containers (using ansible) where appropriate
19:34:26 <mordred> but there's many details in that which warrant a non-tl;dr ;)
19:35:07 <dmsimard> Do we have some generic infra sessions at the forum ? I haven't really booked my schedule yet
19:35:16 <dmsimard> It would likely be pretty productive to discuss this in person
19:35:26 <mordred> given we will not have an infra irc meeting next week because summit, my new goal is have a spec ready for people to have read by 2 weeks from now
19:35:55 <mordred> maybe I'll try to crank on it before the summit so that if we do manage to discuss at summit we have a summary to start from
19:36:31 <fungi> also possibly puppet 5 if phase 2 takes too long and we end up facing puppet 4 eol with lots left to do in ansible land
19:36:58 <fungi> at least sounds like puppet 5 isn't a huge leap from 4 anyway
19:36:59 <mordred> yah
19:38:03 <dmsimard> fungi: we've long outlived the life of puppet 3 :)
19:38:32 <fungi> indeedily-doodily
19:39:09 <dmsimard> mordred: want to take an action to be accountable for that non-tl;dr ? :D
19:40:35 <mordred> dmsimard: yup. that's on me
19:40:42 <dmsimard> ack
19:41:39 <dmsimard> #action mordred to provide an update for combining the modern configuration management specs
19:41:54 <dmsimard> On to general topics
19:42:06 <dmsimard> #topic General Topics
19:42:45 <dmsimard> This is an idea I had in the shower last night (best ideas come from the shower, I tell you.)
19:43:09 <mordred> dmsimard: mine usually come from long road trips with no internet access
19:43:28 <dmsimard> I'd like to gauge interest in this idea where we could leverage ARA for aggregating and processing the vast amounts of ansible-related data we have
19:43:34 <dmsimard> #link https://etherpad.openstack.org/p/ara-aggregation
19:43:55 <fungi> mordred: some of my worst ideas come from nights of binge-drinking... does that count?
19:44:24 <dmsimard> corvus: ^ this might be of interest
19:45:02 <anteaya> I personally like the gentle time about 5am or between waking and sleeping, if we are sharing out experiences
19:45:35 <anteaya> corvus: first of all you have to share what activity nets you your best ideas
19:45:38 <dmsimard> tl;dr, I would leverage firehose.o.o to pick up messages about Zuul finishing a job ( https://review.openstack.org/#/c/535543/ ) , download the ARA database (which are conveniently logged for every jobs) and then send that data to places like graphite, logstash or other things
19:46:00 <fungi> dmsimard: as a general case, would it make more sense for this to just be another zuul job?
19:46:23 <fungi> or an additional job phase even?
19:46:52 <fungi> i mean, i like the creative use of the firehose there, but seems like you could cut out the middle-man as it were
19:46:57 <dmsimard> I don't know if it needs to be a job, I felt that listening on firehose and being async/disconnected was interesting
19:47:35 <dmsimard> I was borderline considering proposing we move to this kind of model for logstash but then I remembered that our gearman ansible module sends custom data to tell logstash what files to index
19:47:45 <fungi> we had at one point talked about obsoleting the logstash and subunit workers by just doing that with zuul jobs/playbooks too
19:49:05 <dmsimard> fungi: so the thing about making it synchronous and in-line is that it introduces latency in every job and has the potential of self-DDoS
19:49:09 <corvus> there's a lot we can talk about the mechanism -- but i'd put that aside for a moment to ask what's the goal?  what does it mean to "send data from ara to graphite" for a check job?
19:49:40 <dmsimard> corvus: that's an excellent question
19:50:09 <dmsimard> corvus: TripleO are leveraging this kind of mechanism today to graph the duration of specific tasks, i.e: https://review.rdoproject.org/grafana/dashboard/db/tripleo-ci
19:50:33 <corvus> dmsimard: have you read the counter spec?
19:50:46 <fungi> the zuul executors have that information too, right? could we add statsd emitters with the playbook times?
19:51:02 * fungi realizes the counters spec probably already has that in it
19:51:16 <dmsimard> corvus: that vaguely rings a bell but other than that no
19:51:22 <dmsimard> is that the several years old spec ?
19:51:38 <corvus> well, it predates zuul v3.  but it has discussion about related problems.  it'd be a good starting point for discussion
19:51:41 <corvus> http://specs.openstack.org/openstack/qa-specs/specs/devstack/counter-inspection.html
19:52:59 <corvus> at least, it's good background for considering questions like "what does it mean for check vs gate?" and "how do you compare performance metrics in a heterogeneous environment?"
19:53:53 <dmsimard> aye, it goes a little bit more in depth (or perhaps a different granularity) than what I was suggesting but the general concepts are the same
19:54:12 <fungi> i'm curious, does this relate to the current ml thread where the tripleo team are wanting to gauge the impact (positive or negative) of chaining more expensive jobs dependent on less expensive jobs?
19:54:41 <dmsimard> I don't feel the counter spec would be mutually exclusive to what I have in mind, though
19:54:55 <fungi> i gather there was a lot of shrugging when it came to actual durations (both of the jobs and the wait times in various queues)
19:55:15 <dmsimard> fungi: I'm afraid I wasn't involved in that thread/discussion
19:55:20 <fungi> ahh
19:56:08 <corvus> dmsimard: i'm not suggesting it as an alternative, i'm suggesting that it provides good background for answering the questions posed earlier.
19:56:25 <dmsimard> I came up with the idea after realizing that we had all the job's ARA databases available for download since the sqlite middleware implementation and I brainstormed a bit about the value we could pull out of those
19:56:39 <dmsimard> corvus: ++
19:56:44 <corvus> our ci system produces an enormous amount of data -- the question is how to turn that into information.
19:57:22 <dmsimard> valuable* information
19:57:58 <dmsimard> I'll sit on the idea for a while, it's not something I'll be able to work on this week or anything
19:58:08 <fungi> #link http://lists.openstack.org/pipermail/openstack-dev/2018-May/130459.html Multi-staged check pipelines for Zuul v3 proposal
19:58:13 <fungi> dmsimard: the thread i was thinking of is this one ^
19:58:14 <dmsimard> With about 2 minutes left, anything for open floor ?
19:58:17 <dmsimard> fungi: thanks
19:58:20 <dmsimard> #topic open floor
19:59:11 <fungi> looking forward to seeing lots of you next week!
19:59:34 <fungi> oh, and i have some changes up for hosting the zuulci.org typo domain
19:59:50 <fungi> #link https://review.openstack.org/#/q/topic:zuulci.org hosting the zuulci.org typo domain
20:00:02 <dmsimard> cool, let's follow up in #openstack-infra, see you next week !
20:00:05 <dmsimard> #endmeeting