19:02:58 <jeblair> #startmeeting infra
19:02:59 <openstack> Meeting started Tue Jul  1 19:02:58 2014 UTC and is due to finish in 60 minutes.  The chair is jeblair. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:03:00 <wenlock> o/
19:03:01 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:03:03 <openstack> The meeting name has been set to 'infra'
19:03:06 <zaro> o/
19:03:11 <jeblair> #link https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting
19:03:11 <krtaylor> o/
19:03:15 <jeblair> agenda ^
19:03:23 <jeblair> #link http://eavesdrop.openstack.org/meetings/infra/2014/infra.2014-06-24-19.02.html
19:03:25 <jeblair> last meeting ^
19:03:35 <jeblair> #topic  Actions from last meeting
19:03:50 <jeblair> so there aren't any on the list, however we did do some renames
19:03:56 <fungi> yeah, that we did
19:04:05 <jeblair> a bunch (but not all) of the murano projects were moved to stackforge-attic
19:04:10 <jeblair> bash8 -> bashate
19:04:14 <jeblair> and designate moved to openstack
19:04:22 <fungi> along with its subprojects
19:04:37 <fungi> (client, specs)
19:04:40 <jeblair> we had to disable the bash8 devstack job temporarily, but i believe that's been re-enabled, yeat?
19:04:42 <jeblair> yeah?
19:04:50 <fungi> yes
19:05:12 <jeblair> cool.  i haven't heard about any problems
19:05:13 <fungi> i approved the revert as soon as i got the last image update to complete (required a retry in one region because the first attempt hung)
19:05:29 <jeblair> we have a few more changes to the rename docs pending as a result of that
19:05:42 <zaro> i think clarkb did trusty and updated build-timeout plugin
19:06:02 <clarkb> I did, and approved the changes to start using the new build timeout plugin
19:06:10 <jeblair> (catching up with puppet procedural changes)
19:06:34 <jeblair> #topic  Announcement: Next Bug Day Tuesday July 8th starting at 17:00 UTC (pleia2)
19:07:11 <jeblair> not sure if pleia2 was here, but i'm sure if she were, she would want to say: "Next Bug Day Tuesday July 8th starting at 17:00 UTC"
19:07:16 <mordred> in a fit of consistency, I will be out in an all-day managers meeting july 8th
19:07:38 <jeblair> mordred: so we can expect quite a bit of work from you as you seek to escape the tedium!
19:07:40 <pleia2> o/
19:08:08 <pleia2> I'll prep the etherpad the evening before so we can get started quickly in the morning of the 8th
19:08:39 <jeblair> we should make sure this is our last bug day. i don't want to be using launchpad by the time the next one rolls around.
19:08:57 <mordred> ++
19:09:01 <pleia2> that would be great
19:09:14 <jeblair> #topic  F20 jobs (ianw 7/1)
19:09:21 <ianw> hi
19:09:44 <ianw> thanks for reviews on https://review.openstack.org/101110 (Track last allocations to ensure forward-progress)
19:09:54 <jeblair> it has 2x+2s now
19:09:59 <ianw> who else can look at it
19:10:18 <jeblair> if anyone else wants to look it over, we can wait, otherwise, we can merge it now
19:10:33 <clarkb> I am reasonably confident in it because the added tests are great :)
19:10:42 <jeblair> derekh_: ^ fyi
19:10:43 <ianw> 2) i've started to look at centos7 testing
19:11:23 <mordred> ianw: thanks for the work on that!
19:11:30 <jeblair> oh, ha! rhel7 released while i was out in the wilderness!
19:11:42 <ianw> https://review.openstack.org/103735 & https://review.openstack.org/103458
19:11:51 <fungi> i too missed that event
19:11:57 <ianw> i would love to get an early experimental job started
19:12:07 <ianw> even if on just one provider
19:12:20 <ianw> however, i'm not sure where i'd get the images from
19:12:43 <ianw> i'd love some contacts for either hp or rackspace who i could talk to
19:12:46 <derekh_> jeblair: seen the review been catching up after 2 weeks off, can give it a whirl later if ye want and add another +1 (hopefully)
19:13:07 <mordred> ianw: we're getting close on having dib-based images for nodepool
19:13:09 <ianw> so we could discuss early centos7 images and how we could test them
19:13:23 <mordred> ianw: do they have published images for early centos7?
19:13:28 <ianw> mordred: yeah, i still have to spin that up and test out fedora builds there
19:13:34 <ianw> next on my todo :)
19:13:37 <mordred> sweet
19:13:51 <clarkb> derekh_: awesome that would be good
19:13:53 <ianw> mordred: yes, there is a centos7 nightly stuff happening
19:14:00 <jeblair> ianw: the ideal is that we will soon provide our own images using dib and glance; but until that works, our process depends on hp and rax having base images
19:14:33 <ianw> mordred: http://buildlogs.centos.org/centos/7/cloud/CentOS-7-Broken-20140620-Nightly.qcow2 is one i've been using
19:14:49 <mordred> ianw: I love that it has broken in the name
19:14:52 <fungi> the filename of that is rather inspiring
19:14:56 <fungi> jinx
19:15:00 <ianw> jeblair: yes, for sure, that's why i'd like to see if hp & rax have any "secret menu" etc where we could get a beta image just to smoke-test
19:15:18 <mordred> but honestly, if that exists, then it's a piece of cake to make a dib image on top of
19:15:48 <jeblair> ianw: *nod*
19:16:23 <fungi> looks like 101110 *just* got a new patchset courtesy of tchaypo
19:16:30 <clarkb> wait what
19:16:50 <fungi> oh, he edited the commit message
19:16:53 <clarkb> but why
19:16:56 <fungi> (through the gerrit webui)
19:17:04 <bcrochet> o/
19:17:17 <fungi> to add Closes-Bug: #1308407
19:17:20 <jeblair> now it doesn't have 2x+2s :(
19:17:26 <clarkb> argle bargle
19:17:31 <clarkb> and no tchaypo here to argue with
19:17:33 <clarkb> :P
19:18:13 <ianw> oh
19:18:41 <jeblair> i think i trust that review comment, so it should be a pretty easy re-review
19:18:53 <clarkb> yeah should be quick
19:18:55 <jeblair> ianw: thanks for taking on centos7!  anything further?
19:19:22 <ianw> so yeah, if anyone has any nicks i can ping on the image issue, that would be great.  otherwise, i'll get to the dib path soon
19:19:29 <ianw> other than that, no, thanks
19:20:05 <jeblair> #topic Zuul Cloner patch, a port to python of the devstack shell script cloning repositories
19:20:10 <nibalizer> o/ im back
19:20:37 <jeblair> hashar: ping!
19:20:39 <hashar> hello
19:20:45 <hashar> Zuul cloner, basically a port to python of the devstack shell script that clones your repositories.
19:21:01 <hashar> at wikimedia we have a similar need to clone various repo and run integration tests of the result
19:21:15 <hashar> I am too lazy to adjust a shell script so went mad and ported it to python
19:21:20 <hashar> #link https://review.openstack.org/#/c/70373/
19:21:29 <hashar> It is very basic for now but should fetch appropriate Zuul ref and fallback to a given branch or master
19:21:29 <hashar> Example output for a job on MediaWiki branch REL1_23:
19:21:29 <hashar> #link https://integration.wikimedia.org/ci/job/mediawiki-core-extensions-integration/48/consoleFull
19:21:47 <clarkb> out of curiousity where did the devstack function break?
19:21:50 <hashar> It has the basic functionalities, would need some help writing some basic integration tests.
19:22:01 <clarkb> it should be reasonably flexible but I am sure there were issues
19:22:05 <hashar> I am hoping you guys would be interested in eventually migrating the devstack shell of doom to that python code base
19:22:15 <jeblair> this is really cool, and i believe in the future we will move to using it for devstack-gate, and then start using it for other integration jobs, and we can stop abusing devstack-gate for that purpose
19:22:23 <mordred> ++
19:22:51 <hashar> it is a long way though. The python part only support very basic functionality.  It does not have hardcoded branches for some specific projects you have. But I believe we have a good base to be build upon.
19:22:58 <fungi> yeah, the integration cloning logic in d-g is not really well suited to shell anyway, and benefits from being in python i think
19:23:58 <hashar> the idea came when looking turbo hipster and talking with Joshua.  He was more or less rewriting part of the devstack script. So I though cloning should be a core feature of Zuul so other third parties could reuse it easily.
19:24:16 <hashar> not much to say. Wanted to make sure all of you guys know about the patch and eventually give it a try :]
19:24:49 <hashar> done (sorry long read)
19:24:56 <jeblair> hashar: thanks, i'm very excited about it -- i'm hoping to review it soon, and maybe help out a bit :)
19:25:22 <clarkb> yaeh I think putting that stuff in zuul is a great idea
19:25:37 <clarkb> I still think it may help me review it if there are some examples of the issues with the other thing
19:25:47 <clarkb> basically items to be on the lookout for when reviewing that change and others
19:26:02 <jeblair> clarkb: i think the biggest issue is that devstack-gate comes with a lot of devstack baggage
19:26:09 <clarkb> oh I see
19:26:39 <jeblair> devstack-gate has unit tests for the cloning functionality, so we may want to make sure that those scenarios end up in the zuul-cloner too
19:26:39 <hashar> I made it project agnostic.  But there is a feature known as clone map  which let you clone a repo at a specific place (I need that for mediawiki)
19:27:34 <jeblair> #topic Puppet 3 Master (nibalizer)
19:27:39 <nibalizer> hi
19:27:47 <nibalizer> so it sounds like i have some consesus to fire up a p3 master
19:28:07 <nibalizer> i think that means we need someone to provision a node, and a core to sit with me and we'll poke it till it works
19:28:11 <jeblair> #link https://review.openstack.org/#/c/83678/
19:28:28 <nibalizer> i'm somewhat concerned that the 3.6.2 feature 'directory environments' will bone us
19:28:37 <nibalizer> so im consideriing ammending that reveiw to pin puppet to 3.4
19:28:43 <nibalizer> since 3.5 was buggy as all heck
19:29:24 <nibalizer> is there anyone who would be willing to work with me on getting the p3 node up?
19:29:25 <jeblair> what does 'directory environments' do?
19:29:48 <nibalizer> it depercates the modulepath environment variable, as well as manifest and a few others
19:30:08 <jeblair> #link http://docs.puppetlabs.com/puppet/latest/reference/environments.html
19:30:10 <nibalizer> now you specify an envionment directory and in that directory each sub directory is an environment and has its own config file to set things like modulepath
19:30:22 <hashar> Wikimedia migrated recently and has puppet 3.4.3 (ships with Ubuntu Precise)
19:30:54 <nibalizer> there has been a lot of confused users on the puppet-users mailing list and on irc having trouble with changes in 3.6.2
19:31:00 <hashar> might be relevant, some of our ops build a tool to compile catalog with 2.7 and 3.4 and output the resulting diff ( we got 3.4.3  ). Came very handy to ensure our puppet were still compatible.
19:31:13 <nibalizer> ya that thing is super sweet
19:31:31 <jesusaurus> whats the name of that tool?
19:31:46 <jesusaurus> is it open source?
19:32:09 <jeblair> nibalizer: i should be able to help spin up a node tomorrow
19:32:11 <hashar> #link http://git.wikimedia.org/tree/operations%2Fsoftware.git/master/compare-puppet-catalogs
19:32:14 <hashar> sorry
19:32:30 <nibalizer> jeblair: excellent, you're PDT right? what time works for you?
19:32:39 <nibalizer> I have stuff in the morning so evening is preferable for me
19:32:50 <hashar> jeblair: license is GPLv2
19:32:57 <hashar> jesusaurus: license is GPLv2
19:33:19 <jesusaurus> awesome, thanks
19:33:42 <jeblair> nibalizer: not evening, so much; i'd want to start before 2pm probably
19:34:23 <nibalizer> start at noon? start at 1?
19:34:33 <jeblair> nibalizer: clarkb or mordred tend to hang around later if you wanted to try to ping them
19:34:40 <jeblair> nibalizer: 1pm wfm
19:34:44 <nibalizer> lets do that
19:34:52 <nibalizer> i meant afternoon anyways
19:34:55 <fungi> i'd be happy to help too, though my timezone doesn't overlap as much so unless you like getting up early ;)
19:35:07 <jeblair> ok cool
19:35:26 <jeblair> #action nibalizer and jeblair to spin up a puppet3 master
19:35:39 <jeblair> #topic  Open discussion
19:36:08 <clarkb> the elasticsearch situation is slightly better this week than last (sdague has done some work to cleanup problem logs)
19:36:12 <clarkb> but still not great
19:36:14 <hashar> nibalizer: feel free to ask about wikimedia puppet 3 migration in #wikimedia-operations
19:36:27 <jeblair> clarkb: how's the ssd test going?
19:36:30 <clarkb> mordred has pinged rax about it to see if we can have a discussion about the best way to deal with the problem
19:36:35 <fungi> since nobody has provided a compelling argument for git-review 2.0, i'm going ahead with 1.24. writing up release notes this evening, i'll link an etherpad in #-infra later for perusal
19:36:50 <clarkb> jeblair: I haven't done the ssd yet because I was hoping mordred would get a quick response but I haven't heard anything so I should probably go ahead and start that this afternoon
19:37:01 <clarkb> any opposition to that? eg to wait a bit longer for word from rax?
19:37:04 * krotscheck has lots of big storyboard patches up.
19:37:12 <jeblair> clarkb: yeah, let's give it a shot
19:37:15 <clarkb> fungi: ++
19:37:25 <jeblair> clarkb: learn what we can
19:37:26 <clarkb> jeblair: ok I will start working on that as the game happens this afternoon
19:37:31 <fungi> also, i missed that we have several git-review bugs where reporters simply attached patches as diffs, and some fix (albeit low-priority) bugs. i suppose they can wait for after release to get git-am'd in
19:38:31 <jeblair> i have nearly, but not quite, worked through my vacation-related review backlog
19:39:09 <clarkb> krotscheck: scary js multi k line changes?
19:39:13 <pleia2> I sent a note to the list earlier I'd like feedback on http://lists.openstack.org/pipermail/openstack-infra/2014-July/001471.html have my local (my hpcloud account) demo of zanata up and running at http://15.126.233.162:8080/zanata/
19:39:23 <krotscheck> clarkb: I tried to break them up!
19:40:04 <jeblair> krotscheck: remind me again, what would be needed to make the docs-draft deploys of the js code work against prod?
19:40:39 <pleia2> I called WildFly the community open source version, but it seems to not entirely be a clone, but instead more like "another java application server that should do what jboss enterprise does"
19:40:39 <clarkb> fungi: ya I think we should get into a habit of releasing more often if git review is going to be active which it has been
19:40:44 <krotscheck> jeblair: Configuration injection into the build, plus CORS support in pecan.
19:40:49 <clarkb> fungi: then the cost of applying a few bugfixes and releasing is low
19:41:04 <clarkb> pleia2: so it is magic?
19:41:12 <fungi> clarkb: agreed. i meant to release more often, but have fallen down on the job (ENOTIME, as is the case for everyone else too)
19:41:17 <pleia2> clarkb: something like that
19:41:49 <jeblair> krotscheck: anybody doing cors in pecan?  and do you think it would be safe to enable if it did?
19:42:04 <fungi> pleia2: awesome, btw!
19:42:16 <jeblair> regarding the meetup: https://wiki.openstack.org/wiki/Qa_Infra_Meetup_2014
19:42:25 <jeblair> there are som potential sprint topics listed
19:42:38 <hashar> fungi: lot of wikimedia/mediawiki developers are solely relying on git-review to interact with Gerrit :]
19:42:40 <jeblair> some of them may be discussions, some of them may be hacking...
19:42:57 <hashar> fungi: definitely helped us to enroll more volunteers. So git-review is time well spent
19:43:16 <jeblair> does anyone have other suggestions, or feedback on those things?  and would it be worthwhile to try to organize that a bit more?  maybe even set a schedule?  or better to play by ear?
19:43:24 <fungi> hashar: as for us too. and thanks for the feedback!
19:43:45 <clarkb> jeblair: a loose scheduel would probably be a good thing
19:43:45 <pleia2> jeblair: I should probably add translation stuff on there too, might be good to have some high bandwidth time
19:43:59 <clarkb> jeblair: especially if people end up skpping timeslots to see darmstadt
19:43:59 <krotscheck> jeblair: Haven’t researched it yet.
19:44:02 <hashar> are qa infra meetup held on a regular basis? Would love to attend one in Europe if you don't mind having me floating around.
19:44:15 <clarkb> (I don't intend on doing that fwiw but have heard rumblings that this may happen)
19:44:46 <jeblair> hashar: no, this is the first joint one; we had a bootcamp a year ago in nyc though
19:44:47 <krotscheck> jeblair: Maybe? I get where you’re going though.
19:45:02 <clarkb> jeblair: I definitely think logstash/elasticsearch sould happen given the recent issues around that
19:45:11 <pleia2> hashar: you should join us :)
19:45:17 <clarkb> (that is me being slightly selfish :) but I think it will be time well spent)
19:45:21 <jeblair> hashar: we'd love to have you join us :)
19:45:22 <fungi> hashar: this one is in europe... fortuitous!
19:45:38 <hashar> I would have if knew about it earlier. I am on vacation that week and rented a place ages ago :/
19:46:01 <fungi> you don't happen to be vacationing in frankfurt that week, by any chance? ;)
19:46:02 <jeblair> hashar: does the place you rented hold 30 people?
19:46:07 <pleia2> hehe
19:46:08 <hashar> lol
19:46:40 <jeblair> well, we always have paris.
19:46:48 <krotscheck> We’ve always got paris!
19:46:51 * fungi groans
19:47:08 * krotscheck fails at references
19:47:12 <hashar> on topic, having past experience attending hackathon. We do it unconference style.  People write down on post-it topics they are interested in, we stick them on a wall of card  and thus form groups by topic
19:47:50 <hashar> we also have some informal short presentations during the day. 40 mins + 20 min qa.  That helps bring everyone on par on those topics.
19:47:58 <jeblair> hashar: good idea, and it may disrupt clarkb's plans to skip out and tour the city :)
19:48:10 <fungi> hashar: that's more or less what we're doing already, i think, just getting an early start with the wiki
19:48:17 <hashar> ex of one i held was: creating your Jenkins job with JJB.  Covered installation, code base, yaml template, our jobs, how to set it up + demo.
19:48:38 <clarkb> jeblair: I am saving the touring for post paris
19:49:03 <fungi> hashar: but yes, i've been to a few barcamps which used that model too and i agree it works quite well
19:49:03 <hashar> we usually tour the city in the evening and end up in a bar. The city tour is a good way to relax before the exhausting round of beers / crazy night discussion
19:49:12 <jeblair> i think we're planning on doing a lot of presentation style stuff during the first 2 days, to try to get as many people up to speed as possible, but then the rest of the week is more freeform
19:49:39 <wenlock> o/ you probably have seen me poking around a bit, i wanted to let folks know that work ive been working on was released last week.  we are downstream from a lot of work you are doing on config.
19:49:59 <wenlock> ty for your help up to now.  this allowed us to create forj.io
19:50:00 <fungi> i do think at least a presentation pitch phase would be helpful, in case there are some presentations which nobody really thinks they're going to benefit from after all
19:50:06 <jeblair> so yeah, maybe we should identify which of those topics would make good mini-presentations, or discussions, and which are hacking, and try to break things up and loosly schedule them
19:50:07 <hashar> fungi: yeah that is similar to bar camps indeed
19:50:10 <pleia2> wenlock: great :)
19:50:27 <fungi> jeblair: agreed
19:50:41 <clarkb> ++
19:50:48 <fungi> wenlock: that's awesome
19:50:58 <hashar> I am usually not productive during such hacking sessions. But get a ton of crazy ideas that are implemented post event.
19:52:07 <wenlock> im hoping with time i can contribute some improvements upstream as well, so we have even lest modules we've had to fork
19:52:15 <wenlock> *less
19:52:15 <hashar> if I take "Nodepool enhancements", you could have a presentation of nodepool, the problem it has, feature it lacks. Then brainstorm to create a backlog of items to produce for the next few months or so.
19:52:46 <jeblair> wenlock: any chance you'll be able to attend the meetup in germany?
19:53:13 <hashar> #link https://wiki.openstack.org/wiki/Qa_Infra_Meetup_2014
19:53:14 <jeblair> wenlock: or oscon the following week?
19:53:26 <mordred> wenlock: either or both of those would be awesome for you to come to
19:53:56 <jeblair> wenlock: one of the things i want to discuss is making sure we have a good roadmap for how to make infra reusable downstream, and thereby facilitate upstream contributions
19:54:14 <jeblair> so would love to have you involved :)
19:54:28 <wenlock> jeblair whish's i knew earlier, i would have asked mordred for a ticket, hehe
19:54:34 <wenlock> *wishes
19:54:44 <mordred> wenlock: I can still go talk to people about getting you approved to go
19:54:53 * fungi suspects mordred can fit it into his budget ;)
19:55:04 <mordred> well, wenlock doesn't work for me, so that's harder ...
19:55:05 <hashar> is that https://www.forj.io/ based on Zuul openstack toolchain ?
19:55:15 <fungi> mordred: ahh, too bad
19:55:19 <mordred> but maybe a trip to portland during oscon wouldn't break wenlock's team's budget
19:55:21 <wenlock> true ture
19:55:42 <wenlock> im going to be up in Portland weekend of Aug 1-4
19:55:46 <wenlock> but for fundraiser stuff
19:55:52 <wenlock> + vacation
19:55:57 <jeblair> oscon is july 20-24
19:56:00 <fungi> it's a great place for a vacation
19:56:08 <wenlock> where is oscon?  links?
19:56:09 <clarkb> also great for brewing beer
19:56:12 <wenlock> i bet i can get that goign
19:56:20 <jeblair> wenlock: http://www.oscon.com/oscon2014
19:56:37 <fungi> wenlock: in the convention center downtown
19:57:05 <wenlock> awesome, i'll be checking that one out.  try to get my approval
19:57:07 <zaro> hashar: yes, it is.
19:57:24 <hashar> zaro: maybe forj.io could entirely replace me so :-]
19:57:31 <jeblair> wenlock: there's probably no need to register for the actual conference if you don't want -- i think we're mostly focusing on the hallway track :)
19:57:36 <jeblair> wenlock: or rather, the bar track
19:57:43 <clarkb> this
19:57:44 <zaro> hashar: unlikely.  more for simpler stuff
19:58:18 <fungi> jeblair: or clarkb's beer barrel track
19:58:27 <clarkb> I haev 15 gallons of the stuff now
19:58:30 <jeblair> right, that's the actual thing
19:58:31 <clarkb> I expect you guys to help
19:58:45 <pleia2> a quick fyi, I'm missing oscon to get my gallbladder out (we finally found out what is wrong)
19:58:47 <fungi> my liver is primed and ready
19:58:51 <clarkb> pleia2: oh no :(
19:58:57 <clarkb> but I guess good to know what is wrong
19:59:02 <pleia2> clarkb: yeah, no fun, but at least a solution!
19:59:04 <fungi> pleia2: oh, ouch
19:59:33 <zaro> clarkb: hmm, maybe worth a short trip :)
19:59:37 <jeblair> pleia2: sorry to hear that (but glad you know);  we'll drink to your health.  :)
19:59:44 <pleia2> jeblair: much appreciated :)
20:00:01 <fungi> and we're at time!
20:00:07 <jeblair> thanks everyone!
20:00:10 <jeblair> #endmeeting