Friday, 2014-03-14

openstackgerritJoe Gordon proposed a change to openstack-infra/elastic-recheck: Delete query 1258682
*** maxbit_ has joined #openstack-infra00:01
*** maxbit has joined #openstack-infra00:01
openstackgerritJoe Gordon proposed a change to openstack-infra/elastic-recheck: Add a footprint for bug 1283522
*** yassine has quit IRC00:01
*** sarob has quit IRC00:02
*** mrodden has quit IRC00:03
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Added support for CI Game plugin
openstackgerritA change was merged to openstack-infra/config: Enable Tempest for Murano dvsm job
openstackgerritSpencer Krum proposed a change to openstack-infra/config: Add PuppetBoard to openstack infra
jogoso neutron has a bunch of fixes in the pipeline that should help gate00:07
nibalizerclarkb: i may even have it now00:07
jogowas thinking maybe we can bump them to the top of the queue00:07
jogoarosen2 and dansmith are working on them00:07
fungijogo: the one failing its unit tests and the other one which depends on it?00:08
jogofungi: yup and they have a patch for the unit test bug too00:08
jogooh wait that didn't work00:08
clarkbjogo: bug me about that when I am not about to drive to portland and I can approve it00:08
clarkbfungi: :P00:08
jogoclarkb: ack00:09
mattoliveraujeblair: So I created a RAX centos box, changed resolv.conf, chattr +i /etc/resov.conf, made the server into an image. Created a new server from the image and all worked as expected no networking errors and my change to resolv.conf was consistent.00:09
jogoarosen2: the unit test failure is a race right?00:09
jogofungi: so the catch with those two patches is that nova needs them before we can push a patch to actually fix the issue00:10
arosen2jogo:  i think so.00:10
jogoso this wouldn't result in an immediate fix, but it would help expedite the solution00:10
* clarkb afks to make that trip00:11
jogofungi: ^00:11
jeblairmattoliverau: cool, thanks00:11
jogofungi: thoughts on a bump on this one?00:11
* arosen2 Looking at the test too 00:11
jogothese two*00:11
fungijogo: i guess they just got sniped out by another upload?00:11
*** SumitNaiksatam has quit IRC00:11
fungithey vanished from the gate moments ago00:12
jogoother way around00:12
jogoeverything ahead flushed out00:12
fungioic, they had already finished all their testing00:13
*** dolphm has quit IRC00:14
*** emagana has quit IRC00:15
*** dolphm has joined #openstack-infra00:15
*** dkranz has quit IRC00:16
*** katyafervent has quit IRC00:16
*** dkranz has joined #openstack-infra00:16
nibalizerclarkb: want to look at when you get a chance?00:18
nibalizerfungi: maybe you to if you get a chance00:18
*** maxbit has quit IRC00:18
*** katyafervent_ has joined #openstack-infra00:18
*** maxbit_ has quit IRC00:18
*** maxbit has joined #openstack-infra00:18
arosen2jogo:  markmcclain is going to approve this: not sure if we can bump it ahead at all00:18
jogoarosen2: if you pray (make a logical and sound arguement) to the infra gods perhaps you can00:18
* fungi hears your pleas00:18
arosen2jogo:  or i guess an under the table cash bribe00:19
fungii also take credit00:19
jogounlike regular gods, the infra gods are honest00:20
jogoand reasonable00:20
* fungi is irregular. see, says right here on my jacket00:20
*** pcrews has quit IRC00:25
*** maurosr has quit IRC00:25
*** zehicle_at_dell has joined #openstack-infra00:25
*** zehicle_at_dell has quit IRC00:27
jogoanyway just finished my beer so time to go outside and enjoy the great weather00:27
jogoarosen2: good luck with all the neutron patches00:27
arosen2jogo:  enjoy!00:27
markmcclainfungi: so it would make sense to promote if there is another gate reset00:27
markmcclainthis test has been randomly failing depending on the actual test ordering00:27
fungimarkmcclain: looks like it's still in check right now, but appears safe enough to bump it straight into the gate at the very least00:27
*** pcrews_ has joined #openstack-infra00:27
*** maurosr- has joined #openstack-infra00:27
*** maurosr- is now known as maurosr00:27
*** zehicle has joined #openstack-infra00:27
*** dstanek_afk has joined #openstack-infra00:27
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use unbound
markmcclainfungi: bumping it to the gate would be nice00:27
fungimarkmcclain: i've popped it into the gate00:27
openstackgerritA change was merged to openstack-infra/elastic-recheck: Delete query 1258682
markmcclainfungi: thanks00:27
jeblairmattoliverau: attempt to add immutable to puppet ^ (because i think we actually will need the same functionality on our long-lived centos servers -- either that, or we would need to update one of the injected files in /etc)00:27
fungimarkmcclain: if i see a good opportunity crop up, i'll send it up to the front00:27
fungibut there's only 5 changes ahead of it currently, so if we hold our teeth right it just might go straight in00:27
*** dstanek has quit IRC00:28
mordrednibalizer: on - shouldn't line 2 of the vhost.erb be an fqdn - or $::fqdn ?00:29
*** matsuhashi has joined #openstack-infra00:31
*** jooools has joined #openstack-infra00:33
*** ryanpetrello has quit IRC00:33
*** mgagne has quit IRC00:35
nibalizermordred: thats fine with me, i'll do that00:36
*** thedodd has joined #openstack-infra00:36
*** zehicle has quit IRC00:37
mattoliveraujeblair: nice, made a comment, but looks good to me.00:44
*** thuc has quit IRC00:45
dims_jamespage, ping00:45
*** thuc has joined #openstack-infra00:45
*** mrodden has joined #openstack-infra00:47
*** milki has quit IRC00:48
*** openstack has joined #openstack-infra00:53
*** dims_ has joined #openstack-infra00:53
*** bogdando has joined #openstack-infra00:54
*** dstufft has joined #openstack-infra00:54
*** rfolco has joined #openstack-infra00:54
*** dstufft has quit IRC00:55
*** dkranz has joined #openstack-infra00:55
*** dstufft has joined #openstack-infra00:55
*** nati_uen_ has quit IRC00:55
*** dstufft has quit IRC00:55
*** dstufft has joined #openstack-infra00:55
*** jasondot_ has quit IRC00:55
*** openstack has joined #openstack-infra00:57
*** openstack has quit IRC00:57
*** openstack has joined #openstack-infra01:00
*** afazekas has quit IRC01:01
*** dstufft has quit IRC01:01
*** hdd_ has joined #openstack-infra01:02
*** jgrimm has quit IRC01:02
*** blamar_ has joined #openstack-infra01:02
*** dstufft has joined #openstack-infra01:02
*** SpamapS has quit IRC01:04
*** blamar has quit IRC01:06
*** dstufft has quit IRC01:07
*** dolphm has quit IRC01:07
*** rpodolyaka1 has joined #openstack-infra01:07
*** dstufft has joined #openstack-infra01:07
*** Daisy_ has joined #openstack-infra01:07
*** sarob has quit IRC01:08
*** matsuhashi has quit IRC01:08
*** SpamapS_ has joined #openstack-infra01:08
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Added paging to list endpoints
*** rpodolyaka has quit IRC01:08
*** dolphm_ has joined #openstack-infra01:08
*** thedodd has quit IRC01:08
*** dolphm_ is now known as dolphm01:08
*** blamar_ is now known as blamar01:08
*** dstufft has quit IRC01:08
*** dstufft has joined #openstack-infra01:08
*** amcrn has quit IRC01:08
*** afazekas has joined #openstack-infra01:08
*** julim has quit IRC01:12
*** matsuhashi has joined #openstack-infra01:12
*** rpodolyaka1 has quit IRC01:14
*** FallenPegasus has joined #openstack-infra01:14
*** Daisy_ has quit IRC01:14
*** openstack has joined #openstack-infra01:15 [freenode-info] channel flooding and no channel staff around to help? Please check with freenode support:
*** unicell has joined #openstack-infra01:15
*** openstack has joined #openstack-infra01:19
*** nosnos has joined #openstack-infra01:19
*** Daisy__ has quit IRC01:24
*** openstack has quit IRC01:24
*** openstack has joined #openstack-infra01:26
*** openstack has joined #openstack-infra15:09
jeblairwow its log is not rotated.15:10
jeblairi'm not sure what happened to its connection15:11
fungisomehow i missed the part message for it in my client log, but the channel logs suggest it went missing shorty after 2014-03-14T01:26:0015:11
mordredmorning all15:12
*** rcleere has joined #openstack-infra15:12
jeblairit was disconnecting and reconnecting a lot for the previous hour or so from then15:12
devanandamordred: hey look!
jeblairand there's some weird async connection stuff going on15:13
*** miqui has quit IRC15:13
mordreddevananda: oooo15:13
devanandait's like we almost have integration tests :)15:13
jeblairINFO 2014-03-14T01:28:31 supybot Join to #openstack-trove on FreeNode synced in 193.02 seconds.15:14
jeblairWARNING 2014-03-14T01:28:37 supybot Error message from FreeNode: Ping sent at 2014-03-14T01:26:36 not replied to.15:14
jeblairINFO 2014-03-14T01:28:37 supybot Reconnecting to FreeNode.15:14
jeblairand then that repeats until the present time15:14
mordreddevananda: I don't even want to know what this is: sudo add-apt-repository cloud-archive:havana do I?15:14
*** yamahata has quit IRC15:14
jeblairnote how long it took to join trive15:14
*** yamahata has joined #openstack-infra15:14
devanandamordred: ask clark. tldr - we need ovs from not-precise15:14
*** adalbas has quit IRC15:15
jeblairi'm guessing the irc server or our network was having problems15:15
mordreddevananda: yeah - I figured it was probably something like that ...15:15
*** rpodolyaka has joined #openstack-infra15:15
devanandamordred: the ssh thing is fun, too. i was just explaining that to jeblair :)15:15
*** lcostantino has joined #openstack-infra15:18
*** dprince has joined #openstack-infra15:18
*** HenryG has quit IRC15:20
*** lcheng has joined #openstack-infra15:21
*** emagana has joined #openstack-infra15:22
*** wenlock has joined #openstack-infra15:22
*** sarob has joined #openstack-infra15:22
sdaguejeblair: is there a reason not to send through now? I'd like to get those experimental runs on the devstack patch to make sure a few things in it will work on gate nodes15:22
*** sdake has joined #openstack-infra15:24
*** smarcet has quit IRC15:24
jeblairsdague, devananda so i'm a little puzzled why the ssh and apt repo stuff is in the job def and not devstack (or the ironic plugin for devstack, or whatever)15:25
openstackgerritA change was merged to openstack-infra/config: run link-logs builder first
fungii suppose that stuff could go in a pre-hook before ovs and similar things get pulled in by devstack itself?15:26
jeblairfungi: oh, is the reason because devstack would install the wrong ovs?15:26
fungithat's what i assumed, but maybe i was misinterpreting15:27
*** sarob has quit IRC15:27
jeblairsdague: is this job to prove this is ready for things landing in devstack?  if so, maybe this can move later on?15:27
sdaguejeblair: yeh, this is to evaluate the devstack patch15:27
fungionce 79794 propagates i'll manually update jjb on all the jenkins masters, since the list of jobs touched is so large i expect the puppet exec on them to time out15:27
jeblairok.  i think that in long run, those things should not be in the job defn15:28
sdagueand I agree, this should not be in there. Honestly, given how all the timelines are lining up I expect that before this is voting we'll just hard require 14.04 for ironic anyway15:28
jeblairso once things settle out, we should move them15:28
jeblair(aprvd btw)15:28
openstackgerritA change was merged to openstack-infra/config: Add experimental virtual-ironic test
sdague14.04 has ovs 2.0, which should dramatically make everything using ovs work better15:28
sdaguewhich means I expect as soon as the gate can handle it we'll just hard require that ovs layer for anything using neutron15:29
Ajaegerfungi, jeblair: Thanks for reviewing and approving 80515!15:29
sdaguejeblair: thanks for the +A15:30
devanandasdague: sounds good. how soon should we start testing all the ironic things with 14.04?15:30
devanandajeblair: fungi: thanks for the help with that patch!15:30
sdaguedevananda: once there is an image that can be run in infra15:30
sdaguewhich means basically once the cloud providers start pushing them up15:31
devanandasdague: gotcha15:31
fungiyeah, doing that now would require starting to use glance so we could upload a beta image15:31
openstackgerritA change was merged to openstack-infra/config: Fix manuals translation jobs
*** zehicle_at_dell has joined #openstack-infra15:31
fungii just checked rax and hpcloud and neither have them published in their catalog15:31
sdaguerax just announced glance, is that actually running ?15:31
sdaguenot that I think it's actually worth spending that time in this part of the cycle15:32
sdaguemostly curious15:32
devanandafungi: about half an hour or so before that virtual-ironic job can be run by anything, ya?15:32
fungidevananda: yeah, i'm going to be forcing job updates everywhere here in just a bit, and after that it should be runnable15:33
devanandafungi: great. I'm going to head into downtown in a bit, back in an hour or two to check on things15:33
*** zehicle has quit IRC15:34
*** HenryG has joined #openstack-infra15:35
*** smarcet has joined #openstack-infra15:38
*** miqui has joined #openstack-infra15:39
*** marun has quit IRC15:39
*** Ryan_Lane has joined #openstack-infra15:41
*** rpodolyaka has quit IRC15:42
*** talluri_ has quit IRC15:42
*** talluri has joined #openstack-infra15:43
*** CaptTofu has quit IRC15:44
*** CaptTofu has joined #openstack-infra15:44
*** thuc has joined #openstack-infra15:44
*** dcramer_ has joined #openstack-infra15:46
*** yaguang has quit IRC15:46
*** talluri has quit IRC15:47
mordredsdague: both hp and rax have glance available - but we'll need to re-tool nodepool a bit to make use of it. also, hp 1.0 doesn't have it - so we still can't REALLY use it until we've moved off of 1.015:47
*** drumkilla is now known as rustlebee15:47
*** thuc_ has quit IRC15:48
sdaguemordred: cool, sounds like early juno fodder15:48
mordredI'm looking forward to it - and looking forward to not being dependent on our providers for image content at all15:48
*** thuc has quit IRC15:48
*** CaptTofu has quit IRC15:48
sdaguewhat... you don't like reverse engineering the differences between their nodes?15:48
*** adalbas has joined #openstack-infra15:48
*** NobodyCam is now known as BlartVersenwald15:49
sdagueto figure out why things work one place and not the other15:49
*** pdmars_ has joined #openstack-infra15:49
*** gyee has joined #openstack-infra15:49
jeblairi'm curious how that interacts with rax doing file injection for network config15:49
sdaguejeblair: yeh... my guess is poorly :)15:49
sdagueopen glance is kind of a loaded gun15:49
*** zns has quit IRC15:49
sdagueplenty of opportunity to shoot yourself in the foot15:50
fungimordred: while we *can* (and almost certainly should) improve nodepool to start using glance, we can make use of the service asap by just uploading the distro-provided cloud images and telling nodepool we want it to use those as its base images, right?15:50
mordredfungi: yes15:50
mordredfungi: good point15:50
sdaguejust a thought, for whenever anyone starts actually doing this15:50
mordredand in fact, we can do that in only one provider15:50
*** markmcclain has quit IRC15:50
fungimordred: i mean, it's what we've done in the tripleo-ci cloud already15:50
mordredfungi wins15:50
*** pdmars has quit IRC15:50
fungiwe already *do* that in one provider ;)15:51
sdaguewe should make sure that whatever process ends up prepping and uploading images, also makes them generally available for people15:51
mordredsdague: I'm guessing we'll need to make the images be able to handle metadata and injectino15:51
sdagueso that we can get them to replicate with the gate image locally15:51
mordredsdague: ++15:51
*** VINOD__ has joined #openstack-infra15:51
mordredI believe it will make local gate replication easier once we15:52
mordredget there15:52
sdaguebecause I'd still like to get us to a place where gate replication locally with vagrant (or something else) is super simple15:52
mordredI whole heartedly agree15:52
*** pblaho has quit IRC15:52
jeblairyeah, i think that's been our plan for like the 2 years we've been yelling^Wasking for glance :)15:52
*** mrodden has quit IRC15:53
sdaguemy goofy multinode localy devstack vagrant setup has actually been super useful to me of late, so I'm all on board building this tooling15:53
*** zns has joined #openstack-infra15:55
*** adalbas has quit IRC15:56
jeblairdoes someone have a link to the rax announcement?15:57
openstackgerritA change was merged to openstack-infra/devstack-gate: Fixed DEVSTACK_GATE_TEMPEST_REGEX.
sdagueit was on the twitters, let me go find it15:59
*** esker has joined #openstack-infra15:59
fungibecause that is apparently the new and highly professional ways to make service update announcements16:00
sdagueyou know, now that I think of it, I think mikal was the RT on that, and it was actually a link into pyrax on github, so maybe not as official as I thought16:02
*** pdmars has joined #openstack-infra16:02
*** lcostantino has quit IRC16:03
*** pdmars_ has quit IRC16:03
*** lcostantino has joined #openstack-infra16:03
sdagueyeh, so I actually have no idea what that means for using upstream tools16:03
jeblairwowie glanceclient pulls in a lot of stuff that nova doesn't16:04
fungijeblair: any objection to ? without that (or something like it), nodepool precise image updates are broken in rax regions for the past 48 hours or so16:04
jeblairin ubuntu, installing glanceclient gets you a glance user.16:04
sdaguejeblair: um.... really?16:04
*** mrodden has joined #openstack-infra16:04
sdaguethat's crazy pants16:04
*** hogepodge_ has joined #openstack-infra16:04
sdaguewhy would you do that on the client and not the server16:05
*** gokrokve_ has quit IRC16:05
clarkbsdague: need to keep that client sandboxed in its own user16:06
jeblairso precise glanceclient bombs with Details: [Errno 113] EHOSTUNREACH16:06
jeblairi'm trying to install python-glanceclient from pypi now16:07
sdaguejeblair: precise original?16:07
sdagueor precise + cloud archive16:07
jeblairsdague: precise orig16:07
sdagueso, essex level code16:07
*** hogepodge has quit IRC16:07
*** hogepodge_ is now known as hogepodge16:07
sdaguenot entirely surprised that would be borked16:07
*** nicedice has joined #openstack-infra16:07
jeblairsdague: not surprising, yes.  but also not good.16:08
sdaguelook, I yell at people about backwards incompat as much as anyone :)16:08
clarkbbtw gerrit >=2.6 has a major bug in it to be fixed in 2.8.3. So we should keep that in mind as part of our testing16:08
clarkbzaro: ^16:08
jeblairRequest returned failure status.16:08
jeblair404 Not Found16:08
fungii'm guessing old glanceclient defaulted to trying to reach a different port number16:09
jeblairsorry, that's new glanceclient ^16:09
fungiright, just surmising as to the reason for the difference in errors16:09
*** adalbas has joined #openstack-infra16:09
fungisounds like this one is actually hitting an api endpoint, which is telling it that what it wasn't isn't there16:10
fungier, that what it wants isn't there16:10
fungisome firewalls may send back icmp host-unreach responses instead of port-unreach or tcp rst in cases where a particular port is blocked16:11
fungi(or icmp admin-prohibited or just not respond at all or... i hate poorly-configured firewalls)16:12
openstackgerritA change was merged to openstack-infra/config: Restrict nodepool memory only when possible
clarkbfungi: they love you back16:12
*** johnthetubaguy has quit IRC16:13
fungionce that ^ gets to nodepool.o.o i'll start some updates of precise images in rax16:13
*** chandan_kumar has quit IRC16:14
*** Ryan_Lane has quit IRC16:15
*** esker has quit IRC16:17
*** jnoller has joined #openstack-infra16:17
clarkbjeblair: fungi: zaro mordred SergeyLukjanov and anyone else. I think we have been avoiding testing 2.8 point releases so far to keep things simple but we really want to avoid this gerrit bug. basically gerrit will think stuff is merged without actually merging it if you have a string of dependent changes16:17
clarkbits teh sort of thing that we probably would have found in the first week of using new gerrit had we been using new gerrit16:18
fungiclarkb: yeah, that does sound... not too great16:18
*** hogepodge has quit IRC16:18
clarkbit will merge the first change, report the otehrs as merged without merging in git :)16:18
*** thuc has joined #openstack-infra16:18
clarkbso I think we should go to 2.8.3 when it is available on review-dev16:18
fungiwe would have found that in the first day in production, if not in the first hour in production16:19
*** thuc has quit IRC16:19
*** thuc has joined #openstack-infra16:19
*** comstud is now known as bearhands16:19
*** thuc has quit IRC16:20
*** rdwrer is now known as marktraceur16:20
*** marktraceur is now known as rdwrer16:20
*** jlibosva has quit IRC16:20
*** dansmith is now known as damnsmith16:20
*** thuc has joined #openstack-infra16:21
*** hogepodge has joined #openstack-infra16:21
jeblairjnoller: do you know if glance support needs to be enabled for accounts on rax?16:22
jeblairjnoller: i just tried "glance image-list" and got "404 Not Found"16:23
*** jgallard has quit IRC16:23
*** sarob has joined #openstack-infra16:23
*** caleb_ has quit IRC16:24
*** nati_ueno has quit IRC16:24
*** rpodolyaka has joined #openstack-infra16:25
*** vkozhukalov_ has joined #openstack-infra16:26
jnollerjeblair: It should be "just on"16:27
*** sarob has quit IRC16:27
jnollerjeblair: can you PM me your account number so I can check with the product team?16:28
openstackgerritA change was merged to openstack-infra/config: Pass branch-override on jobs missing it
*** zns_ has joined #openstack-infra16:29
*** zns has quit IRC16:30
*** gokrokve has joined #openstack-infra16:30
clarkbfungi: what sort of verification went on for that? should we go ahead and flip the switch today?16:31
openstackgerritA change was merged to openstack-infra/storyboard: Adding creator_id field assignment
openstackgerritA change was merged to openstack-infra/storyboard: Fix load_superuser to work with new oslo
clarkbfungi: (leaning on your judgement here as I believe you tested it)16:33
clarkbor checked it16:33
fungiclarkb: it's currently working on review.o.o in dry-run mode, logging the first contribution from each new contributor. it's been checked and confirmed that it's really only firing once per contributor too, on the first patchset of their first change, so at worse we'll expose some code path hidden by --dry-run when we merge that16:33
fungiclarkb: but i'll keep an eye on the logs at that point anyway16:33
zaroclarkb: yeah i just saw that gerrit email.16:34
clarkbfungi: cool I will approve ti then16:34
zarofungi: the gerrit-2.8.3 should be available soon, do you want to merge the changes with what's in openstack/2.8 branch when it becomes available?16:35
clarkbzaro: or create a 2.8.3 branch16:35
clarkbthat might be easier depending on the deltas involved16:35
openstackgerritA change was merged to openstack-infra/storyboard: Fix migration tests
fungiyeah, for other updates like that we basically just re-propose a patch series rebased on the new branch/tag and pencil-whip an approval on it if tests pass16:36
*** talluri has joined #openstack-infra16:36
*** jooools has quit IRC16:37
*** openstackstatus has joined #openstack-infra16:38
jeblairfungi, clarkb: statusbot was running but offline with nothing in debug logs; i restarted it in screen16:38
clarkbjeblair: rgr16:39
*** blamar has quit IRC16:41
clarkbjeblair: reviewed the unbound change. one small thing inline16:42
*** blamar has joined #openstack-infra16:42
openstackgerritSean Dague proposed a change to openstack-infra/elastic-recheck: only display events in irc on gate issues
*** esker has joined #openstack-infra16:42
*** esker has quit IRC16:43
jeblairsdague: you don't want to see known errors in check?16:44
sdaguejeblair: no16:44
sdaguebecause we get things like this:16:44
sdagueopenstack/tempest change: failed check-grenade-dsvm, check-tempest-dsvm-full, check-tempest-dsvm-neutron, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-ironic-nv, check-tempest-dsvm-ironic-postgres-nv, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-16:44
sdaguesahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of:
sdagueand people are completely tuning out the bot at this point16:45
*** adalbas has quit IRC16:46
jeblairsdague: is the signature for that bug inaccurate?16:48
sdaguethe signature is fine, it matches one of those failed tests somewhere16:48
*** Ryan_Lane has joined #openstack-infra16:48
sdaguehowever, the signatures really are only full accurate on otherwise working code16:48
*** esker has joined #openstack-infra16:49
openstackgerritA change was merged to openstack-infra/config: Enable Welcome Message to post review comments
jeblairsdague: *nod*16:49
sdagueif people throw random fuzz at things16:49
sdagueyou'll get hits some times as well16:49
jeblairsdague: also, is the test list very useful in irc? maybe omitting it, and thereby reducing the wall of text may help with attention...16:49
sdaguejeblair: that's a good point as well16:49
*** BobBall is now known as BobBall_Away16:50
sdaguehonestly the thing I'm mostly interested in is a failed gate job that is unclassified16:50
jeblair(or alternately, limiting it to 1, so it's either "failed FOO or failed multiple tests"16:50
*** jgrimm has quit IRC16:50
sdagueyeh, there are a few fixes here that would be also useful. dropping the check queue was already about 50% done so I just did that one16:51
*** rpodolyaka has quit IRC16:51
fungiperhaps if it were slightly smarter, not fire at all if more than one job reported the same failure pattern on the same change/patchset16:53
clarkbmordred: has a comment16:53
*** dims has joined #openstack-infra16:54
* jeblair quietly removes his +2 and hopes no one notices it was ever there16:54
clarkbjeblair: >_>16:54
*** matrohon has quit IRC16:55
clarkbI should actually update my review to have it check in the comprehension /me does that16:56
*** rpodolyaka has joined #openstack-infra16:58
*** hogepodge has quit IRC16:58
openstackgerritDavid Kranz proposed a change to openstack-infra/config: Add periodic tempest non-isolated job with neutron
*** VINOD__ has quit IRC16:59
*** esker has quit IRC16:59
*** dims has quit IRC17:00
*** markmcclain has joined #openstack-infra17:00
*** markmcclain has quit IRC17:00
*** markmcclain has joined #openstack-infra17:01
*** nati_ueno has joined #openstack-infra17:01
clarkbjeblair: if it helps I had written my own version of that change too before I realized mordred wrote oen so was familar with the code17:01
*** hogepodge has joined #openstack-infra17:02
*** Ryan_Lane has quit IRC17:02
* fungi is heading out to grab some sushi, back in an hour or so17:02
*** pdmars has quit IRC17:02
*** harlowja_away is now known as harlowja17:02
fungijjb updates on all the jenkins masters finished btw17:02
openstackgerritSean Dague proposed a change to openstack-infra/elastic-recheck: only display events in irc on gate issues
*** Ryan_Lane has joined #openstack-infra17:06
mordredjeblair: it helps if you don't tell people you're quietly removing things :)17:06
mordredclarkb: thanks - I will definitely move the code - I think I'd like to keep it as an if though - putting it in the comprehension seems slightly harder to read17:06
clarkbmordred: do you know why the split on ',' exists?17:07
clarkbmordred: the if won't handle the split on ',' if you have a list of images delimited by ',' and vmdk shows up in the non last element17:07
clarkbmordred: you could do the comprehension then do a filter on it17:07
mordredah. gotcha. excellent point17:08
clarkba second step filte rwould be readable I think17:08
clarkbyou could even do the filter outside the loop before returning17:08
openstackgerritA change was merged to openstack-infra/gerritbot: Support SSL
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Skip downloading giant VMware images
mordredclarkb: nah - I think the filter in the comprehension is fine if there's an operational reason for it17:10
damnsmithwould it be a welcome change for the progress bar of jobs to change color once we learn that the job is failing?17:10
damnsmithI get that via and it's really helpful to know that something is failing early17:10
*** hogepodge has quit IRC17:11
jeblairdamnsmith: how does it know it's failing?17:11
damnsmithjeblair: don't we know? or do we only know once it has failed?17:11
damnsmithI thought we got early warning now..17:11
damnsmithI don't see the jobs broken out in dash, so maybe I'm confusing failed jobs with failing jobs17:12
jeblairdamnsmith: we have the theoretical possibility of determining that by having something read the subunit stream from testr, but nothing has been written yet to do that17:12
damnsmithjeblair: ohh, I thought it had17:12
jeblairdamnsmith: hopefully we will get that, because it means that zuul could make even earlier decisions about changes...17:12
*** pbelanyi_ has quit IRC17:12
damnsmithjeblair: right, I thought it was now.. okay..17:13
jeblairdamnsmith: but i think at this point that might end up landing in the post-jenkins job runner...17:13
*** samalba has left #openstack-infra17:13
damnsmithjeblair: you trampled my hopes and dreams :(17:13
jeblairdamnsmith: because i'm thinking it might be a bit too much dev for something we decided we want to replace17:13
jeblairdamnsmith: yeah, sounds like dash is looking at overall completion as soon as a single job for the change fails17:14
damnsmithokay, I really thought this was already done17:14
jeblairdamnsmith: (equivalent to the red/green dot)17:14
damnsmithtoo much magic to keep in my head, I guess17:14
*** skraynev is now known as skraynev_afk17:14
clarkbjeblair: is very topical17:15
devanandacommand to trigger an experimental run is "check experimental", yes?17:15
clarkbdevananda: yes17:15
*** SumitNaiksatam has joined #openstack-infra17:16
*** smarcet has left #openstack-infra17:16
mordredclarkb: remember back when we exceeded 60k reviews and thought that was alot?17:17
*** afazekas has quit IRC17:18
jeblairwe should plot review # over time17:19
jeblair  |17:19
jeblairoh there it is ^ :)17:19
*** hashar has quit IRC17:20
*** zns_ has quit IRC17:21
*** morganfainberg_Z is now known as morganfainberg17:21
*** e0ne has quit IRC17:22
jeblairis it possible that selinux is not enforcing on rax, but is on hpcloud?17:24
pleia2 looks to be stuck17:25
*** wenlock has quit IRC17:27
jeblairpleia2: thx reapproved17:27
jeblairselinux is disabled on the rax node i booted, and the errors certainly seem to support in being enabled on the hpcloud ones17:27
pleia2jeblair: & too17:27
*** jp_at_hp1 has joined #openstack-infra17:28
*** zns has joined #openstack-infra17:28
*** jp_at_hp has quit IRC17:28
jeblairi approved the 1st one, and recheck-no-bugged the second one since i didn't actually review it.17:30
*** dprince has quit IRC17:31
bknudsonon this
bknudsone-r says check-tempest-dsvm-neutron-full:
bknudsonbut check-grenade-dsvm-partial-ncpu FAILURE in 22m 04s (non-voting)17:32
clarkbbknudson: the bug signature isn't specific enough and matched the successful test17:32
jeblairrustlebee: i think you had a nova-specs change, but it may have the wrong topic, and it looks like jenkins is -1 on nova-docker17:37
clarkboh right new project friday /me reviews those17:37
*** mriedem has quit IRC17:37
*** mriedem has joined #openstack-infra17:39
openstackgerritBrant Knudson proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1291677
pleia2nibalizer: when you're about, would be nice to have your thought's on clarkb's comment here so we can nudge this along (I think it would be great to get this going):
*** dims has joined #openstack-infra17:40
*** amcrn has joined #openstack-infra17:40
*** khyati has joined #openstack-infra17:42
zarofungi: fungi gerrit-2.8.3 has been released.  could you please update with one of our branches?17:42
*** wenlock has joined #openstack-infra17:43
zaroclarkb: i've commented
*** esker has joined #openstack-infra17:43
clarkbjeblair: is another17:44
clarkbzaro: thank you17:44
*** rpodolyaka has quit IRC17:45
jeblairclarkb: i intend to hold the line on the new-project topic17:45
*** hogepodge has joined #openstack-infra17:45
clarkbjeblair: ok, I will leave a comment on any I see without the topic to add the topic17:45
openstackgerritA change was merged to openstack-infra/gitdm: Add header to output files and allow date ranges
*** CaptTofu has joined #openstack-infra17:46
clarkbjeblair: I think that would be good to get in17:46
clarkbbut I also think gettin larger consensus is good because it changes a big piece of the automated infra17:47
jeblairpleia2: just a reminder, i would like to have puppetboard up and running before we merge any puppet refactor changes17:47
*** thuc has quit IRC17:48
*** jpich has quit IRC17:48
pleia2jeblair: ah, ok17:48
*** thuc has joined #openstack-infra17:48
*** rpodolyaka has joined #openstack-infra17:48
*** homeless_ has quit IRC17:48
jeblair(i want the folks working on the puppet refactor to be able to spot and fix any problems themselves without infra-root being a bottleneck)17:49
pleia2yeah, that makes sense17:49
clarkbjeblair: is the nova specs change.  Iam leaving a commetn about new-project topic17:50
*** zns has quit IRC17:51
*** zns has joined #openstack-infra17:52
openstackgerritDan Smith proposed a change to openstack-infra/devstack-gate: DO NOT MERGE - Test larger largerops
*** vponomaryov has left #openstack-infra17:52
openstackgerritRussell Bryant proposed a change to openstack-infra/config: Create nova-specs git repo
*** thuc has quit IRC17:52
rustlebeejeblair: i don't know why the topic isn't right.  it's "new-project" locally.17:52
openstackgerritRussell Bryant proposed a change to openstack-infra/config: Create nova-specs git repo
rustlebeenm, fixed with manual git push instead of git review ..17:53
jeblairrustlebee: "git review -t new-project" should do it on a new patchset17:54
devanandai'm reading it now17:54
rustlebeejeblair: ah, cool17:54
rustlebeejeblair: i went with git push gerrit HEAD:refs/publish/master/new-project, heh17:54
devanandahrmm. didn't configure nova to use ironic ...17:54
clarkbrustlebee: beat me to it17:55
rustlebeeclarkb: thanks for telling me what i did wrong on the nova-docker one17:55
rustlebeei was stumped17:56
*** sarob has joined #openstack-infra17:56
*** thuc has joined #openstack-infra17:56
openstackgerritKhai Do proposed a change to openstack-infra/config: update docs and All-Project configuration for gerrit 2.8
*** derekh has quit IRC17:58
clarkbbreakfast has arrived /me afks17:58
jeblairclarkb, sdague: i'm not keen on the commit message in
jeblairclarkb, sdague: i think it has some inaccuracies and seems to have been written in reaction to an event that, as it turns out, is unrelated to the change18:00
openstackgerritRussell Bryant proposed a change to openstack-infra/config: Create a nova-docker repo
devanandafungi: have a minute to help me figure out why the experimental run of virtual-ironic looks like it didn't test any of the code in that devstack patch?18:00
*** Ryan_Lane has quit IRC18:01
jeblairin practice, it is not a huge problem (it doesn't happen "very often").  we have a tool to more or less instantly recover from such a situation....18:01
*** khyati has quit IRC18:01
jeblairdocumented here:
*** sarob has joined #openstack-infra18:02
sdaguejeblair: that's fair, colorfulness happens after debugging for 3 - 4hrs before anyone on infra root is up :)18:02
*** dcramer_ has quit IRC18:03
*** amcrn has quit IRC18:03
devanandafungi: the console log shows the right exports, but doesn't seem to indicate they were used18:04
openstackgerritSean Dague proposed a change to openstack-infra/config: offset nodepool update by 12 hrs
devanandaooh. instead of exporting, do all these need to be put in localrc?18:05
*** marun has joined #openstack-infra18:05
devanandasdague: ^ ?18:06
*** homeless has joined #openstack-infra18:06
*** zns has quit IRC18:06
sdaguedevananda: we don't automatically pass the environment all the way through18:06
sdaguesee devstack-gate-wrap18:06
*** zns has joined #openstack-infra18:08
*** adalbas has joined #openstack-infra18:08
devanandasdague: hm. so what's the right way to pass these in?18:08
*** yolanda has quit IRC18:09
*** amcrn has joined #openstack-infra18:10
devanandasdague: i see a lot of similar things in devstack-vm-gate.sh18:11
*** e0ne has joined #openstack-infra18:11
sdagueright, the real question is how much non default configuration this needs18:11
sdagueit feels like the way it should be handled is if VIRT_DRIVER=virtual_ironic it would set up all that behind the scenes18:12
devanandasdague: all those exports which we landed in 8057518:12
devanandasdague: hmm18:13
devanandaso _VM_COUNT and _VM_SPECS_* can be defaulted and then dropped from here18:13
devanandayea, i think all of them could be defaulted18:14
devanandaexcept IRONIC_BAREMETAL_BASIC_OPS18:14
devanandathat enables a big chunk of code specific to this environment, which isn't necessary for just starting the ironic api/conductor services18:14
*** jp_at_hp1 has quit IRC18:14
sdagueI think so. That would be at least consistent in that from a devstack user perspective, if yuo wanted to use it for ironic development, set VIRT_DRIVER=virtual_ironic, and you get a working setup18:14
*** hogepodge has quit IRC18:15
devanandaif you want to test ironic itself, set VIRT_DRIVER. if you want a local ovs bridge and some fake VMs, also set IRONIC_BAREMETAL_BASIC_OPS18:15
devanandacool. i'll refactor that and fix up the infra config and d-g code18:15
sdagueso IRONIC_BAREMETAL_BASIC_OPS would then get handled like all the other export = 1 thing in d-g18:15
*** zns has quit IRC18:15
sdagueyeh, I think that will work18:15
*** sweston has joined #openstack-infra18:16
*** mgagne has joined #openstack-infra18:17
*** mgagne has quit IRC18:17
*** mgagne has joined #openstack-infra18:17
devanandasdague: worth having two flags in d-g? DEVSTACK_GATE_IRONIC => just enable the service. DEVSTACK_GATE_VIRT_DRIVER => mutate the environment to use the nova.virt.ironic driver18:17
*** zns has joined #openstack-infra18:17
sdaguesure, I think that's fine18:18
devanandai think that also makes it explicit -- the ironic service is separate from the nova virt driver (which it is)18:18
*** zns has quit IRC18:22
*** zns has joined #openstack-infra18:22
openstackgerritAlex Gaynor proposed a change to openstack-infra/config: Gate python-glanceclient on PyPy
openstackgerritDevananda van der Veen proposed a change to openstack-infra/config: Fix the virtual-ironic job's export list
openstackgerritDevananda van der Veen proposed a change to openstack-infra/devstack-gate: Configure localrc for virtual-ironic tests
openstackgerritDan Smith proposed a change to openstack-infra/devstack-gate: DO NOT MERGE - Test larger largerops
*** jergerber has joined #openstack-infra18:29
*** rwsu has quit IRC18:30
*** ryanpetrello has quit IRC18:30
*** jgrimm has joined #openstack-infra18:31
fungizaro: openstack-infra/gerrit:upstream/stable-2.8 hasn't picked up the new commit yet, but it should when we rerun manage-projects here in a bit. after that i can branch openstack/2.8.3 from it for you18:32
*** hogepodge has joined #openstack-infra18:32
openstackgerritA change was merged to openstack-infra/config: Skip downloading giant VMware images
fungidevananda: keep in mind that if you *do* decide to try and modify localrc prior to running, we'll probably need a feature like the post_setup hook sdague mentions in his comment on
openstackgerritKhai Do proposed a change to openstack-infra/config: update gerrit project configs to use new Workflow label
openstackgerritKhai Do proposed a change to openstack-infra/config: update trigger to match new workflow label
*** salv-orlando has joined #openstack-infra18:38
*** thomasbiege has joined #openstack-infra18:39
*** rpodolyaka has quit IRC18:39
adam_gdevananda, heads up will likely bite you when things end up running on rackspace.18:42
adam_ggonna see if there is a manual workaround in the meantime while the update to cloud archive package makes its way thru18:42
*** dprince has joined #openstack-infra18:43
*** rpodolyaka has joined #openstack-infra18:47
*** dizquierdo has quit IRC18:48
devanandaadam_g: things are running on rackspace ...18:50
devanandafungi: not sure I follow your comment yet. are you suggesting I change 80653 ?18:51
fungidevananda: no, i was catching up on scrollback and you asked about altering localrc from the job definition... i was just going to point out that there are currently changes in flight to make that possible, but it's not doable yet18:52
devanandafungi: oh. gotcha. so instead, i did the above two patches, which is more in line with what ya'll do today18:52
*** Ryan_Lane has joined #openstack-infra18:53
fungidevananda: makes sense, though i haven't had time to look closely yet18:53
*** caleb_ has joined #openstack-infra18:56
sdaguefungi: while I think we could do this this way, if that hook was available, I don't think it requires the hook the same way that the 3rd party folks need it18:57
fungisdague: agreed, the current solution seems better for a project already in incubarion and targeting integration18:57
devanandaadam_g: that paste is interesting18:58
*** thuc has quit IRC18:59
devanandafungi: thoughts on ?18:59
adam_gthe bug has more details, obviously :)18:59
openstackgerritBrant Knudson proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1291677
devanandaadam_g has pointed out that virtual-ironic tests will fail on rax nodes unless we ^18:59
devanandabug --
*** thuc has joined #openstack-infra18:59
fungidevananda: yeah, i was looking at the bug report briefly. it looks like something you could do as part of your job or in a pre hook19:01
*** esker has quit IRC19:01
*** salv-orlando has quit IRC19:02
fungidevananda: though i wonder if providing our own vm images will mitigate that19:02
*** tstevenson has joined #openstack-infra19:02
adam_git would19:03
adam_gassuming that RAX-specific stuff is not installed in them19:03
*** jogo is now known as flashgordon19:03
clarkbthat bug affects precise?19:03
clarkbwe dont do saucy19:04
*** thuc has quit IRC19:04
devanandaclarkb: we're pulling in libvirt from cloud-archives for this test19:04
clarkboh dont do that19:05
clarkbuca libvirt is currently broken19:05
clarkb(being fixed though \o/)19:05
devanandawe need to...19:05
adam_gclarkb, broken in what regard?19:05
adam_gwell,other than not starting19:05
clarkbadam_g aiui its related to removing the global lock in newer libvirt19:06
fungiadam_g: dims is testing it... we had issues under devstack/tempest and also running nova unit tests with it19:06
clarkband it deadlocks. danpb fixed upstream and uca will include the fix(es) soonish19:06
*** thuc has joined #openstack-infra19:07
devanandaclarkb: so uca will be fixed soonish. so pulling it from there is the right thing to do soonish. so we're doign the right thing, then19:07
fungidevananda: anyway, that's what i meant with my apparently vague reference on your etherpad. once it's been sussed out, i expect we'll have it enabled everywhere and then that part of your job will cease being necessary19:07
adam_gclarkb, do you have a bug? i was going to see if i can expedite bug #1248025 getting pushed out, could make sure they both go out together19:07
*** tstevenson has quit IRC19:08
devanandafungi: that will be great. do yhou think we can proceed with this prior to that fix being pushed out?19:08
fungidevananda: i think it's fair to add a non-voting job to see if it's working for your use19:09
devanandafungi: my goal today is to get 70348 landed in devstack, so ironic developers can start working on tempest tests.19:09
*** che-arne has joined #openstack-infra19:09
fungiadam_g: was one of the two we had open as uca blockers i think19:09
devanandafungi: great :)19:09
fungilooking for the other one now19:09
clarkbadam_g I dont have the bug handy it is filed against ubuntu libvirt and nova iirc19:10
zarofungi: a few of my changes have already merged onto openstack/2.8 branch.  should i cherry pick those changes to openstack/2.8.3 and go thru gerrit review again?19:10
*** marun has quit IRC19:11
fungizaro: yes, we'd be starting fresh from upstream on 2.8.3, so just leave comments on the ones which were already approved once and if they're passing tests i'm happy to approve them without a second +219:11
jeblairclarkb, fungi: i just noticed in my dns tests that hpcloud-az2 builds seem to have puppet timing out a lot...19:12
*** hogepodge has quit IRC19:12
jeblairclarkb, fungi: in the middle of the traceback is this:     from /usr/lib/ruby/site_ruby/1.8/facter/ec2.rb:5:in `metadata'19:12
fungijeblair: that's no good. just az2?19:12
jeblairclarkb, fungi: i'm thinking that the same metadata problem that's causing problems on first boot is also hitting puppet19:12
clarkbhuh thats facter getting metadata facts19:12
clarkbjeblair agree19:12
fungioh, ick19:13
*** pdmars has joined #openstack-infra19:13
* jeblair lunches19:13
*** ryanpetrello has joined #openstack-infra19:13
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Replaced first_name and last_name with full_name.
*** rpodolyaka has quit IRC19:14
fungiadam_g: looking back at my notes i guess it's been mainly 1228977 we've been chasing on the libvirt/uca side of things. the other i had noted was a bug in nova exposed by newer libvirt, already addressed now19:15
adam_gfungi, cool. looks like 1228977 has an update ready for testing for yesterday19:16
*** caleb_ has quit IRC19:16
fungiadam_g: yep, they've seen renewed vigor on it this week so i think we're close to a resolution19:16
*** rpodolyaka has joined #openstack-infra19:17
fungiit's just been a loooong time coming, or else we'd probably have been using uca for close to a year now19:17
flashgordonclarkb: what is involved in getting into ES19:17
flashgordonor is already there19:17
clarkbiirc that is what is missing19:17
fungiadam_g: while the bug itself doesn't stretch back that far, we had a couple instances of trying to add uca in production which were hastily rolled back because everything fell apart and we didn't have great debugging info, so no bugs were filed for a while19:18
openstackgerritDevananda van der Veen proposed a change to openstack-infra/config: Fix the virtual-ironic job's export list
dimsdevananda, latest status - there are two things i am chasing a 0.9.8+ libvirt that will end up in precise-updates itself. 1.2.2+ libvirt which will be in UCA/icehouse/precise. (both made available courtesy of @jamespage and @hallyn). i've validated the 0.9.8+ change and just waiting for them to push it out to official deb repos. 1.2.2+ is still facing a test failure which @jamespage is chasing19:19
jamespagedims, yeah - not much progress on that today - been chasing down bugs in icehouse3 on 14.04 (as zul is still on the beach)19:20
dimsjamespage, thanks!19:20
fungijamespage: sounds like he chose very well when scheduling his holiday19:20
flashgordongot a missing dependency in stable/havana19:21
fungiflashgordon: i believe horizon logs aren't indexed into logstash yet, though i forget the exact rationale19:22
*** CaptTofu has quit IRC19:22
clarkbno timestamps19:22
*** CaptTofu has joined #openstack-infra19:22
* flashgordon sees timestamps on
dimsjamespage, not sure if you saw my prev comment - "[09:15] <dims_> jamespage, some of the tests don't compile. (libvirt_1.2.1-0ubuntu5~cloud0.dsc). abandoning effort for now. i can build 1.2.2 tag direct from git."19:22
*** Ryan_Lane has quit IRC19:23
jamespagedims, I did - thanks for poking it19:23
clarkbtimestamps would be a new development19:23
jamespagedims, I think is specifically todo with the build environment19:23
jamespagedims, however I'm going to get zul to look on Monday19:23
jamespageas penance19:23
dimsjamespage, :)19:23
fungino good vacation goes unpunished19:23
clarkbof course it has its own format yay19:23
* clarkb thinks we should make them oslo log if possible19:24
*** rpodolyaka has quit IRC19:25
flashgordonfungi: it looks like keystone client uses oauthlib but we don't  install it in stable/havana oauthlib19:25
flashgordonhow is that possible19:25
fungiflashgordon: that looks like maybe someone's confused versions of python-oauth and python-oauthlib?19:25
*** rpodolyaka has joined #openstack-infra19:26
flashgordonfungi: link?19:26
flashgordonsdague: ^19:26
fungiflashgordon: i'm digging into it19:26
flashgordonfungi: thanks19:26
*** CaptTofu has quit IRC19:27
sdagueclarkb: horizon logs are tailing apache19:28
flashgordonfungi: ahh ouahtlib is needed by keystone19:28
fungiflashgordon: yeah, stable/havana of global-requirements only has oauth219:28
*** yolanda has joined #openstack-infra19:28
flashgordonand ouath is needed by keystone client19:28
sdagueso they will be whatever they are19:28
*** lcostantino has quit IRC19:28
sdaguebut I would assume that logstash already has a format for that?19:28
*** hashar has joined #openstack-infra19:28
clarkbsdague it does for clr stuff we do it with swift19:29
fungiflashgordon: and master of global-requirements only has oauthlib19:29
flashgordonkeystonecleint: oauthlib>=0.619:29
clarkbthat doesnt look the same. the dates are wrong for instance19:29
flashgordonfungi: be back in a bit have a meeting19:29
fungiflashgordon: k19:29
clarkbI think apache is taking stderr from wsgi and logging that maybe19:29
flashgordonare requirements changes gated on stable?19:29
*** yolanda has quit IRC19:30
flashgordonor at least client changes?19:30
sdagueclarkb: good point19:30
fungiflashgordon: clients don't gate on stable currently, but they should19:30
fungiflashgordon: so right now we test client changes against master everything, but we test stable servers with master clients, so asymmetry there19:31
fungiflashgordon: which allows clients to unfortunately break stable jobs right now19:31
fungiflashgordon: it's relatively known that we need clients to also run backward-compat gating jobs, but nobody's had time to add them yet19:32
zarojeblair: i was wondering if there's gonna be anymore releases to python jenkins?  i've seen a few situations where jjb users could use a method to get plugin versions to stay backwards compatible.19:32
*** nijaba has quit IRC19:33
sdagueclarkb: so that's apache error log format19:33
*** ildikov_ has quit IRC19:33
sdaguewhich is not chanable in the version of apache in precise (in trusty we could change it)19:33
sdagueany idea if logstash already has a baked in template for that?19:33
clarkbit might but I dont remember seeing one. look on the logstash recipes/cookbooks wiki19:34
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard: Replaced first_name and last_name with full_name.
*** zns has quit IRC19:36
*** yolanda has joined #openstack-infra19:37
*** zns has joined #openstack-infra19:37
*** sweston has quit IRC19:38
*** nijaba has joined #openstack-infra19:38
*** nijaba has joined #openstack-infra19:38
*** e0ne has quit IRC19:38
*** rfolco has quit IRC19:39
*** e0ne has joined #openstack-infra19:39
sdagueclarkb: looks like not19:39
*** nijaba has quit IRC19:40
*** Ryan_Lane has joined #openstack-infra19:41
*** yolanda has quit IRC19:43
fungibwahaha... i just stumbled across mangodb ;)19:43
fungioutput = open(os.devnull, 'w')19:44
*** e0ne_ has joined #openstack-infra19:44
*** e0ne has quit IRC19:45
dstufftmangodb is web 3.019:46
fungidstufft: and webSCALE19:46
dstufftalso does swiftclient have terrible docs or am I just too stupid to find them19:46
*** talluri has quit IRC19:46
fungidstufft: what sort of docs are you seeking? man swift?19:47
dstufftI want to use the Python API to do stuff19:47
dstufftI found this :/19:47
*** rpodolyaka has quit IRC19:47
sdagueclarkb: so how does one define a new timestamp format?19:49
*** rpodolyaka has joined #openstack-infra19:49
sdagueI'm completely failing with logstash docs19:49
clarkbsdague: logstash comes with a lot of prebuild time/date matchers19:49
fungidstufft: yeah, the sphinx/autodoc stuff is probably what there is19:49
clarkbsdague: you should be able to use MONTH or DAY etc to match it in grok19:49
fungidstufft: so what you find at
dstufftfungi: :[19:49
clarkbsdague: then lower in the logstash config we normalize all fo them to one format with a filter19:50
jesusaurussdague: what do you mean? where are you trying to use the new timestamp?19:50
fungidstufft: i'm sure nobody would turn down documentation patches ;)19:50
dstufftfungi: well before I do that, I have to actually figure out how to use it :D19:50
*** hogepodge has joined #openstack-infra19:51
fungidstufft: i do remember fiddling with it a while myself the first time i needed to do something19:51
sdagueclarkb: if I add it here -  is that the right place?19:51
*** tomhe has joined #openstack-infra19:51
flashgordonfungi: what needs to be done to add stable jobs to client trunk?19:51
sdaguejesusaurus: we need to index this file -
fungidstufft: you might /join #openstack-swift and inquire if the good stuff is hidden behind the counter or something19:52
*** nijaba has joined #openstack-infra19:52
devanandafungi: 80652 and 80653 await your eyes, when ever you get some time. I've already tossed up the changes to the ironic devstack patch, and will re-run the experimental job once these two land.19:52
dstufftfungi: yea good idea19:52
dstufftthanks :)19:52
fungiflashgordon: probably something using branch overrides like i did for the backward-compat devstack-gate jobs19:52
devanandai'm going to step out for ~ an hour or so, lunch and so on19:52
*** hashar has quit IRC19:53
fungiflashgordon: now that the d-g job templates have grown branch-aware flexibility it shouldn't probably be too tough, unless there's something i'm overlooking19:53
jesusaurussdague: ahh, okay, so you need to add a match to that date filter, but you also need to add an appropriate grok filter for the log lines19:53
flashgordonfungi: well lets sort out the dependency issue first, then I can do the client tests19:54
tomheGuys, did you hear about the nasty Gerrit bug? Will Zuul ever have two or more pending submits to the same branch?19:55
*** talluri has joined #openstack-infra19:56
fungiflashgordon: i think someone just needs to backport the requirement to openstack/requirements:stable/havana and it should unblock19:57
openstackgerritA change was merged to openstack-infra/storyboard: Replaced first_name and last_name with full_name.
fungiflashgordon: ought to be able to test with a requirements patch, unless the requirements integration job on stable/havana has bitrotted too19:58
fungitomhe: it's entirely possible for zuul to encounter that bug if you're using gerrit 2.6 or later, i expect19:58
fungitomhe: we haven't seen it, but then again we run gerrit 2.4.4 in production still19:59
tomhefungi: ok, thanks19:59
tomhefungi: We're at 2.8.3 now19:59
*** sweston has joined #openstack-infra20:00
fungitomhe: our dev environment where zaro's been testing our upgrade migration is getting upgraded to 2.8.3 soon too20:00
*** talluri has quit IRC20:00
fungimainly because of that bug20:01
flashgordonfungi:  how does just adding oauthlib to stable reqs fix it?20:01
flashgordondoesn't something need to install it?20:01
fungiflashgordon: won't the client try to install it?20:01
openstackgerritDonald Stufft proposed a change to openstack-infra/config: Check and gate against Python 2.6 for Barbican
fungior maybe i misread the exception20:01
tomhefungi: Really scary thing. Need to do some consistency checking on our repos. Anyway, thanks fir your input.20:02
dstufftChanges openstack-infra/config get reviewed here?20:02
fungitomhe: good luck20:02
dstufftchanged to*20:02
fungidstufft: yep20:03
dstufftI swear I know how to word20:03
sdagueso when do the new projects on friday get created?20:03
dstufft please 2 review? <320:03
fungisdague: on friday ;P20:03
sdaguefungi: well friday is well on the way to being over here in NY :)20:03
fungisdague: probably starting in a bit once west-coasters are done lunching20:03
clarkbtomhe: yes it is very possible. so possible that I expect if we upgraded to 2.6 we would have caught this bug on day 120:03
flashgordonfungi: keystone client will  try20:03
*** lcheng has quit IRC20:04
flashgordonthat doesn't cause things to fail if a req cannot be installed?20:04
dstufftfungi: thanks sir!20:04
fungiflashgordon: yeah, but i believe devstack is installing the global-requirements list and ignoring the requirements files in the projects, so we can be sure they work with the global requirements20:05
fungidstufft: yw. trivial patch so i didn't bother waiting for a second +2 on that one20:05
flashgordonwe install all of global reqs?20:05
nibalizerpleia2: using fqdn in the hierarchy does not on its own break anything, i'm not at this point suggesting we get away from site.pp20:06
*** sweston has quit IRC20:06
nibalizerI'm mostly excited about the osfamily and operatingsystem hierarchies20:06
pleia2nibalizer: ok cool20:06
flashgordonor do we remove a req during the sync global res if its not there20:07
nibalizerbasically using hiera for node level data is not something i've ever really liked, i see it as next to site.pp not replacing it20:07
jeblairthat's not what i meant to type, but it still conveys the same meaning...20:07
nibalizerbut some crazy people do use hiera as an ENC20:07
*** sweston has joined #openstack-infra20:07
*** yamahata has quit IRC20:07
jeblairclarkb, fungi: i'm back from lunch, ready for npf whenever20:07
*** yassine has quit IRC20:08
clarkbI am being yelled at to do lunch myself20:08
*** tomhe has quit IRC20:08
clarkbbut don't let me stop you20:08
jeblairoh, i have a centos node in az2!  and i _think_ the region-b one is past the tricky part20:09
*** rpodolyaka has quit IRC20:10
fungii'm still hammering at a couple of the rax precise images to get them to build now that we can again20:10
sdaguethe infuriating thing, this is basically rfc822 except missing the tz20:12
clarkbsdague: oh then it should work20:12
openstackgerritChad Lung proposed a change to openstack-infra/config: Removing non-voting for gate-barbican-devstack-dsvm
sdagueeven if the tz is not there?20:12
clarkbsdague: the existing matchter makes tz optional iirc20:12
clarkband assumes utc if not timezone20:12
openstackgerritA change was merged to openstack-infra/config: Check and gate against Python 2.6 for Barbican
sdagueoh, so it does20:12
sdaguewas reading patterns/grok-patterns20:13
clarkbok really lunch time now. I will probably be afk for most of teh afternoon while I eat and hang out with a friend before they move to san diego20:13
fungiflashgordon: i'm not certain--have to reread the requirements part of the devstack-gate functions list20:13
flashgordonlets see what does20:13
*** rcarrillocruz1 has joined #openstack-infra20:13
fungiflashgordon: yep, i figure it's worth starting there20:14
*** rcarrillocruz has quit IRC20:14
fungiflashgordon: though we may have trouble getting that through if the reqs-integration job has rotted away for other reasons (it doesn't get run very often), but at least the devstack job results should be enlightening20:15
*** rpodolyaka has joined #openstack-infra20:15
*** lcheng has joined #openstack-infra20:16
*** rpodolyaka has quit IRC20:16
*** khyati has joined #openstack-infra20:16
openstackgerritA change was merged to openstack-infra/storyboard-webclient: Consume full_name rather than first_name and last_name
*** rpodolyaka has joined #openstack-infra20:18
*** alff has joined #openstack-infra20:18
flashgordonfungi: is it just me or does this delete dependencies not in global reqs:
*** dcramer_ has joined #openstack-infra20:19
*** talluri has joined #openstack-infra20:21
openstackgerritJoe Gordon proposed a change to openstack/requirements: If dependency not in global reqs leave it so install will fail
openstackgerritSean Dague proposed a change to openstack-infra/config: add horizon_error to the indexed logs
flashgordonfungi sdague: does that sound sane ^20:22
jeblairflashgordon: maybe i'm late to the conversation, but how does that cause a failure?20:23
fungiflashgordon: why would install fail there?20:23
jeblairfungi: i think it's you and me on new projects; when do you want to get started?20:23
*** esker has joined #openstack-infra20:23
fungijeblair: i was just finishing reviewing them20:24
flashgordonfungi jeblair: so I am hoping that would mean the file isn't in your pypi mirror and would fail20:24
flashgordonbut not sure if it works that way20:24
jeblairflashgordon: ah, we never delete anything from pypi.o.o, so that's not guaranteed20:24
jeblairflashgordon: if something was in g-r then removed, it would still be in the mirror20:24
fungirustlebee: on 79363 you say creating nova-drivers in gerrit is a prerequisite... any reason? manage-projects is going to create that group for us, them we add you and let you put whoever else you want in there20:24
flashgordonjeblair: in that case how make the script exit out with a 1 and make sure devstack detects that and stops?20:25
*** marun has joined #openstack-infra20:25
sdaguefungi: that should be sufficient for what russellb needs20:25
jeblairflashgordon: (the thinking has shifted and pypi.o.o is no longer seen as a strong enforcement mechanism, only a weak one; requirements check jobs, etc are the real enforcement)20:25
rustlebeefungi: ok, then i lied20:25
fungiflashgordon: in the case of branch-specific problems like this, you're talking about expecting the mirror to not hand havana a package which master uses... that's not something we are able to defend against20:25
sdagueI think he just wanted to make sure it was a different group than -core, because it should be the smaller drivers group20:25
rustlebeefungi: sorry, misunderstood how it worked20:25
rustlebeethough i'm wondering now if i should make it nova-specs-core20:25
*** talluri has quit IRC20:25
flashgordonfungi: right20:26
zarojeblair: had a question for you about python jenkins (in scroolback).20:26
flashgordonthat makes sense20:26
fungirustlebee: no worries, just wanted to make sure before we potentially did stuff out of order for reasons of which i'm unaware20:26
rustlebeei'll remove that from the commit message20:26
rustlebeeand change the patch to use nova-specs-core instead, which i think is better20:26
fungirustlebee: no need, in my opinion20:26
fungirustlebee: that's fine if you want20:26
flashgordonfungi: so this happened because of the hole with changing deps on clients20:26
*** lcheng has quit IRC20:26
fungiflashgordon: correct20:26
rustlebeefungi: meh, ok, i'll leave it then20:26
flashgordonwould it be  better just to fix that instead of making the reqs sync fail hard?20:27
flashgordonor do both20:27
rustlebeefungi: is it a big deal to change group names?20:27
fungiflashgordon: about the only sane way we can deal with that currently is to add backward-compat jobs to clients20:27
jeblairzaro: hrm, i've never made a release; i've done very little with it.  but we can look into it20:27
openstackgerritAndreas Jaeger proposed a change to openstack-infra/config: Change operations-guide and openstack-manuals jobs to not use maven
jeblairrustlebee: if you want the alternate group name, let's go ahead and do it20:27
fungirustlebee: it's a patch and a manual action (a click or two and some quick typing) in gerrit. if you want a different name then now is the time to update that20:27
rustlebeejust in case nova-drivers in launchpad gets differentiated at some point20:27
rustlebeeok, let me just fix it20:27
jeblair(i'm +1 on that too, because we like to have the names != lp group names to reinforce there is no auto-syncing)20:27
flashgordonfungi: agreed that  backwards compat jobs to clients is needed -- but should I abandon this current line of thinking for the script to fail  as well20:28
fungiyeah, people still get confused since they log in through lp and there are sometimes similarly-named groups20:28
*** rwsu has joined #openstack-infra20:28
flashgordonfungi: in other words should I do 'A and B' or just 'A'20:28
sdagueflashgordon: so there is already a check-tempest-dsvm-full-havana that we run on devstack-gate20:28
fungiflashgordon: yeah, we might be able to convince to fail in other ways to help belt and braces this, but that would require some more code there i think20:29
openstackgerritAndreas Jaeger proposed a change to openstack-infra/config: Remove maven from operations-guide and openstack-manuals
sdagueyou can probably just slap that on the clients20:29
openstackgerritRussell Bryant proposed a change to openstack-infra/config: Create nova-specs git repo
flashgordonsdague:  thats what I was thinking20:29
jesusaurussdague: you will need to escape your brackets in your grok filter20:29
sdaguejesusaurus: \[ ?20:29
jesusaurussdague: yep and \] is a good idea too (though closing a spurious ] would be a bug in itself)20:30
openstackgerritSean Dague proposed a change to openstack-infra/config: add horizon_error to the indexed logs
*** ArxCruz has quit IRC20:30
sdaguejesusaurus: ok, how about that20:30
sdagueand thanks20:30
fungirustlebee: did you mean to change the group on 79363 in your last patchset?20:31
fungirustlebee: because if so, it looks like you didn't do so20:31
zarojeblair: yes, please.  I have a pending patch that adds a getPluginInfo method.  would be very useful in jjb.  thanks.20:31
sdagueflashgordon: so for this, please make a client-template20:31
fungirustlebee: nevermind me20:31
jesusaurussdague: perfect20:31
fungirustlebee: i was looking at the old patchset :/ (friday!)20:31
openstackgerritRussell Bryant proposed a change to openstack-infra/config: Note requirement to use 'new-project'
sdagueso when we have to add or remove a check-tempest-dsvm-full- it's one line, and not 1520:32
flashgordonI see check-tempest-dsvm-full-havana is non-voting20:32
jeblairclarkb: OH!  now i know what your comment on my patch meant -- grepping for 10. isn't enough because of the 206.164. thing...20:32
*** ArxCruz has joined #openstack-infra20:33
sdagueflashgordon: yeh, in the current scheme that's true20:34
*** bookwar has left #openstack-infra20:34
sdagueand realize there is going to be some very interesting implications of gating clients on stable servers20:34
sdaguebecause stable servers bit rot20:34
sdagueso clients are going to wedge a lot20:34
flashgordonyeah true20:34
sdaguethe net effect might be that people rush to make stable work more often20:35
fungisdague: flashgordon: in previous conversations where this came up, mordred was always keen on pointing out that it would encourage the client devs to pitch in on stable bitrot problems ;)20:35
sdaguebut it is going to cause some pitch forks20:35
openstackgerritJoe Gordon proposed a change to openstack/requirements: If dependency not in global reqs exit with a 1
sdaguewhich I'm totally fine with, if they are coming after you, flashgordon ;)20:35
flashgordonsdague fungi: well lets start with this20:36
fungibut this is where the tc possibly making sure our support policy strongly underscores client support for supported stable servers as important20:36
sdagueflashgordon: you still don't have test20:36
fungicould help20:36
sdagueI went to great lengths to test the heck out of update.py20:36
sdaguebecause if it ever is wrong20:36
sdagueeverything stops dead20:36
flashgordonsdague: I'll add some tests. but is the general idea sound?20:36
sdagueflashgordon: I came in late in the conversation, explain to me what you are trying to address20:38
*** lcheng has joined #openstack-infra20:38
flashgordonsdague: yes, so stable/havana broke20:38
*** alff has quit IRC20:38
*** Ajaeger has quit IRC20:39
sdagueright, but explain exactly how20:39
flashgordonheat doesn't oauthlib.oauth120:39
flashgordonand neuther does horizon20:39
fungimorganfainberg: quick question on the kds new project request (73074)... you've got it in the openstack namespace (being split out from keystone) but under management of existing groups used currently for management of a stackforge project. is this project going to get listed in as belonging under the control of the20:40
fungiidentity program officially?20:40
flashgordonwhich has the same error20:40
flashgordonsdague: it looks like oauthlib is a keystoneclient dependency, but it wasn't in the global-reqs for stable/havana20:40
flashgordonso it never got installed20:40
morganfainbergfungi, it should be stackforge i think.20:40
flashgordonso the fix should be:
flashgordonthen the follow up is how did this happen20:41
flashgordonand make sure it doesn't any more20:41
morganfainbergfungi, did we (jamielennox and I) forget to fix the namespace?20:41
flashgordonso I figure two parts:20:41
*** rhsu has joined #openstack-infra20:41
morganfainbergfungi, *checks*20:41
fungimorganfainberg: oic, you have stackforge in some places, but openstack on the acl paths20:41
flashgordonif finds a dependency not in global reqs -- which shouldn't happen if we do it right -- then fail hard20:41
flashgordonand secondly gate clients on stable since we use them there20:41
morganfainbergfungi, aha, so we have some mismatch20:41
mordredfungi: aroo?20:41
openstackgerritVictor Seva proposed a change to openstack-infra/jenkins-job-builder: use {obj:key} as a way to pass an object and not only strings to templates
fungimordred: aroo20:42
mordredsdague: I actually think that the stable servers will stop bitrotting, not because of more people - but because we'll more frequently run jobs against them that people will care about20:42
sdaguemordred: sure20:42
fungimorganfainberg: i see other typos too. updating the review in a moment20:42
morganfainbergfungi, yeah, barbican is meant to take over the project20:42
morganfainbergfungi, if it needs to have it's own acls, thats fine20:42
morganfainbergfungi, oh and i see the issue20:43
fungimorganfainberg: it does have its own acls, they just have the wrong file paths20:43
morganfainbergfungi, ok. thanks20:43
morganfainbergfungi, ++ eyah was missed when we updated from openstack to stackforbge20:43
fungimorganfainberg: but i've got other comments going in real quick too if you're about to follow up with another patchset20:43
sdagueflashgordon: ok, sure, that's probably the right fix20:43
sdaguethat will explode early which is good20:44
flashgordonsdague: fail early and fail hard20:44
flashgordonand often ;)20:44
sdaguewe should bring those stable gate jobs back20:44
sdagueor create them based on the d-g template20:44
*** zehicle has joined #openstack-infra20:44
*** zehicle_at_dell has quit IRC20:44
sdaguethough the whole thing kind of remains all kinds of fubar because python packaging is so excellent20:45
sdaguebecause it's going to trigger a lot of desire to change stable global-requirements20:45
sdaguemordred: have you thought through where this crazy train takes us?20:46
fungimorganfainberg: okay, updated20:46
sdaguebecause I'm a little concerned it's not a great place20:46
morganfainbergfungi, thnkx much :)20:46
sdagueflashgordon: actually....20:47
sdagueI'm far more inclined that this is the wrong fix20:47
sdagueand your first change was better20:47
*** sarob has quit IRC20:47
flashgordonsdague: ?20:47
*** julim has quit IRC20:47
mordredsdague: what do you mean?20:47
sdaguebecause global-requirements was brought in largely to handle the crazy wedges of incompatible packages between components20:48
flashgordonthe make update error out20:48
sdaguehowever, if we're talking about a few adds by clients20:48
fungijeblair: okay, i think we can probably dispense with the etherpad since we've only got four patches (counting morganfainberg's on the assumption he's got a patch on the way correcting it)20:48
*** sarob has joined #openstack-infra20:48
*** rpodolyaka has quit IRC20:48
sdagueI think it's better to just let those slide20:48
sdaguebecause we are enforcing they are in g-r in master20:48
fungijeblair: are you aware of any new jeepyb patches we needed to test with this round?20:48
*** jnoller has quit IRC20:48
*** lcheng has quit IRC20:48
mordredsdague: I say that we get WAY stricter with client deps20:48
mordredsdague: because our client libs are used in our stable servers20:48
jeblairmordred: ^ can you address fungi's jeepyb question?20:48
mordredjeblair: looking20:49
clarkbfungi I dont think there are any20:49
sdaguemordred: so what we're saying is you can't add a client lib dependency unless you update g-r in master, n-1, and n-2 ?20:49
sdagueor bump a minimum version20:49
clarkbI think jeepyb is happy now. the cross host orchestration is what hurts us now20:49
mordredsdague: yes20:49
sdagueI think that's got to be an ML thread at least20:50
mordredclarkb, jeblair: I was just talking to jesusaurus about the cross-host orchestration - or at least part of it20:50
sdaguebecause that's going to be a big hammer20:50
mordredsdague: sure. I could be VERY wrong about that, btw20:50
sdagueI'm not sure I disagree with the approach20:50
mordredbut for client libs, I think we should probably e pretty able to not use the latest bleeding edge stuff20:50
sdaguehowever it is a very substantial shift from current20:50
sdagueso I don't think something that can be done by fiat, at minimum needs to be at the project meeting20:51
jeblairfungi: sounds like we're gtg.20:51
jeblairfungi: what's the procedure?20:51
*** lcheng has joined #openstack-infra20:51
mordredsdague: ++20:51
openstackgerritJoe Gordon proposed a change to openstack-infra/config: Test stable branch on client changes
flashgordonsdague mordred: I will let you guys sort out if we want this or not20:52
flashgordonbut that should do it20:52
*** sarob has quit IRC20:52
flashgordonas long as we at least get working I am happty20:52
fungijeblair: at this point, i think merge 'em all and let zuul sort 'em out, then once they're on gerrit (assuming we've got manage-projects disabled there from a puppet perspective), run manage-projects manually and make sure the groups/acls were created in gerrit and try cloning them from all the git servers (and retrigger replication for them if not, but if we wait a but before running that shouldn't20:52
fungii'm double-checking the puppet hypothesis now20:53
*** vkozhukalov_ has quit IRC20:53
jeblairfungi: which is supposed to happen first?  create-cgit-repos or manage-projects?20:53
*** hashar has joined #openstack-infra20:53
fungijeblair: if create-cgit-repos happens first then we're good20:53
fungijeblair: if manage-projects happens first then we don't get replicated to some of the farm20:54
jeblairfungi: ok, so merge, wait for cgit to run, then run mp.20:54
*** mark0n has joined #openstack-infra20:54
fungialso if puppet gets stuck on one of the git servers then create-cgit-repos never runs, so we end up with a problem, but the last time that happened i found we still had github remotes for /opt/jeepyb on a few of them, so corrected that20:55
fungiin theory not likely any longer20:55
fungialso, clarkb took m-p back out of puppet last friday in aa84680 so we should be fine there20:56
clarkbyup and your plan sounds good20:56
*** lcheng has quit IRC20:57
*** esker has quit IRC20:57
*** lcheng has joined #openstack-infra20:57
sdagueflashgordon: so I think we're back to that approach, it needs tests, also an ML thread20:57
sdagueflashgordon: as you found the issue, you want to start that thread?20:57
fungi79900 and 79900 need a second +2 ideally, and i think morganfainberg has a patchset on the way for 7307420:58
*** mbacchi has quit IRC20:59
flashgordonsdague: so many threads20:59
flashgordonI'll start one big thread for both20:59
flashgordonexplaining what happend and the fix20:59
sdaguehey, at least this won't be a vim comments thread20:59
fungisdague: we can make any thread into a vim comments thread if you really want21:00
flashgordonsdague: that one resulted in hours of entertainment for many folks21:00
*** jp_at_hp has joined #openstack-infra21:00
*** caleb_ has joined #openstack-infra21:01
jeblairfungi: 3 changes in the queue.21:01
fungijeblair: saw, thanks21:01
*** salv-orlando has joined #openstack-infra21:02
*** salv-orlando has quit IRC21:02
openstackgerritA change was merged to openstack-infra/config: Create nova-specs git repo
jeblairfungi: conflicts21:02
fungii'll rebase it quick21:02
*** salv-orlando has joined #openstack-infra21:02
*** thomasem has quit IRC21:04
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Create a nova-docker repo
fungirebased ^21:05
fungiit conflicted on layout.yaml because the nova-specs addition was within its context margin21:05
*** mfer has quit IRC21:06
*** pdmars has quit IRC21:07
*** caleb_ has quit IRC21:07
fungilooks like we probably have about a 15 minute wait on check nodes for those two changes21:07
*** ildikov_ has joined #openstack-infra21:08
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Ruby metrics plugin support added
fungithough the graph suggests it may be catching up faster than i expect21:08
jeblairzaro: pushed jjb 0.7.021:08
*** david-lyle has quit IRC21:09
*** sdake_ has quit IRC21:10
jeblairfungi: once we get the sudo stuff through (which is like 1 or 2 more changes), we're probably pretty close to combining bare-precise and devstack-precise into one node type21:10
*** sdake_ has joined #openstack-infra21:10
*** sdake_ has quit IRC21:10
*** sdake_ has joined #openstack-infra21:10
jeblairfungi: which should make nodepool a little more responsive21:10
fungijeblair: that will be swell21:10
lifelessmordred: so - ?21:10
fungijeblair: maybe we can collapse py3k-precise into them too21:10
jeblairfungi: oh could we?  that can coexist?21:11
fungithough that may require a little more bootstrapping in the jobs, so maybe not21:11
fungijeblair: yeah, forget i suggested it ;) nontrivial for now still21:12
jeblairfungi: is it a matter of deb installation?  if we cache it, that shouldn't take long...21:12
jeblairfungi: if there's a mess of puppet stuff, that may be a bit further off.21:12
*** aysyd has quit IRC21:12
sdaguehas there been any thinking about doing the unit tests inside a container in a node, so the node doesn't have to be single use? or is that just way too far down the inception hole.21:12
openstackgerritMartin Konrad proposed a change to openstack-infra/jenkins-job-builder: SCM module: Add support for multiple Git remotes.
fungijeblair: i think we might be able... but for right now it depends on a different puppet class21:12
jeblairsdague: we used to not have single use nodes, if you recall :)21:13
sdaguejeblair: yes21:13
fungijeblair: also, i think with the advent of trusty it will become (maybe mostly) a no-op change21:13
sdaguebut they had the disadvantage of getting cruddy21:13
fungisdague: they also had the disadvantage of jenkins21:13
jeblairsdague: one of the reasons we moved to using them so quickly is to avoid a jenkins problem; when we get rid of jenkins that will no longer be necessary...21:14
sdaguejust one of those random friday thoughts21:14
jeblairsdague: but another reason was to increase flexibility --21:14
jeblairsdague: so stackforge projects can install their mongodb on rails or whatever.21:14
sdagueright, though containers would permit that as well21:15
fungisdague: however, at least some of our jobs currently don't like containers (i believe things testing iscsi for example)21:15
fungiso we'd potentially need a mix21:16
sdagueI guess we just need to actually get containers in openstack integrated at the right level, re: not as virt drivers21:16
*** mriedem has quit IRC21:16
sdaguefungi: does that expose in unit tests?21:16
sdagueand if so.... yuck21:16
fungisdague: which is why i said we'd still need a mix. i don't think the unittests are an issue there though21:16
*** caleb_ has joined #openstack-infra21:17
sdagueyes, definitely21:17
fungithe idea is not without merit, but i think we have other things which need to happen first before we can consider it more strongly21:17
sdaguestill would need a mix21:17
clarkbcontainers arent secure21:17
clarkbyou cant allow root in them and reuse21:17
clarkbat least not with precise/centos kernels21:17
*** thedodd has quit IRC21:17
clarkbtrusty may be better21:18
sdagueclarkb: right, this would need at least the trusty kernel, iirc a bunch got fixed by then21:18
*** esker has joined #openstack-infra21:18
fungiclarkb: true. if we're concerned about jobs leaking root privs to other containers or the host vm, then that's a good reason to avoid them until trusty as well21:18
fungiand centos721:18
sdaguethough you could also take an educated risk and kill them after some number of uses anyway21:18
*** rcleere has quit IRC21:18
fungiwhich i guess is due out soon? (well, rhel7 anyway)21:18
jeblairthere's actually a stub idea in nodepool for a "reset" script that would clean up a node after use instead of destroy21:19
jeblairso the thought has definitely occured21:19
jeblairbut yeah, as long as we have jenkins, it's going to be problematic21:19
jeblairso post-jenkins+post-trusty maybe so21:19
fungiand yeah, something like a max reuse metric for them (akin to apache recycling worker threads)21:19
jeblairpost-precise rather21:19
fungiflashgordon: right, i thought we might have bitrotted there21:20
fungisince that job only gets run on the blue moon when someone updates havana global reqs21:21
*** mkoderer has quit IRC21:21
*** openstackgerrit has quit IRC21:21
sdagueflashgordon: this should run clean now?21:21
sdagueI just hit recheck on it21:21
*** openstackgerrit has joined #openstack-infra21:21
sdagueI would like to see the partial job actually pass21:21
*** talluri has joined #openstack-infra21:22
flashgordonsdague: partial-job needs
flashgordonwhich needs
flashgordonso yeah21:23
openstackgerritA change was merged to openstack-infra/config: Create qa-specs git repo
sdagueflashgordon: oh, so that's all borked in the head21:23
flashgordonsdague: exactly21:23
sdaguewell I'm leaving for beers in 30 minutes21:24
fungilooks like when installing trove the integration job hits "No local packages or download links found for pbr>=0.5.16,<0.6"21:24
fungiis trove expected to be operable on havana?21:24
fungior should it be ripped out of the projects.txt in the openstack/requirements:stable/havana branch?21:25
fungiflashgordon: ^21:25
flashgordonfungi: not sure, sdague ^21:25
openstackgerritAntoine Musso proposed a change to openstack-infra/jenkins-job-builder: Migrate to pbr
mtreinishsdague: cool, we've got a qa specs repo now21:26
*** talluri has quit IRC21:26
fungimtreinish: well, soon anyway. the change merged but we haven't run the creation step manually yet21:27
openstackgerritA change was merged to openstack-infra/config: Create a nova-docker repo
sdaguefungi: trove should be ripped out of stable/havana21:27
sdagueeven in master it's not doing anything in the integrated jobs21:27
fungimtreinish: then i think you probably have to do that in the same patch21:27
flashgordonso now my patch needs that patch too heh21:27
fungier, flashgordon not mtreinish21:28
sdagueso requirements sync in stable/havana for trove is pretty pointless21:28
fungiall you electrons look alike to me21:28
SlickNikfungi: We're not supporting trove on stable/havana, so I think ripping it out of stable/havana is the right call.21:28
jeblairmorganfainberg: are you going to update the kite patch?21:28
sdagueSlickNik: to be honest, it's not actually ripping it out of stable/havana21:28
openstackgerritJoe Gordon proposed a change to openstack/requirements: If dependency not in global reqs exit with a 1
sdagueit's just not going to be in the sync job21:28
fungiSlickNik: okay. it wound up in the projects.txt for requirements stable/havana branch, so that makes some sense that it's been overlooked there21:28
morganfainbergjeblair, in a moment i will21:29
morganfainbergjeblair, sorry i'm fighting with some non-upsteam stuff21:29
morganfainbergjeblair, was distracted21:29
sdagueSlickNik: so ... when you all going to get some tempest tests? :) I noticed a very distinct lack of anything when looking at jobs the other day21:29
fungimorganfainberg: if there's no rush on it, we can just catch it next friday21:29
morganfainbergjeblair, i'll get that updated in the next ~5min21:29
SlickNiksdague: I just merged some basic tempest tests in a week ago.21:30
sdaguethey running in the gate?21:30
*** dstanek is now known as dstanek_afk21:30
SlickNiksdague: Still need to make the experimental (non-voting) job in the gate.21:30
SlickNiksdage: nope; hope to do that shortly.21:31
sdagueSlickNik: why aren't they just added to this job -
flashgordonsdague: can you push up the  fix for trove in stable/havana21:31
sdaguewhich also should probably be voting21:31
SlickNiksdague: That _is_ the job they're added to.21:31
* flashgordon drafts an email21:32
*** zehicle_at_dell has joined #openstack-infra21:32
*** openstackgerrit has quit IRC21:32
SlickNikJust need to submit a patch to infra to make it voting.21:32
SlickNikIt's been running stable for a while now.21:32
*** openstackgerrit has joined #openstack-infra21:32
openstackgerritMorgan Fainberg proposed a change to openstack-infra/config: Add the kite (KDS) project and kiteclient project to gerrit/jenkins
fungimorganfainberg: thanks--reviewing21:34
SlickNiksdague: Also working on adding more tests than the initial set that we have in there. So expect more patches to tempest from us. :)21:34
*** zehicle has quit IRC21:34
openstackgerritSalvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a fingeprint for bug 1283522
*** dims has quit IRC21:34
morganfainbergfungi, i think i got those right21:34
sdagueSlickNik: ok, cool :)21:34
morganfainbergfungi, and i am working on registering the project21:34
morganfainbergin pypi21:35
*** lcheng has quit IRC21:35
fungimorganfainberg: great--that was more of a heads up that when it comes time to release, your upload job will fail without it existing on pypi. also, someone might decide to register it who isn't you21:35
sdagueso the reality is we should probably just put the trove tests into the main batch21:35
hasharyou guys dont have enough VM to run the tests during peak hours :-D21:36
fungihashar: are you offering us more vms? thanks@21:36
hashardoes each job consume a vm ?21:36
fungicurrently, yes21:36
*** vkozhukalov_ has joined #openstack-infra21:37
sdaguealso, damn you for having a nick with a different starting character for casual friday21:38
hasharfungi time for some fundraising ? :-]21:39
fungihashar: we need fancy banners like wikipedia uses!21:39
*** ryanpetrello has quit IRC21:39
hasharand Jenkins Job Builder is migrating to pbr!!!!!! :-D (thank you all)21:40
hasharfungi: yeah that is very effective. Comes from a few years of trial and errors.   Wikimedia even leash its fundraising rounds to avoid raising too much money.21:41
hashar(as I understood it)21:41
jeblairwe'll be able to use some more nodes when we move to hpcloud 1.1...21:41
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Migrate to pbr
jeblairand once we do that, we'll be in a better position to ask for more nodes21:42
openstackgerritSean Dague proposed a change to openstack-infra/devstack-gate: trove is an integrated project add it by default
sdagueSlickNik: so that should add it to the main job21:42
sdaguealso, has the advantage of being self testing21:42
*** zhiyan is now known as zhiyan_21:42
morganfainbergfungi, also added openstackci as index maintainer role21:43
*** jeckersb is now known as jeckersb_gone21:43
fungimorganfainberg: checking it out21:44
SlickNiksdague: Thanks!21:44
jeblairfungi: specs repos are on all git servers, so i think things are working on all of them.  just waiting for kite to show up there.21:45
fungimorganfainberg: pypi registration lgtm21:45
fungijeblair: thanks!21:45
*** lcheng has joined #openstack-infra21:46
*** rpodolyaka has joined #openstack-infra21:47
flashgordonsdague mordred fungi: email draft
fungisilly clients, reqs are for servers!21:48
*** jp_at_hp has quit IRC21:48
clarkbfungi ++21:48
sdaguefungi: so I should just be able to git clone git:// , then add a .gitreview, and git review -s ?21:49
sdaguebecause the last part is failing on me21:49
jeblairsdague: not yet21:49
jeblairsdague: hehe, friday isn't over yet! :)21:49
sdagueit is for me :)21:49
sdaguebeer is calling21:49
sdagueok, I'll play tomorrow morning21:49
fungiutc friday has 2.2 hours remaining21:49
flashgordonany comments on th email before I send it out?21:49
jeblairsdague: probably be working in about 30 mins21:49
openstackgerritA change was merged to openstack-infra/storyboard-webclient: Fixed mobile footer padding.
fungiflashgordon: lgtm21:50
sdaguenight all21:50
funginight sdague!21:50
flashgordonhave a good weekend sdague21:50
jeblairsdague: also, there should already be a .gitreview in the repo once everything is done here21:50
jeblairsdague: cheers!21:50
flashgordonfungi: who do I need to poke to get a reviews on
flashgordonsdague's trove patch21:51
fungiflashgordon: i'm fine, but approving depends on whether it ends up needing to get squashed with yours21:52
flashgordonfungi: oh right sigh21:52
fungiflashgordon: also, it's stable, so you need stable branch manglers21:53
jeblairthat also has the side effect of switching off the mirror for trove and ending requirements sync proposals for trove on havana21:53
*** dims has joined #openstack-infra21:53
fungijeblair: i believe SlickNik and hub_cap have both previously acknowledged that the stable/havana branch of trove is unmaintained21:54
jeblairboth of which are probably ok21:54
jeblairfungi: speaking of which, when is grizzly eol?21:54
openstackgerritA change was merged to openstack-dev/pbr: Fix up some docstrings.
openstackgerritA change was merged to openstack-infra/config: Add the kite (KDS) project and kiteclient project to gerrit/jenkins
*** dkliban has quit IRC21:55
*** changbl has quit IRC21:55
*** bada_ has joined #openstack-infra21:55
fungijeblair: last thursday, but we had a bit of a mismatch between when we agreed to eol it at the last summit (i-3 date) and when the grizzly stable point release schedule said the last point release was going to be, si i just need to get apevec/adam_g/ttx to agree before i pull the trigger21:55
*** bada__ has quit IRC21:56
*** changbl has joined #openstack-infra21:56
*** dkliban has joined #openstack-infra21:56
mordredfungi: I'm quite happy about it going away myself21:57
jeblairfungi: i can't wait.  you know how i like deleting branches!21:57
jeblair_so much_ cruft gets cleaned up21:57
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Task list now shows associated project
fungijeblair: mordred: says "2013.1.5  Mar 20 2014" so worst case, a week out21:57
*** medieval1 has quit IRC21:58
adam_gfungi, apevec has frozen the grizzly branches for new merges and sent out an email preparing for a final release on the 20th21:58
fungithough the vmt has basically stopped considering backports needed for vulnerabilities on it because it was supposed to be eol last week21:58
fungiadam_g: great21:58
openstackgerritA change was merged to openstack-infra/nodepool: Roll up node stats
*** medieval1 has joined #openstack-infra21:59
*** rcarrillocruz has joined #openstack-infra22:00
openstackgerritDan Smith proposed a change to openstack-infra/devstack-gate: DO NOT MERGE - Test larger largerops
*** medieval1_ has joined #openstack-infra22:01
*** rcarrillocruz1 has quit IRC22:02
fungiso, in the next 10 minutes the git servers should (in theory) have the kite project repos, then i'll fire up the engine and watch it spin22:02
openstackgerritA change was merged to openstack-dev/pbr: Return the real class in VersionInfo __repr__.
jeblairfungi: can we screen that?22:02
*** adalbas has quit IRC22:04
openstackgerritA change was merged to openstack-infra/zuul: Fix ZUUL_COMMIT in certain conditions
*** medieval1 has quit IRC22:04
fungijeblair: definitely22:04
openstackgerritAntoine Musso proposed a change to openstack-infra/jenkins-job-builder: Support the Post Build Script plugin
fungijeblair: do we want to just run it for the specific projects being added, or do a full (unqualified) run? also, verbose or debug output?22:06
jeblairfungi: specific + debug |tee log ?22:06
fungiactually i think we want to do a full run anyway, so that remote tracking updates will pick up zaro's desired upstreamgerrit commits22:06
jeblairfungi: maybe do full + verbose after the specific?22:06
fungisounds good22:07
jeblairfungi: or do you want to exercise project creation in the midst of a full run?22:07
jeblairfungi: (i'm joined)22:07
fungijeblair: nah, i'm fine doing the full run separate so we can debug more easily if we hit a snag on project creation for now22:08
fungiwe've done it successfully both ways prior so i'm not too concerned about exercising that22:08
*** zehicle has joined #openstack-infra22:10
*** zehicle_at_dell has quit IRC22:10
openstackgerritA change was merged to openstack-infra/storyboard-webclient: Hide "Create Project" for non-superusers
*** esker has quit IRC22:14
zarohashar: will you go thru pending changes and let people know they need to rebase and move their plugin entried to setup.cfg?22:14
hasharzaro: yeah I am looking at them slowly one by one22:14
*** esker has joined #openstack-infra22:15
zarohashar: thx.22:15
dstufftpeople sure do get angry about gerrit requiring you to squash your commits22:15
fungidstufft: who gets angry? what commits?22:17
hasharzaro: though I am afraid folks would miss review comments  when I rebase22:17
jeblairdstufft: what people?22:17
clarkbbut it doesnt22:17
fungidstufft: gerrit requires you to edit and resubmit your commits to refine them before they're approved, but not quite the same as squashing them22:17
clarkbit requires you to have good commits22:17
jeblairkrotscheck: where on the list is 'adding comments' to storyboard?  also, is there a place i can look to answer that myself?22:17
fungidstufft: that's like getting angry because the lkml wouldn't let you just contribute a thread of additions to a patch22:17
krotscheckjeblair: We're logging everything here:!/project/456/stories22:18
dstufftfungi: jeblair I've seen like 4 or 5 people in random different venues complain about Openstack's Gerrit requiring them to either get each individual commit reviewed or squash everything down to one commit, instead of just reviewing the final diff like Github does :)22:18
dstufft(Note, I'm not one of those people)22:18
dstufftI don't actually care either way22:18
fungidstufft: they're angry that gerrit's workflow is not github's workflow. i think they also wouldn't enjoy submitting kernel patches then22:19
jeblairdstufft: yeah, we're fans of bisectability here, and the idea that each commit should be a good one, that is well thought out and accomplishes what it describes.22:19
fungidstufft: personally i find the github fork/pull workflow horrible22:19
fungibut people get stockholm syndrome from such experiences22:20
jeblairdstufft: i find the 'keep plastering over commits until you get something that works' workflow distasteful.22:20
dstufftHilarously both sides of the coin say it's for better bisectablity :D22:20
dstufftI like the github PR thing better, but that's probably because I edit my history to be logical changes and not just whatever I happened to have to save at the time22:21
jeblairdstufft: yeah, so don't squash beyond bisectability...22:21
dstufftAlthough I do think it would be hilarous to use the github style thing to get developers to run malcious code22:21
funginot only hilarious but easy22:22
jeblairdstufft: if you have 10 logical changes to do something, push all 10 commits to gerrit22:22
*** yjiang5 has quit IRC22:22
jeblairdstufft: (i'm using the royal you here, i realize i'm not talking directly to you)22:22
fungiand gerrit will even notice that they are dependent and will build up a tree of reviews from them22:23
jeblairdstufft: gerrit actually deals fairly well with patch series like that22:23
dstufftjeblair: I would do that, except I feel like a jerk for getting people to review each and every change like that :]22:23
openstackgerritAntoine Musso proposed a change to openstack-infra/jenkins-job-builder: Add workspace scm
fungiignore my sudden extra l's. i think my keyboard overfloweth22:23
*** yjiang5 has joined #openstack-infra22:24
*** talluri has joined #openstack-infra22:24
fungidstufft: i would much rather review individual (good) commits rather than one giant code dump22:24
*** mark0n has quit IRC22:24
fungii suspect that the people who mostly prefer iterative commits followed by one monolithic merge are probably people who don't spend 20+ hours a week reviewing code22:24
*** MarkAtwood has joined #openstack-infra22:26
*** rpodolyaka has quit IRC22:26
dstufftI have an irrational hate for the commits which are like...22:26
dstufft"fix typo"22:26
dstufft"some other crap"22:26
marunfungi: +122:26
jeblairfungi: doing what to code?22:26
*** sarob has joined #openstack-infra22:26
fungidstufft: those are easy approvals22:26
dstufftfungi: No I mean in github when you have a string of like 20 commits with those kinds interleaved between them :)22:27
dstufftin a PR22:27
jeblairdstufft: yeah, i think that's the worst (and natural) outcome of that model22:27
fungioh, right. those should not be interspersed in unrelated patch series22:27
*** talluri has quit IRC22:27
openstackgerritAntoine Musso proposed a change to openstack-infra/jenkins-job-builder: Added support for Exclusion plugin
marunis anyone willing to help me debug the neutron functional job?  It needs to run tox in a devstack-configured environment but the braindead way I wrote it has a permission problem22:29
flashgordonfungi: looks like the trove patch needs to be squashed into mine22:29
fungiMar 14 22:00:40 git01 puppet-agent[1635]: (/Stage[main]/Openstack_project::Git_backend/Exec[create_cgitrepos]) Failed to call refresh: create-cgitrepos returned 1 instead of one of [0] at /opt/config/production/modules/openstack_project/manifests/git_backend.pp:9922:29
fungiflashgordon: yeah, i suspected they would have to22:29
hasharzaro: I have rebased all pending JJB changes that had a new entry point :]22:30
jeblairkrotscheck: i don't see adding comments on that list; i think that was something we discussed in bru; was all that copied over?22:30
fungijeblair: trying it manually--i probably approved a typo22:30
*** zehicle_at_dell has joined #openstack-infra22:30
*** zehicle has quit IRC22:30
jeblairmarun: you may need to sudo to the 'stack' user to unstack22:31
marunjeblair: the unstack isn't failing22:31
jeblairmarun: and possibly run the test22:31
marunjeblair: the tox invocation is22:31
krotscheckjeblair: Comments were specifically not part of the MVP, no. While discussed, we did make the choice to drop it.22:31
marunjeblair: should it be necessary to run tox with sudo?22:32
jeblairkrotscheck: when did we make that choice?22:32
krotscheckjeblair: In brussles? On friday.22:32
*** ArxCruz has quit IRC22:32
jeblairmarun: i'm guessing it can't write to the checkout22:32
*** reed has joined #openstack-infra22:32
marunjeblair: that's correct.22:32
jeblairkrotscheck: was i in the restroom at the time?  i can't imagine using it without comments, and even the POC had comments22:33
reedclarkb, fungi, jeblair: can someone push this in please? it's a simple mlist, should not take too long22:33
jeblairkrotscheck: i'm pretty sure i went on at length about the necessity of comments22:33
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Correct syntax error in review.projects.yaml
fungireed: lgtm22:35
fungijeblair: that ^ was the delay22:35
krotscheckjeblair: we can certainly explore the priority on commenting.22:36
marunjeblair: should tox be run with sudo though?22:36
krotscheckjeblair: more features would get done faster if, say, there was more than one person consistently working on it.22:37
jeblairmarun: it probably needs to be run as the stack user since stack owns the checkout22:37
marunjeblair: ah, so 'sudo -u stack'?22:38
*** mgagne has quit IRC22:38
jeblairkrotscheck: i'm not complaining about the speed of implementation; i'm just looking for where in the project roadmap i can find a feature we agreed on22:39
jeblairkrotscheck: (and i'll repeat what i said in brussels so you might recall how important it is: there was a bug in the POC that prevented adding comments from working, and that _alone_ is the reason we were not using it yet)22:40
krotscheckjeblair: Hoenstly, I've been adding things to the task list as I remember them. At the moment, I'm the only one who's dogfooding storyboard, what with most of the other people who were in brussles having gotten sucked into HP stuff.22:40
jeblairi remembered correctly; it's in the etherpad for mvp user stories22:41
*** caleb_ has quit IRC22:41
jeblairkrotscheck: i'll add a story for it22:41
krotscheckjeblair: So if you'd like to add a feature, add it.22:41
*** mrodden has quit IRC22:41
reedwhere is the dogfood?22:42
* reed lazytypist22:42
*** marun has quit IRC22:42
jeblairkrotscheck: i just want to be perfectly clear: are you saying "if i want comments, i shoud add them?"22:42
reedwhere are you dogfooding storyboard?22:42
krotscheckjeblair: No - add a story22:42
jeblairreed: storyboard.openstack.org22:42
devanandafungi: think you'll have time to circle back to the virtual-ironic changes today?22:42
jeblairkrotscheck: ok, will do.22:42
flashgordonso good news bad news about the requirements update22:42
krotscheckjeblair: That's what it's there for.22:42
flashgordongood  news it apperas to work, bad news we have missing deps22:42
*** lcheng has quit IRC22:42
krotscheckjeblair: Having said that, if you _want_ to help out, I would be more than happy to have you.22:43
krotscheckThe few times you helped me debug things were quite helpful.22:43
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added pagination to webclient.
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Task list now shows associated project
*** hashar has quit IRC22:44
devanandaor jeblair ? i've got 2 small patches up so we can start doign experimental checks22:44
jeblairkrotscheck: i'm trying to get to that point.  :)22:44
fungidevananda: yep, just a sec i'll take a peek22:44
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added pagination to webclient.
devanandafungi: awesome, thanks22:44
jeblairkrotscheck: i put (MVP) in the description since we lack priority22:44
krotscheckjeblair: Fair warning - the project dropdown still fails ahrd.22:45
krotscheckjeblair: It's much easier to use the create story button from the actual project.22:45
jeblairkrotscheck: that's what i did, so i didn't notice the problem22:45
jeblairreed:!/project/456/stories are storyboard's own stories22:46
jeblairapparently we're about to break the 250 project mark22:47
jeblairin a few minutes22:47
openstackgerritA change was merged to openstack-infra/config: Added list for Content team
jeblairreed: ^22:48
reedwonderful, thanks22:48
*** zns has quit IRC22:49
*** rpodolyaka has joined #openstack-infra22:49
jeblairfungi: haha -- my dns image testing finally managed to get through the critical section, only to be stymied by trying to clone the projects that aren't in gerrit yet22:51
fungijeblair: yeah, this apparently impacts third-party test rigs using nodepool too22:52
fungiArxCruz noticed last week or the week before22:52
jeblairfungi: well, if they are using our setup scripts22:52
openstackgerritAlex Gaynor proposed a change to openstack-infra/config: Simplify gate jobs for python-openstacksdk and add docs
*** esker has quit IRC22:53
*** Ryan_Lane has quit IRC22:53
openstackgerritA change was merged to openstack-infra/config: Correct syntax error in review.projects.yaml
*** zns has joined #openstack-infra22:55
openstackgerritA change was merged to openstack-infra/config: enable doc generation for pycadf
openstackgerritJoe Gordon proposed a change to openstack/requirements: Add missing requirements
openstackgerritJoe Gordon proposed a change to openstack/requirements: If dependency not in global reqs exit with a 1
flashgordonthis is going be a fun game of detect the missing dependency  ^22:56
*** reed has quit IRC22:56
fungiflashgordon: playing catch-up always is. i suspect the hole in devstack was hiding these from us for a while22:57
damnsmithflashgordon: is there any better way to test that than rechecking a bunch? that ends up running a lot of tests and taking lots of resources for just two actual results that we care about23:00
flashgordondamnsmith: yeah add an silent queue job23:01
jeblairor experimental23:01
damnsmithcan you link me to an example?23:01
flashgordondamnsmith: but we should hold off on this until juno23:01
flashgordonas we have plenty of other things to deal with23:01
*** Ryan_Lane has joined #openstack-infra23:01
*** wenlock has quit IRC23:01
*** mrodden has joined #openstack-infra23:02
damnsmithflashgordon: well, we don't have to merge it any time soon, but I want to gather the data to show that we're better off now23:02
flashgordonshuold point you in the right direction23:02
damnsmithyep, thanks23:02
*** che-arne has quit IRC23:03
flashgordondumb bash foo/devstack question23:03
damnsmithflashgordon: er, does that let me change the d-g code? that's what I need to do...23:04
flashgordonif returns a 1 in devstack today will that cause devstack to stop23:04
fungiflashgordon: it's run in devstack, within the errexit, so i believe so23:05
*** miqui has quit IRC23:05
*** miqui has joined #openstack-infra23:05
fungiflashgordon: you could always submit a wip patch which adds exit 1 at the end and just see what happens23:05
flashgordondamnsmith: yeah you will need to change d-g code as well23:06
flashgordonor something23:06
fungijeblair: okay, all 5 git servers have the kite repos now, so we can procede to run m-p i think23:06
flashgordonfungi: good idea23:06
jeblairfungi: ++23:07
*** mrodden has quit IRC23:07
fungijeblair: nevermind... ha!23:07
fungireview.o.o hasn't pulled that change yet ;)23:07
*** khyati has quit IRC23:07
fungiso that i don't have to wait for it23:08
*** wenlock has joined #openstack-infra23:08
jeblairfungi: you can do that without stopping puppet i believe; it will error out with a helpful msg if its already running23:08
openstackgerritJoe Gordon proposed a change to openstack/requirements: TESTING, DO NOT MERGE
flashgordonfungi: ^ that should confirm23:08
jeblairfungi: 1 sec23:08
flashgordonthanks for the idea23:08
fungijeblair: okay23:09
fungijeblair: yeah, the gerrit module is in need of cleanup there23:09
jeblairfungi: all good23:09
fungiready then?23:09
*** maxbit has joined #openstack-infra23:10
*** mrodden has joined #openstack-infra23:10
krotscheckIncidentally, jeblair, that entire exchange we just had left a nasty taste in my mouth.23:10
markmcclainquestion about strange gerrit issue:
markmcclainwhen I click review I can't score.  Is that as sign there are later drafts?23:11
jeblairkrotscheck: sorry.  perhaps we need to get better at communicating with each other in irc.  i don't think it needed to be that way.23:12
krotscheckjeblair: That's fair, and apologies from my end as well.23:12
krotscheckjeblair: Don't get me wrong - I get the need for comments.23:13
openstackgerritA change was merged to openstack-infra/storyboard-webclient: Task list now shows associated project
fungiadded rustlebee to nova-specs-core23:13
krotscheckjeblair: But blargh getting defensive on a text-based medium.23:13
jeblairkrotscheck: so we're on the same page; i'm not badgering you at all, i have no issues with the speed of implementation or your work or anything.  i think it's excellent all around actually.23:14
fungilooks like we eventually need members added to nova-docker-core, barbican-ptl and barbican-milestone23:15
*** Ryan_Lane has quit IRC23:15
fungirustlebee: morganfainberg: ^ who should be the initial members of those23:15
morganfainbergfungi, mordred ^23:16
mordredmorganfainberg: sup/23:17
jeblairfungi: i see kite repos with content on all 5 git mirrors23:18
morganfainbergfungi, pinged me instead of you a couple lines up23:18
morganfainbergi'm sure23:18
morganfainbergfungi, unless you really meant me?23:18
mordredfungi: oy. how would I know?23:18
* morganfainberg is confused23:18
jeblairmorganfainberg: nope you.  :)  we made your kite project and want to know who should be in the barbican-ptl and barbican-milestone groups23:19
morganfainbergyou mean barbican didn't already have that stuff?23:19
jeblairmorganfainberg: apparently not.  we don't really keep track of that for stackforge projects.23:19
jeblairself-service and all23:19
morganfainbergjeblair, heh23:19
morganfainbergwell barbican-ptl should be the barbican ptl ;)23:20
morganfainberglet me see about what milestone is supposed to be23:20
*** e0ne has joined #openstack-infra23:21
jeblairfungi: yay, that happened in time for the next run of my test to succeed.  i now have 3 centos images in 3 providers23:22
fungimorganfainberg: no worries--since you drafted a change mentioning those groups, i thought you might know who to put in the empty ones. we *can* wait for them to just eventually ask us23:22
fungijeblair: perfect23:22
morganfainbergfungi, i think we should just wait on those.23:22
*** e0ne_ has quit IRC23:22
morganfainbergfungi, originally we were using the keystone structure but i think some stuff doesn't translate 1:1 for barbican23:22
openstackgerritDan Smith proposed a change to openstack-infra/devstack-gate: Allow configurable large-ops tests
fungijeblair: also all the projects seemed to get the right stuff created on the gerrit side, so i'll fire up the full run now (which should hopefully just be updating remote tracking branches)23:23
morganfainbergfungi, since this falls under barbican now, when the review was updated i think that stuff was left in assuming barbican was the same structure wise23:23
fungimorganfainberg: fair enough! it's not urgent23:23
morganfainbergfungi, i'll dig up the infor and ping you guys as we get a bit further with this stuff :)23:23
*** talluri has joined #openstack-infra23:23
flashgordonso trove is still being troublesome
*** che-arne has joined #openstack-infra23:25
*** sweston has quit IRC23:27
*** flaper87 is now known as flaper87|afk23:27
*** gokrokve has quit IRC23:28
openstackgerritVictor Seva proposed a change to openstack-infra/jenkins-job-builder: use {obj:key} as a way to pass an object and not only strings to templates
*** talluri has quit IRC23:28
flashgordonit looks like the job isn't using stable branches23:29
*** e0ne has quit IRC23:30
*** e0ne has joined #openstack-infra23:30
* flashgordon wonders where the bottom of this rabbit hole is23:31
*** zehicle_at_dell has quit IRC23:32
*** rpodolyaka has quit IRC23:32
*** zehicle_at_dell has joined #openstack-infra23:32
*** openstackstatus has quit IRC23:36
*** e0ne has quit IRC23:37
*** jergerber has quit IRC23:38
*** jdurgin has quit IRC23:38
*** rwsu has quit IRC23:38
*** unicell has quit IRC23:38
*** miqui has quit IRC23:38
*** jeckersb_gone has quit IRC23:38
*** dstufft has quit IRC23:38
*** openstackgerrit has quit IRC23:39
*** rossella_s has quit IRC23:39
*** changbl has quit IRC23:42
*** miqui has joined #openstack-infra23:42
*** nati_ueno has quit IRC23:42
*** HenryG has quit IRC23:42
*** thuc has quit IRC23:43
*** sdake has quit IRC23:43
*** mwagner_lap has quit IRC23:45
*** julienvey has quit IRC23:45
*** zehicle_at_dell has quit IRC23:45
*** wenlock has quit IRC23:45
*** sarob has quit IRC23:45
*** yjiang5 has quit IRC23:46
*** rcarrillocruz has quit IRC23:46
*** jepoy has quit IRC23:46
*** yongli has quit IRC23:46
*** SpamapS has quit IRC23:46
*** dkranz has quit IRC23:46
*** bogdando has quit IRC23:46
*** cody-somerville has quit IRC23:46
*** dangers_` has quit IRC23:46
*** milki has quit IRC23:46
*** maurosr has quit IRC23:46
*** katyafervent_ has quit IRC23:46
*** KurtMartin has quit IRC23:46
*** Hefeweizen has quit IRC23:46
*** krotscheck has quit IRC23:46
*** sdague has quit IRC23:46
*** dragondm has quit IRC23:46
*** rainya has quit IRC23:46
*** che-arne has quit IRC23:46
*** zns has quit IRC23:46
*** medieval1_ has quit IRC23:46
*** Alex_Gaynor has quit IRC23:46
*** dolphm has quit IRC23:46
*** dstanek_afk has quit IRC23:46
*** spredzy has quit IRC23:46
*** mkerrin1 has quit IRC23:46
*** tristanC has quit IRC23:46
*** cody-somerville- has quit IRC23:46
*** exz has quit IRC23:46
*** tian has quit IRC23:46
*** weiwei has quit IRC23:46
*** cyeoh has quit IRC23:46
*** JoshNang has quit IRC23:46
*** mtreinish has quit IRC23:46
*** Daviey has quit IRC23:46
*** sc68cal has quit IRC23:46
*** hughsaunders has quit IRC23:46
*** mrodden has quit IRC23:46
*** maxbit has quit IRC23:46
*** bada_ has quit IRC23:46
*** vkozhukalov_ has quit IRC23:46
*** nicedice has quit IRC23:46
*** harlowja has quit IRC23:46
*** bknudson has quit IRC23:46
*** chmouel has quit IRC23:46
*** EntropyWorks has quit IRC23:46
*** damnsmith has quit IRC23:46
*** ianw has quit IRC23:46
*** moted has quit IRC23:46
*** antonym has quit IRC23:46
*** mrda has quit IRC23:46
*** loquacities has quit IRC23:46
*** nibalizer has quit IRC23:46
*** asadoughi has quit IRC23:46
*** shortstop has quit IRC23:46
*** weshay has quit IRC23:46
*** fungi has quit IRC23:46
*** JayF has quit IRC23:46
*** simonmcc has quit IRC23:46
*** EmilienM has quit IRC23:46
*** locke105 has quit IRC23:46
*** AaronGr has quit IRC23:46
*** psedlak has quit IRC23:46
*** Adri2000 has quit IRC23:46
*** pabelanger has quit IRC23:46
*** odyi has quit IRC23:46
*** jd__ has quit IRC23:46
*** vhoward- has quit IRC23:46
*** mancdaz has quit IRC23:46
*** gondoi has quit IRC23:46
*** zhiyan_ has quit IRC23:46
*** salv-orlando has quit IRC23:46
*** dprince has quit IRC23:46
*** jgrimm has quit IRC23:46
*** homeless has quit IRC23:46
*** blamar has quit IRC23:46
*** pcrews_ has quit IRC23:46
*** wchrisj has quit IRC23:46
*** dripton has quit IRC23:46
*** esmute has quit IRC23:46
*** ilyashakhat has quit IRC23:46
*** vipul has quit IRC23:46
*** saschpe has quit IRC23:46
*** sbadia has quit IRC23:46
*** dkehn has quit IRC23:46
*** dhellmann has quit IRC23:46
*** dachary has quit IRC23:46
*** saper has quit IRC23:46
*** morgabra has quit IRC23:46
*** gaelL_ has quit IRC23:46
*** Shrews has quit IRC23:46
*** clarkb has quit IRC23:46
*** kevinbenton has quit IRC23:46
*** dcramer_ has quit IRC23:46
*** amcrn has quit IRC23:46
*** rhsu has quit IRC23:46
*** SumitNaiksatam has quit IRC23:46
*** markwash has quit IRC23:46
*** alexpilotti has quit IRC23:46
*** sahumada has quit IRC23:46
*** sandywalsh_ has quit IRC23:46
*** mikal has quit IRC23:46
*** gmoro has quit IRC23:46
*** juice has quit IRC23:46
*** davidlenwell has quit IRC23:46
*** mattoliverau has quit IRC23:46
*** jaypipes has quit IRC23:46
*** kashyap has quit IRC23:46
*** yamahata_ has quit IRC23:46
*** jamielennox|away has quit IRC23:46
*** flaper87|afk has quit IRC23:46
*** bradm has quit IRC23:46
*** koolhead17 has quit IRC23:46
*** cp16net has quit IRC23:46
*** nijaba has quit IRC23:46
*** hogepodge has quit IRC23:46
*** gyee has quit IRC23:46
*** emagana has quit IRC23:46
*** atiwari has quit IRC23:46
*** gothicmindfood has quit IRC23:46
*** enikanorov_ has quit IRC23:46
*** mugsie has quit IRC23:46
*** ProfFalken has quit IRC23:46
*** Mithrandir has quit IRC23:46
*** wayneeseguin has quit IRC23:46
*** skraynev_afk has quit IRC23:46
*** dkehn_ has quit IRC23:46
*** pfallenop has quit IRC23:46
*** fbo has quit IRC23:46
*** msuriar_ has quit IRC23:46
*** jeblair has quit IRC23:46
*** jpeeler has quit IRC23:46
*** jeremyb has quit IRC23:46
*** soren has quit IRC23:46
*** gema has quit IRC23:46
*** alaski has quit IRC23:46
*** NikitaKonovalov has quit IRC23:46
*** michchap has quit IRC23:46
*** westmaas has quit IRC23:46
*** DuncanT- has quit IRC23:46
*** SergeyLukjanov has quit IRC23:46
*** ruhe has quit IRC23:46
*** DinaBelova has quit IRC23:46
*** ogelbukh has quit IRC23:46
*** notmyname has quit IRC23:46
*** pleia2 has quit IRC23:46
*** bearhands has quit IRC23:46
*** Vivek has quit IRC23:46
*** niska has quit IRC23:46
*** rahmu has quit IRC23:46
*** plomakin_ has quit IRC23:46
*** rdwrer has quit IRC23:46
*** dosaboy has quit IRC23:46
*** vladan has quit IRC23:46
*** spiffxp_ has quit IRC23:46
*** jcooley has quit IRC23:46
*** belliott has quit IRC23:46
*** adarazs_ has quit IRC23:46
*** flashgordon has quit IRC23:46
*** guitarzan has quit IRC23:46
*** mburned has quit IRC23:46
*** aude has quit IRC23:46
*** Steap has quit IRC23:46
*** gilliard has quit IRC23:46
*** lifeless has quit IRC23:46
*** mika has quit IRC23:46
*** jesusaurus has quit IRC23:46
*** primeministerp has quit IRC23:46
*** gpocentek has quit IRC23:46
*** mordred has quit IRC23:46
*** dtroyer has quit IRC23:46
*** sileht has quit IRC23:46
*** zigo has quit IRC23:46
*** tteggel has quit IRC23:46
*** jgriffith has quit IRC23:46
*** StevenK has quit IRC23:46
*** jraim has quit IRC23:46
*** GheRivero has quit IRC23:46
*** vishy has quit IRC23:46
*** isviridov has quit IRC23:46
*** openstack has joined #openstack-infra23:47 [freenode-info] channel trolls and no channel staff around to help? please check with freenode support:
*** openstackstatus has joined #openstack-infra23:47
*** dstufft has quit IRC23:47
*** dstufft has joined #openstack-infra23:47
*** zns has joined #openstack-infra23:49
*** mkerrin has joined #openstack-infra23:51
*** Alex_Gaynor has joined #openstack-infra23:53
*** che-arne has joined #openstack-infra23:53
*** dstufft has quit IRC23:53
*** dstanek_afk has joined #openstack-infra23:54
*** bada_ has joined #openstack-infra23:54
*** cody-somerville- has joined #openstack-infra23:56
*** dstufft has joined #openstack-infra23:57
*** cyeoh has joined #openstack-infra23:58

Generated by 2.14.0 by Marius Gedminas - find it at!