Tuesday, 2016-06-21

*** thorst_ has joined #openstack-infra00:00
fungiright now, with reprepro generating the index _from_ the set of packages on disk, there's no consistency issue at a single point in time, but really the most painful race we had was dealing with mirrors which were actually just plain inconsistent, often for long spans of time. i have to assume they were not using the officially prescribed mechanisms for mirroring00:01
*** aysyd has quit IRC00:01
SpamapSMaybe just mirror from the official sources?00:01
fungii don't see how any magic in apt solved _that_ problem00:01
mordredSpamapS: we do mirror from the official sources to make our mirror00:02
mordredSpamapS: before we did that, we had per-cloud mirrors run by the clouds in question that were out of date and inconsistent00:02
SpamapSI remember those00:02
mordredthose were worse than unhelpful00:02
fungiSpamapS: maybe, though ubuntu's "official" mirrors tended to have that issue from time to time, i think because they were hosted locally in cloud providers and masquerading as normal mirrors or something (or the providers could have been doing some terrible cached proxying at their border or something?)00:02
SpamapSyeah that's not really official00:03
SpamapSI mean it's maintained.. but.. archive.ubuntu.com is the one you want.00:03
*** xyang1 has quit IRC00:03
fungianyway, i know we'd hit it on archive.u.o for spans of time far too long to be a short update race00:03
SpamapSeven tho those local ones are super fast :-P00:03
lifelessfungi: http front-end accelerators can mess up repos even when they are actually consistent on disk00:03
fungier, archive.u.c00:03
mordredyah. the o.o mirrors have been very pleasant since we put them in00:03
SpamapSRight, they're pleasant, but not usable for production.00:04
mordredlifeless:  I mean, I agree with SpamapS00:04
lifelessI mean why aren't they00:04
mordredin their current state, they are useful for the gate, but not for external parties wanting to run production00:04
mordredlifeless: no auth - could be MITM'd00:04
*** fungi_lounge has joined #openstack-infra00:04
lifelessmordred: what about the signing key?00:05
fungigranted, we didn't bill our ci mirrors as "recommended for production cloud deployments" or anything00:05
*** fungi_lounge has quit IRC00:05
mordredlifeless: we don't sign them currently00:05
*** sree has joined #openstack-infra00:05
lifelessoh, because not mirrored. right00:05
mordredlifeless: yah00:05
*** ffrank has quit IRC00:05
*** zhurong has quit IRC00:05
SpamapSA more sinister person might see that as a problem for the gate too.. like.. mitm in a python package which disguises broken things in OpenStack's security code.00:05
mordredso - fungi is working on our signing infrastructure in general, so we could choose to start signing them00:05
mordredSpamapS: that would be an exceptionally hard MITM to pull off00:05
*** ffrank has joined #openstack-infra00:06
mordredgiven that to be successful they'd have to MITM the AFS protocol between rax dfw and all of the other cloud regions we have00:06
SpamapSmordred: all nodes don't mount afs00:07
mordredgranted, not technically unpossible - but still I'd expect the gain to be higher than a library that would fail-open00:07
SpamapSmordred: so, all they have to do is mitm the http between nodes and the mirror.00:07
mordredSpamapS: we have an HTTP host in each cloud region00:07
mordredthat mounts AFS00:07
*** fungi_lounge has joined #openstack-infra00:07
mordredso either way they'd need to do the MITM across all 10 cloud-regions we have00:07
SpamapSnot so crazy thing to do if you can run arbitrary code in the check queue on the same network as those nodes.00:08
*** jamesmcarthur has joined #openstack-infra00:08
SpamapSAnd then make it serve your "yes I do validate SSL certs" version of XYZ python lib, and then introduce patches to subtly break that.......00:08
SpamapSbut the tests pass and verify SSL certs are validating00:08
SpamapSYes I know00:09
*** thorst_ has quit IRC00:09
mordredsure - but you can't request a region - so you'd have to submit code into the code review system that implemented an exploit - and run it/ recheck it enough that it ran in all 10 regions00:09
SpamapSstraw man army ;)00:09
mordredI mean, I hear you00:09
fungiactually, our trusted job workers and afs mirrors are in a separate tenant from the nodes that run arbitrary code00:09
mordredI'm just not going to lose any sleep over that one00:09
*** sree has quit IRC00:09
lifelessmordred: MITM HTTP is not hard00:09
*** sree has joined #openstack-infra00:09
mordredlifeless: didn't say it was00:09
lifelessmordred: just saying, that the AFS step isn't relevant00:09
fungithe nodes that run arbitrary code are sandboxed in their own tenant, but yes the provider in question puts them on a shared provider network so it's all a question of how good their arp spoofing and such is00:10
SpamapSNo not losing sleep. Just pointing out that having them signed isn't _that_ hard, and should be done as a dot on an i, not as the lynchpin of OpenStacks' code integrity. :)00:10
fungier, arp spoofing prevention00:10
mordredI'm saying that MITMing 10 different regions at the same time is a bit steep given the dubious gain you'd get00:10
lifelessmordred: they just have to MITM the node->http->mirror link00:10
mordredlifeless: yes. in 10 places00:10
SpamapSyou don't even need arp00:10
lifelessmordred: sure, but since you'd be basically automating any such attack...00:10
SpamapSbut yeah, the many many regions does help00:10
fungiSpamapS: right, so also ip source address spoofing prevention comes into play00:11
SpamapSprovides more "wtf why is this patch over here having so many rechecks?"00:11
SpamapSthis has been fun00:11
SpamapSI am hoping I can help00:11
SpamapSit's down to 106F outside00:12
SpamapSso I'm going to jump in my 79F pool now00:12
mordredSpamapS: woot!00:12
fungibasically node on the can-run-anything tenant needs to somehow convince the provider to reroute traffic between two instances on the ci control plane tenant through itself00:12
mordredSpamapS: ours is up to 86F now00:12
pleia2SpamapS: ugh, that temperature is not ok, enjoy the pool00:12
SpamapSyeah the water over the baja bench is up to 82F :)00:12
mordredfungi: and needs to convince 6 different providers to do that00:12
jeblairianw: yep00:13
mordredfungi: at least one of whom runs a very different network stack from the others00:13
fungimordred: not if they just want to temporarily backdoor a release tarball00:13
lifelessthere was a great discussion about dns poisoning attacks in ubuntu-devel recently00:13
mordredfungi: I don't think mitm'ing the mirrors has any bearing on backdooring a release tarball00:13
lifelessdns is how I'd do it, no ip spoofing needed.00:13
fungii'm sure it's not impossible (with my tenure on the vmt i've seen plenty of network issues in openstack that would make layer 2 or layer 3 traffic redirection to accomplish a mitm possible00:14
*** sree has quit IRC00:14
mordredI'm not saying this is not possible00:14
mordredI'm saying at best MITM'ing the mirrors might get you a false positive/false negative on some gate tests00:14
openstackgerritIan Wienand proposed openstack-infra/zuul: Make console log output use deliminator  https://review.openstack.org/33189200:14
mordredthat's the best outcome you can hope for as an attacker - it's not really worth the effort00:14
lifelessthe idea would be to land known vulnerable code00:14
ianwjeblair: ^ something like that00:15
lifelesshow you'd keep it from being discovered shortly thereafter I have no idea00:15
fungilifeless: true, dns response flooding the builder in question would be a possibility, and doesn't require access to that network00:15
mordredlifeless: right. but to do such a thing by poisoning a dependency library ... and yes, then keeping it secret00:15
lifelesssince if it needs an active attack to pass tests00:15
*** hosanai has joined #openstack-infra00:15
mordredin this context00:15
mordredis way harder than just socially engineering the bad code in through normal means00:15
mordredlifeless: exactly00:15
fungior yes, dns cache poisoning (google's resolvers since those are the ones we're using)00:16
jeblairianw: oh, er, i don't think i want to lose precision on the timestamp.  i thought only the '|' was at issue.00:16
mordredif we build rpms or debs or go/rust binaries or something, such that an attack could get bad code into a release tarball, that would be a different story I think00:16
lifelesssingle emitter somewhere on the internet with ip source spoofing off, flood of src= packets.00:16
*** notmars-lounge has joined #openstack-infra00:17
ianwjeblair: well, i think most things we have (os-loganalyze & logstash) will handle the extra precision ... but it is a change.  what's more important?00:17
ianwcompatability or microseconds?00:17
*** sree has joined #openstack-infra00:17
*** amotoki has joined #openstack-infra00:17
jeblairianw: microseconds :)00:17
jeblairianw: so both things are at issue?  the ts format and the | ?00:18
ianwjeblair: the thing that causes the weird coloring on,e.g. http://logs.openstack.org/14/331514/1/check/gate-os-loganalyze-python27/3c70514/console.html  is just the missing " | " i *think*00:19
ianwi'll have to check with the higher precision timestamp, but i think osloganalyze has regexes that handle that00:19
mordredI can be down with adding a | to get colors00:19
jeblairianw: okay.  i (perhaps this is unusual) have no strong opinion on the '|' and will happily approve a patch to add it to zuul_runner (but also don't object if we want to leave it as is and change os-loganalyze)00:20
jeblairianw: i'm fond of high-precision timestamps and would like to keep them if possible.00:20
ianwmordred: actually, it's adding a | to get *rid* of colors.  osloganalyze's oslo-highlighting regex is picking up the console logs and trying to color them, badly00:20
mordredianw: I should have said "correct" colors00:21
fungii guess it boils down to whether it's faster to update osla or to roll a zuul patch out to all the launchers and gracefully restart them00:21
ianwjeblair: ok, let me test osloganalyze with higher precision timestamps and if so, i'll update the zuul to put in the " | " and we should be OK00:21
*** sree has quit IRC00:21
ianwlogstash does handle it with it's built-in timestamp regex stuff00:21
jeblairfungi: meh.  this isn't that urgent.  i think we can do the thing we want in this case.  whatever that is.  :)00:21
jeblairianw: sounds like a plan00:21
fungitrue. a few more days of miscolorization (if that's what it takes) isn't the end00:22
jeblairfungi: plus, who lets websites tell their browser what color to use anyway? ;)00:22
fungion the other hand, fixing osla to not expect the | means that teh logs we've already uploaded will get colored correctly, rather than only the logs we upload after the zuul behavior change00:22
*** amotoki has quit IRC00:22
*** sree has joined #openstack-infra00:23
fungijeblair: yep, i have gertty pull job logs up in lynx, so i don't actually see the colorization regardless00:23
*** jamesmcarthur has quit IRC00:23
jamielennoxfungi: when you have a minute do you know why: https://review.openstack.org/#/c/330845/ didn't work?00:24
ianwcolor is a bit meh, but i kind of found this when trying to go through kibana to pull out some logs, where we're not matching too00:24
fungijamielennox: oh, yep!00:25
ianwhttps://review.openstack.org/#/c/331526/ <- actually, i kind of think it makes sense to have the log in one message anyway, but it is unintentional00:25
fungijamielennox: we need to manually switch it back from read-only first00:25
*** salv-orlando has joined #openstack-infra00:25
fungijamielennox: catch-22, if the project is read-only, manage-projects can't update the acl for it00:26
fungii always forget about this00:26
fungidoesn't come up very often00:26
openstackgerritKhai Do proposed openstack-infra/puppet-gerrit: Add configurations for gerrit its plugins  https://review.openstack.org/33092200:26
jamielennoxfungi: excellent - because i was coming up with weird ideas that didn't make sense even to me00:26
fungijamielennox: most of my ideas don't make sense to me, so you'd be in fine company00:26
nibalizerfungi: anteaya this is passing now if you've got a sec to reapply votes00:27
*** pvaneck has quit IRC00:27
fungijamielennox: i've switched it from read-only to active, and will manually run manage-projects for that repo now00:27
*** sree has quit IRC00:28
jamielennoxfungi: thank you00:28
*** burgerk has quit IRC00:28
fungiso that i can make sure it's really and truly fixed00:28
fungijamielennox: lgtm now00:28
*** yamahata has quit IRC00:29
*** ffrank has quit IRC00:29
jamielennoxfungi: yep, lgtm https://review.openstack.org/#/admin/projects/openstack/oslo.context,access00:29
*** jamesmcarthur has joined #openstack-infra00:29
openstackgerritKhai Do proposed openstack-infra/puppet-gerrit: Add configurations for gerrit its plugins  https://review.openstack.org/33092200:30
*** salv-orlando has quit IRC00:32
openstackgerritSpencer Krum proposed openstack-infra/vinz: First (hacky) poke at vinz  https://review.openstack.org/33189400:32
*** sshnaidm has quit IRC00:33
*** sshnaidm has joined #openstack-infra00:34
openstackgerritKhai Do proposed openstack-infra/system-config: Configure its-storyboard plugin  https://review.openstack.org/33092500:34
*** savihou has quit IRC00:34
*** savihou has joined #openstack-infra00:34
*** lvdongbing has joined #openstack-infra00:38
*** hosanai_ has quit IRC00:41
*** masayukig has quit IRC00:41
*** rfolco has quit IRC00:42
*** thorst_ has joined #openstack-infra00:42
*** sree has joined #openstack-infra00:43
*** sdake_ has quit IRC00:44
*** gouthamr_ has quit IRC00:44
*** masayukig has joined #openstack-infra00:45
*** sree_ has joined #openstack-infra00:46
*** markvoelker has quit IRC00:46
*** sree_ is now known as Guest9063100:47
*** amitgandhinz has joined #openstack-infra00:47
*** markvoelker has joined #openstack-infra00:47
*** sree has quit IRC00:47
*** sree has joined #openstack-infra00:47
*** ddieterly has joined #openstack-infra00:50
*** Guest90631 has quit IRC00:50
ianwahh unittest -- 30 levels of backtrace, and not one of them in code i've written00:51
*** sree has quit IRC00:52
*** amitgandhinz has quit IRC00:52
*** yamamot__ has joined #openstack-infra00:53
*** sree has joined #openstack-infra00:54
*** rajinir has quit IRC00:54
funginibalizer: gate-project-config-nodepool for 331780 wanged off a POST_FAILURE and bounced out. i've rechecked it00:56
nibalizerfungi: thanks00:57
*** yamamot__ has quit IRC00:58
*** sree has quit IRC00:58
*** ddieterly is now known as ddieterly[away]00:58
*** sree has joined #openstack-infra01:00
*** zul has joined #openstack-infra01:00
*** asselin has quit IRC01:00
*** tkelsey has joined #openstack-infra01:02
*** sree has quit IRC01:04
*** hosanai_ has joined #openstack-infra01:05
*** jamesmcarthur has quit IRC01:05
*** jamesmcarthur has joined #openstack-infra01:05
*** yamahata has joined #openstack-infra01:06
openstackgerritIan Wienand proposed openstack-infra/os-loganalyze: Test high-precision timestamps in console log  https://review.openstack.org/33189801:06
*** tkelsey has quit IRC01:06
*** penguinolog has quit IRC01:07
*** penguinolog has joined #openstack-infra01:07
*** weshay has quit IRC01:07
*** ddieterly[away] has quit IRC01:08
*** zhurong has joined #openstack-infra01:09
*** jamesmcarthur has quit IRC01:10
*** tphummel has quit IRC01:10
*** sree has joined #openstack-infra01:12
*** ddieterly has joined #openstack-infra01:12
openstackgerritIan Wienand proposed openstack-infra/zuul: Make console log output use deliminator  https://review.openstack.org/33189201:13
ianwjeblair: ^ think that should do it01:13
*** hosanai_ has quit IRC01:15
*** mrunge has quit IRC01:15
*** sree has quit IRC01:16
*** sree has joined #openstack-infra01:16
*** thorst_ has quit IRC01:18
*** thorst_ has joined #openstack-infra01:19
*** amotoki has joined #openstack-infra01:20
*** sree has quit IRC01:20
openstackgerritJeremy Stanley proposed openstack-infra/infra-specs: Priority specs cleanup/update for Newton cycle  https://review.openstack.org/33190301:21
*** mrunge has joined #openstack-infra01:21
openstackgerritMerged openstack-infra/project-config: Add infra puppet gem dependency holder repo  https://review.openstack.org/33178001:22
*** sree has joined #openstack-infra01:23
*** amotoki has quit IRC01:24
*** xenogear has quit IRC01:25
*** thorst_ has quit IRC01:28
*** sree has quit IRC01:28
*** sree has joined #openstack-infra01:29
*** raunak has quit IRC01:29
*** zhurong has quit IRC01:31
*** zhurong has joined #openstack-infra01:32
*** sree has quit IRC01:33
*** Apoorva has quit IRC01:34
*** sree has joined #openstack-infra01:36
*** salv-orlando has joined #openstack-infra01:37
*** kzaitsev_mb has quit IRC01:38
*** zhurong has quit IRC01:38
*** amotoki has joined #openstack-infra01:40
*** sree has quit IRC01:40
*** zhurong has joined #openstack-infra01:42
*** salv-orlando has quit IRC01:44
*** roxanaghe has quit IRC01:45
anteayanibalizer: sorry thunderstorm took me out, coming in off my phone hope my back-up modem works, will test tomorrow01:45
*** xarses has joined #openstack-infra01:46
*** gouthamr_ has joined #openstack-infra01:47
*** amitgandhinz has joined #openstack-infra01:49
*** zz_dimtruck is now known as dimtruck01:53
*** yanyanhu has joined #openstack-infra01:53
*** amitgandhinz has quit IRC01:53
openstackgerritgreghaynes proposed openstack/diskimage-builder: Fix sphinx-build to not depend on diskimage-builder  https://review.openstack.org/33191101:54
*** ddieterly is now known as ddieterly[away]01:54
*** ddieterly[away] is now known as ddieterly01:54
anteayaoff again, hoping the internets heal themselves and let me connect tomorrow01:57
*** jamesmcarthur has joined #openstack-infra01:59
*** yamamoto has joined #openstack-infra02:00
*** savihou has quit IRC02:00
*** amotoki has quit IRC02:00
*** zhurong has quit IRC02:00
*** savihou has joined #openstack-infra02:01
*** claudiub has quit IRC02:02
*** zhurong has joined #openstack-infra02:03
*** jamesmcarthur has quit IRC02:03
*** kzaitsev_mb has joined #openstack-infra02:03
*** yamahata has quit IRC02:06
*** zhurong has quit IRC02:08
*** zhurong has joined #openstack-infra02:09
*** ddieterly has quit IRC02:09
*** pt_15 has joined #openstack-infra02:10
*** zhs has joined #openstack-infra02:10
*** dimtruck is now known as zz_dimtruck02:11
*** hosanai_ has joined #openstack-infra02:15
*** erlon has quit IRC02:16
*** amotoki has joined #openstack-infra02:16
*** kzaitsev_mb has quit IRC02:16
jeblairianw: thanks!02:19
jeblairpabelanger, mordred: all zls are running02:19
*** hongbin has joined #openstack-infra02:19
*** hosanai_ has quit IRC02:20
*** rhallisey has quit IRC02:25
*** mriedem has quit IRC02:26
*** thorst_ has joined #openstack-infra02:26
funginibalizer: openstack-infra/puppet-openstack_infra_spec_helper is live now02:30
*** raunak has joined #openstack-infra02:32
*** thorst_ has quit IRC02:33
*** psilvad has quit IRC02:33
*** zhs_s has joined #openstack-infra02:34
*** PollyZ has joined #openstack-infra02:35
fungiokay, gonna knock off for the night. tomorrow is meeting day!02:36
funginight all02:36
pleia2night fungi02:37
*** zhurong has quit IRC02:37
*** zhs has quit IRC02:37
*** PollyZ has quit IRC02:38
*** PollyZ has joined #openstack-infra02:40
nibalizerthansk fungi02:43
*** PollyZ has quit IRC02:44
*** zhurong has joined #openstack-infra02:48
*** amitgandhinz has joined #openstack-infra02:49
*** csomerville has joined #openstack-infra02:50
*** yuanying has quit IRC02:51
*** zhurong has quit IRC02:52
*** zhurong has joined #openstack-infra02:53
*** zhs_ss has joined #openstack-infra02:53
*** stevemar has quit IRC02:54
*** zz_dimtruck is now known as dimtruck02:54
*** amitgandhinz has quit IRC02:54
*** yuanying has joined #openstack-infra02:54
*** zhs_s has quit IRC02:56
*** zhurong has quit IRC02:58
*** zhurong has joined #openstack-infra02:59
*** PollyZ has joined #openstack-infra03:02
*** yamahata has joined #openstack-infra03:02
*** sree has joined #openstack-infra03:06
*** sree has quit IRC03:06
*** sree has joined #openstack-infra03:06
*** salv-orlando has joined #openstack-infra03:08
*** sree_ has joined #openstack-infra03:09
*** sree_ has quit IRC03:10
*** sputnik13 has joined #openstack-infra03:12
*** zhs_ss has quit IRC03:12
*** gouthamr_ has quit IRC03:12
*** sree has quit IRC03:12
*** sputnik13 has quit IRC03:12
*** sree_ has joined #openstack-infra03:12
*** kzaitsev_mb has joined #openstack-infra03:12
*** sree_ is now known as Guest9798603:13
*** sputnik13 has joined #openstack-infra03:13
*** sputnik13 has quit IRC03:14
*** salv-orlando has quit IRC03:14
*** sputnik13 has joined #openstack-infra03:16
*** raunak has quit IRC03:16
*** hosanai_ has joined #openstack-infra03:16
*** kzaitsev_mb has quit IRC03:17
*** sputnik13 has quit IRC03:18
*** hosanai_ has quit IRC03:21
*** julim has joined #openstack-infra03:23
*** csomerville has quit IRC03:23
nibalizercrinkle: EmilienM https://review.openstack.org/#/c/331923/03:25
*** kushal has joined #openstack-infra03:27
*** zhurong has quit IRC03:29
*** tphummel has joined #openstack-infra03:29
openstackgerritSpencer Krum proposed openstack-infra/project-config: Notify puppet-openstack_infra_spec_helper events in the infra channel  https://review.openstack.org/33192603:29
*** raunak has joined #openstack-infra03:30
*** rbrndt has joined #openstack-infra03:32
*** raunak has quit IRC03:32
*** thorst_ has joined #openstack-infra03:32
*** julim has quit IRC03:33
*** zhurong has joined #openstack-infra03:34
*** Srinu has joined #openstack-infra03:34
*** Nikolay_St has joined #openstack-infra03:35
*** zhurong has quit IRC03:35
SrinuNikolay_St: While running jobs in Openstack CI i am getting this error http://paste.openstack.org/show/520842/03:36
*** thorst_ has quit IRC03:37
*** zhurong has joined #openstack-infra03:39
openstackgerritKhai Do proposed openstack-infra/puppet-gerrit: Add configurations for gerrit its plugins  https://review.openstack.org/33092203:42
openstackgerritFei Long Wang proposed openstack-infra/project-config: Enable install-guide job for zaqar  https://review.openstack.org/33192803:44
*** zhurong has quit IRC03:45
*** raunak has joined #openstack-infra03:47
*** zhurong has joined #openstack-infra03:49
*** amitgandhinz has joined #openstack-infra03:50
*** shahid__ has joined #openstack-infra03:51
*** amitgandhinz has quit IRC03:55
*** hongbin has quit IRC03:59
*** yuanying has quit IRC04:02
*** yuanying has joined #openstack-infra04:03
*** tkelsey has joined #openstack-infra04:04
*** PollyZ has quit IRC04:04
*** zhurong has quit IRC04:05
*** zhurong has joined #openstack-infra04:06
*** tkelsey has quit IRC04:08
*** zhurong has quit IRC04:12
*** kzaitsev_mb has joined #openstack-infra04:13
*** hosanai_ has joined #openstack-infra04:14
openstackgerritHisashi Osanai proposed openstack-infra/project-config: Fix Monasca Analytics python27 job  https://review.openstack.org/33153504:15
*** links has joined #openstack-infra04:17
*** kzaitsev_mb has quit IRC04:18
*** savihou has quit IRC04:21
*** _nadya_ has joined #openstack-infra04:21
*** salv-orlando has joined #openstack-infra04:22
*** hosanai_ has quit IRC04:25
*** zhurong has joined #openstack-infra04:25
zarobeaker-ubuntu-trusty job failing with "basic openstackci installation of packages Package "apache2" should be installed” message.  anybody know what’s up with that?04:26
*** salv-orlando has quit IRC04:29
*** maishsk has joined #openstack-infra04:30
*** ramishra has joined #openstack-infra04:30
openstackgerritMathieu Mitchell proposed openstack-infra/project-config: (WIP) Enable ironic multinode grenade as non-voting  https://review.openstack.org/33193604:30
*** psachin has joined #openstack-infra04:31
nibalizerzaro: yep04:32
*** Nikolay_St has quit IRC04:32
nibalizerbasically all of those jobs will fail until we land 331923 then patch all the modules04:32
*** Jeffrey4l_ has quit IRC04:33
zaronibalizer: gotcha, thanks.04:34
*** rbrndt has quit IRC04:34
*** thorst_ has joined #openstack-infra04:34
*** denisra has quit IRC04:35
*** denisra has joined #openstack-infra04:36
*** maishsk has quit IRC04:41
*** thorst_ has quit IRC04:42
*** zhurong has quit IRC04:45
*** zhurong has joined #openstack-infra04:46
*** dimtruck is now known as zz_dimtruck04:46
*** asselin__ has quit IRC04:46
*** dfflanders has joined #openstack-infra04:47
*** _nadya_ has quit IRC04:48
*** ramishra has quit IRC04:50
*** amitgandhinz has joined #openstack-infra04:51
*** _nadya_ has joined #openstack-infra04:52
*** banix has joined #openstack-infra04:53
*** yamamoto has quit IRC04:53
*** zhurong has quit IRC04:54
*** zz_dimtruck is now known as dimtruck04:54
*** Jeffrey4l_ has joined #openstack-infra04:55
*** amitgandhinz has quit IRC04:55
crinklenibalizer: are you sure you want all of that copied into the infra gem? right now infra only needs the gemspec part04:55
crinkle95% of that is not relevant to infra04:56
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Use noop-jobs template  https://review.openstack.org/33180504:57
*** _nadya_ has quit IRC04:57
nibalizercrinkle: right this is why I wanted reviews04:57
nibalizerit seems there is some testing built in to this thing that it would be nice to take advantge of04:58
nibalizerit looks like the 'unit test' is basically install the gems and make sure that works, which seems pretty sane and useful04:58
nibalizerI don't think we need the release notes stuff04:58
*** abregman has joined #openstack-infra05:00
*** zhurong has joined #openstack-infra05:04
*** ramishra has joined #openstack-infra05:07
*** slaweq has quit IRC05:11
*** amotoki has quit IRC05:11
*** javeriak has joined #openstack-infra05:13
*** ramishra has quit IRC05:14
*** amotoki has joined #openstack-infra05:14
*** ramishra has joined #openstack-infra05:15
*** salv-orlando has joined #openstack-infra05:15
*** tphummel has quit IRC05:16
*** roxanaghe has joined #openstack-infra05:23
*** hosanai_ has joined #openstack-infra05:26
*** savihou has joined #openstack-infra05:27
*** roxanaghe has quit IRC05:27
*** jaosorior has joined #openstack-infra05:28
*** rbrndt has joined #openstack-infra05:29
*** claudiub has joined #openstack-infra05:30
*** hosanai_ has quit IRC05:30
*** javeriak has quit IRC05:31
*** aeng has quit IRC05:32
*** banix has quit IRC05:33
*** zhurong has quit IRC05:34
*** xarses_ has joined #openstack-infra05:35
*** isaacb has joined #openstack-infra05:35
*** zhurong has joined #openstack-infra05:35
*** rbrndt has quit IRC05:35
*** xarses_ has quit IRC05:35
*** xarses_ has joined #openstack-infra05:36
*** ramishra has quit IRC05:37
*** rbrndt has joined #openstack-infra05:37
*** ramishra has joined #openstack-infra05:37
*** andreww has joined #openstack-infra05:38
*** zhurong has quit IRC05:38
*** xarses has quit IRC05:39
*** salv-orl_ has joined #openstack-infra05:39
*** zhurong has joined #openstack-infra05:39
*** thorst_ has joined #openstack-infra05:40
AJaegerjhesketh: https://review.openstack.org/331563 is wrong, fungi commented here in IRC and I added that as comment to the review. Please abandon.05:40
*** xarses_ has quit IRC05:41
*** armax has joined #openstack-infra05:41
*** andreww is now known as xarses05:41
jheskethAJaeger: thanks05:41
*** salv-orlando has quit IRC05:41
*** yamamot__ has joined #openstack-infra05:41
AJaegerjhesketh, ianw: If either of has time for a review, https://review.openstack.org/331928 would be appreciated - adding Install Guide for Zaqar.05:42
jheskethAJaeger: lgtm05:43
AJaegerthanks, jhesketh !05:43
*** rlandy has quit IRC05:43
*** rbrndt has quit IRC05:43
*** javeriak has joined #openstack-infra05:44
*** armax has quit IRC05:46
*** thorst_ has quit IRC05:47
*** aeng has joined #openstack-infra05:49
*** rossella_s has quit IRC05:50
*** amitgandhinz has joined #openstack-infra05:52
*** Julien-zte has joined #openstack-infra05:52
*** _nadya_ has joined #openstack-infra05:53
*** yolanda has joined #openstack-infra05:55
openstackgerritMerged openstack-infra/project-config: Enable install-guide job for zaqar  https://review.openstack.org/33192805:55
*** rcernin has joined #openstack-infra05:56
*** amitgandhinz has quit IRC05:56
*** _nadya_ has quit IRC05:58
*** mohankumar has joined #openstack-infra06:00
*** jtomasek has joined #openstack-infra06:02
*** _nadya_ has joined #openstack-infra06:03
*** mohankumar has quit IRC06:05
*** zhurong has quit IRC06:08
*** raunak has quit IRC06:09
*** pgadiya has joined #openstack-infra06:11
*** zhurong has joined #openstack-infra06:11
*** Nikolay_St has joined #openstack-infra06:12
*** armax has joined #openstack-infra06:12
*** isaacb has quit IRC06:14
*** javeriak has quit IRC06:14
*** maishsk has joined #openstack-infra06:14
*** kzaitsev_mb has joined #openstack-infra06:15
*** zhurong has quit IRC06:16
*** linuxmaniac_ has joined #openstack-infra06:16
*** linuxmaniac has quit IRC06:16
*** sarob has joined #openstack-infra06:17
*** armax has quit IRC06:17
*** mohankumar has joined #openstack-infra06:17
*** sputnik13 has joined #openstack-infra06:19
*** sputnik13 has quit IRC06:19
*** kzaitsev_mb has quit IRC06:20
*** armax has joined #openstack-infra06:21
*** slaweq has joined #openstack-infra06:22
*** garyk1 has joined #openstack-infra06:23
*** bhavik has quit IRC06:23
*** roxanaghe has joined #openstack-infra06:24
*** wanghua has quit IRC06:25
*** hosanai_ has joined #openstack-infra06:26
*** dfflanders has quit IRC06:26
openstackgerritHisashi Osanai proposed openstack-infra/project-config: Fix Monasca Analytics python27 job  https://review.openstack.org/33153506:27
*** zhurong has joined #openstack-infra06:28
*** roxanaghe has quit IRC06:29
*** sarob has quit IRC06:29
*** sarob has joined #openstack-infra06:29
*** prabu has joined #openstack-infra06:31
*** rwsu_ is now known as rwsu06:32
prabucould someone help me on this error  ERROR gerrit.GerritWatcher: Exception on ssh event stream:06:32
*** yolanda has quit IRC06:32
*** aeng has quit IRC06:34
*** markvoelker has quit IRC06:35
*** andreas_s has joined #openstack-infra06:36
*** hosanai_ has quit IRC06:37
*** dimtruck is now known as zz_dimtruck06:37
*** rwsu has quit IRC06:40
*** zhurong has quit IRC06:40
*** zhurong has joined #openstack-infra06:41
*** hosanai_ has joined #openstack-infra06:43
*** lucasagomes has quit IRC06:44
*** thorst_ has joined #openstack-infra06:45
*** javeriak has joined #openstack-infra06:47
prabucould someone help me on this error  ERROR gerrit.GerritWatcher: Exception on ssh event stream:06:47
*** isaacb has joined #openstack-infra06:48
*** slaweq has quit IRC06:51
*** lucasagomes has joined #openstack-infra06:51
*** rwsu has joined #openstack-infra06:51
AJaegerprabu: please provide some context on where you hit this error06:51
*** thorst_ has quit IRC06:52
*** pcaruana has joined #openstack-infra06:52
*** amitgandhinz has joined #openstack-infra06:52
*** _nadya_ has quit IRC06:54
*** amitgandhinz has quit IRC06:57
SrinuJenkins jobs which are in queue are not waiting until slave node is online and running in the same old nodepool vm and while the job in progress it is deleting the old nodepool vm and results failure of the job06:57
*** jpich has joined #openstack-infra07:00
*** aeng has joined #openstack-infra07:03
*** sarob has quit IRC07:08
*** matrohon has joined #openstack-infra07:09
*** zhurong has quit IRC07:11
*** pt_15 has quit IRC07:13
*** larainema has quit IRC07:13
prabuAJaeger: zuul service is running in the system, now i wanted to read gerrit event stream and trigger a jenkins job as configured in layout.yaml file07:13
*** zhurong has joined #openstack-infra07:13
prabui am verifying the connection by 2 methods07:13
*** jargonmonk has joined #openstack-infra07:13
prabu1. ssh -p 29418 user@review.openstack.org07:14
prabuits perfectly working fine and i am able to read the gerrit streams07:14
prabussh -p 29418 user@review.openstack.org gerrit stream-events07:14
AJaegerprabu: so, this is a third party system? There's a separate channel for that, please use that one. Btw. right now nobody from the OpenStack infra team is around to help either.07:14
*** ociuhandu has joined #openstack-infra07:15
AJaegerprabu: channel is openstack-third-party-ci07:15
prabuyes it's a third party system07:15
prabuthank you so much Ajaeger07:15
SrinuAJaeger: Hi. jenkins jobs which are in queue are not waiting until slave node is online and running in the same old nodepool vm and while the job in progress it is deleting the old nodepool vm and results failure of the job07:16
AJaegerSrinu: Do you speak about zuul-launcher? jeblair, pabelanger, and others are asleep now. If you have some more information - including log files, please leave them here, so that they can followup once they are awake (US time).07:16
AJaegerSrinu: no need to repeat yourself07:16
SrinuAJaeger: Which logs i have to sencd07:17
AJaegerSrinu: Those that show the behaviour. This is OpenStack CI - or a third-pary system?07:18
*** zhurong has quit IRC07:18
AJaegerFor third-party system, see my answer to prabu.07:18
AJaegerIf it's OpenStack CI, use paste.openstack.org or a link to log files.07:18
*** zhurong has joined #openstack-infra07:18
*** woodster_ has quit IRC07:18
*** amoralej|off is now known as amoralej07:19
SrinuAJaeger: i am using third party openstack ci07:20
AJaegerSrinu: with zuul-launcher or jenkins?07:20
SrinuAJaeger: Jenkins07:22
*** netsin has quit IRC07:22
AJaegerSrinu: ok. Read my answer to prabu, please and discuss with other third party operators...07:23
* AJaeger cannot help more in this area than giving these pointers07:24
*** roxanaghe has joined #openstack-infra07:25
*** ihrachys has joined #openstack-infra07:25
*** Na3iL has joined #openstack-infra07:26
*** chlong|rhce_trng has quit IRC07:27
*** roxanaghe has quit IRC07:30
*** kaisers has joined #openstack-infra07:32
*** yaume has joined #openstack-infra07:35
*** markvoelker has joined #openstack-infra07:36
*** jed56 has joined #openstack-infra07:37
*** kushal has quit IRC07:37
*** infra-red has joined #openstack-infra07:38
*** infra-red has quit IRC07:38
*** mohankumar has quit IRC07:38
*** vincentll has joined #openstack-infra07:40
*** hashar has joined #openstack-infra07:40
*** nmagnezi has joined #openstack-infra07:40
*** kaisers has quit IRC07:40
*** markvoelker has quit IRC07:41
*** kaisers has joined #openstack-infra07:41
*** chem|off has quit IRC07:41
*** zhurong has quit IRC07:42
*** chem has joined #openstack-infra07:44
*** lucasagomes has quit IRC07:46
*** lucasagomes has joined #openstack-infra07:47
*** mikelk has joined #openstack-infra07:48
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Fix tempest configuration  https://review.openstack.org/33199707:50
*** thorst_ has joined #openstack-infra07:50
*** mohankumar has joined #openstack-infra07:51
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE TESTING  https://review.openstack.org/31643607:51
*** amitgandhinz has joined #openstack-infra07:53
*** salv-orl_ has quit IRC07:53
*** esikachev has joined #openstack-infra07:55
*** zhurong has joined #openstack-infra07:56
*** chem has quit IRC07:56
*** _nadya_ has joined #openstack-infra07:56
*** thorst_ has quit IRC07:57
*** chem has joined #openstack-infra07:57
*** binbincong has joined #openstack-infra07:58
*** amitgandhinz has quit IRC07:58
*** eezhova has joined #openstack-infra07:59
*** zzzeek has quit IRC08:00
*** _nadya__ has joined #openstack-infra08:00
*** mhickey has joined #openstack-infra08:00
*** _nadya_ has quit IRC08:01
esikachevAJaeger: hi!08:02
*** zzzeek has joined #openstack-infra08:03
*** dizquierdo has joined #openstack-infra08:04
*** shardy has joined #openstack-infra08:05
AJaegersometimes, esikachev ;)08:05
esikachevAJaeger: about https://review.openstack.org/#/c/329497/9/zuul/layout.yaml, how specify job in 2 projects without dublicates?08:07
AJaegerwe have many jobs like these, e.g. gate-swift-dsvm-functional-nv08:08
* esikachev looking...08:08
AJaegeresikachev: or gate-grenade-dsvm-neutron-multinode08:08
esikachevok, thanks!08:09
AJaegeresikachev: since your two jobs are just duplicates - there's not any difference between them AFAIU - let's not have twins but a single job that is triggered from two places.08:09
AJaegeresikachev: JJB gives you a library of jobs, zuul tells you when to schedule them...08:10
AJaegerhope that helps, I'll be back later..08:10
*** e0ne has joined #openstack-infra08:11
*** matrohon has quit IRC08:12
*** dfflanders has joined #openstack-infra08:12
*** zhurong has quit IRC08:13
*** Julien-zte has quit IRC08:15
*** kzaitsev_mb has joined #openstack-infra08:16
*** hosanai_ has quit IRC08:17
*** zhurong has joined #openstack-infra08:17
*** dmk0202 has joined #openstack-infra08:18
*** Julien-zte has joined #openstack-infra08:19
*** salv-orlando has joined #openstack-infra08:19
*** zhurong has quit IRC08:19
*** kzaitsev_mb has quit IRC08:21
*** zhurong has joined #openstack-infra08:23
*** jlanoux has joined #openstack-infra08:23
openstackgerritEvgeny Sikachev proposed openstack-infra/project-config: Migrated sahara-client-functional-job  https://review.openstack.org/32949708:23
*** eezhova has quit IRC08:24
*** hosanai_ has joined #openstack-infra08:25
*** roxanaghe has joined #openstack-infra08:26
*** hosanai_ has quit IRC08:26
*** larainema has joined #openstack-infra08:29
*** roxanaghe has quit IRC08:30
*** shardy has quit IRC08:32
*** mohankumar has quit IRC08:32
*** ociuhandu has quit IRC08:34
*** dfflanders has quit IRC08:35
*** yolanda has joined #openstack-infra08:39
*** dmk0202 has quit IRC08:42
*** zeih has joined #openstack-infra08:43
*** armax has quit IRC08:44
electrofelixzaro: don't know what RTD refers to?08:44
*** derekh has joined #openstack-infra08:44
*** _degorenko|afk is now known as degorenko08:45
*** HeOS has quit IRC08:45
*** acoles_ is now known as acoles08:45
*** mohankumar has joined #openstack-infra08:46
*** pt_15 has joined #openstack-infra08:46
*** zhurong has quit IRC08:46
*** permalac__ has quit IRC08:47
openstackgerritMerged openstack-infra/project-config: Notify puppet-openstack_infra_spec_helper events in the infra channel  https://review.openstack.org/33192608:48
*** jargonmonk has quit IRC08:49
*** maishsk has quit IRC08:49
openstackgerritMerged openstack-infra/project-config: Run Swift functional tests in Identity v3-only  https://review.openstack.org/31365908:49
*** permalac has joined #openstack-infra08:50
*** BobBall_AWOL is now known as BobBall08:52
*** markusry has joined #openstack-infra08:53
*** vincentll has quit IRC08:53
*** vincentll has joined #openstack-infra08:53
openstackgerritMerged openstack-infra/project-config: Add stable/mitaka branch for #openstack-ansible  https://review.openstack.org/33177508:53
openstackgerritMerged openstack-infra/project-config: Run Magnum functional tests in Identity v3-only  https://review.openstack.org/31511808:54
*** amitgandhinz has joined #openstack-infra08:54
*** zhurong has joined #openstack-infra08:57
*** amitgandhinz has quit IRC08:58
*** roxanaghe has joined #openstack-infra09:01
*** maishsk has joined #openstack-infra09:02
*** ociuhandu has joined #openstack-infra09:02
*** javeriak has quit IRC09:04
openstackgerritMerged openstack-infra/project-config: Make ironic grenade voting  https://review.openstack.org/33142209:04
*** dfflanders has joined #openstack-infra09:04
*** roxanaghe has quit IRC09:05
openstackgerritMerged openstack-infra/project-config: Run cinderclient functional tests in Identity v3-only  https://review.openstack.org/31356209:06
openstackgerritMerged openstack-infra/project-config: Use ironic-core for ironic-python-agent ACLs  https://review.openstack.org/33142509:07
*** jaosorior has quit IRC09:09
*** jaosorior has joined #openstack-infra09:10
*** javeriak has joined #openstack-infra09:13
*** ramishra has quit IRC09:13
*** ramishra has joined #openstack-infra09:13
*** matrohon has joined #openstack-infra09:15
*** eezhova has joined #openstack-infra09:17
rdx_mattI've mistakenly named my branch bb/<blueprint_name> instead of bp/<blueprint_name>. Should submit a new review? If so, how do I delete the old review?09:17
*** openstackgerrit has quit IRC09:18
*** openstackgerrit has joined #openstack-infra09:18
*** rdxMatt has joined #openstack-infra09:18
rdxMattI've mistakenly named my branch bb/<blueprint_name> instead of bp/<blueprint_name>. Should submit a new review? If so, how do I delete the old review?09:19
*** zhurong has quit IRC09:19
*** eezhova has quit IRC09:22
*** zhurong has joined #openstack-infra09:23
*** ramishra has quit IRC09:23
*** ociuhandu has quit IRC09:23
*** ociuhandu_ has joined #openstack-infra09:23
*** ociuhandu_ is now known as ociuhandu09:24
*** amotoki has quit IRC09:24
*** mohankumar has quit IRC09:26
AJaegerrdxMatt: just go to review.openstack.org/YOUR_CHANGE and adjust in the UI09:27
AJaegerChange the TOPIC in the ui and all is fine...09:28
*** ifarkas has joined #openstack-infra09:28
*** ramishra has joined #openstack-infra09:29
*** HeOS has joined #openstack-infra09:32
*** ianychoi has joined #openstack-infra09:33
*** rdxMatt has quit IRC09:34
openstackgerritMerged openstack-infra/project-config: Add networking-extreme project to neutron  https://review.openstack.org/32280309:34
*** armax has joined #openstack-infra09:34
*** eezhova has joined #openstack-infra09:35
*** esikachev has quit IRC09:37
*** locust has joined #openstack-infra09:37
*** esikachev has joined #openstack-infra09:37
*** markvoelker has joined #openstack-infra09:37
locusthello, I am setting up CI and I am loking for documentation09:37
locustI know about http://docs.openstack.org/infra/openstackci/third_party_ci.html#configure-masterless-puppet09:38
openstackgerritMerged openstack-infra/project-config: add networking-sfc gate for functional tests and tempest tests  https://review.openstack.org/32413509:38
*** eezhova has quit IRC09:38
locustbut it looks that is not enough for me to understand what is going on09:38
openstackgerritMerged openstack-infra/project-config: Add Rally job against deployment with existing users  https://review.openstack.org/32824909:39
locustmay be there is some pre definde configs or tutorial of step by step with configs and description?09:39
openstackgerritMerged openstack-infra/project-config: Add devstack-plugin-mariadb as new project  https://review.openstack.org/32098309:41
rcarrillocruzlocust: check puppet-openstackci project09:42
rcarrillocruzit's a project specifically for deploying 3rd party CI09:42
*** markvoelker has quit IRC09:43
openstackgerritMerged openstack-infra/project-config: Enable Tempest test job also for sahara-tests  https://review.openstack.org/32959009:43
*** ociuhandu_ has joined #openstack-infra09:47
*** ociuhandu has quit IRC09:47
*** ociuhandu_ is now known as ociuhandu09:47
*** zhurong has quit IRC09:47
*** zhurong has joined #openstack-infra09:48
*** winggundamth has quit IRC09:49
*** zhurong has quit IRC09:50
*** Julien-zte has quit IRC09:50
*** kzaitsev_mb has joined #openstack-infra09:51
*** Guest97986 has quit IRC09:52
*** sree has joined #openstack-infra09:52
*** eezhova has joined #openstack-infra09:54
*** ociuhandu has quit IRC09:54
*** amitgandhinz has joined #openstack-infra09:55
*** maishsk has quit IRC09:56
*** genunix has joined #openstack-infra09:56
*** kzaitsev_mb has quit IRC09:56
openstackgerritHisashi Osanai proposed openstack-infra/project-config: Fix Monasca Analytics python27 job  https://review.openstack.org/33153509:56
*** sree has quit IRC09:56
*** armax has quit IRC09:58
*** pilgrimstack has joined #openstack-infra09:58
*** amitgandhinz has quit IRC09:59
*** ramishra has quit IRC09:59
locustrcarrillocruz: thank you10:00
locustmay be you know any shared CI git repo for single node10:00
locustthat can be used as example?10:01
*** roxanaghe has joined #openstack-infra10:02
rcarrillocruznot really, because the 3rd party cis conf are really private10:03
rcarrillocruzcontain keys etc10:03
rcarrillocruzcheck the contrib folder10:03
openstackgerritDmitry Ilyin proposed openstack-infra/project-config: Add puppet-4.5 unit test to the fuel-library  https://review.openstack.org/33204910:03
rcarrillocruzit contains a hiera.yml10:03
rcarrillocruzwith dummy values10:03
rcarrillocruzjust put yours10:03
rcarrillocruzyou should be good10:03
*** eantyshev has joined #openstack-infra10:03
eantyshevjhesketh, jeblair: Hello! Our CI suffers from occasional hangups, which are dealt with here: https://review.openstack.org/#/c/23898810:04
*** ramishra has joined #openstack-infra10:04
*** tqtran has quit IRC10:05
locustrcarrillocruz: can I setup log server on the same machine with Puppet?10:06
rcarrillocruzi guess so10:06
*** roxanaghe has quit IRC10:06
rcarrillocruzjust put the relevant class on the same node you put the other stuff like nodepool, zuul etc10:06
locustand it is not the part of puppet10:06
*** mohankumar has joined #openstack-infra10:07
*** yolanda has quit IRC10:08
*** sshnaidm is now known as sshnaidm|afk10:08
*** lvdongbing has quit IRC10:09
*** eezhova has quit IRC10:11
*** hosanai has quit IRC10:12
openstackgerritDmitry Ilyin proposed openstack-infra/project-config: Change puppet-4.0 unit test to puppet-4.5  https://review.openstack.org/33204910:13
openstackgerritDmitry Ilyin proposed openstack-infra/project-config: Change puppet-4.0 unit test to puppet-4.5  https://review.openstack.org/33204910:14
*** amotoki has joined #openstack-infra10:17
openstackgerritEvgeny Sikachev proposed openstack-infra/project-config: Migrated sahara-client-functional-job  https://review.openstack.org/32949710:18
andreafhere: a project-config change for review please https://review.openstack.org/#/c/329723/10:22
*** ihrachys has quit IRC10:22
*** winggundamth has joined #openstack-infra10:23
milanHey Infra, I'm running grenade for Ironic inspector and grenade/inc/upgrade:save_mysql_dbs fails for me because of wrong DB name (ironic-inspector --- the project name). Correct db name is ironic_inspector. I'm wondering where is the best spot to override the database name. Thanks a lot!10:23
AJaegerandreaf: since it runs against tempest, please get tempest folks to review it - ask openstack QA team to review and +1.10:23
AJaegermilan: grenade is done by QA team - I suggest to ask on #openstack-qa channel10:24
milanAJaeger, thx, will do10:24
andreafAJaeger: ok, sure - I discussed this in http://lists.openstack.org/pipermail/openstack-dev/2016-June/097383.html and with few folks already but I'll ask them to +110:25
*** sambetts|afk is now known as sambetts10:25
openstackgerritEvgeny Sikachev proposed openstack-infra/project-config: Migrated sahara-client-functional-job  https://review.openstack.org/32949710:26
AJaegerandreaf: add that link to your commit message - or at least as comment. That will help in evaluating the change...10:26
*** pilgrimstack has quit IRC10:27
*** tosky has joined #openstack-infra10:27
*** maishsk has joined #openstack-infra10:27
*** winggundamth has quit IRC10:28
*** eezhova has joined #openstack-infra10:28
*** amrith has quit IRC10:28
andreafheh, I should have done that already10:29
*** amrith has joined #openstack-infra10:29
openstackgerritKota Tsuyuzaki proposed openstack-infra/project-config: WIP: Enable storlets to show test_result  https://review.openstack.org/33206510:30
*** yanyanhu has quit IRC10:30
openstackgerritZara proposed openstack-infra/python-storyboardclient: Add boards  https://review.openstack.org/29944510:32
*** eezhova has quit IRC10:32
*** amrith has quit IRC10:32
*** eezhova has joined #openstack-infra10:33
*** amrith has joined #openstack-infra10:33
*** rfolco has joined #openstack-infra10:35
*** rfolco has quit IRC10:35
*** pilgrimstack has joined #openstack-infra10:38
*** ramishra has quit IRC10:38
*** ramishra has joined #openstack-infra10:38
*** markvoelker has joined #openstack-infra10:38
*** winggundamth has joined #openstack-infra10:40
*** zeih has quit IRC10:41
*** markvoelker has quit IRC10:43
*** sdague has joined #openstack-infra10:43
*** eezhova has quit IRC10:49
*** chlong|rhce_trng has joined #openstack-infra10:50
openstackgerritAdam Coldrick proposed openstack-infra/system-config: Add Zara to StoryBoard Admin  https://review.openstack.org/32909210:50
openstackgerritAdam Coldrick proposed openstack-infra/system-config: Add Adam Coldrick to StoryBoard Admin  https://review.openstack.org/33207210:50
*** hdaniel has joined #openstack-infra10:51
*** eezhova has joined #openstack-infra10:52
*** amitgandhinz has joined #openstack-infra10:55
*** ArxCruz has joined #openstack-infra10:56
*** eezhova has quit IRC10:56
*** eezhova has joined #openstack-infra10:59
*** rodrigods has quit IRC11:00
*** rodrigods has joined #openstack-infra11:00
*** amitgandhinz has quit IRC11:00
*** roxanaghe has joined #openstack-infra11:02
*** Julien-zte has joined #openstack-infra11:03
*** kzaitsev_mb has joined #openstack-infra11:03
*** ikhudoshyn has quit IRC11:05
*** armax has joined #openstack-infra11:06
*** makowals has joined #openstack-infra11:06
*** lucasagomes is now known as lucas-afk11:06
*** yaume has quit IRC11:07
*** roxanaghe has quit IRC11:07
*** sree__ has joined #openstack-infra11:07
*** eezhova has quit IRC11:09
*** abregman is now known as abregman|afk11:10
openstackgerritKota Tsuyuzaki proposed openstack-infra/project-config: Enable storlets to show test_result  https://review.openstack.org/33206511:10
*** eezhova has joined #openstack-infra11:11
*** sree__ has quit IRC11:11
*** sorantis has joined #openstack-infra11:11
*** kushal has joined #openstack-infra11:12
*** pilgrimstack has quit IRC11:13
*** abregman|afk is now known as abregman|pto11:13
*** javeriak has quit IRC11:14
*** sorantis has quit IRC11:14
openstackgerritMário Santos proposed openstack-infra/shade: Add support for changing metadata of compute instances  https://review.openstack.org/31939511:17
*** javeriak has joined #openstack-infra11:18
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Add quick non-permanent filter buttons for timeline events  https://review.openstack.org/33174811:19
*** amotoki has quit IRC11:22
*** dfflanders has quit IRC11:23
*** sree has joined #openstack-infra11:23
*** pilgrimstack has joined #openstack-infra11:24
*** ihrachys has joined #openstack-infra11:24
*** ihrachys has quit IRC11:26
*** sree has quit IRC11:27
*** prabu has quit IRC11:29
*** zeih has joined #openstack-infra11:31
*** sree has joined #openstack-infra11:32
*** ArxCruz has quit IRC11:32
*** sorantis has joined #openstack-infra11:32
*** armax has quit IRC11:33
*** zhurong has joined #openstack-infra11:33
*** ihrachys has joined #openstack-infra11:34
*** links has quit IRC11:35
*** ldnunes has joined #openstack-infra11:35
*** sree has quit IRC11:36
openstackgerritEvgeny Sikachev proposed openstack-infra/project-config: Migrated sahara-client-functional-job  https://review.openstack.org/32949711:36
*** sree has joined #openstack-infra11:36
*** sree___ has joined #openstack-infra11:37
*** makowals has quit IRC11:38
*** salv-orl_ has joined #openstack-infra11:39
openstackgerritDmitry Ilyin proposed openstack-infra/project-config: Add puppet-4.5 unit tests to the fuel-library  https://review.openstack.org/33204911:39
*** makowals has joined #openstack-infra11:39
openstackgerritOfer Ben-Yacov proposed openstack-infra/project-config: Add new project: networking-icc  https://review.openstack.org/33209111:39
*** markvoelker has joined #openstack-infra11:39
*** makowals has quit IRC11:39
*** makowals has joined #openstack-infra11:40
*** sree has quit IRC11:41
*** salv-orlando has quit IRC11:41
*** pilgrimstack has quit IRC11:41
*** ociuhandu has joined #openstack-infra11:42
*** ccamacho is now known as ccamacho|lunch11:42
*** sree___ has quit IRC11:42
*** weshay has joined #openstack-infra11:43
*** armax has joined #openstack-infra11:43
openstackgerritKirill Zaitsev proposed openstack-infra/project-config: Make murano-agent py34 job voting  https://review.openstack.org/33209311:44
*** rfolco has joined #openstack-infra11:45
*** pilgrimstack has joined #openstack-infra11:45
*** markvoelker has quit IRC11:46
*** links has joined #openstack-infra11:47
*** dizquierdo has quit IRC11:48
*** ldnunes_ has joined #openstack-infra11:49
*** javeriak has quit IRC11:49
openstackgerritDmitry Ilyin proposed openstack-infra/project-config: Change puppet-4.0 unit test to puppet-4.5  https://review.openstack.org/33204911:50
*** ociuhandu has quit IRC11:51
*** ociuhandu_ has joined #openstack-infra11:51
*** ociuhandu_ is now known as ociuhandu11:51
openstackgerritKirill Zaitsev proposed openstack-infra/project-config: Make murano-agent py34 job voting  https://review.openstack.org/33209311:53
AJaegermordred: What kind of code is this today?11:54
*** salv-orl_ has quit IRC11:54
*** ldnunes_ has quit IRC11:54
*** sree has joined #openstack-infra11:55
AJaegerhttp://www.fileformat.info/info/unicode/char/4f645/index.htm - 325189 is not valid unicode ;)11:55
*** amitgandhinz has joined #openstack-infra11:56
*** aysyd has joined #openstack-infra11:56
*** hdaniel has quit IRC11:57
*** Julien-zte has quit IRC11:59
*** sree has quit IRC11:59
*** pilgrimstack has quit IRC11:59
*** sshnaidm|afk is now known as sshnaidm12:00
*** sree has joined #openstack-infra12:00
*** amitgandhinz has quit IRC12:00
*** dtantsur|afk is now known as dtantsur12:01
*** haaj has joined #openstack-infra12:02
*** roxanaghe has joined #openstack-infra12:03
*** haaj has quit IRC12:03
*** haaj has joined #openstack-infra12:03
*** esikachev has quit IRC12:04
*** shardy has joined #openstack-infra12:04
*** shardy has quit IRC12:04
*** psilvad has joined #openstack-infra12:04
*** rossella_s has joined #openstack-infra12:05
*** raildo-afk is now known as raildo12:07
openstackgerritMerged openstack-infra/zuul: Add support for bindep  https://review.openstack.org/32551912:07
*** dprince has joined #openstack-infra12:07
*** roxanaghe has quit IRC12:08
*** yolanda has joined #openstack-infra12:08
*** ramishra has quit IRC12:08
*** pilgrimstack has joined #openstack-infra12:09
*** psilvad has quit IRC12:10
*** hdaniel has joined #openstack-infra12:10
*** krtaylor has quit IRC12:11
*** tpsilva has joined #openstack-infra12:13
*** ramishra has joined #openstack-infra12:13
openstackgerritMerged openstack-infra/zuul: Make console log output use deliminator  https://review.openstack.org/33189212:13
*** pilgrimstack has quit IRC12:13
amrithtimeout errors on launchpad, doesn't bode well for the rest of the day ...12:14
*** krtaylor has joined #openstack-infra12:15
*** markvoelker has joined #openstack-infra12:16
*** pilgrimstack has joined #openstack-infra12:16
*** pradk has joined #openstack-infra12:18
*** ifarkas has quit IRC12:18
*** ifarkas has joined #openstack-infra12:19
*** esikachev has joined #openstack-infra12:19
*** kgiusti has joined #openstack-infra12:19
*** salv-orlando has joined #openstack-infra12:20
*** psilvad has joined #openstack-infra12:20
*** armax has quit IRC12:20
*** gordc has joined #openstack-infra12:22
*** ddieterly has joined #openstack-infra12:22
*** garyk1 has quit IRC12:23
*** yamamot__ has quit IRC12:23
*** rossella_s has quit IRC12:24
*** ramishra has quit IRC12:24
*** kushal has quit IRC12:24
*** ccamacho|lunch is now known as ccamacho12:25
*** amoralej is now known as amoralej|lunch12:25
*** armax has joined #openstack-infra12:27
*** eezhova has quit IRC12:28
*** pilgrimstack has quit IRC12:29
*** rossella_s has joined #openstack-infra12:29
*** yaume has joined #openstack-infra12:32
*** links has quit IRC12:33
*** zhurong has quit IRC12:35
*** armax has quit IRC12:35
*** zul has quit IRC12:35
*** armax has joined #openstack-infra12:35
*** ddieterly has quit IRC12:35
*** zul has joined #openstack-infra12:35
*** tkelsey has joined #openstack-infra12:37
*** esikachev has quit IRC12:38
openstackgerritDmitry Ilyin proposed openstack-infra/project-config: Change puppet-4.0 unit test to puppet-4.5  https://review.openstack.org/33204912:39
mordredAJaeger: I'm bad at not hitting a button my laptop :)12:40
*** daemontool has joined #openstack-infra12:40
*** ArxCruz has joined #openstack-infra12:41
*** matt-borland has joined #openstack-infra12:41
*** ramishra has joined #openstack-infra12:42
*** rlandy has joined #openstack-infra12:42
*** baoli has joined #openstack-infra12:43
*** esikachev has joined #openstack-infra12:46
*** links has joined #openstack-infra12:46
pabelangermordred: I added a comment to 331426 about extra_vars, mind checking when you had bandwidth?12:46
*** trown|outtypewww is now known as trown12:47
*** cbader has quit IRC12:49
mordredpabelanger: OH! my bad - in the new world where we don't actually read the timeout in the timeout plugin but instead just register elapsed time, this totally works now12:49
*** ociuhandu_ has joined #openstack-infra12:49
*** ociuhandu has quit IRC12:49
*** ociuhandu_ is now known as ociuhandu12:49
*** baoli_ has joined #openstack-infra12:49
*** ArxCruz has quit IRC12:50
derekhIs there anybody around to take a look at adding a new cloud rh2 for tripleo ci https://review.openstack.org/#/c/330514/312:51
derekhand a experimental job to test it out https://review.openstack.org/#/c/330515/312:51
*** baoli has quit IRC12:52
mordredderekh: did you see pabelanger's comment?12:52
*** rhallisey has joined #openstack-infra12:52
mordredderekh: on 330514 - re: AFS mirror12:53
pabelangermordred: Oh, right. Now I see what you were saying about extra_vars.12:53
openstackgerritMerged openstack-infra/storyboard: Allow tags to be filtered by name  https://review.openstack.org/33102712:53
pabelangermordred: I should see if that could be updated to use the combined vars.  But, not the case any more12:53
*** ArxCruz has joined #openstack-infra12:54
*** ArxCruz has quit IRC12:55
*** erlon has joined #openstack-infra12:55
mordredpabelanger: yup. yay!12:55
derekhmordred: i hadn't, pabelanger I'll create a flavor with 100G in a few minutes12:55
*** ddieterly has joined #openstack-infra12:56
*** jcoufal has joined #openstack-infra12:56
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Use gear module from pip  https://review.openstack.org/33212312:57
*** kushal has joined #openstack-infra12:57
*** grassass has quit IRC13:00
*** ramishra has quit IRC13:00
*** cbader has joined #openstack-infra13:01
*** eezhova has joined #openstack-infra13:01
andreafAJaeger: got a couple of +1 from QA on https://review.openstack.org/#/c/329723/13:03
*** roxanaghe has joined #openstack-infra13:04
*** Julien-zte has joined #openstack-infra13:05
*** ramishra has joined #openstack-infra13:05
AJaegerandreaf: I'd like to have one from mtreinish or oomichi as well, especially after reading oomichi's mail in your thread.13:06
*** ddieterly has quit IRC13:06
derekhpabelanger:  flavor created on rh2 "d1.medium", can't we approve rh2 anyways and the experimental job so that other tests can happen in parallel?13:07
*** salv-orlando has quit IRC13:07
*** tlian has joined #openstack-infra13:07
derekhpabelanger: gotta run, back later tonight13:08
andreafAJaeger: sure13:08
*** roxanaghe has quit IRC13:08
*** amitgandhinz has joined #openstack-infra13:09
pabelangerderekh: you can remove the configure_mirror.sh script, that'll get the nodes online13:09
*** edmondsw has joined #openstack-infra13:10
*** afred312 has quit IRC13:11
*** grassass has joined #openstack-infra13:12
*** locust has quit IRC13:12
*** zz_dimtruck is now known as dimtruck13:13
*** esikachev has quit IRC13:13
*** Srinu has quit IRC13:13
*** esikachev has joined #openstack-infra13:14
*** sigmavirus24_awa is now known as sigmavirus2413:14
*** amitgandhinz has quit IRC13:14
*** Srinu has joined #openstack-infra13:15
*** haaj has quit IRC13:15
*** eezhova has quit IRC13:15
*** gouthamr has joined #openstack-infra13:16
mtreinishAJaeger: which patch?13:16
Srinu gouthamr: Hi13:16
*** mdrabe has joined #openstack-infra13:16
*** psilvad has quit IRC13:16
gouthamrSrinu: hi!13:16
AJaegermtreinish: https://review.openstack.org/#/c/329723/13:16
Srinu gouthamr: i am using Openstack CI in that i am running jenkins job13:17
Srinu gouthamr:jenkins jobs which are in queue are not waiting until slave node is online and running in the same old nodepool vm and while the job in progress it is deleting the old nodepool vm and running another job that is in queue.13:17
*** xyang1 has joined #openstack-infra13:17
mtreinishAJaeger: oh that one, yeah +1. I was the one who asked andreaf to start with the layer4 job so we didn't effect anyone elses gate as we ramped that up13:18
Srinugouthamr: this is the nodepool log http://paste.openstack.org/show/520897/13:18
*** Julien-zte has quit IRC13:18
AJaegermtreinish: thanks, mtreinish. andreaf, will +2 now...13:18
*** sree_ has joined #openstack-infra13:18
*** esikachev has quit IRC13:19
*** sree has quit IRC13:19
*** sree_ is now known as Guest5548913:19
gouthamrSrinu: your question's in the right channel, but i'm not too familiar with nodepool... probably fungi, AJaeger can help?13:19
*** amoralej|lunch is now known as amoralej13:20
*** fifieldt has joined #openstack-infra13:20
*** bhavik has joined #openstack-infra13:20
* AJaeger cannot13:20
*** esikachev has joined #openstack-infra13:21
SrinuAJaeger: Hi.jenkins jobs which are in queue are not waiting until slave node is online and running in the same old nodepool vm and while the job in progress it is deleting the old nodepool vm and running another job that is in queue. this is the nodepool logs  http://paste.openstack.org/show/520897/13:21
*** ramishra has quit IRC13:22
*** esberglu has joined #openstack-infra13:22
*** links has quit IRC13:23
pabelangerSrinu: Have you set OFFLINE_NODE_WHEN_COMPLETE? http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/openstack_functions.py#n4713:24
*** inc0 has joined #openstack-infra13:24
openstackgerritMerged openstack-infra/project-config: Zun: rename higgins to zun (1)  https://review.openstack.org/32630613:26
*** e0ne has quit IRC13:27
*** _ari_ has joined #openstack-infra13:27
Srinupabelanger: in openstack_functions.py file i used  single_use_node function in that i used params['OFFLINE_NODE_WHEN_COMPLETE'] = '1'13:27
*** mriedem has joined #openstack-infra13:30
Srinupabelanger: can you check this nodepool log http://paste.openstack.org/show/520897/13:31
*** e0ne has joined #openstack-infra13:31
*** akshai has joined #openstack-infra13:32
*** burgerk has joined #openstack-infra13:32
*** pgadiya has quit IRC13:33
Srinupabelanger : i am using this code in openstack_function.py can you check this and tell me anything to change http://paste.openstack.org/show/520939/13:33
*** burgerk_ has joined #openstack-infra13:33
*** dizquierdo has joined #openstack-infra13:34
*** BigWillie has joined #openstack-infra13:34
*** ddieterly has joined #openstack-infra13:34
*** hichihara has joined #openstack-infra13:34
*** afred312 has joined #openstack-infra13:35
*** ajmiller has quit IRC13:35
*** rossella_s has quit IRC13:35
*** ajmiller has joined #openstack-infra13:35
*** sdake has joined #openstack-infra13:36
*** yamamoto has joined #openstack-infra13:36
*** derekh has quit IRC13:36
*** burgerk has quit IRC13:37
*** tkelsey has quit IRC13:38
*** eharney has joined #openstack-infra13:39
Srinufungi: Hi13:39
*** nwkarsten has joined #openstack-infra13:41
*** binbincong has quit IRC13:41
*** e0ne has quit IRC13:41
*** binbincong has joined #openstack-infra13:41
Srinunwkarsten: hi13:41
pabelangerSrinu: your nodepool.log file is raising some exceptions, so nodepool is likely not working correctly. If you look at jenkins logs you'll also be able to see why the jobs are running back to back13:43
*** yolanda has quit IRC13:43
*** shahid__ has quit IRC13:45
*** sshnaidm_ has joined #openstack-infra13:46
*** PollyZ has joined #openstack-infra13:46
*** afred312 has quit IRC13:46
*** M-docaedo_vector has quit IRC13:47
*** afred312 has joined #openstack-infra13:47
Srinupabelanger: this is the jenkins log http://paste.openstack.org/show/520941/13:47
*** openstackgerrit has quit IRC13:48
*** openstackgerrit has joined #openstack-infra13:48
*** sshnaidm has quit IRC13:48
*** haaj has joined #openstack-infra13:48
pabelangerif I had to guess, jenkins is not putting the node into offline mode, next job runs then nodepool eventually deletes it mid run.  For some reason OFFLINE_NODE_WHEN_COMPLETE is not setting up properly.13:49
*** ajmiller has quit IRC13:50
pabelangerwhat version of jenkins are you using? Wonder if this is related to CSRF Protection13:50
*** abregman|pto has quit IRC13:51
*** e0ne has joined #openstack-infra13:51
*** sorantis has quit IRC13:51
*** jkraj has joined #openstack-infra13:51
openstackgerritVictor Ryzhenkin proposed openstack-infra/project-config: Add openstack/fuel-plugin-murano-tests project  https://review.openstack.org/33215113:51
Srinupabelanger: i am using Jenkins ver. 1.651.213:51
*** afred312 has quit IRC13:51
*** kencjohnston has joined #openstack-infra13:51
*** kencjohnston has quit IRC13:51
zaroelectrofelix: re RTD, wanted to add you as maintainer of https://readthedocs.org/projects/jenkins-job-builder/13:52
*** ajmiller has joined #openstack-infra13:52
zigoWhat's wrong with my patch?13:52
zigoI'm stairing at the console logs, but can't find what's wrong.13:52
*** rbrndt has joined #openstack-infra13:52
*** ajmiller has quit IRC13:52
zigofungi: ^13:52
pabelangerSrinu: I recommend reading up on CSRF Protection, there was a recent thread on our openstack-infra ML. It might be possible you having problems with that13:53
*** ddieterly is now known as ddieterly[away]13:53
Srinupabelanger: If the Jobs are not in queue it is deleting the node if the job is success or failure13:54
*** bhavik has quit IRC13:55
*** amitgandhinz has joined #openstack-infra13:55
*** eezhova has joined #openstack-infra13:56
pabelangerSrinu: I am not sure I understand. Jenkins slaves will be deleted regardless of success or failure, nodepool will ensure a fresh slave is online for the next job13:56
Srinupabelanger: but in my case it is taking another job in queue13:56
*** kencjohnston_ is now known as kencjohnston13:57
Srinupabelanger: and deleting the slave when it is progress13:57
Srinupabelanger: it is making offline and creating another13:57
*** kencjohnston has quit IRC13:57
pabelangerSrinu: there is some confusion, what do you expect to happen?13:58
*** kencjohnston has joined #openstack-infra13:58
*** afred312 has joined #openstack-infra14:00
Srinupabelanger: the first job in queue is passed all the test cases and success, the job in queue is taking same slave and running the job and nexts it is deleting that slave and the test is failed14:01
*** fifieldt has quit IRC14:01
*** jaosorior has quit IRC14:01
*** isaacb has quit IRC14:01
*** M-docaedo_vector has joined #openstack-infra14:02
*** ArxCruz has joined #openstack-infra14:02
*** ddieterly[away] is now known as ddieterly14:02
*** PollyZ_ has joined #openstack-infra14:02
pabelangerSrinu: right, that is how I understand your problem.  Which, I have listed a potential fix.  See my comments about CSRF Protection14:02
*** eezhova has quit IRC14:03
*** kencjohnston has quit IRC14:03
*** kencjohnston has joined #openstack-infra14:04
*** M-docaedo_vector has quit IRC14:04
*** roxanaghe has joined #openstack-infra14:05
*** zeih has quit IRC14:05
*** PollyZ_ has quit IRC14:05
Srinupabelanger: Can you suggest me a link for CSRF Protection14:05
*** PollyZ has quit IRC14:05
*** PollyZ has joined #openstack-infra14:07
*** kencjohnston has quit IRC14:07
*** salv-orlando has joined #openstack-infra14:08
*** afred312 has quit IRC14:08
*** haaj has quit IRC14:08
*** afred312 has joined #openstack-infra14:08
Srinupabelanger: in my Jenkins CSRF Protection is not enable14:08
*** kencjohnston has joined #openstack-infra14:08
*** yamamoto has quit IRC14:08
*** yamamoto has joined #openstack-infra14:09
*** eezhova has joined #openstack-infra14:09
*** roxanaghe has quit IRC14:09
*** bhavik has joined #openstack-infra14:10
*** zeih has joined #openstack-infra14:11
*** kencjohnston has quit IRC14:11
*** stevemar has joined #openstack-infra14:12
*** kencjohnston has joined #openstack-infra14:12
*** M-docaedo_vector has joined #openstack-infra14:13
*** kencjohnston has quit IRC14:13
*** kencjohnston has joined #openstack-infra14:14
*** salv-orlando has quit IRC14:14
*** amitgandhinz has quit IRC14:14
*** amitgandhinz has joined #openstack-infra14:15
*** kencjohnston has quit IRC14:15
*** wfoster has quit IRC14:16
*** dimtruck is now known as zz_dimtruck14:16
*** kencjohnston has joined #openstack-infra14:16
fungizigo: in the gate14:17
zigofungi: What do you mean? No pb with my patch, that's just an issue with the gate?14:17
fungi-project-config-jenkins-project log it has "yaml.scanner.ScannerError: mapping values are not allowed here" so my guess is some syntax issue. looking at the patch now14:17
fungizigo: stray newline in my message14:17
fungisorry for the confusion14:17
*** wfoster has joined #openstack-infra14:18
fungii was cutting and pasting from lynx and it had wrapped the job name in the log so i didn't notice i was copying a newline14:18
fungizigo: oh, when adding parameters, we need a : after the publisher name. left a comment inline14:20
fungisorry for missing that earlier14:20
zigoOh, I see...14:20
zigoI'm still not completely used to yaml, thanks for your patience.14:21
openstackgerritThomas Goirand proposed openstack-infra/project-config: Fix upload path to tarballs.o.o for packaging  https://review.openstack.org/33100814:22
*** armax has quit IRC14:23
*** snarwade has quit IRC14:23
*** snarwade has joined #openstack-infra14:23
pabelangerSrinu: http://lists.openstack.org/pipermail/openstack-infra/2016-May/004284.html14:23
*** jkraj has quit IRC14:24
*** yujunz has joined #openstack-infra14:25
*** rajinir has joined #openstack-infra14:25
Srinupabelanger: How can i check is Single Use Slave Plugin enable14:26
pabelangerSrinu: you'll need to see if https://git.openstack.org/cgit/openstack-infra/zmq-event-publisher is enabled14:27
*** tonytan4ever has joined #openstack-infra14:28
*** amrith has quit IRC14:28
pabelangerwhile not a single use plugin, it is the interface used between nodepool and jenkins14:28
*** yujunz has quit IRC14:28
*** amrith has joined #openstack-infra14:28
*** isaacb has joined #openstack-infra14:29
*** dizquierdo has quit IRC14:29
Srinupabelanger: it is in enable ZMQ Event Publisher14:30
Srinu pabelanger:  can you check this link https://wiki.jenkins-ci.org/display/JENKINS/Single+Use+Slave+Plugin14:30
fungiSrinu: pabelanger: if the question is how to find out whether you're set for single-use slaves, it's the jenkins-gearman plugin which adds that feature, and you need to pass a parameter OFFLINE_NODE_WHEN_COMPLETE=114:30
*** nmagnezi has quit IRC14:31
fungiSrinu: oh, i was unaware of the Single+Use+Slave+Plugin14:31
Srinufungi: i added the parameter OFFLINE_NODE_WHEN_COMPLETE=114:31
fungiwe don't use that at all, so couldn't tell you much about it14:31
pabelangerYa, never seen that before14:31
Srinufungi: how can i check OFFLINE_NODE_WHEN_COMPLETE=1 is exported correctly14:31
*** eezhova has quit IRC14:32
pabelangerlook at the job parameters in jenkins14:32
pabelangeryou should see it14:32
fungiSrinu: i think it's probably a very stripped-down fork of the gearman plugin for people who aren't actually using zuul/gearman14:32
*** hongbin has joined #openstack-infra14:32
Srinupabelanger: i have to ZMQ Events inZMQ Event Publisher14:34
*** xarses has quit IRC14:34
Srinupabelanger: i have to enable  ZMQ Events inZMQ Event Publisher14:34
fungiwow, looking at the singleuseslave-plugin source code, there are no licenses and no copyright statements...14:35
rdx_mattDoes someone know how to integrate Hacking+Flacke8 into PyCharm?14:35
pabelangerSrinu: yes: http://docs.openstack.org/infra/nodepool/installation.html#jenkins14:35
fungirdx_matt: no clue. most of us use free software. not a lot of pycharm users around here14:36
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Support colorized output  https://review.openstack.org/33217314:36
AJaegeranteaya: what do you think of the above ^14:36
*** Guest55489 has quit IRC14:37
rdx_mattfungi: Thanks. Which software do you use?14:37
fungirdx_matt: i use vim. some use emacs14:37
Srinupabelanger: i enabled that ZMQ publisher but in job whether i have to enable the ZMQ events are not14:37
mordredrdx_matt: I also use vim. although sometimes I use emacs too14:37
fungirdx_matt: i think in general not a lot of people use an ide for python since its interpreter is already quite suitable for interactive code validation and it's easy to invoke a debugger for it directly as well14:39
pabelangerSrinu: check if we should publish events for all jobs should be enabled14:39
fungirdx_matt: so it's not hard to get by with a bog standard text editor plus syntax highlighting and maybe some minimal plugins14:40
openstackgerritTimofey Durakov proposed openstack-infra/project-config: Moving live-migration job to check queue for nova  https://review.openstack.org/33217514:40
rdx_mattfungi: Well, It's a matter of personal taste. As a Java developer, I feel much more in control of my code with PyCharm. I will try to make it work. Thanks again guys14:41
fungirdx_matt: for example, there's at least one vim plugin which can run pep8/flake8 on the fly in the background and highlight failing lines for you as you type14:41
*** Nikolay_St has quit IRC14:41
fungithough i find that sort of thing extremely annoying so i don't use it14:41
AJaegersdague: could you review https://review.openstack.org/332174 , please? It's needed for os-api-ref.14:41
*** jamesmcarthur has joined #openstack-infra14:42
fungirdx_matt: yeah, i can see how coming from java you might get accustomed to needing a much more robust ide14:42
rdx_mattfungi: Yeah, all of those plugins are just a pain. I'd rather have syntax highlighting and some auto formatting.14:42
*** sorantis has joined #openstack-infra14:43
fungibut if all you want is syntax highlighting, autoindenting and the ability to trigger custom commands in the background on demand, just about any popular text editor can do that14:43
rdx_mattfungi: vim is way too much hardcore for me :) I will try sublime though14:44
*** zeih has quit IRC14:45
*** afred312 has quit IRC14:46
*** nmagnezi has joined #openstack-infra14:46
*** sree has joined #openstack-infra14:46
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Support colorized output for our tests  https://review.openstack.org/33217314:47
*** zz_dimtruck is now known as dimtruck14:47
*** yolanda has joined #openstack-infra14:47
*** lucas-afk is now known as lucasagomes14:48
fungirdx_matt: yep, vi-derived text editors are mainly popular with people who started out as unix sysadmins on platforms where there was frequently no other editor available (except maybe ed)14:48
fungiold habits die hard14:48
*** Volundr has quit IRC14:49
AJaegerfungi, pabelanger, mordred : I fear with merging https://review.openstack.org/331892 , we lost the coloring of console output that we had for a few days.14:49
sdagueAJaeger: +214:49
AJaegerhave a look at http://logs.openstack.org/91/332091/1/check/gate-project-config-projects-yaml/4f0d803/console.html#_2016-06-21_11_43_53_541588 and compare with http://logs.openstack.org/51/332151/1/check/gate-project-config-gerrit/e451e7a/console.html#_2016-06-21_14_35_00_80221614:49
AJaegerthanks, sdague14:49
AJaegerfungi, pabelanger, mordred, so what's correct? And is my change 332173 needed/correct in this case?14:50
*** sree_ has joined #openstack-infra14:50
*** xarses has joined #openstack-infra14:51
*** sree has quit IRC14:51
*** sree_ is now known as Guest4399014:51
pabelangerAJaeger: I actually like the lack of color :)  So, not sure which way the the correct way14:51
mordredI think before we had a problem where things were getting colored incorrectly14:51
mordredand now we may have a problem where things are not getting colored incorrectly14:52
fungipabelanger: well, what's SUPPOSED to happen is that the lines which start with WARNING are supposed to be highlighted in yellow, and those starting with ERROR in red, but not all the lines which come after them14:52
*** rhochmuth has joined #openstack-infra14:52
*** dtardivel has joined #openstack-infra14:52
fungiAJaeger: 332173 is probably an improvement, but isn't going to bring back the intended colorization until we fix whatever other matching issue we have14:53
fungiianw: had speculated that osla may also be thrown off by now having microsecond precision in the timestamps14:53
*** alfredodeza has joined #openstack-infra14:53
pabelangerfungi: Good to know14:54
mordredyah. I bet it's the microsecond14:54
*** jordanP has joined #openstack-infra14:54
fungiDATEFMT = '\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}((\.|\,)\d{3,6})?'14:55
*** Guest43990 has quit IRC14:55
pabelangernice find14:55
mordredfungi: it doesn't seem to have support for the | in the string14:55
fungilooks like that _should_ handle microseconds14:55
*** zeih has joined #openstack-infra14:56
*** ArxCruz has quit IRC14:56
openstackgerritTimofey Durakov proposed openstack-infra/project-config: Moving live-migration job to check queue for nova  https://review.openstack.org/33217514:57
fungiand yeah, i'm not finding where | is expected14:57
alfredodezaA while ago I was doing a bit of research trying to see if zuul had integration with Github, and fungi mentioned that a rest API in zuul would be the first step for such a thing14:57
*** abregman has joined #openstack-infra14:57
alfredodezaI wanted to ping again and see if there was anything there today for github integration :)14:57
mordredalfredodeza: the code has been written but merging it is on the backburner pending other things14:57
alfredodezamordred: that sounds similar to what I recall from about a year ago14:58
alfredodezaI guess it hasn't moved too much (understandably so)14:58
alfredodezamordred: do you have the link to the review handy?14:58
mordredalfredodeza: it's on the list for sure, there are legit needs we have for it - we just haven't gotten to it yet :)14:59
*** ramishra has joined #openstack-infra14:59
*** amitgandhinz has quit IRC14:59
*** Srinu_ has joined #openstack-infra14:59
mordredalfredodeza: https://review.openstack.org/#/c/325300/2 and the 19 changes before it14:59
alfredodezamordred: thank you15:00
*** julim has joined #openstack-infra15:00
*** Jeffrey4l_ has quit IRC15:00
openstackgerritMonty Taylor proposed openstack-infra/os-loganalyze: Add support for optional vertical bar  https://review.openstack.org/33218515:00
mordredfungi, pabelanger: ^^ ?15:00
*** amitgandhinz has joined #openstack-infra15:00
*** makowals has quit IRC15:00
*** zeih has quit IRC15:01
fungimordred: could that be taking things the wrong direction? seems like a reaction to https://review.openstack.org/33189215:01
openstackgerritDan Prince proposed openstack-infra/tripleo-ci: Website: Switch to planet-venus code  https://review.openstack.org/33218615:01
openstackgerritDan Prince proposed openstack-infra/tripleo-ci: Add dprince to planet  https://review.openstack.org/33218715:01
*** daemontool has quit IRC15:02
pabelangerfungi: mordred: there is also some line where are still missing |.  http://logs.openstack.org/51/332151/1/check/gate-project-config-gerrit/e451e7a/console.html#_2016-06-21_14_34_41_65461815:03
pabelangerzuul_log was likely not updated too15:03
*** abregman is now known as abregman|afk15:03
*** amitgandhinz has quit IRC15:04
*** ociuhandu has quit IRC15:04
*** amitgandhinz has joined #openstack-infra15:04
electrofelixzaro: ah, I'll sign up15:05
*** dimtruck is now known as zz_dimtruck15:05
*** timothyb89 has quit IRC15:05
*** swat30 has quit IRC15:06
*** timothyb89 has joined #openstack-infra15:06
*** ajmiller has joined #openstack-infra15:06
*** swat30 has joined #openstack-infra15:06
*** piet_ has joined #openstack-infra15:06
*** sorantis has quit IRC15:07
*** isaacb has quit IRC15:07
*** Srinu_ has quit IRC15:08
fungilooking at the page source for a current log, timestamp parsing looks correct, just the ERROR/WARNING matches aren't happening (all lines are span class='NONE ...'15:08
*** Srinu_ has joined #openstack-infra15:08
fungilooks like that's supposed to be matched by the STATUSFMT subexpression, which is only used by OSLO_LOGMATCH15:10
fungiso i'm going to guess that before zuul-launcher we had no colorization on console logs15:10
fungiwhen zuul-launcher came along and didn't use | we had some lines starting to sort of match the OSLO_LOGMATCH pattern15:11
AJaegerfungi, I only saw it recently, so zuul-launcher might have given it to us.15:11
*** hichihara has quit IRC15:11
*** PollyZ has quit IRC15:11
openstackgerritJames E. Blair proposed openstack-infra/zuul: Ansible launcher: copy lib/plugins at startup  https://review.openstack.org/33219815:11
*** PollyZ has joined #openstack-infra15:11
*** PollyZ has quit IRC15:11
fungimy vote would be to get zuul-launcher logs looking more like oslo log format rather than emulating jenkins console logs as closely15:11
AJaegerand then ianw proposed https://review.openstack.org/#/c/331526/15:11
AJaegerwhere I suggested to make them look like jenkins console log ;/15:12
jeblairmost console lines don't have severity15:12
jeblairwhat's the severity of a bash command?15:12
fungithat's a good point. it looks like oslo/syslog format have a severity on every line15:12
*** kzaitsev_mb has quit IRC15:12
fungiwhich the multi-line colorizing relies on15:12
*** jlanoux has quit IRC15:13
fungiso that explains why we got the behavior we did on the console logs15:13
*** vhosakot has joined #openstack-infra15:13
*** zul_ has joined #openstack-infra15:13
fungiit assumes any lines with no severity are continuation lines not new log entries15:13
Srinu_Fungi: how can we export param offline node with complete manually15:14
openstackgerritJames E. Blair proposed openstack-infra/zuul: Ansible launcher: add '|' to zuul_log  https://review.openstack.org/33220115:14
fungiSrinu_: i don't know what you're asking. you mean instead of having zuul set it?15:14
*** Volundr has joined #openstack-infra15:16
*** PollyZ has joined #openstack-infra15:16
*** kzaitsev_mb has joined #openstack-infra15:16
Srinu_Fungi: yes instead of zuul to set offline node with complete. Can we set manually offline node with complete=115:16
jeblairmordred, pabelanger: i'm going to stack a bunch of the zuul patches and test them all locally15:16
*** zul has quit IRC15:17
fungiSrinu_: it looks like that's probably what that singleuseslave-plugin was developed for, but i don't really know since we're not using it15:17
fungiwe set it with zuul15:17
jeblairSrinu_: if this is to fix the jenkins permissions problem, you should know that none of the other zuul parameters will be available, so the job won't actually work15:17
Srinu_Fungi: how to set with zuul can you tell15:18
jeblairSrinu_: the only solution to that is to disable the check as described in the mailing list post15:18
*** salv-orlando has joined #openstack-infra15:18
pabelangerjeblair: sounds good15:18
mordredjeblair: ++15:18
*** Julien-zte has joined #openstack-infra15:18
pabelangerjeblair: let me know if you revisit ssh_known_hosts patch. Interested to see what the issues are15:19
jeblairpabelanger: i'll throw that one on the pile too15:19
*** vincentll has quit IRC15:19
mordredmmm. pile of zuul15:19
Srinu_Jeblair: when jenkind jobs are queue it is executing first job and queued job not waiting until the deletion of slave i15:20
*** mohankumar has quit IRC15:20
*** hrubi has quit IRC15:20
Srinu_Jelblair: and after some time deleting the old slave and jobs fails15:21
*** sree has joined #openstack-infra15:21
fungiSrinu_: that matches the symptom i would expect from OFFLINE_NODE_WHEN_COMPLETE not getting set15:22
*** andreas_s has quit IRC15:23
openstackgerritzzxwill proposed openstack-infra/zuul: Rename jenkins to zuul  https://review.openstack.org/33085315:23
jeblairSrinu_: you need to tell jenkins to allow the gearman plugin to receive job parameters (i don't know how to do that, but the mailing list thread says how).  your sigle_use_node function sets the parameter correctly, but you need to tell zuul to call that in your layout.yaml file.15:23
fungiSrinu_: if you're using jenkins 1.651.2 or later, you probably need to read the mailing list thread jeblair mentioned, starting at http://lists.openstack.org/pipermail/openstack-infra/2016-May/004284.html15:23
*** piet_ has quit IRC15:24
*** azvyagintsev has joined #openstack-infra15:25
*** shardy has joined #openstack-infra15:25
*** e0ne has quit IRC15:25
*** sree has quit IRC15:26
*** hrubi has joined #openstack-infra15:26
*** e0ne has joined #openstack-infra15:26
*** hashar is now known as hasharAway15:26
*** javeriak has joined #openstack-infra15:27
*** ajmiller has quit IRC15:27
*** sree has joined #openstack-infra15:27
*** ajmiller has joined #openstack-infra15:28
*** hdaniel has quit IRC15:29
*** bhavik has quit IRC15:29
openstackgerritJames E. Blair proposed openstack-infra/zuul: Move job timeout into playbook as vars  https://review.openstack.org/33142615:29
openstackgerritJames E. Blair proposed openstack-infra/zuul: Ansible launcher: add support for node revocation  https://review.openstack.org/33127315:29
openstackgerritJames E. Blair proposed openstack-infra/zuul: Add local_tmp to ansible.cfg  https://review.openstack.org/33179715:29
openstackgerritJames E. Blair proposed openstack-infra/zuul: Remove unused timeout args for _makeBuilderTask  https://review.openstack.org/33142915:29
openstackgerritJames E. Blair proposed openstack-infra/zuul: Enable keep_remote_files in ansible.cfg  https://review.openstack.org/33181915:29
Srinu_Fungi:i used keepundefinedparameters=true but it is not work15:29
jeblairmordred, pabelanger: no changes there ^ only rebased into a stack (which i tested and they all look good)15:29
jeblairmordred, pabelanger: i'm going to do the ssh change separately now15:30
*** melwitt_ is now known as melwitt15:30
fungiSrinu_: i don't know what to suggest. we never used that newer version of jenkins and we don't use jenkins at all now, so hopefully someone who is can provide you with additional troubleshooting suggestions15:30
*** Na3iL has quit IRC15:30
*** melwitt is now known as Guest6187315:30
*** javeriak_ has joined #openstack-infra15:31
openstackgerritAdam Coldrick proposed openstack-infra/storyboard: Store history of a comment when editing  https://review.openstack.org/33220815:32
*** sree has quit IRC15:32
Shrewsmordred: if you get the chance, could you provide thoughts on the comment I left on Ghe's shade change: https://review.openstack.org/33073915:32
*** Guest61873 is now known as melwitt15:32
*** javeriak has quit IRC15:33
*** salv-orlando has quit IRC15:33
*** zz_dimtruck is now known as dimtruck15:33
*** Nikolay_St has joined #openstack-infra15:33
*** jordanP has quit IRC15:33
openstackgerritAdam Coldrick proposed openstack-infra/storyboard: Fix GET /v1/worklists/:id/filters/:id  https://review.openstack.org/33221015:33
*** Srinu_ has quit IRC15:34
*** Srinu_ has joined #openstack-infra15:34
*** e0ne has quit IRC15:34
stajkowskiAJaeger: hey! ty again, we'll need to place myself into the quark-core and quark-release groups so I can add others, per the guide15:34
Srinu_Jeblair: can you tell me how to overcome that issue15:35
*** pcaruana has quit IRC15:35
Srinu_Fungi: can you tell me how to overcome that issue15:36
rcarrillocruzShrews: besides the operator to openstack move15:36
rcarrillocruzthe change has domain_id plus name_or_id, should only have name_or_id15:36
rcarrillocruzand also15:36
rcarrillocruzkeystone domains are unique both by name and ids15:36
rcarrillocruztherefore doesn't make much sense to have filters15:36
rcarrillocruzif you get the domain, you get it..15:36
AJaegerstajkowski: I don't have those permissions. Other in the team will need to do it. Please provide a link to the review that created the repo for reference15:36
*** nmagnezi has quit IRC15:37
*** piet has joined #openstack-infra15:37
*** amotoki has joined #openstack-infra15:38
jeblairSrinu_: the only suggestion i have is to look at the parameters jenkins got for that build and see if offline_node_when_complete is there15:39
*** jaosorior has joined #openstack-infra15:39
stajkowskiAJaeger: ty, here is the patch https://review.openstack.org/#/c/319368/915:39
AJaegercan any infra-root help stajkowski , please? ^15:40
Srinu_Jeblair: where can i find the parameters of jenkins15:40
pabelangerAJaeger: sure, let me look15:40
*** dizquierdo has joined #openstack-infra15:40
stajkowskipabelanger: just need myself added to the quark-core and quark-release groups so I can add others, ty!15:41
fungiSrinu_: when you're logged into the jenkins webui and looking at the page for a recent job run status, you should see a link on the left side for "job parameters" or something like that. we don't use jenkins at all any more so i don't have an easy way to check what it's called exactly15:42
pabelangerAJaeger: stajkowski: added15:42
*** sheeprine_ has joined #openstack-infra15:42
openstackgerritgreghaynes proposed openstack-infra/puppet-gerrit: Remove text/html as a safe mimetype  https://review.openstack.org/33221915:43
stajkowskipabelanger: ty!15:43
openstackgerritMerged openstack-infra/tripleo-ci: Website: Switch to planet-venus code  https://review.openstack.org/33218615:44
*** jlanoux has joined #openstack-infra15:44
openstackgerritMerged openstack-infra/tripleo-ci: Add dprince to planet  https://review.openstack.org/33218715:44
*** Julien-zte has quit IRC15:46
*** maishsk has quit IRC15:47
*** amotoki has quit IRC15:48
*** sputnik13 has joined #openstack-infra15:48
*** sputnik13 has quit IRC15:49
*** sheeprine_ has quit IRC15:49
*** sheeprine_ has joined #openstack-infra15:50
*** sheeprine_ has quit IRC15:50
*** sheeprine_ has joined #openstack-infra15:51
*** sheeprine_ has quit IRC15:51
*** salv-orlando has joined #openstack-infra15:51
*** ddieterly is now known as ddieterly[away]15:51
*** amitgandhinz has quit IRC15:51
*** sheeprine_ has joined #openstack-infra15:51
*** amitgandhinz has joined #openstack-infra15:51
*** sheeprine_ has quit IRC15:51
*** tphummel has joined #openstack-infra15:52
*** sheeprine_ has joined #openstack-infra15:52
openstackgerritJames E. Blair proposed openstack-infra/zuul: Revert "Revert "Enabled host_key_checking""  https://review.openstack.org/33222415:52
*** yamamoto has quit IRC15:52
jeblairpabelanger: okay, i can't find anything wrong with it now :)15:52
*** yamamoto has joined #openstack-infra15:52
pabelangerjeblair: excellent!15:52
jeblairpabelanger: maybe there was something wrong in my test environment.  sorry15:52
jeblair(or maybe something else changed)15:52
jeblairpabelanger: anyway, take a look at that patch because i had to fix several conflicts15:53
pabelangerNo, that's fine. I'd rather tested it before we upgrade zuul-launchers15:53
*** gyee has joined #openstack-infra15:53
*** afred312 has joined #openstack-infra15:53
*** dmk0202 has joined #openstack-infra15:54
jeblairmordred: would you please review https://review.openstack.org/332198 and https://review.openstack.org/331819 ?15:54
*** ddieterly[away] is now known as ddieterly15:55
*** mhickey has quit IRC15:55
*** bhavik has joined #openstack-infra15:55
*** psachin has quit IRC15:55
*** Srinu_ has quit IRC15:55
mordredjeblair: oh - funny story - that's open in gertty right now15:56
pabelangerjeblair: looks good, left it open for you to approve when ready15:56
mordredthen devananda suckered me into a conversation15:56
jeblairpabelanger: i also tested that the ssh key thing does what we want -- i regenerated keys on my test host mid-job, and it exited with code 315:56
jeblairpabelanger: that means that zuul would re-launch the job15:56
jeblairpabelanger: so ... hrm15:56
mordredjeblair: done15:57
jeblairpabelanger: yeah, that's good if we run into the thing we did before... except, we will never notice it, and if we have a job that regenerates host keys, it will loop forever15:57
pabelangerRight, we maybe we should have it fail for now, just to see how often the issue happens15:57
*** chem is now known as chem|off15:57
jeblairpabelanger: yeah, but i don't know if we can do that?15:58
*** ihrachys has quit IRC15:58
pabelangerYa, exit code 3 is the issue atm15:58
jeblairbecause it looks just like the case where we can't *initially* connect to the node (where we do want to retry)15:58
jeblairanyway, i'm okay with it going in as-is, and maybe we can improve it later15:59
jeblairso when those 8 patches land, let's run your restart playbook15:59
*** sheeprine_ has quit IRC15:59
openstackgerritMerged openstack-infra/tripleo-ci: Use gear module from pip  https://review.openstack.org/33212315:59
anteayafungi: I have no internets, I'm coming in on my phone16:01
*** sheeprine_ has joined #openstack-infra16:01
openstackgerritMerged openstack-infra/puppet-gerrit: Remove text/html as a safe mimetype  https://review.openstack.org/33221916:01
*** sheeprine_ has quit IRC16:01
fungiinfra-root: i'm going to hand-apply that ^ on review.openstack.org and perform an emergency gerrit restart16:02
*** sheeprine_ has joined #openstack-infra16:02
fungithanks greghaynes for bringing it to my attention16:02
anteayafungi: I haven't read all the backscroll on storyboard/gerrit accounts but the bottom line is I don't care what things are called but would really like the integration in place for tomorrow16:02
anteayathank you16:02
*** alfredodeza has left #openstack-infra16:02
*** esikachev has quit IRC16:03
*** yolanda has quit IRC16:04
fungi#status notice Gerrit is being restarted now to apply an emergency security-related configuration change16:04
openstackstatusfungi: sending notice16:04
mordredfungi: ++16:04
fungiit should be on its way back up now16:05
-openstackstatus- NOTICE: Gerrit is being restarted now to apply an emergency security-related configuration change16:05
*** sheeprine_ has quit IRC16:05
d0ugalYup, it is back16:06
*** sheeprine_ has joined #openstack-infra16:06
fungigreghaynes is working on a quick advisory about that for the infra ml and oss-sec ml so people will be aware to update their configs if they were relying on ours16:06
*** sree has joined #openstack-infra16:06
*** Apoorva has joined #openstack-infra16:07
openstackstatusfungi: finished sending notice16:07
openstackgerritMerged openstack-infra/zuul: Ansible launcher: copy lib/plugins at startup  https://review.openstack.org/33219816:07
jeblairgreghaynes: thanks for that.  i wonder if you could provide a citation for your first sentence in the commit message?  i'd like to read more.16:07
*** yamamoto has quit IRC16:08
greghaynesjeblair: Ya, I found it on the gerrit ML last night, let me find it...16:08
*** mikelk has quit IRC16:08
greghaynesjeblair: https://groups.google.com/forum/#!topic/repo-discuss/h7Bvgns5cyY last comment16:08
jeblairgreghaynes: thanks!16:08
*** piet has quit IRC16:09
Shrewsjeblair: pabelanger: why does that host_key_checking change use pre_tasks?16:09
greghaynesthat actually is about why .zip instead of just setting mimetype, but that is protecting against the same issue... so I guess its not directly about that16:09
*** _nadya__ has quit IRC16:10
fungiyeah, https://review.openstack.org/Documentation/config-gerrit.html#mimetype doesn't do a good job of warning about that possibility16:10
*** sheeprine_ has quit IRC16:10
*** sree_ has joined #openstack-infra16:10
*** sree_ is now known as Guest9522316:10
fungia good upstream patch to gerrit might be to add a sentence about it to the documentation there16:10
*** sheeprine_ has joined #openstack-infra16:10
*** sree has quit IRC16:10
greghaynesIMO gerrit just shouldnt let you set that16:10
jeblairyeah, that was the first place i looked16:11
greghaynesespecially for text/html16:11
*** mdrabe has quit IRC16:11
fungii wonder if there are any vectors via text/xml16:11
fungier, application/xml i mean16:11
pabelangerjeblair: Shrews: doesn't need to, but felt moving it to a pre_task would break up the this run before a job, this runs during a job16:11
greghaynesheh, I was wondering the same. Its really going to come down to browser behavior. IMO all of those should be unset as they are an attack surface area (there have been libjpg vuls which could be exploited this way too)16:12
*** yamamoto has joined #openstack-infra16:12
greghaynesbut thats only if you have something that could be considered another vulnerability16:12
fungiyeah, the html one is a pretty obvious (in retrospect) risk16:13
*** raunak has joined #openstack-infra16:13
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Support colorized output for our tests  https://review.openstack.org/33217316:13
fungiwe definitely don't want gerrit serving user-supplied html to people16:13
openstackgerritMerged openstack-infra/zuul: Ansible launcher: add '|' to zuul_log  https://review.openstack.org/33220116:14
greghayneshah, yep, certainly not16:14
*** sheeprine_ has quit IRC16:15
*** Guest95223 has quit IRC16:15
*** sheeprine_ has joined #openstack-infra16:15
*** esikachev has joined #openstack-infra16:15
fungiif any of you have viewed html files from an untrusted submitter via download links in gerrit, you may want to consider the possibility that your gerrit account is compromised16:16
anteaya<-- walking uptown to use public wifi at the library hope to be online by meeting time16:16
*** sree has joined #openstack-infra16:16
mordredanteaya: have a good walk!16:16
*** trown is now known as trown|lunch16:17
bkerofungi: any clue how long the vulnerability has been around?16:17
bkeroLike if I viewed ahtml from an untrusted committer sometime last year?16:17
fungibkero: likely since the dawn of our gerrit configuration, but i'm digging in our git history now to find out16:17
bkerooh awesome16:18
*** piet has joined #openstack-infra16:18
zarofungi: https://review.openstack.org/#/c/262605/16:19
fungihttps://review.openstack.org/151376 seems to have added it16:19
fungirather, switched it from a broken value to a "working" one16:19
*** esikachev has quit IRC16:20
fungiat least, i'm assuming that back when it was application/html it didn't render?16:20
*** haaj has joined #openstack-infra16:20
*** Sukhdev_ has joined #openstack-infra16:20
*** sree has quit IRC16:21
fungiotherwise, it dates back to https://review.openstack.org/9310816:21
AJaegerfungi, see 262605 which mentioned that html was broken...16:21
*** Sukhdev_ has quit IRC16:21
*** Sukhdev_ has joined #openstack-infra16:22
fungiso without application/xml, text/html doesn't actually work16:22
*** dmk0202 has quit IRC16:22
zaroyeah, i think the one i referenced was when it was actually enabled.16:22
fungiwhich means it's only been in place since our first restart after 2016-01-0416:22
*** yaume_ has joined #openstack-infra16:24
zigofungi: It passed checks this time: https://review.openstack.org/#/c/331008/16:26
zigoCan you review it so I can attempt the next step?16:27
*** yaume has quit IRC16:27
fungigreghaynes: jeblair: zaro: https://review-dev.openstack.org/cat/5453%2C1%2Cxss-vuln-example.html^016:28
greghaynesfungi: oh, I have a poc16:29
greghaynesfungi: if youre looking for one16:29
*** stevemar has quit IRC16:29
fungigreghaynes: not looking for one, that's a pretty clear demonstration (runs a javascript alert window box)16:29
greghaynesah, yea16:29
*** amotoki has joined #openstack-infra16:30
*** links has joined #openstack-infra16:30
openstackgerritKien Ha proposed openstack-infra/jenkins-job-builder: Add plugin="junit" attribute for JUnit plugin  https://review.openstack.org/33223816:31
*** e0ne has joined #openstack-infra16:32
*** chem|off has quit IRC16:32
*** kien-ha has joined #openstack-infra16:33
*** mdrabe has joined #openstack-infra16:34
fungigreghaynes: also, i've checked your poc and it's properly serving me a zipfile from review.o.o now16:35
greghaynesI never thought i'd actually *want* to be served a zipfile16:36
fungibetter than being served a steaming pile of compromised account16:36
jeblaircan we use rar instead?16:36
jeblairi hear it's better than zip16:36
*** akshai_ has joined #openstack-infra16:36
nibalizerjeblair: zing16:37
*** shardy has quit IRC16:37
mordredjeblair: no shar?16:37
*** degorenko is now known as _degorenko|afk16:37
* greghaynes has flashbacks16:37
SpamapSlzip ftw?16:37
pabelanger<3 rar16:37
* rcarrillocruz votes for arj16:37
* SpamapS wants ARJ's16:37
SpamapSrcarrillocruz: ^5!16:37
SpamapStakin it back to the old school cause we're old fools who are so cool16:38
*** ramishra has quit IRC16:38
*** david-lyle has quit IRC16:38
* mordred hands SpamapS a cpio16:38
*** baoli_ has quit IRC16:38
*** david-lyle has joined #openstack-infra16:38
*** akshai has quit IRC16:39
*** sree has joined #openstack-infra16:39
SpamapSOn my first HP-UX box ever, there was a post-it on the console:  "cpio -ocvBdum / > /dev/st0"16:39
SpamapS"We run this every night at 5 before we go home, after we put a new tape in."16:40
*** jaosorior has quit IRC16:40
fungijeblair: i was hoping for split uuencode16:40
SpamapS"Why don't you use cron?" "What's cron?"16:40
mordredbecause I'm a dork, I have now run across this: http://www.nordier.com/v7x86/index.html16:40
mordredand kind of want to build a dib image for it so I can boot it in openstack16:41
*** dizquierdo has quit IRC16:41
nibalizergreghaynes: thanks for chasing this down16:41
SpamapSyoink.. wow...16:41
SpamapSmordred: can't be any less useful than cirros.. ;)16:42
zarojeblair: i don’t see an option for other compression formats.16:42
fungimordred: ooh! i wonder if it's possible to boot openvms as a vm16:42
mordredfungi: I bet openvms doesn't use systemd16:42
jeblairzaro: sorry, i was joking :)16:42
fungimordred: i'm quite certain it does not, in fact16:42
*** genunix has left #openstack-infra16:42
*** sree has quit IRC16:43
jeblairInventory script (/etc/ansible/hosts/openstack) had an execution error: /usr/local/lib/python2.7/dist-packages/os_client_config/config.py:476: UserWarning: Couldn't find the vendor profile 'hp', for the cloud 'openstackjenkins-hpcloud'16:43
*** sree has joined #openstack-infra16:44
Shrewswe removed the HP profile in 1.1516:44
mordredjeblair: where is that?16:45
jeblairmordred: when i 'sudo ansible' on puppetmaster16:45
*** tonytan4ever has quit IRC16:45
jeblairmordred: but not when i 'sudo su -; ansible'16:45
mordredjeblair: oh - it's probably looking at an old clouds.yaml in your home dir?16:46
jeblairmordred: oh, i thought it only looked in .config16:46
*** savihou has quit IRC16:46
mordredI meant in ~/.config16:46
*** piet has quit IRC16:47
mordredjeblair: but it will also use a clouds.yaml in the current dir if there is one16:47
jeblairmordred: no ~/.config is clean -- you were right, it was looking at ~/clouds.yaml16:47
mordredjeblair: awesome16:47
jeblairmordred: i was in ~16:47
jeblairi'm going to say "clouds.yaml in current dir" is a mildly surprising behavior :)16:48
*** sree has quit IRC16:48
jeblair(especially so far removed from ansible)16:48
jeblairjust some user feedback.  take it or leave it.  :)16:48
mordredjeblair: yah - I hear that :)16:49
mordredjeblair: I'm not sure I'd do the same thing if I were doing it over again today16:49
openstackgerritMerged openstack-infra/project-config: Fix upload path to tarballs.o.o for packaging  https://review.openstack.org/33100816:50
*** yamamoto has quit IRC16:51
*** Srinu_ has joined #openstack-infra16:51
*** armax has joined #openstack-infra16:51
*** jlanoux has quit IRC16:52
*** piet has joined #openstack-infra16:53
*** jpich has quit IRC16:54
nibalizerjeblair: i've tripped there too16:55
nibalizerbut once you know about it its really nice to be able to plop one in your cwd for testing and poking things16:56
*** Srinu_ has quit IRC16:56
*** amotoki has quit IRC16:57
*** zhurong has joined #openstack-infra16:58
*** sarob has joined #openstack-infra16:58
openstackgerritMerged openstack-infra/zuul: Add local_tmp to ansible.cfg  https://review.openstack.org/33179716:58
*** dmsimard is now known as dmsimard|afk16:59
*** inc0 has quit IRC17:01
*** inc0 has joined #openstack-infra17:02
*** xenogear has joined #openstack-infra17:02
*** e0ne has quit IRC17:02
*** Guest32736 has quit IRC17:03
*** ihrachys has joined #openstack-infra17:03
*** matt-borland has quit IRC17:03
openstackgerritMerged openstack-infra/zuul: Enable keep_remote_files in ansible.cfg  https://review.openstack.org/33181917:05
openstackgerritMerged openstack-infra/zuul: Ansible launcher: add support for node revocation  https://review.openstack.org/33127317:05
openstackgerritMerged openstack-infra/zuul: Move job timeout into playbook as vars  https://review.openstack.org/33142617:05
openstackgerritMerged openstack-infra/zuul: Remove unused timeout args for _makeBuilderTask  https://review.openstack.org/33142917:05
*** akshai has joined #openstack-infra17:05
openstackgerritMerged openstack-infra/zuul: Revert "Revert "Enabled host_key_checking""  https://review.openstack.org/33222417:05
jeblairpabelanger: okay, how do we run that thar playbook you got?17:06
jeblairpabelanger: you want to do it?17:07
pabelangerjeblair: I tested it from puppetmaster.o.o as root, but you could run it locally, but I need to do a patch for that17:07
*** dprince has quit IRC17:07
pabelangerjeblair: I can run it if needed17:07
pabelangerjeblair: how many zuul-launchers to we want to rotate at a time?17:08
*** akshai_ has quit IRC17:08
jeblairpabelanger: let's just do 1 at a time17:08
pabelangerjeblair: okay, let me get started17:09
openstackgerritPaul Belanger proposed openstack-infra/zuul: Enable pipelining for ansible-playbook  https://review.openstack.org/33149917:09
pabelangerjeblair: I'll start with zlstatic0117:10
*** BobBall is now known as BobBall_AWOL17:11
jeblairpabelanger: if you start with static01, make sure the changes have landed on disk first17:12
jeblair(since it might be able to cycle quickly)17:13
*** ihrachys_ has joined #openstack-infra17:13
pabelangerjeblair: ack17:13
fungizaro: playing around with https://review-dev.openstack.org/cat/5453%2C1%2Cxss-vuln-example.html^0 it looks like 262605 must have been a red herring. i still get a vulnerable page served even without application/xml or text/xml in the config, with just text/html in there17:13
*** qb_ has joined #openstack-infra17:15
*** ihrachys has quit IRC17:16
*** piet has quit IRC17:17
nibalizercrinkle: so for 331923 maybe we should just drop the entire lib and releasenotes directories17:18
*** yujunz has joined #openstack-infra17:18
nibalizersince we're not getting a lot of value out of that stuff17:18
AJaegernibalizer: you can always add them in again later ;)17:19
*** dizquierdo has joined #openstack-infra17:21
*** yujunz has quit IRC17:21
openstackgerritSpencer Krum proposed openstack-infra/puppet-openstack_infra_spec_helper: Initial commit  https://review.openstack.org/33192317:21
*** ddieterly is now known as ddieterly[away]17:21
*** PollyZ has quit IRC17:21
nibalizerAJaeger: cool17:22
*** psilvad has joined #openstack-infra17:22
*** sputnik13 has joined #openstack-infra17:23
*** electrofelix has quit IRC17:23
*** sarob_ has joined #openstack-infra17:24
cineramamordred: hi! can we have a glean release please if it's not too much trouble?17:24
*** sdake_ has joined #openstack-infra17:24
*** mriedem has quit IRC17:25
*** PollyZ has joined #openstack-infra17:25
fungizaro: so some further testing on review-dev, with 93108 i don't get rendered html from the download. with 151376 i do. 262605 doesn't seem to actually affect it one way or the other17:25
*** rbradfor has quit IRC17:25
*** yamahata has quit IRC17:26
*** sdake has quit IRC17:27
*** sarob has quit IRC17:28
zarofungi: hmm, not sure why it wasn't working for me before. could be that code mirror was updated17:28
*** trown|lunch is now known as trown17:28
*** mriedem has joined #openstack-infra17:28
zaroi think with 151376 we were still on gerrit 2.8?17:29
fungizaro: right. likely this behavior changed with our upgrade to 2.1117:30
*** PollyZ has quit IRC17:30
*** kien-ha has quit IRC17:30
fungiso our configuration in that puppet module was potentially vulnerable starting in february 2015, but our production deployment was unimpacted by it until december 2015 when we upgraded from 2.8 to 2.1117:30
fungiat least that's my guess17:31
fungiso other sites deploying newer gerrit releases with our puppet module would potentially have been impacted all the way back to when 151376 merged17:31
*** matt-borland has joined #openstack-infra17:32
*** ffrank has joined #openstack-infra17:32
*** dprince has joined #openstack-infra17:32
*** ihrachys_ has quit IRC17:32
*** ayoung has quit IRC17:33
*** ihrachys has joined #openstack-infra17:33
*** baoli has joined #openstack-infra17:34
*** Sukhdev_ has quit IRC17:34
crinklenibalizer: agree17:34
zaroyeah, i think that's correct.17:35
*** tosky has quit IRC17:35
nibalizercrinkle: cool17:35
nibalizeralso yes I did sync the gemfile from a puppet module with the gemspec so we should be good17:35
*** sree_ has joined #openstack-infra17:35
*** sree_ is now known as Guest6142417:36
openstackgerritVictor Ryzhenkin proposed openstack-infra/project-config: Add openstack/fuel-plugin-murano-tests project  https://review.openstack.org/33215117:36
*** stevemar has joined #openstack-infra17:36
* zaro is afk for kids field trip. bb in about 4hrs17:36
nibalizerzaro: have fun17:37
pabelangerjeblair: zlstatic01 restarted17:37
pabelangermoving to zl03 now17:37
pabelangeransible-playbook choice17:37
*** salv-orl_ has joined #openstack-infra17:38
*** sputnik13 has quit IRC17:39
*** Guest61424 has quit IRC17:40
*** sree has joined #openstack-infra17:40
*** pcaruana has joined #openstack-infra17:40
*** salv-orlando has quit IRC17:41
*** rcernin has quit IRC17:43
*** sree has quit IRC17:44
*** piet has joined #openstack-infra17:45
openstackgerritJames E. Blair proposed openstack-infra/zuul: Ansible launcher: move ansible log to ansible root  https://review.openstack.org/33226317:46
*** ihrachys has quit IRC17:46
openstackgerritPaul Belanger proposed openstack-infra/system-config: Use become for service restart for zuul-launcher playbooks  https://review.openstack.org/33226417:46
*** rbrndt has quit IRC17:47
jeblairpabelanger: do you need a 'become_user' for that?17:47
pabelangerjeblair: no, the default is root17:48
*** binbincong has quit IRC17:48
pabelangerhowever, we can add it to be safe17:49
anteaya<-- at the library17:49
openstackgerritPaul Belanger proposed openstack-infra/system-config: Use root to restart our zuul-launcher service  https://review.openstack.org/33226417:50
*** binbincong has joined #openstack-infra17:51
*** bhavik has quit IRC17:51
openstackgerritAndreas Florath proposed openstack/diskimage-builder: Refactor: block-device handling (partitioning)  https://review.openstack.org/32267117:51
*** yamamoto has joined #openstack-infra17:52
*** PollyZ has joined #openstack-infra17:52
*** armax has quit IRC17:53
*** aviau has quit IRC17:57
*** aviau has joined #openstack-infra17:57
openstackgerritOpenStack Proposal Bot proposed openstack-dev/hacking: Updated from global requirements  https://review.openstack.org/33226817:57
openstackgerritOpenStack Proposal Bot proposed openstack-dev/pbr: Updated from global requirements  https://review.openstack.org/33226917:57
openstackgerritOpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements  https://review.openstack.org/33228117:58
openstackgerritBrian Stajkowski proposed openstack-infra/project-config: Quark Project Non-Voting Py3 We will need to make pu3 jobs non-voting until we can address some issues.  https://review.openstack.org/33228717:59
*** _nadya_ has joined #openstack-infra17:59
*** nwkarsten has quit IRC17:59
stajkowskipabelanger: AJaeger we'll need one last thing then we're good17:59
stajkowskipabelanger: AJaeger https://review.openstack.org/#/c/332287/ non-voting on py3 for now17:59
*** nwkarsten has joined #openstack-infra18:00
*** SamYaple has joined #openstack-infra18:00
*** yamamoto has quit IRC18:01
*** netsin has joined #openstack-infra18:02
*** yamahata has joined #openstack-infra18:02
AJaegerstajkowski: that commit message is broken, please add an empty line after the title18:02
AJaegerstajkowski: and that's not how you do it either - I'll comment18:02
dhellmannfungi , clarkb : ttx and I are probably going to need more guidance on https://review.openstack.org/#/c/322863/ w.r.t. whether you think this is a useful direction to be trying to go, or if we should just stick with our current system18:02
stajkowskiAJaeger: my bad ty18:02
openstackgerritOpenStack Proposal Bot proposed openstack/os-testr: Updated from global requirements  https://review.openstack.org/33233518:04
*** javeriak_ has quit IRC18:04
*** nwkarsten has quit IRC18:04
*** cody-somerville has joined #openstack-infra18:05
*** ayoung has joined #openstack-infra18:05
*** pvaneck has joined #openstack-infra18:05
*** maishsk has joined #openstack-infra18:06
*** Rockyg has joined #openstack-infra18:06
*** zhurong has quit IRC18:06
stajkowskiAJaeger: so the check doesn't actually launch a job, just basically ignores it?18:06
AJaegerstajkowski: ?18:07
*** nwkarsten has joined #openstack-infra18:07
stajkowskiAJaeger: the job doesn't actually exist per say18:07
fungidhellmann: looking over that, it's similar to what we discussed as the primary alternative to mass acl changes back when we tried this solution18:07
fungidhellmann: i have a possibly simpler idea though...18:07
AJaegerstajkowski: I'm not getting what you reference18:07
fungidhellmann: groups can "include" groups, and we can change that inclusion (add/remove) through the gerrit api18:08
clarkbfungi: thats what this proposal uses aiui, make a group that doesnt change then change the membership of it ad apptopriate depending on cycle time18:08
fungiclarkb: hrm, rereading again before what i suggest turns out to be in the lines i somehow missed18:09
stajkowskiAJaeger: i only see gate-drgonflow-python34 referenced in zuul/layout so the job isn't an actual jenkins job, just a dummy placeholder?18:09
dhellmannthere are open questions about how to name the groups18:09
*** haaj has quit IRC18:10
*** markusry has quit IRC18:11
*** markusry has joined #openstack-infra18:11
*** markusry has quit IRC18:11
*** amoralej is now known as amoralej|off18:11
*** kzaitsev_mb has quit IRC18:11
AJaegerstajkowski: It's a job for sure. Defined via python-jobs in jenkins directory.18:12
AJaegerstajkowski: Do it exactly like dragonflow has done it...18:12
*** rbrndt has joined #openstack-infra18:12
AJaegerstajkowski: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n585018:12
AJaegerstajkowski: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n171018:12
AJaegerstajkowski: and do not use the python3-jobs template. That's all you need to do.18:12
fungidhellmann: clarkb: yep, so this proposal seems fine to me. basically switch from reverting a mass acl change immediately after release to updating group memberships immediately after release and cleaning up the acl down the line18:13
*** sree has joined #openstack-infra18:13
*** maishsk has quit IRC18:14
fungiand either we live with an ever growing pile of defunct groups (i'd love to come up with a mechanism for finding and cleaning those up in gerrit anyway, we already have a bunch) or renaming them each release18:15
fungiwe don't actually need to rotate through more than one set as long as we revert the branch-specific acl pattern for the previous release prior to adjusting the group inclusion and adding the acl pattern for the coming release branch18:16
dhellmannyeah, the only part I don't like is leaving cruft in gerrit18:16
*** akshai has quit IRC18:16
fungidhellmann: i don't think that leaves cruft in gerrit, it just creates a set of groups which are unused for some portion of each cycle18:16
fungibut we can recycle those same set of groups every cycle without needing to rename them18:17
clarkbfungi: good point18:17
clarkbn-1-nova-stable and n-2-nova-stable18:17
*** droyo has joined #openstack-infra18:17
fungiwe just need to stop referencing them in the acl before we switch their included group from the stable team to the release team, and then propose the acl for the coming release which uses that group on the not-yet-existent branch18:18
*** sree has quit IRC18:18
dhellmannthat sounds a bit like the alternative I propose around line 99, is that what you mean?18:18
openstackgerritBrian Stajkowski proposed openstack-infra/project-config: Quark Project Non-Voting Py3  https://review.openstack.org/33228718:18
openstackgerritSpencer Krum proposed openstack-infra/project-config: Infra spec helper is in openstack-infra  https://review.openstack.org/33241218:19
fungiexcept with "n different groups" being just 118:19
*** javeriak has joined #openstack-infra18:19
stajkowskiAJaeger: ok updated ty18:19
fungidhellmann: starting at the steady state (1) refs/heads/stable/* is granted to nova-stable-maint18:19
*** e0ne has joined #openstack-infra18:19
*** sputnik13 has joined #openstack-infra18:20
*** maishsk has joined #openstack-infra18:20
fungidhellmann: then we prepare for release (2) by adding refs/heads/stable/newton controlled by nova-new-stable-maint18:20
fungidhellmann: then (3) we make release managers the only team included in nova-new-stable-maint18:21
openstackgerritMerged openstack/os-testr: Add documentation for generate-subunit  https://review.openstack.org/32835218:21
fungidhellmann: then (4) the rc period is upon us and we create the stable/newton branch in nova18:21
*** sree_ has joined #openstack-infra18:21
*** ddieterly[away] has quit IRC18:21
*** sree_ is now known as Guest911818:21
fungidhellmann: then the release happens (5) and we replace release managers with nova-stable-maint as included in the nova-new-stable-maint group18:21
AJaegerstajkowski: looks fine. Just a cosmetic nit: Could you move the voting: false line above the mutex entries so that we have some order in the file, please?18:21
fungidhellmann: then sometime later in the cycle, as immediately as you like, (6) we merge a change to remove refs/heads/stable/newton from the acl so the nova-new-stable-maint group is no longer referenced18:22
*** pcaruana has quit IRC18:22
fungidhellmann: after that, (7) replace remove nova-stable-maint as included in the nova-new-stable-maint group and add release managers back to being included in that group18:23
dhellmannI like that. it leaves us with no cruft.18:23
dhellmannand we don't even need to do step 7 until we are ready to do step 2 again18:23
fungidhellmann: and that brings it full circle for ocata18:23
dhellmannso it could just be part of the script for step 318:24
dhellmannsteps 2 and 3 could be handled by the same script (set up the group memberships in gerrit and build the patch for project-config)18:24
dhellmannor separate, it doesn't matter18:24
dhellmannok, I'll paste this into a comment on the review for ttx to update18:24
fungidhellmann: i can rehash it there, just a sec18:25
*** Guest9118 has quit IRC18:25
*** ifarkas has quit IRC18:26
openstackgerritSpencer Krum proposed openstack-infra/project-config: Infra spec helper is in openstack-infra  https://review.openstack.org/33241218:26
*** dangers_away has quit IRC18:30
*** dangers_away has joined #openstack-infra18:31
nibalizerEmilienM: crinkle does https://review.openstack.org/#/c/331923/3 look good to go?18:32
fungidhellmann: my suggested plan is now outlined (slightly more clearly) in the comment i just added to 32286318:33
crinklenibalizer: it's failing tests?18:34
crinklenibalizer: maybe we want to get rid of those jobs18:34
*** rhochmuth has quit IRC18:34
nibalizercrinkle: experimental tests18:34
dhellmannfungi : great, thank you!18:34
EmilienMnibalizer: looking18:34
nibalizercrinkle: https://review.openstack.org/#/c/332412/ should fix the tests18:35
nibalizeror at least allow them to go forward18:35
*** _milan_ has joined #openstack-infra18:35
nibalizerbut right now we can't make any changes to infra puppet, so there is some desire to gogo on this18:35
nibalizerfungi: had to manually verify gregs security patch earlier18:35
stajkowskipabelanger: AJaeger all should be set to go now on https://review.openstack.org/#/c/332287/18:35
anteayanibalizer: or system-config18:35
fungidhellmann: i think this is also remarkably similar to the current process, but uses group manipulation to move the cleanup down the road so it can happen post-release18:35
*** ajmiller has quit IRC18:36
anteayanibalizer: https://review.openstack.org/#/c/329092/18:36
dhellmannfungi : yeah, the issue we were trying to address was the tight scheduling synchronization18:36
*** links has quit IRC18:36
EmilienMnibalizer: +1 & commented18:37
AJaegerstajkowski: did you want to upload a new version with my suggested nit fixed?18:37
openstackgerritMerged openstack-dev/hacking: Updated from global requirements  https://review.openstack.org/33226818:37
*** milan has quit IRC18:37
sdagueanyone know why our specs tree has blank documents - https://specs.openstack.org/openstack/nova-specs/specs/newton/approved/api-no-more-extensions.html ?18:38
sdaguethat was valid content last week18:38
crinklenibalizer: did we decide we want to keep release notes?18:40
*** PollyZ has quit IRC18:40
crinklenibalizer: i don't think we're ever going to release this thing (i don't think puppet-openstack even releases theirs)18:40
anteayasdague: looking at https://specs.openstack.org/openstack/nova-specs/specs/newton/approved/ only that file has a size of 018:40
sdagueanteaya: yep18:40
*** HeOS has quit IRC18:41
sdaguewhy is the real question18:41
sdaguedid some disk fill up and bork the publishing?18:41
*** ddieterly has joined #openstack-infra18:41
clarkbsdague: have you tried a local build to see if it is consistent?18:41
sdagueclarkb: yes, local build is fine18:42
jeblairi don't see anything interested on 6-14 on https://wiki.openstack.org/wiki/Infrastructure_Status18:43
jeblairer interesting18:43
sdaguels -la doc/build/html/specs/newton/approved/api-no-more-extensions.html18:43
sdague-rw-rw-r-- 1 sdague sdague 22893 Jun 21 14:42 doc/build/html/specs/newton/approved/api-no-more-extensions.html18:43
sdaguejust rebuilt18:43
*** piet has quit IRC18:43
*** PollyZ has joined #openstack-infra18:44
clarkbwas it possibly lost when the static host disappeared?18:44
*** r-mibu has quit IRC18:45
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Quark Project Non-Voting Py3  https://review.openstack.org/33228718:45
*** dizquierdo has quit IRC18:45
fungiyeah, it's possible that got truncated or something18:45
*** r-mibu has joined #openstack-infra18:45
jeblairclarkb: no that was 2016-06-20 05:24:05 according to the status page18:45
fungi-rw-rw-r-- 1 jenkins jenkins 0 Jun 14 09:33 /srv/static/specs/openstack/nova-specs/specs/newton/approved/api-no-more-extensions.html18:46
fungifrom the server18:46
clarkbwould fsck possibly do that though? or other recovery steps18:46
AJaegerclarkb: could you look at https://review.openstack.org/330821 - this adds back the files rules for project-config.18:46
jeblairi don't see anything interesting in the job18:47
sdaguethat file has definitely been live and working previously, I've shared that link18:47
*** roaet has joined #openstack-infra18:47
clarkbAJaeger: I have already said my piece if others want to take the time to review and keep those up tondate thats fine18:47
sdagueso it was good, then got truncated sometime after wed18:47
fungithough we have ick in syslog like: Jun 19 13:47:01 static kernel: [30671148.000318] EXT4-fs (dm-0): error count since last fsck: 1196218:47
*** rhochmuth has joined #openstack-infra18:48
sdagueI was offline thu -> sun, so that's as good as I can narrow it :)18:48
*** htruta` is now known as htruta18:48
jeblairthen yeah, fallout from the hardware failure seems to make the most sense to me18:48
sdagueit would be helpful if we could get things regenerated regardless18:48
AJaegerthanks, clarkb. jeblair, fungi, do we have then consensus on 330821?18:49
* AJaeger is willing to help keeing this up to date18:49
openstackgerritMerged openstack/os-testr: Sort failed tests at the top on HTML result page  https://review.openstack.org/32184118:49
fungisdague: if it's going to be a while before another commit merges to the nova-specs repo, i can reenqueue the branch tip for it in post18:49
zigofungi: I'm not sure what happened, now we have a http://tarballs.openstack.org/openstack-pkg-tools/, but it is empty.18:49
sdaguefungi: it might be, we're past non priority spec freeze18:50
fungizigo: getting ready for our weekly meeting in 10 minutes so don't have time to look at the moment18:50
fungisdague: i've reenqueued it into post, though looks like we have a ~5.5 hour backlog there so it's not going to update straight away18:52
stajkowskiAJaeger: thank you!18:52
sdaguefungi: cool, thank you18:52
sdagueit should at least be there by tomorrow morning18:52
*** jaosorior has joined #openstack-infra18:52
*** sputnik13 has quit IRC18:52
*** ayoung has quit IRC18:54
*** PollyZ_ has joined #openstack-infra18:55
*** sdake has joined #openstack-infra18:56
*** piet has joined #openstack-infra18:56
*** PollyZ has quit IRC18:56
*** sree has joined #openstack-infra18:56
*** javeriak has quit IRC18:57
*** mriedem1 has joined #openstack-infra18:58
*** mriedem has quit IRC18:59
*** ayoung has joined #openstack-infra18:59
*** sdake_ has quit IRC18:59
*** mriedem1 is now known as mriedem19:00
*** ayoung has quit IRC19:00
fungiit's that (weekly infra team meeting) time again! find us in #openstack-meeting for the next hour19:00
*** sree has quit IRC19:01
*** zul_ has quit IRC19:01
* anteaya imagines jeblair with kermit arms19:01
*** zul has joined #openstack-infra19:02
*** ajmiller has joined #openstack-infra19:02
fungipabelanger: you're not in #openstack-meeting19:04
*** jaosorior has quit IRC19:05
*** amitgandhinz has quit IRC19:05
*** amitgandhinz has joined #openstack-infra19:05
*** acoles is now known as acoles_19:07
*** akshai has joined #openstack-infra19:10
anteayamorning here too jhesketh19:11
*** Sukhdev has joined #openstack-infra19:13
*** dmsimard|afk is now known as dmsimard19:15
*** bhavik has joined #openstack-infra19:15
*** ajmiller has quit IRC19:19
*** haaj has joined #openstack-infra19:19
*** _nadya_ has quit IRC19:19
*** gyee has quit IRC19:19
*** joaotargino has joined #openstack-infra19:20
*** sree_ has joined #openstack-infra19:22
*** sree_ is now known as Guest281919:22
openstackgerritMatthew Treinish proposed openstack/os-testr: Remove the TODO file  https://review.openstack.org/32835719:23
*** abregman|afk has quit IRC19:24
*** abregman has joined #openstack-infra19:24
*** akshai has quit IRC19:25
*** xenogear has quit IRC19:25
*** ihrachys has joined #openstack-infra19:25
*** Guest2819 has quit IRC19:26
*** rdx_matt_ has joined #openstack-infra19:27
*** permalac_ has joined #openstack-infra19:27
*** esberglu_ has joined #openstack-infra19:27
*** claudiub|2 has joined #openstack-infra19:27
*** xarses_ has joined #openstack-infra19:27
*** jcoufal_ has joined #openstack-infra19:27
*** esberglu has quit IRC19:28
*** jpmaxman has quit IRC19:28
*** esberglu_ is now known as esberglu19:28
*** e0ne_ has joined #openstack-infra19:28
*** e0ne_ has quit IRC19:28
*** piet has quit IRC19:29
*** kaisers1 has quit IRC19:29
*** aysyd has quit IRC19:29
*** scottynomad has quit IRC19:29
*** rdx_matt has quit IRC19:29
*** arif-ali has quit IRC19:29
*** toabctl has quit IRC19:29
*** inc0_ has joined #openstack-infra19:29
*** rossella_s has joined #openstack-infra19:29
*** e0ne has quit IRC19:30
*** david-lyle has quit IRC19:30
*** jcoufal has quit IRC19:30
*** pradk has quit IRC19:30
*** claudiub has quit IRC19:30
*** leifmadsen has quit IRC19:30
*** vsaienko has quit IRC19:30
*** aderyugin has quit IRC19:30
*** mandre has quit IRC19:30
*** ioni has quit IRC19:30
*** cinerama has quit IRC19:30
*** pfallenop has quit IRC19:30
*** SotK has quit IRC19:30
*** aarefiev has quit IRC19:30
*** gabriel-bezerra has quit IRC19:30
*** esikachev has joined #openstack-infra19:30
*** inc0 has quit IRC19:31
*** xarses has quit IRC19:31
*** permalac has quit IRC19:31
*** tristanC has quit IRC19:31
*** sc` has quit IRC19:31
*** tonyb has quit IRC19:31
*** ajmiller has joined #openstack-infra19:32
*** pfallenop has joined #openstack-infra19:32
*** tristanC has joined #openstack-infra19:32
*** sc` has joined #openstack-infra19:32
*** tonyb has joined #openstack-infra19:32
*** ioni has joined #openstack-infra19:32
*** ioni is now known as Guest2389919:33
*** sree has joined #openstack-infra19:33
*** ldnunes has quit IRC19:34
*** amitgandhinz has quit IRC19:34
*** amitgandhinz has joined #openstack-infra19:34
*** cinerama has joined #openstack-infra19:34
*** toabctl has joined #openstack-infra19:35
*** arif-ali has joined #openstack-infra19:35
*** scottynomad has joined #openstack-infra19:35
*** leifmadsen has joined #openstack-infra19:35
*** aderyugin has joined #openstack-infra19:35
*** ldnunes has joined #openstack-infra19:36
*** mandre has joined #openstack-infra19:36
*** SotK has joined #openstack-infra19:36
fungipabelanger: you're still not in #openstack-meeting and we're at your topic now19:37
*** flepied1 has joined #openstack-infra19:37
*** aysyd has joined #openstack-infra19:37
*** sree has quit IRC19:38
*** _nadya_ has joined #openstack-infra19:38
openstackgerritDoug Hellmann proposed openstack-infra/project-config: import the release tools that need to run on secure nodes  https://review.openstack.org/33244719:39
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: move the release tools that need to run on secure nodes  https://review.openstack.org/33244819:39
*** flepied has quit IRC19:39
*** ayoung has joined #openstack-infra19:40
*** david-lyle has joined #openstack-infra19:41
*** haaj has quit IRC19:41
*** kzaitsev_mb has joined #openstack-infra19:42
*** jpmaxman has joined #openstack-infra19:42
*** gabriel-bezerra has joined #openstack-infra19:43
*** pradk has joined #openstack-infra19:43
*** aarefiev has joined #openstack-infra19:43
*** vsaienko has joined #openstack-infra19:44
*** kaisers1 has joined #openstack-infra19:45
*** BigWillie has quit IRC19:45
*** sarob has joined #openstack-infra19:46
*** maishsk_ has joined #openstack-infra19:47
*** maishsk has quit IRC19:48
*** maishsk_ is now known as maishsk19:48
*** sree has joined #openstack-infra19:48
*** banix has joined #openstack-infra19:49
*** sarob_ has quit IRC19:49
*** rhochmuth has quit IRC19:49
*** HeOS has joined #openstack-infra19:50
*** dtantsur is now known as dtantsur|afk19:51
*** PollyZ_ has quit IRC19:51
*** sree has quit IRC19:52
*** sree has joined #openstack-infra19:53
*** akshai has joined #openstack-infra19:54
*** tphummel has quit IRC19:56
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Don't try to get stack details if the stack doesn't exist  https://review.openstack.org/31933719:56
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: TEST: Delete the overcloud when finished  https://review.openstack.org/29732819:56
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Test overcloud deletion in periodic job  https://review.openstack.org/29676519:56
openstackgerritDoug Hellmann proposed openstack-infra/project-config: import the release tools that need to run on secure nodes  https://review.openstack.org/33244719:56
*** rhochmuth has joined #openstack-infra19:57
*** rhochmuth has quit IRC19:57
*** sree has quit IRC19:57
*** PollyZ has joined #openstack-infra19:58
*** rhochmuth has joined #openstack-infra19:59
*** akshai has quit IRC20:00
*** sree has joined #openstack-infra20:01
*** amitgandhinz has quit IRC20:01
*** _nadya_ has quit IRC20:01
*** amitgandhinz has joined #openstack-infra20:01
fungipabelanger: when you're around again, see meeting log. pleia2 is going to update your upgrade notification to be july 1st and get the announcement out in short order. i gather it's a holiday in canada but there are several of us over the border who plan to be around and are happy to take care of the upgrades20:02
*** PollyZ has quit IRC20:02
fungiand that way we have a week+ of advance notice to the community that we're taking stuff offline for hours20:03
*** marcusvrn_ has joined #openstack-infra20:03
*** ihrachys has quit IRC20:04
*** sree has quit IRC20:05
*** Nikolay_St has quit IRC20:06
*** sree has joined #openstack-infra20:07
rcarrillocruzpabelanger: https://review.openstack.org/#/c/332435/1 , generating a keypair on-the-fly in the launcher role20:08
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Add other-requirements to files list for project-config  https://review.openstack.org/33246020:08
*** Apoorva has quit IRC20:10
*** PollyZ has joined #openstack-infra20:11
pleia2downtime announcement sent20:11
*** sree has quit IRC20:12
*** kgiusti has left #openstack-infra20:12
*** Sukhdev has quit IRC20:13
*** rcernin has joined #openstack-infra20:13
AJaegerthanks, pleia2 ! That was quick!20:13
* Zara makes note, thanks :)20:13
*** sree has joined #openstack-infra20:14
*** rossella_ has joined #openstack-infra20:14
pleia2it was already drafted :)20:14
*** PollyZ has quit IRC20:16
*** dtardivel has quit IRC20:17
*** rossella_s has quit IRC20:17
*** sree has quit IRC20:18
*** kzaitsev_mb has quit IRC20:19
nibalizerthansk pleia220:20
*** sree has joined #openstack-infra20:20
*** kzaitsev_mb has joined #openstack-infra20:22
openstackgerritMerged openstack/os-testr: Remove the TODO file  https://review.openstack.org/32835720:23
jrollmordred: ircops won't bump the channel limit? :/20:23
mordredjroll: it's global currently20:23
mordredjroll: not per-accout20:23
jrollah, I thought there was a per-user override20:23
nibalizerso are we gonna have a fleet of gerritbots then?20:23
mordredjroll: used to be - doesn't seem to be any more20:23
mordrednibalizer: yah. I think we should name them fun things20:24
clarkbI would prefer a "listen here " chennel to multiple robots20:24
mordrednibalizer: like openstackgerrit-spencer20:24
njohnstonFYI review.openstack.org comes up as having broken HTTPS in my web client - Chrome 51.0.2704.84 - because SHA-1 signatures are used: http://imgur.com/Q4w33jt20:24
nibalizermordred: im going for 'openstackgerrit-ping-mordred'20:24
*** sputnik13 has joined #openstack-infra20:24
*** matbu has quit IRC20:25
*** sree has quit IRC20:25
kzaitsev_mbcan anyone help me with https://review.openstack.org/#/c/331754/3 — as far as I understand it should have been merged long ago, but the gate check just doesn't start20:25
*** PollyZ has joined #openstack-infra20:25
mordrednjohnston: oh. uhm20:25
rcarrillocruzmordred , clarkb : can you look https://review.openstack.org/#/c/327789/ ? got a +2 from nibalizer ....20:25
mordredfungi: ^^20:25
mordredfungi: if new chrome doesn't like our ssl certs, that's going to probably hit more people soonish20:26
clarkbwe also refreshed our certs all together a year ago iirc20:26
clarkbso likely need to do it across the baord20:26
anteayapleia2: thanks for sending the announcement20:27
*** banix has quit IRC20:27
*** markvoelker has quit IRC20:27
AJaegerkzaitsev_mb: could you toggle your workflow, please? Remove it and add it back again.20:27
fungioh sigh20:27
AJaegerkzaitsev_mb: I see: +2W and then a rebase, this looks strange20:27
jeblairmordred, njohnston, fungi: i see a sha-256 signature.20:27
*** Apoorva has joined #openstack-infra20:27
funginjohnston: are you behind a proxy of some kind?20:28
AJaegerkzaitsev_mb: let's check zuul status first20:28
kzaitsev_mbAJaeger: just did that and the gate job started. phew.20:28
openstackgerritSpencer Krum proposed openstack-infra/puppet-openstack_infra_spec_helper: Initial commit  https://review.openstack.org/33192320:28
kzaitsev_mbI actually thought this trick doesn't work (I thought I did it and it had no effect)20:29
AJaegerkzaitsev_mb: changeset 2 had the +2W and then it was rebased and now it's changeset 3. Not sure what happened there...20:29
njohnstonfungi: I'm not supposed to be... let me check and see if my corporate network put up a transparent proxy without telling me20:29
jeblairnjohnston: could you paste the output from: openssl s_client -showcerts -connect review.openstack.org:44320:29
*** PollyZ has quit IRC20:29
AJaegerkzaitsev_mb: why did you rebase at all? gerrit rebases for you...20:30
kzaitsev_mbit's always like this — you try everything, then come to #infra and someone asks to do an obvious thing and everything works %)20:30
openstackgerritJef Mallal proposed openstack-infra/jenkins-job-builder: Add support for retry in multijob plugin  https://review.openstack.org/32916920:30
kzaitsev_mbAJaeger: because the gate job didn't start even though it had 2+W20:30
AJaegerkzaitsev_mb: always check first zuul queue length...20:31
*** ldnunes_ has joined #openstack-infra20:31
*** matt-borland has quit IRC20:31
*** mdrabe has quit IRC20:31
funginjohnston: http://paste.openstack.org/show/520987/ "Signature Algorithm: sha256WithRSAEncryption"20:31
AJaegerrebasing is seldom the solution, it will just force it back into check queue20:31
* AJaeger waves good bye20:32
kzaitsev_mbAJaeger: it was based on 2d patchset of a different patch, the 3d patchset of that change got merged and I thought that retriggering all the things might force it back into check and then into gate20:32
funginight AJaeger!20:32
kzaitsev_mbok, will know, that it's a bad idea20:32
*** bhavik1 has joined #openstack-infra20:32
kzaitsev_mbthanks for help +)20:32
*** matbu has joined #openstack-infra20:32
njohnstonfungi: http://paste.openstack.org/show/520988/  I don't see a "Signature Algorithm" line20:33
*** tphummel has joined #openstack-infra20:33
*** ldnunes has quit IRC20:33
*** bhavik has quit IRC20:33
*** bhavik1 is now known as bhavik20:33
*** cody-somerville has quit IRC20:34
jeblairnjohnston, fungi: that appears to be the correct cert and have a sha256 signature20:35
jeblairso at least openssl gets the right cert20:35
jeblairnjohnston: can you ask chrome to give you a copy of the certificate it received?20:35
njohnstonYeah, I just tried from a computer not on my corporate network, and after refreshing it looks fine, the warning disappears from Chrome.  The bastards implemented a transparent ssl proxy on me.  :-(20:35
jeblairnjohnston: apparently one that is active for chrome, but not command line openssl...20:35
funginjohnston: and even more insecure of a proxy than it needed to be apparently!20:35
SamYaplenjohnston: not so transparent there20:36
funginjohnston: is it possible to get chrome to give you details on the cert? as a not-chrome user i don't know the answer there20:36
SamYaplefungi: yea you can20:36
*** sree_ has joined #openstack-infra20:36
SamYapleclick on the lock in the top left by the url bar and then "details"20:36
clarkbyou click the green(or not green) lock20:36
*** sree_ is now known as Guest220920:37
fungiyou should be able to tell what the reissued cert looks like. presumably you've allowed them to update your browser's trust store to include whatever the proxy's (terroble) cert is20:37
*** maishsk has quit IRC20:37
*** jefmallal has joined #openstack-infra20:38
njohnstonnow I can't get it to give me the insecure certificate icon anymore.  It's as if there are multiple servers behind a VIP, and at first I hit a real server that had a bad cert, but now I'm on a good real server.20:38
*** ayoung has quit IRC20:38
*** PollyZ has joined #openstack-infra20:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-accessbot: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33246620:39
fungiin the entire cert chain from your paste, our cert's signature is the _shortest_ with sha256. the two others further up the chain are sha384 even20:39
openstackgerritSpencer Krum proposed openstack-infra/puppet-openstack_infra_spec_helper: Initial commit  https://review.openstack.org/33192320:39
*** maishsk has joined #openstack-infra20:39
clarkbnjohnston: that is a common deployment method for large corp proxies20:40
*** Guest2209 has quit IRC20:41
*** sree has joined #openstack-infra20:41
SamYaplenjohnston: call corporate IT and ask them to turn the servers on and off again20:41
*** ddieterly is now known as ddieterly[away]20:42
*** rossella__ has joined #openstack-infra20:43
openstackgerritMerged openstack-infra/project-config: Infra spec helper is in openstack-infra  https://review.openstack.org/33241220:43
*** PollyZ has quit IRC20:43
*** bhavik1 has joined #openstack-infra20:45
*** sree has quit IRC20:45
*** pt_15 has quit IRC20:46
*** rossella_ has quit IRC20:46
*** matbu has quit IRC20:47
*** matbu has joined #openstack-infra20:47
anteayaso I have an appointment to have internets again, let's hope the gods are smiling oh whoever is assigned the ticket to fix my internets tomorrow20:48
*** hemanthm is now known as hemanthm|afk20:48
anteayaI'm packing up to go get something to eat then will come back to the library in a bit20:48
openstackgerritSean Roberts proposed openstack-infra/project-config: add craton project  https://review.openstack.org/33247020:48
*** bhavik has quit IRC20:49
*** bhavik1 is now known as bhavik20:49
*** sputnik13 has quit IRC20:51
*** pradk has quit IRC20:51
*** maishsk has quit IRC20:52
*** dprince has quit IRC20:53
*** pilgrimstack has joined #openstack-infra20:53
*** sputnik13 has joined #openstack-infra20:54
*** sarob_ has joined #openstack-infra20:54
*** rossella_ has joined #openstack-infra20:55
*** Sukhdev_ has joined #openstack-infra20:56
*** shashank_hegde has joined #openstack-infra20:57
*** rossella__ has quit IRC20:58
*** sarob has quit IRC20:59
*** gyee has joined #openstack-infra21:00
*** raunak has quit IRC21:00
dhellmannfungi : I've put the release tools that we'll need for the automation in a subdir of jenkins/scripts, is that going to work OK?21:00
dhellmannfungi : https://review.openstack.org/#/c/332447/21:01
*** pilgrimstack has quit IRC21:01
*** jefmallal has quit IRC21:01
*** _ari_ has quit IRC21:02
fungidhellmann: yep, that's where they belong21:02
*** sputnik13 has quit IRC21:02
*** ddieterly[away] is now known as ddieterly21:04
*** raildo is now known as raildo-afk21:04
*** rfolco has quit IRC21:04
*** PollyZ has joined #openstack-infra21:06
*** kzaitsev_mb has quit IRC21:06
*** yolanda has joined #openstack-infra21:06
*** trown is now known as trown|outtypewww21:07
*** eharney has quit IRC21:07
rajinirZuul: Having a wierd problem in our CI. Zuul is not  passing a range of built-in parameters (eg: ZUUL_PROJECT) . Any clues where to  debug this problem?21:08
*** psilvad has quit IRC21:08
*** matt-borland has joined #openstack-infra21:09
*** sputnik13 has joined #openstack-infra21:10
*** armax has joined #openstack-infra21:10
*** kzaitsev_mb has joined #openstack-infra21:10
*** inc0_ has quit IRC21:10
*** rossella__ has joined #openstack-infra21:10
*** esikachev has quit IRC21:11
*** baoli has quit IRC21:11
*** baoli has joined #openstack-infra21:12
dhellmannfungi : ok, I wasn't sure about the new subdirectory21:12
*** gouthamr has quit IRC21:12
*** rhallisey has quit IRC21:14
*** rossella_ has quit IRC21:14
*** rossella_ has joined #openstack-infra21:15
fungidhellmann: oh, new directory? i haven't pulled up the change yet21:15
fungiahh, jenkins/scripts/release-tools/21:15
pleia2oh, forgot to mention earlier, I'm away Mon-Wednesday next week, will be at CodeConfLA21:15
*** sree has joined #openstack-infra21:16
jpmaxmanok fungi - I think I'm going to try recaptcha to see if it successfully stops the spam21:16
fungidhellmann: we've not done subtrees under there in the past, but with that many scripts it makes some sense to keep 'em separated21:16
jpmaxmanI've been watching the server off and on21:16
fungijpmaxman: great, thanks for the heads-up! happy to test21:16
jpmaxmanand one thing I notice21:16
jpmaxman2001:4800:7810:512:3bc3:d7f6:ff04:5371 - - [21/Jun/2016:21:15:19 +0000] "POST /w/index.php?title=Special%3ARunJobs&tasks=jobs&maxjobs=1&sigexpiry=1466543724&signature=6e25ad7850a068071bfb103182b594f701003350 HTTP/1.1" 302 725 "-" "-"21:16
jpmaxmancontinuous posts to this url in the logs21:16
jpmaxmanis that something you're aware of?21:16
*** rossella__ has quit IRC21:16
jpmaxmani don't think it is related to the spam21:17
jpmaxmanjust thought I'd point it out21:17
*** sree_ has joined #openstack-infra21:17
fungijpmaxman: i think there's a cron job on the server calling that to do some sort of periodic wiki maintenance. checking now21:17
dhellmannfungi : that's what I thought re organization, but I wasn't sure how those files are "installed" on a node and if that extra level would cause an issue. I may end up putting a little readme in there, too, since sometimes it's necessary/useful to run those scripts by hand21:17
pleia2speaking of the wiki, I still could use some eyes on the upgrade spec https://review.openstack.org/#/c/328455/21:17
*** sree_ is now known as Guest9225821:17
jpmaxmanyeah I found this21:17
pleia2jpmaxman: could use input from you as well, in case we want to include any anti-spam stuff in the spec21:18
pleia2(not sure if we do)21:18
*** ajmiller_ has joined #openstack-infra21:19
pleia2fungi: I think you also mentioned that pabelanger may not have time to work on this? so I think we need to change the primary assignee21:19
fungipleia2: yep, i'm happy to be the primary assignee on it21:19
pleia2ok, I'll make a comment to update it in the next revision21:19
fungipleia2: pabelanger and nibalizer had volunteered to work on it at the summit but have since been pulled in other directions21:20
* pleia2 nods21:20
*** sree has quit IRC21:20
nibalizeroh the wiki?21:20
*** gouthamr has joined #openstack-infra21:20
nibalizeryea I had hoped to help pabelanger not drive21:20
pleia2yeah, wiki upgrade21:21
nibalizeri.e. be around for reviews and bounce ideas and stuff21:21
pleia2nibalizer: can you look at the spec?21:21
nibalizergenerally 'hand me a task' and ill do it21:21
*** mdrabe has joined #openstack-infra21:21
*** sree has joined #openstack-infra21:21
nibalizerbut like planning and figuring out what needs to be done... don't depend on me21:21
nibalizerbtw i am actually trying to fix puppet still yall21:21
*** markvoelker has joined #openstack-infra21:21
*** Guest92258 has quit IRC21:22
jpmaxmanpleia2 yup - I think it is all a good idea regardless21:22
pleia2fix puppet \o/ (that's part of the spec)21:22
jpmaxmanI'm not sure of the current state of puppetization21:22
jpmaxmanI know that the settings file was added21:22
*** ajmiller has quit IRC21:22
pleia2jpmaxman: puppet does still need a bit of work, but I haven't dug into it21:23
clarkbthe puppet installs apache and php, elasticsearch, and the config but does not control the mediawiki install iirc21:23
pleia2yeah, it doesn't install the mediawiki software itself21:23
pleia2that was all done manually21:23
clarkbbasically the only but missing is the nebulous install mediawiki and plugins21:23
clarkband possibly db management byt I think thats automagic with mediawiki?21:24
*** ccamacho has left #openstack-infra21:24
pleia2yeah, I think it has a script21:24
*** ldnunes_ has quit IRC21:25
pleia2I think we need to still create the database and user though21:25
pleia2the script just populates it21:25
*** sree has quit IRC21:25
clarkbpleia2: currently thats trove iirc which isnt part of puppet for our things21:25
*** ramishra has joined #openstack-infra21:26
jpmaxmanright I think it is using a local mysql21:27
pleia2no, clarkb is right, we are using trove (just checked)21:28
fungialso the mediawiki installation there can be massively simplified21:28
fungithe whole red/blue paradigm is serious additional complication if we're planning to also haev a wiki-dev to test out new versions21:29
*** ddieterly is now known as ddieterly[away]21:29
*** jcoufal_ has quit IRC21:30
*** amitgandhinz has quit IRC21:30
fungia brief window every few weeks where the wiki is offline because files are being untarred is tolerable for this system21:30
*** baoli has quit IRC21:30
fungigenerally going to be on the order of seconds21:30
*** amitgandhinz has joined #openstack-infra21:30
fungiso i really don't want to overengineer it21:30
*** markusry has joined #openstack-infra21:30
*** burgerk_ has quit IRC21:30
pabelangerfungi: pleia2: sorry for missing our meeting, got pulled away from computer for personal reason21:30
pleia2pabelanger: no problem, it got you out of a maintenance window :D21:31
fungipabelanger: no apologies needed! just wanted to point you to maintenance updates in the meeting log so you're aware21:31
pleia2(enjoy Canada Day)21:31
mtreinishfungi: any amount of downtime and someone will notice and come in channel to complain21:31
*** sputnik13 has quit IRC21:31
fungiwhy ask why, drink canada dry?21:31
* fungi can't remember the canada dry slogan21:32
pabelangerfungi: pleia2: sure, looking at the meeting logs now21:32
fungimtreinish: yep, and we'll refund them every cent they pay for wiki access21:32
nibalizerpleia2: spec looks solid21:32
pleia2nibalizer: thanks for looking <321:32
nibalizeryah no prblem21:32
pabelangerjeblair: zl03 was restarted correctly by our ansible-playbook. moving to zl01 now21:33
*** gouthamr has quit IRC21:33
mtreinishfungi: https://flic.kr/p/A154HN I tried that during my first trip to japan21:33
*** gouthamr has joined #openstack-infra21:33
mtreinishfungi: I don't think they sold it for very long, I haven't been able to find it since21:34
bkerommm hot ginger ale21:34
fungimtreinish: oh, interesting! however we have plenty of very spicy ginger ales available in the usa made with far better ingredients21:35
mtreinishbkero: heh, it's not as good as it sounds. It was actually kinda gross :)21:35
openstackgerritElizabeth K. Joseph proposed openstack-infra/infra-specs: Add wiki modernization spec  https://review.openstack.org/32845521:35
*** sree has joined #openstack-infra21:35
*** ddieterly[away] is now known as ddieterly21:35
bkeromtreinish: Blenheim is my favorite hot ginger ale21:35
bkerosuper spicy21:35
mtreinishfungi: hot as in serving temperature, not how spicy it was :21:35
fungimtreinish: WEIRD21:35
SpamapStoo bad about Quorrobolong21:35
SpamapSthat's a FUNNY word21:36
pleia2hot ginger ale is a thing?21:36
mtreinishyes, it was very strange. That's why I had to try it :)21:36
fungimtreinish: well, maybe not so weird. i drink hot ginger broth ("tea") frequently21:36
pleia2had alcoholic ginger ale recently, enjoyed21:36
mtreinishpleia2: I only ever saw it during that trip. I think they learned their lesson21:36
*** jpeeler has quit IRC21:36
pleia2fungi: yeah, I think it's hot+carbonated that's throwing me off21:37
pleia2ginger tea is tastey21:37
fungitake fresh ginger root, slice thin, simmer for a few minutes, filter and consume. add a little honey if you like21:37
fungibut hot carbonation seems physically infeasible21:37
pabelangerpleia2: regarding wiki, that's currently last on my list for upgrades.  Let me loop back to it tomorrow and see what our current status on the precise upgrades are21:37
fungisolubility of gasses goes down as the solution temperature increases21:38
mtreinishI thought it'd be like warm ginger ale, which can be soothing, especially if you're sick21:38
pleia2pabelanger: thanks, I'd appreciate your eyes on the upgrade spec when you do that21:38
mtreinishbut increase the temperature too much and it just tastes bad21:38
*** dimtruck is now known as zz_dimtruck21:38
pabelangerpleia2: sure, I'll read up on it too21:38
jeblairpabelanger: that seems like zl03 took a long time?21:39
*** bhavik has quit IRC21:39
*** markvoelker has quit IRC21:39
bkerofungi: your ginger tea sounds fantsatic21:39
*** sputnik13 has joined #openstack-infra21:39
fungibkero: in the summer i just drop sliced ginger in a pitcher of water and keep it in the fridge for a day or so before drinking21:40
*** bhavik has joined #openstack-infra21:40
*** sree has quit IRC21:40
fungiadding a lemon wedge is great too21:40
* pleia2 adds bourbon to fungi's ginger tea21:40
pleia2how pirate of you21:40
fungiwe have a pretty drinkable local rum here now (well, brewed on the next island over, but near)21:41
*** ayoung has joined #openstack-infra21:41
*** rcernin has quit IRC21:41
pabelangerjeblair: zl03 took 2.3 hours to restart using the playbook.  It actually tried to do the next server in the list, but I had not accepted the ssh host key from puppetmaster.o.o under my username.21:41
bkerofungi: is the ginger good for more than one steep?21:41
jeblairpabelanger: 2.3 is still a long time :(21:42
bkeroI didn't think the gingery-ness would come out if you don't steep it in hot water21:42
*** abregman has quit IRC21:42
fungibkero: the cold variety i will sometimes top back up with water at least once. the hot less so (heat tends to extract the sap more rapidly)21:43
bkeroI'll probably try that when I get home21:43
pabelangerjeblair: not sure how we can speed that up with graceful21:43
fungiyeah, you need to slice thinly or mince finely and leave the water sitting on it for a good 24 hours to steep it sufficiently for a cold brew. also i think the slight acidity from the lemon juice helps21:44
bkeroMaybe add some sugar and yeast and time to make some carbonated ginger tea beer21:44
*** jamesmca_ has joined #openstack-infra21:44
bkeroI have a hard time slicing ginger thinly. I have a pretty sharp mandolin, but it's not good enough to slice ginger :(21:44
fungibkero: yeah, making ginger beer is pretty easy. used to do it all the time growing up. also birch beer and sassafrass beer21:44
*** amitgandhinz has quit IRC21:45
fungimore of the latter two since they grew all over the mountains where i grew up21:45
fungiginger we had to buy21:45
jeblairpabelanger: well, i wonder what jobs it was running?21:45
*** sputnik13 has quit IRC21:46
pabelangerjeblair: let me check21:46
openstackgerritSpencer Krum proposed openstack-infra/project-config: More typos in infra_spec_helper job definition  https://review.openstack.org/33248621:46
jeblairi like this idea of being upfront in the commit message with commits that add typos21:46
*** clayton has quit IRC21:46
*** jamesmcarthur has quit IRC21:47
*** stevemar has quit IRC21:47
*** clayton has joined #openstack-infra21:47
jeblairit's like setting the evil bit on your packets21:47
*** rossella__ has joined #openstack-infra21:47
*** gouthamr has quit IRC21:47
bkerofungi: none of that grows around here sadly. We just make gin with the stuff that grows around here21:48
fungijpmaxman: strangely, we have a todo comment in the mw config to add a cronjob to post to the RunJobs url http://git.openstack.org/cgit/openstack-infra/puppet-mediawiki/tree/templates/Settings.php.erb#n5621:48
nibalizerjeblair: its a story of the discovery of more typos21:48
*** chlong|rhce_trng has quit IRC21:48
nibalizeri only rechecked it like 4 times to make sure!21:48
*** zz_dimtruck is now known as dimtruck21:49
fungijpmaxman: but that ipv6 address calling it is definitely the one from the local eth0 interface21:49
*** sputnik13 has joined #openstack-infra21:49
nibalizerpleia2: unfortunately, it doesn't look like we have a lot of options for a puppet mediawiki module21:49
*** markvoelker has joined #openstack-infra21:50
pleia2nibalizer: yeah, I did a basic review of the two that exist, hoping that their out of dateness wouldn't matter, but it does :(21:50
pleia2someone should dig deeper, but I don't think we can reasonably use either one21:50
*** dimtruck is now known as zz_dimtruck21:50
pleia2and I'm not keen on adopting an upstream and maintaining things we don't use21:50
*** rossella_ has quit IRC21:51
fungiright, i similarly evaluated the three mentioned in the puppetforge and even the most "active" one hadn't seen a commit to its master branch in over two years21:51
fungiand the one we have is partway there anyway21:51
* pleia2 nods21:52
pleia2and we can keep it simple21:52
fungiwe can certainly borrow relevant ideas from those others too if we hit a head scratcher21:52
pabelangerjeblair: last 2 jobs we on tripleo-test-cloud-rh1, which took about 1 hour more to finish then final job from rax-ord.21:52
*** sree has joined #openstack-infra21:53
*** sputnik13 has quit IRC21:53
*** Rockyg has quit IRC21:55
*** dfflanders has joined #openstack-infra21:55
*** claudiub|2 has quit IRC21:55
*** jordan__ has joined #openstack-infra21:57
*** zeih has joined #openstack-infra21:57
*** sree has quit IRC21:57
*** sputnik13 has joined #openstack-infra21:58
*** jordan__ has quit IRC21:59
*** jamesmca_ has quit IRC22:00
*** esberglu has quit IRC22:00
*** cody-somerville has joined #openstack-infra22:00
*** cody-somerville has joined #openstack-infra22:00
*** sree has joined #openstack-infra22:01
openstackgerritSam Betts proposed openstack-infra/project-config: Add Ironic multinode job  https://review.openstack.org/33249022:01
*** sree_ has joined #openstack-infra22:02
*** sambetts is now known as sambetts|afk22:02
*** sputnik13 has quit IRC22:03
*** sree_ is now known as Guest6418122:03
*** mriedem has quit IRC22:03
*** yaume_ has quit IRC22:05
*** raunak has joined #openstack-infra22:05
*** sputnik13 has joined #openstack-infra22:05
nibalizerany cores wanna push https://review.openstack.org/#/c/332486/ through?22:05
*** sree has quit IRC22:06
*** matt-borland has quit IRC22:07
*** Guest64181 has quit IRC22:07
*** sputnik13 has quit IRC22:07
*** mdrabe has quit IRC22:08
pleia2nibalizer: lgtm22:08
*** ramishra has quit IRC22:10
*** banix has joined #openstack-infra22:10
*** markusry has quit IRC22:11
mtreinishpleia2, nibalizer: any idea why this doesn't ever want to land: https://review.openstack.org/328467 it seems to keep getting stuck on random unrelated failures22:11
mtreinishalthough I'm not the best at reading the output from beaker tests22:11
*** xarses_ is now known as xarses22:12
pleia2mtreinish: hm, there were some issues yesterday, but I thought it had gotten sorted22:12
anteayabeaker tests are currently broken22:13
pleia2ah, still :\22:13
nibalizermtreinish: yea they are broken22:13
nibalizerstill working on it22:13
anteayathis is what nibalizer is trying to fix with the new repo22:13
nibalizerso close though22:13
nibalizerya we're headed for the 'real solution' instead of the quick fix22:13
nibalizerjust experienceing some pain from that22:13
anteayayeah basically some new gem release broke all the things and pinning in all the infra modules is painful22:13
nibalizerpleia2: thanks22:13
anteayaso new repo is a gem that is the source of the pins for all infra puppte modules gems22:14
anteayago go new repo22:14
mtreinishnibalizer: :/ ok thanks22:14
* mtreinish just wants his 1 line config change to land22:14
anteayamtreinish: I hear ya22:14
*** sree has joined #openstack-infra22:15
nibalizermtreinish: yea22:15
*** Sukhdev_ has quit IRC22:16
*** njohnston is now known as njohnston|afk22:17
*** sree has quit IRC22:19
*** tphummel has quit IRC22:19
*** ddieterly is now known as ddieterly[away]22:19
*** vhosakot has quit IRC22:19
*** Sukhdev_ has joined #openstack-infra22:20
mordredmtreinish: I'm secretly blocking your patch just to mess with you ... but sssh, don't tell mtreinish22:21
mtreinishmordred: lol22:21
mtreinishmordred: I think it's been failing to get through the gate since you were away22:22
pleia2mordred: Your father's a scoundrel, and so, it seems, are you.22:22
openstackgerritMerged openstack-infra/project-config: More typos in infra_spec_helper job definition  https://review.openstack.org/33248622:22
*** marcusvrn_ has quit IRC22:23
*** tphummel has joined #openstack-infra22:24
pleia2(too much hamilton soundtrack)22:24
*** shashank_hegde has quit IRC22:24
*** edmondsw has quit IRC22:25
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Fix tempest configuration  https://review.openstack.org/33199722:25
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE TESTING  https://review.openstack.org/31643622:25
mordredpleia2: I'm young, scrappy and hungry22:26
mordredpleia2, notmorgan: btw, Lin-Manuel, Leslie Odom and Phillipa Sou are all leaving July 922:27
pleia2yeah, I just learned about Phillipa Sou :\22:27
pleia2but they said they'd record it before then22:27
mordredI saw all of them except Lin22:27
mordredooh. I would watch the video recording constantly22:28
*** javeriak has joined #openstack-infra22:28
mordredpleia2: fwiw, the understudy is _amazing_ ... so it should still be quite good enough once they22:28
mordredare gone22:28
pleia2that's good22:28
jpmaxmanfungi ok - maybe we can circle back around to that in the wiki upgrade22:29
jpmaxmanso can you suspend the updating of the settings file?22:29
*** haaj has joined #openstack-infra22:29
*** sree has joined #openstack-infra22:30
mordredjpmaxman: one sec, I can help with that22:30
jpmaxmanthanks :D22:30
jpmaxmanpleia2 I took a look at the upgrade doc - it looks good to me and certainly seems a good place to start.  I'm sure there will be other things that will come up, but the sooner it is started...22:31
pleia2jpmaxman: thanks, want to leave a +1? :)22:32
*** banix has quit IRC22:32
openstackgerritMonty Taylor proposed openstack-infra/system-config: Disable puppet on wiki.openstack.org  https://review.openstack.org/33249722:33
mordredpleia2: ^^22:33
pleia2mordred: need a colon in there?22:33
mordredpleia2: thank you22:34
openstackgerritMonty Taylor proposed openstack-infra/system-config: Disable puppet on wiki.openstack.org  https://review.openstack.org/33249722:34
mordrednibalizer: ^^ you got a sec for that?22:34
*** sree has quit IRC22:34
jpmaxmanpleia2 yeah did :)22:35
pleia2jpmaxman: thank you22:35
*** sree has joined #openstack-infra22:35
*** zeih has quit IRC22:35
nibalizermordred: why not just use the emergency file?22:36
mordrednibalizer: cause I think it's gonna be turned off for a bit22:36
mordrednibalizer: while they figure out the sitch22:36
nibalizeri love that we have a random uuid22:36
*** markvoelker has quit IRC22:36
mordredof course we do :)22:37
mordred(I think that's new-gerrit, yeah?)22:37
anteayadoes anyone know what that uuid is?22:37
anteayanew-gerrit? how old is new gerrit?22:37
anteayais there backscroll I need to read?22:37
mordredanteaya: we're doing an OS upgrade in a couple of weeks, right?22:38
*** shashank_hegde has joined #openstack-infra22:39
openstackgerritPaul Belanger proposed openstack-infra/zuul: Remove unused success extra_vars from post_playbook  https://review.openstack.org/33249922:39
*** rcarrillocruz has quit IRC22:39
*** sree has quit IRC22:40
anteayamordred: oh obviously I have reading to do22:40
anteayamordred: I must have missed this bit entirely22:40
*** rcarrillocruz has joined #openstack-infra22:40
openstackgerritPaul Belanger proposed openstack-infra/zuul: Remove unused success extra_vars from post_playbook  https://review.openstack.org/33249922:40
*** sree has joined #openstack-infra22:41
mordredanteaya: nope. I suck.22:41
mordredanteaya: we're updating static and zuul - not review22:41
mordredanteaya: I'm back to not knowing what uuid is22:42
*** ddieterly[away] is now known as ddieterly22:43
anteayamordred: okay22:43
anteayamordred: I'm in sync with you again22:43
*** shashank_hegde has quit IRC22:44
*** ajmiller_ has quit IRC22:44
fungijpmaxman: sorry, stepped away for dinner, but looks like mordred took care of you. as for the runjobs call, i have a strong suspicion it's the cirrussearch plugin, since it runs elasticsearch in a persistent daemon jvm, so it may be making local callouts to mw to do something, just need to take some time to research what22:44
jpmaxmanok cool22:44
*** droyo has quit IRC22:45
jpmaxmanand no problem - all set - we'll see what we can do22:45
*** sree has quit IRC22:45
*** hongbin has quit IRC22:45
*** sree has joined #openstack-infra22:46
*** rossella_ has joined #openstack-infra22:46
*** rbrndt has quit IRC22:47
anteayajenkins failed on the beaker test, or sorry zuul did22:49
anteayaso how long before we make beaker tests on system-config non voting?22:49
fungijpmaxman: suspicion confirmed, sort of... https://www.mediawiki.org/wiki/Thread:Extension_talk:CirrusSearch/How_does_CirrusSearch_refresh_the_index%3F22:49
anteayaI'm in the now camp if anyone is counting votes22:49
*** armax has quit IRC22:50
*** rossella__ has quit IRC22:50
fungiapparently mw calls runjobs _itself_ on each pageload by default, and cirrussearch adds a job hook in there to do its index refreshes22:50
*** sputnik13 has joined #openstack-infra22:51
anteayathis is the beaker is busted issue: http://logs.openstack.org/97/332497/2/check/gate-openstackci-beaker-ubuntu-trusty/83411e8/console.html#_2016-06-21_22_42_14_46751322:51
*** sree has quit IRC22:51
anteayawhich nibalizer is trying to fix but won't be in a postion to have the things fixed for a day or so22:51
fungijpmaxman: which explains the todo comment in our config... if you set it to 0 then you can make a cron job to periodically call that instead, otherwise it calls back to that itself on each page load (highly inefficient) https://www.mediawiki.org/wiki/Thread:Extension_talk:CirrusSearch/How_does_CirrusSearch_refresh_the_index%3F22:51
anteayaestimate * typos22:52
jpmaxmanyes - pleia2 maybe should make that part of the upgrade to move to a cron job22:52
*** xyang1 has quit IRC22:53
pleia2jpmaxman: moving or adding?22:53
openstackgerritMerged openstack-infra/system-config: Disable puppet on wiki.openstack.org  https://review.openstack.org/33249722:53
pleia2we should probably add specifics like this to the etherpad we had going22:53
fungiadding a cron job (moving from having mw call that itself to having a cron job call it)22:53
pleia2oh I understand22:54
jpmaxmanright moving away from mw calls22:54
fungii manually bypassed the broken beaker job for 332497 (which is why it just merged) but i agree with anteaya we should probably make these jobs nonvoting until the fix lands if nibalizer estimates another day for them22:54
anteayafungi: thank you22:54
pleia2adding an Upgrade section to https://etherpad.openstack.org/p/wiki.openstack.org22:55
nibalizerwell its not a day22:55
*** sree has joined #openstack-infra22:55
nibalizerits 'this should have taken a day'22:55
funginibalizer: anything i can help get rolled through to wrap it up?22:55
nibalizerim waiting for a job to be updated in zuul-launcher22:56
nibalizerthen gonna check experimental on this; https://review.openstack.org/#/c/331923/; land it; then land this https://review.openstack.org/#/c/332466/22:57
anteayanibalizer: then create all the patches to fix the jobs, yeah?22:57
nibalizerand once 332466 works and is known to work, then i run explode.sh (hopefully for the last time) to update all the other puppet modules the same as 332466 and then its fixed22:57
anteayanibalizer: how much time do you think?22:57
*** ddieterly has quit IRC22:57
*** tpsilva has quit IRC22:57
fungii'm happy to use gertty batch reviewing on them if they have a consistent topic when you create them all22:58
anteayagiven this issue started morning of Monday and it is end of day Tuesday22:58
* fungi doesn't get enough excuses for batch reviewing22:58
nibalizeranteaya: engineering estimates are the best estimates22:59
anteayanibalizer: always22:59
anteayapersonally I'm thinking Friday22:59
nibalizerbut i think (as long as we have cores around to approve the masses) we should be done by today22:59
anteayathe librarians will kick me out in an hour22:59
*** sree has quit IRC22:59
anteayabut I'm not the core you need for mass approval23:00
funginibalizer: you were waiting for the typos change to propagate i guess? that merged about 40 minutes ago so hopefully is everywhere now23:00
nibalizerfungi: i believe there were more typos23:01
*** javeriak has quit IRC23:01
openstackgerritSpencer Krum proposed openstack-infra/puppet-openstack_infra_spec_helper: Initial commit  https://review.openstack.org/33192323:01
nibalizera couple minor updates to that23:01
*** sree has joined #openstack-infra23:02
nibalizertechnically we can land it now and go with it, but I was trying to get this little test working first23:02
mordrednibalizer: I'm ready to mass-approve23:02
nibalizermordred: im glad23:02
*** chlong has joined #openstack-infra23:02
nibalizerill start working on the explosion23:02
fungii'm on board to help get these through. i have a 750ml bottle of hananomai sake "katana" chilling, but it can stay in the fridge until we're through with this23:03
*** rhochmuth has quit IRC23:03
fungiwhee. i can't load that change because gertty doesn't know about the new repo yet23:04
fungibrowser for now i guess so i don't lose my gertty stack23:04
anteayacrinkle: have you time for a pass of https://review.openstack.org/#/c/331923/623:05
jeblairfungi: have you set up a 'recently seen changes' dashboard?23:05
anteayacrinkle: now with less cruft23:05
fungijeblair: i have, but...23:05
nibalizerwe need to remove the depends on  I think23:05
nibalizerbut the depends on does allow the experimental test to pass23:05
notmorganmordred: =/23:05
anteayanibalizer: I agree, when you are ready to remove it23:05
*** sree has quit IRC23:06
notmorganmordred: i am sad i wont see it with the original cast though!23:06
nibalizeroh i think its totally working23:06
*** stevemar has joined #openstack-infra23:07
crinkleanteaya: sure23:07
nibalizercrinkle: thanks23:07
*** sdague has quit IRC23:08
*** sree_ has joined #openstack-infra23:08
jpmaxmanok I've enabled recaptcha and disabled the new user creation23:09
anteayafungi: would you feel like shoehorning this patch in so Zara can be a storyboard Admin for tomorrow in case it is needed: https://review.openstack.org/#/c/329092/ same beaker failure23:09
*** sree_ is now known as Guest9362223:09
jpmaxmanironically in testing this I had to work really hard to convince launchpad I was not a bot23:09
anteayafungi: sorry to jump on the wagon here23:09
fungijpmaxman: funny indeed23:10
jpmaxmani'm going to monitor this https://wiki.openstack.org/wiki/Special:RecentChanges23:10
fungianteaya: yep, will look in just a sec23:10
jpmaxmanunfortunately it is the old ugly recapthca23:10
anteayafungi: yup, thanks23:10
jpmaxmanI think because confirmedit is bundled with mediawiki and we are on an old version23:10
jpmaxmanbut I'd still like to see how it holds23:11
fungijpmaxman: even ugly old recaptcha can't be any uglier than the built-in captcha, so thanks23:11
funginibalizer: looks like it's passed23:11
*** stevemar has quit IRC23:12
fungiall three experimental jobs passing there23:12
*** Guest93622 has quit IRC23:12
*** gouthamr has joined #openstack-infra23:13
fungianteaya: i thought the idea was that we were going to make the superusers file a new class parameter and have a custom one for storyboard-dev23:14
*** sree has joined #openstack-infra23:14
crinklenibalizer: sorry i should have looked more closely at the gemspec before, there's one gem that's going to cause failures in some modules, otherwise it's fine23:14
anteayafungi: I didn't know that23:14
fungirather than the current state of affairs where the superusers list is shared by both storyboard and storyboard-dev23:14
anteayafungi: oh okay, where might I find an example to point Zara at so she can fix the patch?23:15
fungianteaya: i can update these. it's almost certainly eod for Zara and SotK23:15
fungiworking on it now23:15
anteayathanks so much23:15
anteayaI'm grateful23:15
nibalizercrinkle: ok23:15
fungianteaya: no need to thank me. i want the next couple days for sb to go as smoothly as possible23:16
anteayaoh thank you, me too23:16
nibalizercrinkle: i've wired it up so the 'unit test' just tries to do a bundle install, basically makes sure bundle install works, i figure this tests that we haven't asked for an imposible set of gems to be installed and that we have all the dev libraries needed23:17
nibalizerwe can make it a better name after things are unbroken (since we have to go through project-config to get it fixed)23:17
crinklenibalizer: oh okay23:18
crinklenibalizer: sounds good23:18
*** stevemar has joined #openstack-infra23:18
*** sree has quit IRC23:18
openstackgerritSpencer Krum proposed openstack-infra/puppet-openstack_infra_spec_helper: Initial commit  https://review.openstack.org/33192323:19
nibalizercrinkle: updated23:19
*** sree has joined #openstack-infra23:19
*** sputnik13 has quit IRC23:20
anteayacrinkle: thank you for your fantastic reviews!23:21
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Give storyboard-dev its own superusers list  https://review.openstack.org/33250523:22
fungianteaya: Zara: SotK: i'll rebase the other two changes on that, moving them to the new dev superusers list23:22
nibalizercrinkle: if you're good with it, plaese land 33192323:23
*** sree has quit IRC23:23
nibalizeri'm using the gemfile-sync topic23:23
fungimordred: nice otp23:24
nibalizerreviewers remember that a couple of these look a little different23:24
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Add Zara to StoryBoard Dev Admin  https://review.openstack.org/32909223:24
mordredfungi: yah man. I'm good with them23:24
mordredAND- have now moved the location of the thing23:24
anteayafungi: I think you got the file name wrong in storyboard.pp, or did I misunderstand?23:24
* nibalizer logs into mordreds instagram, posts picture of bud light lime23:24
nibalizercaption "why have I never tried this? It's delicious"23:25
mordrednibalizer: where did you get such a picture?23:25
*** julim has quit IRC23:25
nibalizernot a picture of you drinking it, that would be too hard to forge23:25
pleia2anteaya: the dev.pp is where we want to the dev storyboard one linked (and it is)23:25
* mordred hides the bud light23:25
*** dingyichen has joined #openstack-infra23:25
anteayapleia2: oh so I'm confused?23:25
anteayaokay, thanks23:25
pleia2anteaya: the storyboard.pp file changes are just setting up the environment so $superusers is a variable23:25
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Add Adam Coldrick to StoryBoard Admin  https://review.openstack.org/33207223:25
fungianteaya: pleia2: there's the whole stack done. i'll have a look now and check over it myself23:26
anteayapleia2: ah23:26
anteayathanks fungi23:26
openstackgerritJeremy Stanley proposed openstack-infra/system-config: Add Adam Coldrick to StoryBoard Dev Admin  https://review.openstack.org/33207223:26
fungiforgot to correct the subject on that last one initially23:27
*** sputnik13 has joined #openstack-infra23:27
pleia2fungi: thanks, all lgtm23:27
*** sputnik13 has quit IRC23:27
anteayathank you fungi for patches and to pleia2 and mordred for reviews23:28
fungianteaya: i explained a little in my inline comment23:28
anteayafungi: thanks, I'll read that23:28
mordredfungi, pleia2 - need a +3 on 33207223:28
fungii'll likely still need to bypass the beaker jobs on these changes in the near term23:29
*** shashank_hegde has joined #openstack-infra23:29
fungibut let's see them pass the apply tests first23:29
pleia2makes sense23:29
nibalizercan folks land https://review.openstack.org/#/c/331923/ please23:30
anteayais crinkle happy?23:31
anteayacrinkle: are you happy?23:31
*** zz_dimtruck is now known as dimtruck23:31
anteayaI'm happy if she's happy23:31
*** Jeffrey4l_ has joined #openstack-infra23:32
fungilgtm. looks like ps7 addresses crinkle's comments23:32
anteayanibalizer: do you know if crinkle is around to say she is happy?23:33
anteayanibalizer: or should we move ahead?23:33
openstackgerritMerged openstack-infra/puppet-openstack_infra_spec_helper: Initial commit  https://review.openstack.org/33192323:33
anteayawell that's done then23:33
fungiher comments seemed to be addressed, and if they aren't sufficient we can always add a new change23:34
anteayacool, thanks23:34
anteayanibalizer: what next?23:34
fungibracing for nibalizer's asplode.sh channelspam23:34
*** aysyd has quit IRC23:34
nibalizerseconds away23:35
fungiand then watching us consume every last ounce of nodepool quota23:35
*** sree has joined #openstack-infra23:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-ansible: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33250923:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-apparmor: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33251023:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-apps_site: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33251123:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-askbot: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33251223:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-asterisk: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33251323:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-bandersnatch: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33251423:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-bugdaystats: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33251523:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-bup: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33251623:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-cgit: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33251723:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-ciwatch: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33251823:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-diskimage_builder: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33252023:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-drupal: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33252123:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-elastic_recheck: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33252223:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-elasticsearch: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33252323:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-etherpad_lite: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33252423:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-exim: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33252523:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-gerrit: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33252623:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-gerritbot: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33252723:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-github: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33252823:36
openstackgerritSpencer Krum proposed openstack-infra/puppet-grafyaml: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33252923:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-graphite: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33253023:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-haveged: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33253123:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-hound: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33253223:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-httpd: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33253323:37
*** PollyZ has quit IRC23:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-infracloud: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33253423:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-iptables: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33253523:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-jeepyb: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33253623:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-jenkins: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33253723:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-kerberos: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33253823:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-kibana: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33253923:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-lodgeit: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33254023:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-log_processor: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33254123:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-logrotate: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33254223:37
Shrewsoh my23:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-logstash: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33254323:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-mailman: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33254423:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-mediawiki: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33254523:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-meetbot: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33254623:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-mysql_backup: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33254723:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-nodepool: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33254823:37
nibalizerthis is hopefully the last time I have to do this23:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-odsreg: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33254923:37
funginibalizer: i guess 332466 was updated too soon and needs a recheck?23:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-openafs: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33255023:37
openstackgerritSpencer Krum proposed openstack-infra/puppet-openstack_health: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33255123:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-openstackid: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33255223:38
pleia2^5 nibalizer23:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-os_client_config: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33255323:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-packagekit: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33255423:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-pgsql_backup: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33255523:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-phabricator: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33255623:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-pip: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33255723:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-planet: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33255823:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-project_config: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33255923:38
fungiriiiiide the walrus23:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-puppet: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33256023:38
nibalizerfungi: that was the speciimen, yes it needs a recheck23:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-redis: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33256123:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-refstack: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33256223:38
nibalizerwe shoulda rechecked it before i submitted all these23:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-releasestatus: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33256323:38
nibalizerin case they all fail23:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-reviewday: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33256423:38
nibalizeroh well23:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-simpleproxy: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33256523:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-snmpd: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33256623:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-ssh: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33256723:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-ssl_cert_check: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33256823:38
Shrewsnibalizer: you gonna kill da interwebs!23:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-stackalytics: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33256923:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-statusbot: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33257023:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-storyboard: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33257123:38
openstackgerritSpencer Krum proposed openstack-infra/puppet-subunit2sql: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33257223:39
openstackgerritSpencer Krum proposed openstack-infra/puppet-sudoers: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33257323:39
openstackgerritSpencer Krum proposed openstack-infra/puppet-tmpreaper: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33257423:39
openstackgerritSpencer Krum proposed openstack-infra/puppet-ulimit: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33257523:39
openstackgerritSpencer Krum proposed openstack-infra/puppet-unattended_upgrades: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33257623:39
openstackgerritSpencer Krum proposed openstack-infra/puppet-unbound: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33257723:39
pabelangerjeblair: okay, I see the confusion on 332499, the missing "{{ }}" confused me.23:39
openstackgerritSpencer Krum proposed openstack-infra/puppet-user: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33257823:39
openstackgerritSpencer Krum proposed openstack-infra/puppet-vcsrepo: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33257923:39
openstackgerritSpencer Krum proposed openstack-infra/puppet-vinz: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33258023:39
openstackgerritSpencer Krum proposed openstack-infra/puppet-yum: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33258123:39
openstackgerritSpencer Krum proposed openstack-infra/puppet-zanata: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33258223:39
openstackgerritSpencer Krum proposed openstack-infra/puppet-zuul: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33258323:39
*** salv-orlando has joined #openstack-infra23:39
Nakatojeblair: https://review.openstack.org/#/c/293194/23:39
anteayanibalizer: I hope they pass23:40
nibalizeranteaya:  me too23:40
*** sree has quit IRC23:41
anteayawell nothing more for me to do tonight I don't think, I'm going to pack up and leave the library23:41
anteayapleia2: text me if you want me to come online once I get home23:41
anteayag'night and thank you for all the help23:41
*** salv-orl_ has quit IRC23:42
funginibalizer: puppet-lint and beaker-rspec-centos-7 seem unhappy on them so far23:42
nibalizeror T_T23:42
nibalizeri'm not super familiar with that emoji23:42
nibalizerokay lets see what we borke23:42
fungiacross the board, so hopefully something simple?23:42
nibalizerreally really shoulda rechecked the accessbot one before sending the hoard23:42
mtreinishnibalizer: what's the fun in doing that? ;)23:43
*** yamahata has quit IRC23:43
nibalizermtreinish: ikr23:43
funginibalizer: beaker-ubuntu-trusty failing on them too23:43
* bkero waits for patchset 2 to every module23:44
pleia2anteaya: have a good evening23:44
*** amitgandhinz has joined #openstack-infra23:45
*** sree has joined #openstack-infra23:45
*** salv-orlando has quit IRC23:46
*** hosanai has joined #openstack-infra23:47
nibalizeri think ivet got it23:47
nibalizeryep patchset to every module23:47
nibalizerpuppet-accessbot to start23:47
openstackgerritSpencer Krum proposed openstack-infra/puppet-accessbot: Use new infra_spec_helper for gem dependencies  https://review.openstack.org/33246623:47
fungitesting one first: golden ;)23:47
nibalizerfungi: okay lets see if that works23:48
fungionce our test resources recover sufficiently from the first round anyway23:48
fungithough i have a somewhat unorthodox solution to speed this up. just a sec23:49
*** sree has quit IRC23:50
nibalizerhaha yesss23:50
*** amitgandhinz has quit IRC23:51
openstackgerritKhai Do proposed openstack-infra/system-config: disable puppet to install its-storyboard plugin  https://review.openstack.org/33258523:51
fungiit's testing that one with gate priority now23:51
zaropleia2: ^ would you mind? i forgot we need to turn off puppet on review-dev to install the plugin23:51
fungiif it works, you can napalm-carpet the others with new patchsets so the old ones get aborted quicker and free resources back to the pool23:52
*** sree has joined #openstack-infra23:52
zarothanks nibalizer23:53
nibalizerzaro: o723:53
nibalizerfungi: yep23:53
zaronibalizer: is this the pattern to move params into it's own file?23:55
nibalizerwell thats all puppet 4 stuff23:55
*** sree has quit IRC23:56
nibalizerto move params to its own file23:56
fungiall green, waiting for the last job to complete23:56
fungino, wait23:56
*** sree has joined #openstack-infra23:57
funginibalizer: "LoadError: cannot load such file -- puppet" http://logs.openstack.org/66/332466/2/gate/gate-puppet-accessbot-puppet-lint/d4237ae/console.html#_2016-06-21_23_55_32_82136323:57
zaronibalizer: looks like it's the same.23:58
funginibalizer: also failed gate-openstackci-beaker-ubuntu-trusty23:58
zaronibalizer: looking at https://github.com/puppetlabs/puppetlabs-apache/blob/master/manifests/init.pp it looks like all vars need to be in the class that consumes the params.  doesn't seem like much benefit.23:59
mat128PSTN line is busy23:59
fungipabelanger: ^ any chance you can check on pbx.o.o?23:59
zaronibalizer: or am i missing something?23:59
*** tphummel has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!