| *** s-shiono has joined #openstack-infra | 00:01 | |
| *** ijw__ has quit IRC | 00:01 | |
| clarkb | but it looks like the sys v compat generator system respects ordering (unlike upstarts) | 00:01 |
|---|---|---|
| *** jascott1 has quit IRC | 00:05 | |
| *** SumitNaiksatam has quit IRC | 00:05 | |
| *** Apoorva_ has quit IRC | 00:05 | |
| *** gmann has quit IRC | 00:06 | |
| *** gmann has joined #openstack-infra | 00:06 | |
| *** Apoorva has joined #openstack-infra | 00:06 | |
| *** s-shiono_ has joined #openstack-infra | 00:07 | |
| *** yee37922 has joined #openstack-infra | 00:09 | |
| *** openstack has joined #openstack-infra | 00:16 | |
| *** szaher has quit IRC | 00:16 | |
| *** jcook has joined #openstack-infra | 00:16 | |
| *** s-shiono_ has quit IRC | 00:17 | |
| *** neiljerram has quit IRC | 00:17 | |
| *** zigo has quit IRC | 00:17 | |
| *** gyee has quit IRC | 00:17 | |
| *** aviau has quit IRC | 00:17 | |
| *** jaosorior has quit IRC | 00:17 | |
| *** laurelm has quit IRC | 00:17 | |
| *** schang has quit IRC | 00:17 | |
| *** vgnbkr has quit IRC | 00:17 | |
| *** peterstac has quit IRC | 00:17 | |
| *** Keitaro has quit IRC | 00:17 | |
| *** mikal has quit IRC | 00:17 | |
| *** vipul has quit IRC | 00:17 | |
| *** aviau has joined #openstack-infra | 00:17 | |
| *** neiljerram has joined #openstack-infra | 00:17 | |
| *** sambetts_ has quit IRC | 00:17 | |
| *** patriciadomin_ has quit IRC | 00:17 | |
| *** charz has quit IRC | 00:17 | |
| *** charz has joined #openstack-infra | 00:18 | |
| *** puiterwi1 is now known as puiterwijk | 00:18 | |
| *** dingyichen has joined #openstack-infra | 00:19 | |
| *** Keitaro has joined #openstack-infra | 00:19 | |
| *** breton has joined #openstack-infra | 00:19 | |
| *** atarakt has joined #openstack-infra | 00:19 | |
| *** pfallenop has joined #openstack-infra | 00:19 | |
| *** pfallenop has joined #openstack-infra | 00:19 | |
| *** sambetts_ has joined #openstack-infra | 00:19 | |
| *** patriciadomin has joined #openstack-infra | 00:20 | |
| *** PramodJ has quit IRC | 00:22 | |
| *** niska has quit IRC | 00:23 | |
| *** strigazi has joined #openstack-infra | 00:24 | |
| *** s-shiono_ has joined #openstack-infra | 00:24 | |
| *** zigo has joined #openstack-infra | 00:24 | |
| *** gyee has joined #openstack-infra | 00:24 | |
| *** laurelm has joined #openstack-infra | 00:24 | |
| *** schang has joined #openstack-infra | 00:24 | |
| *** vgnbkr has joined #openstack-infra | 00:24 | |
| *** peterstac has joined #openstack-infra | 00:24 | |
| *** vipul has joined #openstack-infra | 00:24 | |
| clarkb | I'm trying to test a unit but its slow going because my test machine decided to get all slow | 00:25 |
| *** szaher has joined #openstack-infra | 00:25 | |
| *** thorst has joined #openstack-infra | 00:26 | |
| *** thorst has quit IRC | 00:27 | |
| *** niska has joined #openstack-infra | 00:33 | |
| *** Apoorva_ has joined #openstack-infra | 00:35 | |
| *** yuanying has quit IRC | 00:36 | |
| *** xarses_ has joined #openstack-infra | 00:37 | |
| *** Apoorva has quit IRC | 00:37 | |
| *** xarses has quit IRC | 00:39 | |
| *** jaosorior has joined #openstack-infra | 00:39 | |
| *** Apoorva_ has quit IRC | 00:39 | |
| *** andreww has joined #openstack-infra | 00:41 | |
| *** ijw has joined #openstack-infra | 00:41 | |
| *** LindaWang has joined #openstack-infra | 00:41 | |
| *** amotoki_away is now known as amotoki | 00:41 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] use ci mirrors https://review.openstack.org/475607 | 00:42 |
| *** ijw has quit IRC | 00:43 | |
| *** ijw has joined #openstack-infra | 00:43 | |
| *** xarses_ has quit IRC | 00:44 | |
| *** askb has joined #openstack-infra | 00:46 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] use ci mirrors https://review.openstack.org/475607 | 00:47 |
| *** bradm_ is now known as bradm | 00:47 | |
| *** rwsu has quit IRC | 00:48 | |
| *** aviau has quit IRC | 00:48 | |
| *** aviau has joined #openstack-infra | 00:48 | |
| *** Apoorva has joined #openstack-infra | 00:49 | |
| mordred | clarkb, fungi: seems like info we know at *sigh* glean time | 00:51 |
| *** claudiub has quit IRC | 00:52 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] use ci mirrors https://review.openstack.org/475607 | 00:52 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: Use IP version appropriate dns servers https://review.openstack.org/475608 | 00:52 |
| *** Julien-zte has joined #openstack-infra | 00:52 | |
| mordred | oh neat | 00:52 |
| clarkb | mordred: I think ^ will work but its not tested in a dib and boot context | 00:53 |
| mordred | clarkb: ok. that seems non-terrible | 00:53 |
| clarkb | so far just in a modify running machine context | 00:53 |
| *** Apoorva has quit IRC | 00:53 | |
| clarkb | I need to pop out for dinner now, but will try to test in the morning | 00:53 |
| clarkb | that also won't affect trusty but meh | 00:53 |
| *** Apoorva has joined #openstack-infra | 00:54 | |
| *** stakeda has joined #openstack-infra | 00:54 | |
| clarkb | mordred: also I don't think glean should bother with this, since glean handles the default scenario just fine which is configure it given what we know from cloud/dhcp | 00:55 |
| clarkb | mordred: we explicitly want to override that behavior so seems to make sense to not have it in glean | 00:55 |
| *** amotoki is now known as amotoki_away | 00:55 | |
| tonyb_ | 2~ | 00:55 |
| *** cuongnv has joined #openstack-infra | 00:56 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] use ci mirrors https://review.openstack.org/475607 | 00:56 |
| *** tonyb_ is now known as tonyb | 00:57 | |
| *** Apoorva has quit IRC | 00:58 | |
| *** jkilpatr has joined #openstack-infra | 00:59 | |
| *** ijw has quit IRC | 01:02 | |
| *** harlowja has joined #openstack-infra | 01:04 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] use ci mirrors https://review.openstack.org/475607 | 01:04 |
| *** jascott1 has joined #openstack-infra | 01:05 | |
| *** timburke has quit IRC | 01:07 | |
| *** charz has quit IRC | 01:07 | |
| clarkb | I just did a quick reboot test (still no dib and just on xenial) and that actually seems to work | 01:09 |
| *** ijw has joined #openstack-infra | 01:09 | |
| clarkb | double checked with unbound-control list_fowards too | 01:10 |
| clarkb | so I think the concept is worth review | 01:10 |
| *** sree has joined #openstack-infra | 01:11 | |
| *** xyang1 has quit IRC | 01:12 | |
| *** rwsu has joined #openstack-infra | 01:12 | |
| *** liujiong has joined #openstack-infra | 01:13 | |
| *** namnh has joined #openstack-infra | 01:14 | |
| *** jamesmca_ has quit IRC | 01:14 | |
| *** ijw has quit IRC | 01:15 | |
| *** sree has quit IRC | 01:16 | |
| *** zzzeek has quit IRC | 01:18 | |
| *** zzzeek has joined #openstack-infra | 01:18 | |
| *** ijw has joined #openstack-infra | 01:21 | |
| *** StevenK_ is now known as StevenK | 01:22 | |
| *** cshastri has joined #openstack-infra | 01:23 | |
| ianw | i think mirror.sto2.citycloud.openstack.org is upset again | 01:24 |
| ianw | it's downloading a < 100kb/s or so | 01:25 |
| *** ijw has quit IRC | 01:26 | |
| *** markvoelker has quit IRC | 01:27 | |
| *** Kevin_Zheng has joined #openstack-infra | 01:30 | |
| clarkb | thats normal after a vos release :( | 01:30 |
| ianw | oh, yeah that might be it i guess | 01:30 |
| clarkb | ianw: Ome thing I'd like to try is the reverse proxy setup but back to the dfw mirror since it is adjacent to the afs servers | 01:31 |
| clarkb | and see if that performs better | 01:31 |
| *** Swami has quit IRC | 01:31 | |
| *** xinliang has quit IRC | 01:31 | |
| *** zz_dimtruck is now known as dimtruck | 01:32 | |
| *** mriedem has quit IRC | 01:33 | |
| *** xinliang has joined #openstack-infra | 01:34 | |
| *** xinliang has joined #openstack-infra | 01:34 | |
| *** thorst has joined #openstack-infra | 01:38 | |
| *** rfolco has joined #openstack-infra | 01:39 | |
| *** ijw has joined #openstack-infra | 01:40 | |
| *** xinliang_ has joined #openstack-infra | 01:44 | |
| *** Goneri has joined #openstack-infra | 01:45 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] use ci mirrors https://review.openstack.org/475607 | 01:47 |
| *** ijw has quit IRC | 01:48 | |
| *** yamahata_ has quit IRC | 01:53 | |
| *** thorst has quit IRC | 01:56 | |
| *** thorst has joined #openstack-infra | 01:56 | |
| *** esberglu has quit IRC | 02:00 | |
| *** thorst has quit IRC | 02:00 | |
| *** bobh has joined #openstack-infra | 02:01 | |
| *** amotoki_away is now known as amotoki | 02:01 | |
| openstackgerrit | Yolande Amate proposed openstack-infra/jenkins-job-builder master: Update run_param to make use of convert_mapping_to_xml() https://review.openstack.org/475613 | 02:02 |
| *** xinliang has quit IRC | 02:02 | |
| *** dhinesh has quit IRC | 02:03 | |
| *** amotoki is now known as amotoki_away | 02:11 | |
| *** dpaterson has quit IRC | 02:12 | |
| *** dpaterson has joined #openstack-infra | 02:13 | |
| *** yamamoto has joined #openstack-infra | 02:17 | |
| openstackgerrit | Yolande Amate proposed openstack-infra/jenkins-job-builder master: Update extended_choice_param to make use of convert_mapping_to_xml() https://review.openstack.org/475616 | 02:19 |
| *** armax has joined #openstack-infra | 02:19 | |
| *** cuongnv_ has joined #openstack-infra | 02:21 | |
| *** pahuang has quit IRC | 02:22 | |
| *** gomarivera has joined #openstack-infra | 02:23 | |
| *** Goneri has quit IRC | 02:23 | |
| *** cuongnv has quit IRC | 02:24 | |
| *** pahuang has joined #openstack-infra | 02:27 | |
| *** Guest8722 is now known as sdake | 02:29 | |
| *** sdake has joined #openstack-infra | 02:29 | |
| *** hongbin has joined #openstack-infra | 02:31 | |
| openstackgerrit | Yolande Amate proposed openstack-infra/jenkins-job-builder master: Update validating_string_param to make use of convert_mapping_to_xml() https://review.openstack.org/475620 | 02:32 |
| *** raginbajin has quit IRC | 02:33 | |
| *** masber has joined #openstack-infra | 02:34 | |
| *** ramishra has joined #openstack-infra | 02:35 | |
| *** yuanying has joined #openstack-infra | 02:35 | |
| *** amotoki_away is now known as amotoki | 02:36 | |
| *** bobh has quit IRC | 02:36 | |
| *** masuberu has quit IRC | 02:37 | |
| *** masuberu has joined #openstack-infra | 02:38 | |
| *** raginbajin has joined #openstack-infra | 02:39 | |
| *** gyee has quit IRC | 02:41 | |
| *** masber has quit IRC | 02:42 | |
| openstackgerrit | Yolande Amate proposed openstack-infra/jenkins-job-builder master: Update maven_metadata_param to make use of convert_mapping_to_xml() https://review.openstack.org/475623 | 02:44 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a static driver for Nodepool https://review.openstack.org/468624 | 02:46 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Extend Nodepool configuration syntax to support multiple drivers https://review.openstack.org/468751 | 02:46 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Collect request handling implementation in an OpenStack driver https://review.openstack.org/468750 | 02:46 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Abstract Nodepool provider management code https://review.openstack.org/468749 | 02:46 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Abstract Nodepool request handling code https://review.openstack.org/468748 | 02:46 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement an OpenContainer driver https://review.openstack.org/468753 | 02:46 |
| *** cuongnv_ has quit IRC | 02:57 | |
| *** gomarivera has quit IRC | 02:58 | |
| *** gongysh has joined #openstack-infra | 02:59 | |
| *** gomarivera has joined #openstack-infra | 03:01 | |
| *** yuanying has quit IRC | 03:04 | |
| *** armax has quit IRC | 03:04 | |
| *** yuanying has joined #openstack-infra | 03:04 | |
| *** aeng has quit IRC | 03:07 | |
| *** cuongnv has joined #openstack-infra | 03:08 | |
| *** dhajare has joined #openstack-infra | 03:08 | |
| *** yuanying has quit IRC | 03:09 | |
| *** armax has joined #openstack-infra | 03:10 | |
| *** rfolco has quit IRC | 03:13 | |
| *** dhinesh has joined #openstack-infra | 03:16 | |
| openstackgerrit | Yolande Amate proposed openstack-infra/jenkins-job-builder master: Update builds_chain_fingerprinter to use of convert_mapping_to_xml() https://review.openstack.org/475631 | 03:17 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] use ci mirrors https://review.openstack.org/475607 | 03:20 |
| openstackgerrit | Yolande Amate proposed openstack-infra/jenkins-job-builder master: Update builds_chain_fingerprinter to use convert_xml() https://review.openstack.org/475631 | 03:21 |
| *** amotoki is now known as amotoki_away | 03:24 | |
| *** nicolasbock has joined #openstack-infra | 03:27 | |
| *** ykarel has joined #openstack-infra | 03:28 | |
| *** markvoelker has joined #openstack-infra | 03:28 | |
| *** tristanC has quit IRC | 03:29 | |
| *** tristanC has joined #openstack-infra | 03:31 | |
| *** dhajare has quit IRC | 03:40 | |
| mtreinish | clarkb: I saw your ping from last week about that and reviewed it already :) | 03:41 |
| mtreinish | clarkb: and I'm sort of back online, I'm at linuxcon china right now. I head back to the US on thurs | 03:41 |
| openstackgerrit | Yolande Amate proposed openstack-infra/jenkins-job-builder master: Update least_load to make use of convert_mapping_to_xml() https://review.openstack.org/475635 | 03:42 |
| clarkb | mtreinish: oh did you? I completely missed ut | 03:46 |
| *** winggundamth has joined #openstack-infra | 03:46 | |
| *** gomarivera has quit IRC | 03:48 | |
| cloudnull | clarkb: none that I'm aware of. | 03:48 |
| cloudnull | ^RE: firewall throttling | 03:49 |
| *** gomarivera has joined #openstack-infra | 03:49 | |
| *** Guest66617 has quit IRC | 03:55 | |
| cloudnull | clarkb: are the instances failing with dns issues also running systemd-resolvd ? | 03:56 |
| *** dhajare has joined #openstack-infra | 03:56 | |
| cloudnull | asking because i found this https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1449001 maybe related failure | 03:56 |
| openstack | Launchpad bug 1449001 in systemd (Ubuntu) "systemd-resolved: please do not use Google public DNS by default" [Low,Triaged] | 03:56 |
| *** thorst has joined #openstack-infra | 03:57 | |
| *** links has joined #openstack-infra | 03:58 | |
| *** thorst has quit IRC | 04:01 | |
| clarkb | cloudnull: no they shouldnt be since ubuntu 16.04 doesnt have it enables by default | 04:03 |
| *** gouthamr has quit IRC | 04:03 | |
| cloudnull | but the instances are still using google DNS ? | 04:04 |
| cloudnull | its not hitting opendns ? | 04:04 |
| clarkb | cloudnull: using both looks lik unbound doesnt prioritize by orsrr | 04:04 |
| clarkb | *by orser | 04:04 |
| clarkb | gah order | 04:05 |
| cloudnull | ah | 04:05 |
| clarkb | and so uses ipv4 and ipv6 too | 04:05 |
| clarkb | and thinking ipv4 through nat is problem | 04:05 |
| clarkb | I'm working a change to only use ipv6 | 04:05 |
| cloudnull | its possible ipv4 nat is the issue | 04:05 |
| cloudnull | the instance count in the cloud has been growing recently so maybe its hitting some throughput limitation? | 04:07 |
| cloudnull | that said, i've been having dns issues in rax cloud too, so maybe there's some larger issue going on ? | 04:08 |
| cloudnull | the rax cloud was ORD and the osic is in DFW | 04:09 |
| clarkb | yq could be other things still trying to narrow it down | 04:11 |
| *** csomerville has joined #openstack-infra | 04:14 | |
| *** cody-somerville has quit IRC | 04:17 | |
| *** hongbin has quit IRC | 04:17 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] use ci mirrors https://review.openstack.org/475607 | 04:19 |
| openstackgerrit | Tin Lam proposed openstack-infra/project-config master: Adding linter and check to OpenStack-Helm-AddOn repo https://review.openstack.org/475641 | 04:23 |
| *** eranrom has joined #openstack-infra | 04:28 | |
| *** gomarivera has quit IRC | 04:29 | |
| *** gomarivera has joined #openstack-infra | 04:30 | |
| *** gomarivera has quit IRC | 04:33 | |
| *** gomarivera has joined #openstack-infra | 04:33 | |
| openstackgerrit | Tin Lam proposed openstack-infra/project-config master: Adding linter and check to openstack-helm-addons repo https://review.openstack.org/475641 | 04:33 |
| *** gildub has quit IRC | 04:34 | |
| *** bhavik1 has joined #openstack-infra | 04:35 | |
| *** mdrabe has quit IRC | 04:40 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] use ci mirrors https://review.openstack.org/475607 | 04:45 |
| *** amotoki_away is now known as amotoki | 04:48 | |
| *** salv-orlando has joined #openstack-infra | 04:48 | |
| *** salv-orlando has quit IRC | 04:49 | |
| *** salv-orlando has joined #openstack-infra | 04:49 | |
| *** gildub has joined #openstack-infra | 04:51 | |
| *** salv-orlando has quit IRC | 04:51 | |
| *** salv-orlando has joined #openstack-infra | 04:51 | |
| *** winggundamth has quit IRC | 04:56 | |
| *** makowals has quit IRC | 04:56 | |
| *** gildub has quit IRC | 04:56 | |
| *** gongysh has quit IRC | 04:56 | |
| *** yamahata_ has joined #openstack-infra | 04:58 | |
| *** amotoki is now known as amotoki_away | 05:00 | |
| *** kzaitsev_mb has joined #openstack-infra | 05:02 | |
| *** winggundamth has joined #openstack-infra | 05:08 | |
| *** psachin has joined #openstack-infra | 05:09 | |
| *** pleia2_ is now known as pleia2 | 05:09 | |
| *** yuanying has joined #openstack-infra | 05:11 | |
| *** makowals has joined #openstack-infra | 05:12 | |
| *** pcaruana has joined #openstack-infra | 05:14 | |
| *** salv-orl_ has joined #openstack-infra | 05:15 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] use ci mirrors https://review.openstack.org/475607 | 05:16 |
| *** ltomasbo|away is now known as ltomasbo | 05:17 | |
| *** salv-orlando has quit IRC | 05:18 | |
| *** gomarivera has quit IRC | 05:20 | |
| *** gomarivera has joined #openstack-infra | 05:21 | |
| *** prateek has joined #openstack-infra | 05:21 | |
| *** winggundamth has quit IRC | 05:25 | |
| *** gomarivera has quit IRC | 05:25 | |
| *** sree has joined #openstack-infra | 05:29 | |
| *** eranrom has quit IRC | 05:29 | |
| *** pcaruana has quit IRC | 05:30 | |
| *** gongysh has joined #openstack-infra | 05:30 | |
| kornicameister | hello once again | 05:34 |
| kornicameister | looking for some help around tox_install utility | 05:34 |
| *** frickler_ is now known as frickler | 05:35 | |
| *** cshastri has quit IRC | 05:38 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add kafka plugin to the gate tests (non-voting) https://review.openstack.org/473528 | 05:43 |
| openstackgerrit | Merged openstack-infra/project-config master: networking-midonet: Add experimental jobs for centos-7 https://review.openstack.org/467925 | 05:43 |
| *** e0ne has joined #openstack-infra | 05:44 | |
| openstackgerrit | Merged openstack-infra/project-config master: networking-midonet: Disable rally jobs for stable/newton https://review.openstack.org/467515 | 05:45 |
| openstackgerrit | Merged openstack-infra/project-config master: networking-midonet: remove -nv from job regex https://review.openstack.org/467866 | 05:45 |
| AJaeger | kornicameister: what's the problem? | 05:47 |
| kornicameister | AJaeger: hi, the problem is that python gates do not take Depends-On into account while setting up tox environment within gate | 05:48 |
| kornicameister | I had this problem here => https://review.openstack.org/#/c/465532/ | 05:48 |
| kornicameister | and finally managed to solve it via custom code | 05:48 |
| kornicameister | it seems like this "issue" (at least from my POV) is shared across python jobs | 05:48 |
| *** e0ne has quit IRC | 05:49 | |
| AJaeger | kornicameister: did you read https://docs.openstack.org/infra/manual/developers.html#limitations-and-caveats ? | 05:49 |
| kornicameister | AJaeger, nope...checking now | 05:49 |
| kornicameister | AJaeger: so how can I make gates pass in changes like the one I posted ? | 05:50 |
| AJaeger | kornicameister: it really depends on what you want - test against *released* tar balls on pypi or against master. | 05:50 |
| kornicameister | well in this particular case I wouldn't be able to close this change down (see link above) because it needs latest code changes | 05:51 |
| kornicameister | otherwise it fails | 05:51 |
| AJaeger | tox_install is what most projects do if they *always* want to test against master | 05:51 |
| AJaeger | kornicameister: you could: Merge the dependency, do a release - and then merge your change | 05:51 |
| AJaeger | and that's the way the consumers of your code will do it. | 05:51 |
| AJaeger | So, for this case, I strongly advise to relase the python-monascaclient first so that everybody using the UI will have the dependency | 05:52 |
| kornicameister | AJaeger: thx, I guess I will need to talk about that with other monasca guys, to see how we can proceed | 05:53 |
| kornicameister | thx | 05:53 |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Add an experimental grenade job with v2 plugin https://review.openstack.org/464172 | 05:54 |
| *** cshastri has joined #openstack-infra | 05:55 | |
| *** kzaitsev_mb has quit IRC | 05:56 | |
| *** thorst has joined #openstack-infra | 05:58 | |
| *** markvoelker has quit IRC | 06:01 | |
| *** thorst has quit IRC | 06:02 | |
| kornicameister | AJaeger: if you don't mind I sent you PM about other issue we have in monasca | 06:02 |
| *** cshastri has quit IRC | 06:04 | |
| *** gaurangt has quit IRC | 06:05 | |
| *** gaurangt has joined #openstack-infra | 06:05 | |
| *** yolanda has joined #openstack-infra | 06:06 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Add an experimental grenade job with v2 plugin https://review.openstack.org/464172 | 06:10 |
| *** florianf has joined #openstack-infra | 06:10 | |
| *** armax has quit IRC | 06:10 | |
| *** cshastri has joined #openstack-infra | 06:16 | |
| *** dhinesh has quit IRC | 06:21 | |
| *** markus_z has joined #openstack-infra | 06:23 | |
| *** claudiub has joined #openstack-infra | 06:26 | |
| *** armax has joined #openstack-infra | 06:27 | |
| *** amotoki_away is now known as amotoki | 06:31 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at using CI mirrors for fedora/centos https://review.openstack.org/475607 | 06:32 |
| *** cshastri has quit IRC | 06:39 | |
| *** cshastri has joined #openstack-infra | 06:40 | |
| *** tesseract has joined #openstack-infra | 06:40 | |
| *** gongysh has quit IRC | 06:41 | |
| *** tumbarka has quit IRC | 06:45 | |
| *** umbarkar has joined #openstack-infra | 06:45 | |
| *** amotoki is now known as amotoki_away | 06:49 | |
| *** links has quit IRC | 06:49 | |
| *** pcaruana has joined #openstack-infra | 06:49 | |
| *** nunchuck has joined #openstack-infra | 06:53 | |
| *** bhavik1 has quit IRC | 06:53 | |
| *** rcernin_ has joined #openstack-infra | 06:53 | |
| *** salv-orl_ has quit IRC | 07:01 | |
| *** salv-orlando has joined #openstack-infra | 07:02 | |
| *** gongysh has joined #openstack-infra | 07:04 | |
| *** e0ne has joined #openstack-infra | 07:04 | |
| *** hashar has joined #openstack-infra | 07:05 | |
| *** links has joined #openstack-infra | 07:05 | |
| *** amotoki_away is now known as amotoki | 07:06 | |
| *** jgrassle1 is now known as jgrassler | 07:06 | |
| *** salv-orlando has quit IRC | 07:07 | |
| *** gaurangt has quit IRC | 07:08 | |
| *** cshen has quit IRC | 07:09 | |
| *** chason_ has quit IRC | 07:09 | |
| *** salv-orlando has joined #openstack-infra | 07:09 | |
| *** MatthewLi1 has joined #openstack-infra | 07:11 | |
| *** MatthewLi has quit IRC | 07:12 | |
| *** MatthewLi1 is now known as MatthewLi | 07:12 | |
| *** alexchadin has joined #openstack-infra | 07:13 | |
| openstackgerrit | Merged openstack-infra/subunit2sql master: Dailycount should not need build_queue to exist. https://review.openstack.org/474701 | 07:13 |
| openstackgerrit | Merged openstack-infra/subunit2sql master: Add vcs status to version option https://review.openstack.org/475275 | 07:13 |
| openstackgerrit | Merged openstack-infra/subunit2sql master: Update docs theme https://review.openstack.org/470849 | 07:13 |
| *** gaurangt has joined #openstack-infra | 07:14 | |
| *** jpich has joined #openstack-infra | 07:15 | |
| *** e0ne has quit IRC | 07:16 | |
| *** cmurphy_ is now known as cmurphy | 07:18 | |
| *** yamahata_ has quit IRC | 07:18 | |
| *** ijw has joined #openstack-infra | 07:21 | |
| *** amotoki is now known as amotoki_away | 07:22 | |
| *** eranrom has joined #openstack-infra | 07:25 | |
| *** ijw has quit IRC | 07:25 | |
| *** amotoki_away is now known as amotoki | 07:29 | |
| *** dtantsur|afk is now known as dtantsur | 07:30 | |
| *** dave-mccowan has quit IRC | 07:30 | |
| *** rkukura has quit IRC | 07:31 | |
| *** sbezverk has quit IRC | 07:31 | |
| *** markus_z has quit IRC | 07:31 | |
| *** rkukura has joined #openstack-infra | 07:32 | |
| *** dizquierdo has joined #openstack-infra | 07:32 | |
| *** pcaruana|afk| has joined #openstack-infra | 07:35 | |
| *** pcaruana|afk| has quit IRC | 07:35 | |
| *** pcaruana has quit IRC | 07:36 | |
| *** dave-mccowan has joined #openstack-infra | 07:36 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at using CI mirrors for fedora/centos https://review.openstack.org/475607 | 07:37 |
| *** rkukura has quit IRC | 07:37 | |
| *** jpena|off is now known as jpena | 07:37 | |
| *** hashar is now known as hasharAway | 07:38 | |
| *** amotoki is now known as amotoki_away | 07:39 | |
| *** pcaruana has joined #openstack-infra | 07:40 | |
| *** hasharAway has quit IRC | 07:41 | |
| *** markus_z has joined #openstack-infra | 07:50 | |
| *** ralonsoh has joined #openstack-infra | 07:50 | |
| *** rkukura has joined #openstack-infra | 07:50 | |
| *** amotoki_away is now known as amotoki | 07:53 | |
| openstackgerrit | Daniel Mellado proposed openstack-infra/project-config master: Create tempest non-voting gate in kuryr-kubernetes https://review.openstack.org/474993 | 07:54 |
| *** links has quit IRC | 07:55 | |
| *** SpamapS has quit IRC | 07:56 | |
| *** rossella_ has joined #openstack-infra | 07:58 | |
| *** thorst has joined #openstack-infra | 07:58 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** rossella_s has joined #openstack-infra | 08:02 | |
| *** thorst has quit IRC | 08:03 | |
| *** rossella_ has quit IRC | 08:04 | |
| *** oomichi has quit IRC | 08:05 | |
| *** oomichi has joined #openstack-infra | 08:07 | |
| *** amoralej|off is now known as amoralej | 08:08 | |
| *** links has joined #openstack-infra | 08:08 | |
| openstackgerrit | Merged openstack-infra/project-config master: Make the Ironic multinode grenade job voting https://review.openstack.org/475521 | 08:10 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test https://review.openstack.org/475696 | 08:12 |
| *** Guest39557 is now known as asettle | 08:16 | |
| *** rvba has quit IRC | 08:18 | |
| openstackgerrit | Merged openstack-infra/project-config master: networking-midonet: Add an experimental grenade job with v2 plugin https://review.openstack.org/464172 | 08:20 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Use cirros images on host https://review.openstack.org/475576 | 08:20 |
| openstackgerrit | Merged openstack-infra/project-config master: define a unified project documentation build job https://review.openstack.org/471881 | 08:21 |
| *** cuongnv has quit IRC | 08:21 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: testing featurset of containers https://review.openstack.org/473082 | 08:22 |
| openstackgerrit | Merged openstack-infra/project-config master: Adding linter and check to openstack-helm-addons repo https://review.openstack.org/475641 | 08:22 |
| openstackgerrit | Merged openstack-infra/project-config master: Move manila cephfs jobs to check queue https://review.openstack.org/475411 | 08:22 |
| openstackgerrit | Merged openstack-infra/project-config master: Bump Ansible 2.1 from 2.1.5 to 2.1.6 for ARA gate jobs https://review.openstack.org/475575 | 08:24 |
| openstackgerrit | Merged openstack-infra/project-config master: new project - Adjutant https://review.openstack.org/474867 | 08:25 |
| *** mhickey has joined #openstack-infra | 08:26 | |
| *** rvba has joined #openstack-infra | 08:26 | |
| *** rvba has quit IRC | 08:26 | |
| *** rvba has joined #openstack-infra | 08:26 | |
| *** s-shiono_ has quit IRC | 08:26 | |
| *** priteau has joined #openstack-infra | 08:27 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add neutron-fwaas-dashboard project https://review.openstack.org/475163 | 08:32 |
| *** derekh has joined #openstack-infra | 08:32 | |
| *** bakbad has joined #openstack-infra | 08:33 | |
| *** lucas-afk is now known as lucasagomes | 08:34 | |
| andreaf | mtreinish, sdague: https://review.openstack.org/#/c/473851/ ? | 08:35 |
| andreaf | e-r review ^^^ | 08:35 |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder master: disk-image-create flag that receive packages to be removed before install phase https://review.openstack.org/475701 | 08:35 |
| openstackgerrit | Merged openstack-infra/project-config master: Add neutron-vpnaas-dashboard project https://review.openstack.org/475165 | 08:38 |
| *** tmorin has joined #openstack-infra | 08:39 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add new repo openstack-helm-infra under OSH https://review.openstack.org/469079 | 08:39 |
| *** efoley has joined #openstack-infra | 08:39 | |
| *** fabo_ is now known as fabo | 08:41 | |
| *** namnh has quit IRC | 08:42 | |
| openstackgerrit | Merged openstack-infra/project-config master: A Devstack-plugin that installs a VMAX backend https://review.openstack.org/474924 | 08:45 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/project-config master: Drop neutron-full-ssh job https://review.openstack.org/468718 | 08:45 |
| *** markus_z has quit IRC | 08:46 | |
| *** dingyichen has quit IRC | 08:47 | |
| *** tosky has joined #openstack-infra | 08:47 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add new puppet-ganesha project https://review.openstack.org/474474 | 08:48 |
| openstackgerrit | Merged openstack-infra/project-config master: Removed a bunch of identity-v3-only jobs https://review.openstack.org/474733 | 08:48 |
| openstackgerrit | Merged openstack-infra/project-config master: Add scenario jobs to tripleo repositories https://review.openstack.org/475482 | 08:48 |
| *** hrubi has quit IRC | 08:48 | |
| *** openstackgerrit has quit IRC | 08:48 | |
| *** openstackgerrit has joined #openstack-infra | 08:48 | |
| openstackgerrit | Merged openstack-infra/project-config master: Added Freezer tempest plugin repo https://review.openstack.org/472732 | 08:48 |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder master: add flag -r that receive packages to be removed before install phase https://review.openstack.org/475701 | 08:49 |
| openstackgerrit | Merged openstack-infra/project-config master: Propose a job for keystone rolling upgrades https://review.openstack.org/471419 | 08:49 |
| *** tmorin has left #openstack-infra | 08:50 | |
| *** hrubi has joined #openstack-infra | 08:52 | |
| *** bakbad has quit IRC | 08:52 | |
| openstackgerrit | Merged openstack-infra/project-config master: Enable horizon for designate grenade gate https://review.openstack.org/475337 | 08:53 |
| openstackgerrit | Merged openstack-infra/project-config master: migrate periodic jobs on stable branches to oooq https://review.openstack.org/474736 | 08:53 |
| *** stakeda has quit IRC | 08:55 | |
| *** electrofelix has joined #openstack-infra | 08:56 | |
| *** SpamapS has joined #openstack-infra | 08:56 | |
| openstackgerrit | Merged openstack-infra/project-config master: Added Freezer OpenStack-ansible repo https://review.openstack.org/469938 | 08:56 |
| *** markus_z has joined #openstack-infra | 08:58 | |
| *** sree has quit IRC | 09:03 | |
| *** sree has joined #openstack-infra | 09:04 | |
| *** sree has quit IRC | 09:08 | |
| *** salv-orlando has quit IRC | 09:09 | |
| *** salv-orlando has joined #openstack-infra | 09:09 | |
| *** pfallenop has quit IRC | 09:09 | |
| *** pfallenop has joined #openstack-infra | 09:10 | |
| *** pfallenop has quit IRC | 09:11 | |
| *** pfallenop has joined #openstack-infra | 09:11 | |
| *** salv-orlando has quit IRC | 09:13 | |
| *** eranrom has quit IRC | 09:16 | |
| openstackgerrit | Pratik Shah proposed openstack-infra/project-config master: Added changes to run unit tests for Omni project using Zuul https://review.openstack.org/475300 | 09:18 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at using CI mirrors for fedora/centos https://review.openstack.org/475607 | 09:18 |
| *** sambetts_ is now known as sambetts | 09:19 | |
| openstackgerrit | Yuval Brik proposed openstack-infra/irc-meetings master: Karbor: turn irc meetings into biweekly https://review.openstack.org/475709 | 09:19 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack-infra/project-config master: jenkins: jobs: ansible-role-jobs: Add openSUSE 42.2 https://review.openstack.org/475710 | 09:22 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack-infra/project-config master: zuul: layout: Add openSUSE jobs to OpenStack Ansible roles and AIO https://review.openstack.org/475711 | 09:22 |
| MasterofJOKers | I'm locally trying to build an image for debian stretch with tools/build_image.sh, but it takes ages because it always refreshes the cache. I've exported DIB_OFFLINE=1, but that doesn't help. Any advice? | 09:23 |
| *** openstackgerrit has quit IRC | 09:33 | |
| cmurphy | MasterofJOKers: check the advice on this line http://git.openstack.org/cgit/openstack-infra/project-config/tree/tools/build-image.sh#n70 | 09:37 |
| *** openstackgerrit has joined #openstack-infra | 09:37 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/irc-meetings master: Free up SWG meeting slot https://review.openstack.org/475712 | 09:37 |
| MasterofJOKers | cmurphy, thank you, I'll try that :) | 09:38 |
| *** shardy has joined #openstack-infra | 09:39 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml https://review.openstack.org/475713 | 09:39 |
| *** yamamoto has quit IRC | 09:41 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack-infra/project-config master: jenkins: jobs: ansible-role-jobs: Add openSUSE 42.2 https://review.openstack.org/475710 | 09:42 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack-infra/project-config master: zuul: layout: Add openSUSE jobs to OpenStack Ansible roles and AIO https://review.openstack.org/475711 | 09:42 |
| *** eranrom has joined #openstack-infra | 09:43 | |
| *** ijw has joined #openstack-infra | 09:44 | |
| *** jascott1 has quit IRC | 09:45 | |
| *** jascott1 has joined #openstack-infra | 09:46 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Fix periodic jobs uploads https://review.openstack.org/475715 | 09:48 |
| *** sshnaidm|afk is now known as sshnaidm | 09:48 | |
| *** thorst has joined #openstack-infra | 09:49 | |
| *** ijw has quit IRC | 09:49 | |
| *** jascott1 has quit IRC | 09:51 | |
| *** quite has quit IRC | 09:52 | |
| *** hrubi has quit IRC | 09:52 | |
| *** alexchadin has quit IRC | 09:53 | |
| *** alexchadin has joined #openstack-infra | 09:53 | |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: docs: Don't specify pbr version in 'setup.py' https://review.openstack.org/475719 | 09:53 |
| *** hrubi has joined #openstack-infra | 09:54 | |
| *** dtantsur is now known as dtantsur|bbl | 09:55 | |
| *** yamamoto has joined #openstack-infra | 09:56 | |
| *** quite has joined #openstack-infra | 09:58 | |
| *** thorst has quit IRC | 09:59 | |
| *** liujiong has quit IRC | 10:02 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Fix periodic jobs uploads https://review.openstack.org/475715 | 10:04 |
| *** yamamoto has quit IRC | 10:06 | |
| *** hrubi has quit IRC | 10:06 | |
| *** alexchadin has quit IRC | 10:07 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Add linebreak to generated passwd/group file https://review.openstack.org/475723 | 10:09 |
| *** Julien-zte has quit IRC | 10:15 | |
| *** dims has quit IRC | 10:16 | |
| openstackgerrit | Kevin Benton proposed openstack-infra/project-config master: Move rally-py35 to experimental neutron https://review.openstack.org/475725 | 10:17 |
| *** thorst has joined #openstack-infra | 10:17 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add the grenade test to gate https://review.openstack.org/474824 | 10:18 |
| *** gongysh has quit IRC | 10:18 | |
| *** dims has joined #openstack-infra | 10:21 | |
| *** gomarivera has joined #openstack-infra | 10:22 | |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate master: remove etcd3 from subnode install https://review.openstack.org/475726 | 10:23 |
| *** yamamoto has joined #openstack-infra | 10:25 | |
| *** gomarivera has quit IRC | 10:26 | |
| *** jpena is now known as jpena|off | 10:27 | |
| openstackgerrit | aditi sharma proposed openstack-infra/project-config master: Add install-guide-jobs for watcher https://review.openstack.org/471601 | 10:30 |
| *** jpena|off is now known as jpena | 10:31 | |
| *** adisky__ has joined #openstack-infra | 10:32 | |
| *** rhallisey has quit IRC | 10:33 | |
| adisky__ | AJaeger: hii | 10:33 |
| *** andreas_s has joined #openstack-infra | 10:36 | |
| AJaeger | what's up adisky__ ? | 10:36 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at using CI mirrors for fedora/centos https://review.openstack.org/475607 | 10:36 |
| adisky__ | AJaeger: I am not sure about branches | 10:37 |
| *** umbarkar has quit IRC | 10:37 | |
| adisky__ | AJaeger: we have included it after ocata | 10:37 |
| *** tushar has joined #openstack-infra | 10:37 | |
| AJaeger | adisky__: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n2860 is the generic condition used - so if you change in stable/ocata the tox.ini file, it will be run | 10:38 |
| *** panda|off has quit IRC | 10:38 | |
| AJaeger | Add a specific override just for your own project further down - with a branch condition to not run it on newton and ocata | 10:38 |
| adisky__ | AJaeger: ok got it thanks | 10:39 |
| *** hrubi has joined #openstack-infra | 10:39 | |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder master: add flag -r that receive packages to be removed before install phase https://review.openstack.org/475701 | 10:40 |
| *** kzaitsev_mb has joined #openstack-infra | 10:41 | |
| *** hrubi has quit IRC | 10:42 | |
| openstackgerrit | Merged openstack-infra/project-config master: Create tempest non-voting gate in kuryr-kubernetes https://review.openstack.org/474993 | 10:45 |
| *** dtantsur|bbl is now known as dtantsur | 10:45 | |
| *** hrubi has joined #openstack-infra | 10:45 | |
| *** jascott1 has joined #openstack-infra | 10:46 | |
| *** jkilpatr has quit IRC | 10:47 | |
| *** hrubi has quit IRC | 10:48 | |
| *** jascott1 has quit IRC | 10:51 | |
| *** hrubi has joined #openstack-infra | 10:51 | |
| *** hrubi has quit IRC | 10:54 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack-infra/project-config master: jenkins: scripts: propose_update: Turn on OSA updates on all repositories https://review.openstack.org/475734 | 10:55 |
| *** cshastri has quit IRC | 10:56 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: Disable telemetry from fakeha-caserver job https://review.openstack.org/475736 | 10:57 |
| *** hrubi has joined #openstack-infra | 10:57 | |
| *** hrubi has quit IRC | 11:00 | |
| openstackgerrit | Merged openstack-infra/project-config master: Normalize projects.yaml https://review.openstack.org/475713 | 11:00 |
| *** hrubi has joined #openstack-infra | 11:03 | |
| *** tumbarka has joined #openstack-infra | 11:06 | |
| *** tushar has quit IRC | 11:06 | |
| *** rossella_ has joined #openstack-infra | 11:07 | |
| *** neiljerram has quit IRC | 11:08 | |
| *** neiljerram has joined #openstack-infra | 11:09 | |
| *** rossella_s has quit IRC | 11:10 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix access before assignment in zuul streamer https://review.openstack.org/475741 | 11:10 |
| *** amoralej is now known as amoralej|lunch | 11:10 | |
| *** wolverineav has joined #openstack-infra | 11:10 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix typo in localhost check in zuul streamer https://review.openstack.org/475742 | 11:10 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix typo in localhost check in zuul streamer https://review.openstack.org/475742 | 11:12 |
| *** thorst has quit IRC | 11:16 | |
| *** MatthewLi has quit IRC | 11:19 | |
| *** MatthewLi has joined #openstack-infra | 11:20 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/project-config master: WIP: enable tripleo periodic containers job https://review.openstack.org/475746 | 11:22 |
| *** eric_young has quit IRC | 11:23 | |
| *** jkilpatr has joined #openstack-infra | 11:24 | |
| *** lucasagomes is now known as lucas-hungry | 11:24 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at using CI mirrors for fedora/centos https://review.openstack.org/475607 | 11:27 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: DONT MERGE: containers periodic test https://review.openstack.org/475747 | 11:32 |
| *** edwarnicke has quit IRC | 11:33 | |
| *** edwarnicke has joined #openstack-infra | 11:36 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move ironic-agent test to fedora-minimal https://review.openstack.org/475749 | 11:37 |
| *** ijw has joined #openstack-infra | 11:37 | |
| *** e0ne has joined #openstack-infra | 11:38 | |
| *** rhallisey has joined #openstack-infra | 11:40 | |
| *** ijw has quit IRC | 11:42 | |
| *** kgiusti1 has quit IRC | 11:45 | |
| openstackgerrit | Tom Barron proposed openstack-infra/project-config master: Experimental py35 dsvm for ceph devstack plugin https://review.openstack.org/454890 | 11:46 |
| *** e0ne has quit IRC | 11:46 | |
| *** thorst has joined #openstack-infra | 11:47 | |
| *** rossella_ has quit IRC | 11:48 | |
| *** thorst has quit IRC | 11:52 | |
| *** ldnunes has joined #openstack-infra | 11:53 | |
| *** dnikishov has joined #openstack-infra | 11:53 | |
| dnikishov | hello, can I get some reviews please @ https://review.openstack.org/#/c/475323/ ? | 11:54 |
| *** jpena is now known as jpena|lunch | 11:54 | |
| gibi_ | hi infra! Is it just me or the paste.openstack.org is down? I constantly get HTTP 502 | 11:55 |
| *** martinkopec has joined #openstack-infra | 11:56 | |
| sdague | can we get https://review.openstack.org/#/c/475726 d-g change for etcd3 not installing on the subnodes | 11:59 |
| *** flepied has quit IRC | 11:59 | |
| mordred | sdague: lgtm | 12:00 |
| sdague | mordred: thanks | 12:00 |
| *** rossella_ has joined #openstack-infra | 12:03 | |
| frickler | gibi_: infra-root: same for me | 12:04 |
| MasterofJOKers | can someone please point me in a direction so I can reproduce this: http://lists.alioth.debian.org/pipermail/openstack-devel/2017-March/020998.html the CONFIG_REF stuff seems to be removed. how do you run the puppet changes? do I have to boot the image created via tools/build-image.sh and run puppet manually? | 12:08 |
| *** thorst has joined #openstack-infra | 12:08 | |
| *** chlong has joined #openstack-infra | 12:10 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/project-config master: Revert "Add scenario jobs to tripleo repositories" https://review.openstack.org/475757 | 12:10 |
| EmilienM | sshnaidm: please explain why you revert this maybe? https://review.openstack.org/#/c/475757/ | 12:13 |
| EmilienM | sshnaidm: file a bug in launchpad, etc | 12:13 |
| sshnaidm | EmilienM, just prepared | 12:13 |
| EmilienM | sshnaidm: could it be $PROJECTS not container these repos? | 12:13 |
| sshnaidm | EmilienM, it blocks now all gates in tripleo-ci, oooq and oooq-extras | 12:14 |
| EmilienM | sshnaidm: yeah I know | 12:14 |
| EmilienM | sshnaidm: instead of revert, move it to experimental | 12:14 |
| *** rfolco has joined #openstack-infra | 12:14 | |
| EmilienM | sshnaidm: so we can figure out afterward and fix it | 12:14 |
| EmilienM | sshnaidm: without blocking gate | 12:14 |
| sshnaidm | EmilienM, ok | 12:14 |
| *** flepied has joined #openstack-infra | 12:16 | |
| *** MatthewLi has quit IRC | 12:17 | |
| *** salv-orlando has joined #openstack-infra | 12:19 | |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder master: add flag -r that receive packages to be removed before install phase https://review.openstack.org/475701 | 12:19 |
| *** hashar has joined #openstack-infra | 12:20 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/project-config master: Move tripleo scenario jobs to experimental https://review.openstack.org/475759 | 12:20 |
| sshnaidm | EmilienM, ^ | 12:20 |
| EmilienM | sshnaidm: all good | 12:20 |
| *** MatthewLi has joined #openstack-infra | 12:20 | |
| EmilienM | AJaeger, fungi: tripleo team has an outstanding request, if possible please review https://review.openstack.org/#/c/475759/ when possible - thanks | 12:21 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Use cirros images on host https://review.openstack.org/475576 | 12:22 |
| mordred | infra-root: shade 1.22.0 is out - I'd like to upgrade nodepool and restart it so I can watch the logs for a bit and make sure we didn't miss anything | 12:26 |
| *** gomarivera has joined #openstack-infra | 12:27 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Free up SWG meeting slot https://review.openstack.org/475712 | 12:28 |
| Shrews | mordred: fine with me | 12:29 |
| *** jaypipes has joined #openstack-infra | 12:30 | |
| *** rlandy has joined #openstack-infra | 12:31 | |
| Shrews | mordred: oy, big changes since last release | 12:31 |
| mordred | Shrews: yah - no more novaclient | 12:32 |
| mordred | mmm. the log is much nicer now | 12:32 |
| *** lucas-hungry is now known as lucasagomes | 12:32 | |
| EmilienM | sshnaidm: do you have logs of the failing task that try to clone https://git.openstack.org/openstack/puppet-ganesha/ ? | 12:33 |
| sshnaidm | EmilienM, http://logs.openstack.org/76/475576/2/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-puppet/e9c7a15/logs/devstack-gate-setup-workspace-new.txt | 12:34 |
| sshnaidm | EmilienM, puppet-ganesha was added yesterday to PROJECTS | 12:34 |
| EmilienM | mordred: do you know what times does it take to create a git repo after an addition in project-config to create a project? | 12:35 |
| EmilienM | https://git.openstack.org/openstack/puppet-ganesha/ is not created after https://review.openstack.org/474474 and it breaks our CI | 12:35 |
| sshnaidm | https://github.com/openstack-infra/project-config/commit/9eea80c9c613fda42db343462569752bf6d7bc93 | 12:35 |
| Shrews | mordred: exception in nodepool-deleter | 12:36 |
| mordred | Shrews: yah - poking at that now | 12:36 |
| mordred | Shrews: I see two different things wrong with that | 12:36 |
| mordred | Shrews: a) list_floating_ips should not throw an exception on not-found - it should return an empty list ... b) it shouldn't be calling floating ip things on internap in the first place | 12:36 |
| EmilienM | also, I got a bunch of Gerrit timeouts & 502 | 12:36 |
| EmilienM | we probably should restart it? | 12:37 |
| *** esberglu has joined #openstack-infra | 12:37 | |
| *** gordc has joined #openstack-infra | 12:37 | |
| sshnaidm | EmilienM, it may be temporary error.. I see 004 scenario runs now and puppet-ganesha is just empty | 12:38 |
| *** amoralej|lunch is now known as amoralej | 12:38 | |
| *** mriedem has joined #openstack-infra | 12:38 | |
| Shrews | mordred: i see a cert exception on internap and the fip thing on rax | 12:38 |
| EmilienM | well, https://git.openstack.org/openstack/puppet-ganesha/ got 404 | 12:38 |
| Shrews | also | 12:38 |
| *** hrubi has quit IRC | 12:38 | |
| EmilienM | sshnaidm: but https://github.com/openstack/puppet-ganesha exists :/ | 12:38 |
| mordred | Shrews: is the cert exception from before the restart? | 12:38 |
| EmilienM | sshnaidm: maybe a git server issue | 12:38 |
| Shrews | mordred: oh, yah, that may be | 12:39 |
| *** esberglu has quit IRC | 12:39 | |
| mordred | Shrews: the requests update moved the location of the bundled cert :( | 12:39 |
| sshnaidm | EmilienM, or it didn't exist earlier | 12:39 |
| *** esberglu has joined #openstack-infra | 12:39 | |
| *** kzaitsev_mb has quit IRC | 12:40 | |
| *** trown|outtypewww is now known as trown | 12:40 | |
| sshnaidm | EmilienM, actually it did exist, http://logs.openstack.org/97/474197/4/check/gate-tripleo-ci-centos-7-scenario004-multinode-oooq-puppet/ab59b5c/logs/devstack-gate-setup-workspace-new.txt.gz#_2017-06-20_09_43_53_360 | 12:41 |
| *** therve_ is now known as therve | 12:41 | |
| *** hrubi has joined #openstack-infra | 12:42 | |
| *** kzaitsev_mb has joined #openstack-infra | 12:42 | |
| *** mdrabe has joined #openstack-infra | 12:42 | |
| EmilienM | sshnaidm: I would wait a little bit for infra folks to have a look | 12:43 |
| *** gomarivera has quit IRC | 12:43 | |
| *** kgiusti has joined #openstack-infra | 12:43 | |
| sshnaidm | EmilienM, sure.. either we can delete it from PROJECTS | 12:43 |
| *** esberglu has quit IRC | 12:44 | |
| EmilienM | sshnaidm: also. Please abandon other patches that moved the jobs, now that we know what happens | 12:44 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Return an empty list on FIP listing failure https://review.openstack.org/475769 | 12:45 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Don't try to delete fips on non-fip clouds https://review.openstack.org/475770 | 12:45 |
| mordred | Shrews: ^^ | 12:45 |
| * Shrews waits for gerrit | 12:45 | |
| EmilienM | sshnaidm: we'll probably need puppet-ganesha later in tripleo | 12:47 |
| EmilienM | sshnaidm: IIRC it was created to deploy some NFS things, for an integration in tripleo | 12:47 |
| mordred | Shrews: I think we should rollback until we get those landed and out | 12:48 |
| odyssey4me | argh, gerrit appears to be hitting a slow down | 12:48 |
| mordred | Shrews: otherwise I think we're going to run out of nodes on rackspace and internap | 12:48 |
| Shrews | mordred: yup | 12:48 |
| Shrews | trying to review, but gerrit is sad | 12:48 |
| *** hrubi has quit IRC | 12:49 | |
| *** lbragstad_ has quit IRC | 12:49 | |
| EmilienM | odyssey4me: yes, we're waiting for some infra-root to have a look | 12:49 |
| mordred | it's running fine for me? | 12:49 |
| *** gomarivera has joined #openstack-infra | 12:49 | |
| mordred | oh - there it goes | 12:49 |
| EmilienM | mordred: you're probably on the right node | 12:49 |
| *** hrubi has joined #openstack-infra | 12:49 | |
| EmilienM | 1) Gerrit is slow and sends 502 sometimes | 12:50 |
| *** LindaWang has quit IRC | 12:50 | |
| *** gomarivera has quit IRC | 12:50 | |
| EmilienM | 2) https://git.openstack.org/openstack/puppet-ganesha/ doesn't exist and it breaks TripleO CI | 12:50 |
| Shrews | mordred: maybe you should move the try:except: around FIP to inside the _nova_list_floating_ips() method since there is more than one call to that method | 12:50 |
| mordred | EmilienM: is https://git.openstack.org/openstack/puppet-ganesha/ a new repo? | 12:50 |
| mordred | Shrews: ++ | 12:50 |
| *** esberglu has joined #openstack-infra | 12:51 | |
| *** hrubi has quit IRC | 12:52 | |
| *** salv-orlando has quit IRC | 12:52 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Return an empty list on FIP listing failure https://review.openstack.org/475769 | 12:52 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Don't try to delete fips on non-fip clouds https://review.openstack.org/475770 | 12:52 |
| *** salv-orlando has joined #openstack-infra | 12:52 | |
| EmilienM | mordred: yes, since a few hours (it's already on github) | 12:52 |
| *** mugsie_ is now known as mugsie | 12:52 | |
| *** mugsie has quit IRC | 12:53 | |
| *** mugsie has joined #openstack-infra | 12:53 | |
| mordred | EmilienM: nod | 12:53 |
| *** alexchadin has joined #openstack-infra | 12:54 | |
| mordred | #status notice restarting gerrit to address slowdown issues | 12:54 |
| openstackstatus | mordred: sending notice | 12:54 |
| *** hrubi has joined #openstack-infra | 12:55 | |
| -openstackstatus- NOTICE: restarting gerrit to address slowdown issues | 12:56 | |
| *** gardlt has quit IRC | 12:56 | |
| dmsimard | How are secret credentials passed down to jobs right now ? Never had to use one and I don't see any JJB around credentials-binding ? | 12:56 |
| *** salv-orlando has quit IRC | 12:56 | |
| mordred | dmsimard: they aren't really - for any jobs where we need something to be able to do something in a trusted manner, we have a dedicated node for it and we write special jobs | 12:57 |
| mordred | dmsimard: this is one of the things that changes with zuul v3 where there will be a mechanism for jobs to request access to credentials | 12:57 |
| openstackstatus | mordred: finished sending notice | 12:57 |
| *** jpena|lunch is now known as jpena | 12:57 | |
| mordred | dmsimard: what sort of credential do you need / what do you want to do? | 12:58 |
| *** efoley has quit IRC | 12:58 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/project-config master: WIP: enable tripleo periodic containers job https://review.openstack.org/475746 | 12:58 |
| *** efoley has joined #openstack-infra | 12:58 | |
| dmsimard | It's a token so a TripleO job can push and tag containers to a registry | 12:58 |
| dmsimard | (After it's tested them) | 12:58 |
| EmilienM | sshnaidm: https://git.openstack.org/cgit/openstack/puppet-ganesha/ see this link works fine | 12:59 |
| EmilienM | sshnaidm: I see multinode jobs working now https://review.openstack.org/#/c/462916/ | 12:59 |
| EmilienM | sshnaidm: maybe it was just temporary, can you recheck please? | 12:59 |
| *** dprince has joined #openstack-infra | 12:59 | |
| mordred | dmsimard: well, I havne't been following it - but I know there has been work around pushing containers to a registry related to the kolla jobs that you can probably re-use | 13:00 |
| dmsimard | mordred: I guess you could say it'd be the same thing as Kolla jobs pushing to DockerHub -- except iirc they push containers without automation and with humans | 13:00 |
| sshnaidm | EmilienM, yeah, looking now | 13:00 |
| mordred | dmsimard: I believe that's been changing / people are workign on that | 13:00 |
| dmsimard | mordred: yeah, I'll poke around. Thanks. | 13:01 |
| dmsimard | pabelanger, inc0: is the registry for containers ready now ? haven't kept up with the news | 13:01 |
| mordred | dmsimard: the general pattern is to have the job upload $something to tarballs.o.o then to have a trusted job pull that thing down and upload it to the place that needs credentials | 13:01 |
| dmsimard | mordred: makes sense | 13:01 |
| dmsimard | mordred: sort of ends up taking twice the space, however | 13:01 |
| mordred | indeed. but in the current system it's unavoidable | 13:02 |
| dmsimard | agreed | 13:02 |
| mordred | this is one of the things that will be WAY better in v3 | 13:02 |
| dmsimard | I dream about v3 at night | 13:02 |
| dmsimard | <3 | 13:03 |
| mordred | dmsimard: you and me both :) | 13:03 |
| mordred | EmilienM: yah - I think you just hit unlucky timing with puppet-ganesha | 13:03 |
| EmilienM | mordred: thanks | 13:04 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/project-config master: Drop neutron-full-ssh job https://review.openstack.org/468718 | 13:05 |
| *** hichihara has joined #openstack-infra | 13:07 | |
| *** flepied has quit IRC | 13:07 | |
| *** panda has joined #openstack-infra | 13:08 | |
| *** kzaitsev_mb has quit IRC | 13:08 | |
| *** flepied has joined #openstack-infra | 13:09 | |
| *** yamamoto has quit IRC | 13:10 | |
| *** yamamoto has joined #openstack-infra | 13:11 | |
| *** salv-orlando has joined #openstack-infra | 13:12 | |
| *** dizquierdo has quit IRC | 13:13 | |
| *** gouthamr has joined #openstack-infra | 13:13 | |
| *** alexchadin has quit IRC | 13:13 | |
| *** yamamoto has quit IRC | 13:15 | |
| *** links has quit IRC | 13:18 | |
| *** yolanda has quit IRC | 13:19 | |
| *** bobh has joined #openstack-infra | 13:19 | |
| *** baoli has joined #openstack-infra | 13:20 | |
| *** eharney has joined #openstack-infra | 13:23 | |
| *** yolanda has joined #openstack-infra | 13:24 | |
| andreaf | AJaeger, gmann, sdague: https://review.openstack.org/#/c/468718 - removing the -ssh job | 13:26 |
| *** jcoufal has joined #openstack-infra | 13:26 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add Shrews to statusbot user whitelist https://review.openstack.org/475787 | 13:27 |
| mordred | Shrews: ^^ | 13:27 |
| openstackgerrit | Arx Cruz proposed openstack-infra/devstack-gate master: Enable process_stackviz function to be optional https://review.openstack.org/475788 | 13:28 |
| *** yamamoto has joined #openstack-infra | 13:29 | |
| *** yamamoto has quit IRC | 13:29 | |
| *** mhickey has quit IRC | 13:30 | |
| *** jrist_ is now known as jrist | 13:31 | |
| *** jrist has quit IRC | 13:31 | |
| *** jrist has joined #openstack-infra | 13:31 | |
| *** bobh has quit IRC | 13:31 | |
| *** yamamoto has joined #openstack-infra | 13:31 | |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack-dev/pbr master: Revert "Add binding option for WSGI server" https://review.openstack.org/475790 | 13:33 |
| *** Goneri has joined #openstack-infra | 13:33 | |
| *** yamahata_ has joined #openstack-infra | 13:34 | |
| *** hjensas has joined #openstack-infra | 13:36 | |
| *** gomarivera has joined #openstack-infra | 13:40 | |
| *** gardlt has joined #openstack-infra | 13:40 | |
| *** camunoz has joined #openstack-infra | 13:41 | |
| *** ykarel has quit IRC | 13:42 | |
| *** apuimedo has quit IRC | 13:43 | |
| openstackgerrit | Merged openstack-infra/project-config master: Drop neutron-full-ssh job https://review.openstack.org/468718 | 13:44 |
| *** ianychoi has joined #openstack-infra | 13:44 | |
| *** mhickey has joined #openstack-infra | 13:45 | |
| *** xarses has joined #openstack-infra | 13:46 | |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack-dev/pbr master: Restore previous IP binding https://review.openstack.org/475790 | 13:46 |
| *** gomarivera has quit IRC | 13:46 | |
| *** xarses has quit IRC | 13:47 | |
| *** xarses has joined #openstack-infra | 13:48 | |
| *** sree has joined #openstack-infra | 13:51 | |
| *** dayou has joined #openstack-infra | 13:51 | |
| *** bh526r has joined #openstack-infra | 13:52 | |
| *** gardlt_ has joined #openstack-infra | 13:54 | |
| openstackgerrit | Hengqing Hu proposed openstack-infra/project-config master: Make octavia dashboard nodejs4 npm run test voting https://review.openstack.org/475804 | 13:56 |
| *** gardlt has quit IRC | 13:56 | |
| *** gouthamr has quit IRC | 13:58 | |
| *** Julien-zte has joined #openstack-infra | 13:59 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Fix periodic jobs release detection https://review.openstack.org/475805 | 14:00 |
| *** gomarivera has joined #openstack-infra | 14:02 | |
| *** gomarivera has quit IRC | 14:04 | |
| *** gomarivera has joined #openstack-infra | 14:04 | |
| *** prateek has quit IRC | 14:05 | |
| *** dhajare has quit IRC | 14:06 | |
| *** Julien-zte has quit IRC | 14:06 | |
| *** Julien-zte has joined #openstack-infra | 14:07 | |
| *** Julien-zte has quit IRC | 14:08 | |
| *** bobh has joined #openstack-infra | 14:08 | |
| *** felipemonteiro has joined #openstack-infra | 14:09 | |
| *** Julien-zte has joined #openstack-infra | 14:09 | |
| *** jamesmcarthur has joined #openstack-infra | 14:09 | |
| *** rbrndt has joined #openstack-infra | 14:09 | |
| openstackgerrit | Ian Y. Choi proposed openstack-infra/project-config master: Modifies job for i18n repo to build translated doc https://review.openstack.org/471242 | 14:10 |
| *** mhickey has quit IRC | 14:12 | |
| *** eharney has quit IRC | 14:14 | |
| *** dizquierdo has joined #openstack-infra | 14:14 | |
| openstackgerrit | Merged openstack-infra/shade master: Return an empty list on FIP listing failure https://review.openstack.org/475769 | 14:15 |
| openstackgerrit | Merged openstack-infra/shade master: Don't try to delete fips on non-fip clouds https://review.openstack.org/475770 | 14:16 |
| *** bkero has quit IRC | 14:16 | |
| mordred | infra-root: just landed fix for this morning's nodepool-deleter issue with 1.22 - as soon as the release is cut, we can do the upgrade again | 14:18 |
| *** yamahata_ has quit IRC | 14:23 | |
| *** salv-orlando has quit IRC | 14:24 | |
| openstackgerrit | Javier Peña proposed openstack-infra/tripleo-ci master: Install rpmdevtools for DLRN https://review.openstack.org/475818 | 14:24 |
| *** hrubi has quit IRC | 14:25 | |
| *** mhickey has joined #openstack-infra | 14:25 | |
| *** dhajare has joined #openstack-infra | 14:26 | |
| openstackgerrit | Javier Peña proposed openstack-infra/tripleo-ci master: Install additional required packages for DLRN https://review.openstack.org/475818 | 14:27 |
| *** salv-orlando has joined #openstack-infra | 14:28 | |
| *** isaacb has joined #openstack-infra | 14:28 | |
| *** pblaho has quit IRC | 14:29 | |
| *** gouthamr has joined #openstack-infra | 14:30 | |
| *** salv-orlando has quit IRC | 14:30 | |
| *** salv-orlando has joined #openstack-infra | 14:30 | |
| sdague | any other infra-root / d-g reviewers around - as this etcd3 fix would be good to get in - https://review.openstack.org/#/c/475726/ | 14:31 |
| *** rossella_ has quit IRC | 14:32 | |
| *** eharney has joined #openstack-infra | 14:33 | |
| *** colettecello has quit IRC | 14:33 | |
| *** gothicmindfood has joined #openstack-infra | 14:34 | |
| clarkb | sdague: done. Also mtreinish reviewed https://review.openstack.org/474036 if you want to take another look at it | 14:34 |
| sdague | clarkb: and done | 14:35 |
| sdague | clarkb: thanks | 14:35 |
| cmurphy | clarkb: I took another stab at the zuul/systemd problem if you want to look https://review.openstack.org/#/c/473106/ | 14:35 |
| clarkb | sdague: re tooz not installing those libs by default maybe want to consider changing that if etcd is considered a common base service now. | 14:36 |
| sdague | clarkb: yeh, but that's not my call | 14:36 |
| sdague | however, there should be no issue just always installing those libs | 14:37 |
| sdague | so... win | 14:37 |
| *** thorst has quit IRC | 14:37 | |
| *** annegentle has joined #openstack-infra | 14:38 | |
| *** lbragstad_ has joined #openstack-infra | 14:38 | |
| *** felipemonteiro has quit IRC | 14:41 | |
| *** lbragstad_ is now known as lbragstad | 14:42 | |
| *** thorst has joined #openstack-infra | 14:42 | |
| *** yamahata_ has joined #openstack-infra | 14:43 | |
| *** felipemonteiro has joined #openstack-infra | 14:44 | |
| *** mhayden has quit IRC | 14:46 | |
| *** hongbin has joined #openstack-infra | 14:46 | |
| *** mhayden has joined #openstack-infra | 14:46 | |
| *** thorst has quit IRC | 14:47 | |
| *** rcernin_ has quit IRC | 14:47 | |
| *** palimarium has joined #openstack-infra | 14:47 | |
| *** rossella_ has joined #openstack-infra | 14:48 | |
| inc0 | dmsimard: ad registry, current plan is to build, test, publish to dockerhub | 14:48 |
| inc0 | and that's what I'm working on today | 14:49 |
| dmsimard | inc0: from infra jobs ? | 14:49 |
| inc0 | yes | 14:49 |
| *** markvoelker has joined #openstack-infra | 14:49 | |
| dmsimard | inc0: curious to see how you'll handle authentication without credentials bindings | 14:49 |
| dmsimard | inc0: that's what I was looking to solve | 14:49 |
| inc0 | more detailed plan is, we'll have periodic job that will build and publish to tarballs/kolla/images/tmp | 14:49 |
| inc0 | then we'll have slew of test jobs | 14:50 |
| inc0 | we'll have authentication - push will be handled by secure node | 14:50 |
| dmsimard | inc0: so you'll push to tarballs.o.o and then run a push job from that secure node ? | 14:50 |
| dmsimard | for context, I'm also looking to be pushing containers so it's likely the flow would be similar :) | 14:51 |
| inc0 | today we have this: http://tarballs.openstack.org/kolla/images/ | 14:51 |
| dmsimard | also, how do you ensure that job order makes sense so that containers are built before they are tested (periodically) ? | 14:51 |
| inc0 | there are tared docker registries | 14:51 |
| fungi | dmsimard: zuul job dependency ordering | 14:52 |
| inc0 | dmsimard: infra already has plumbing for that | 14:52 |
| dmsimard | fungi: ah, dependency also works for periodic jobs ? | 14:52 |
| dmsimard | I thought it was only for check/gate pipeline, silly me | 14:52 |
| fungi | dmsimard: we do dependency ordering in tag-based pipelines like release | 14:53 |
| *** annegentle has quit IRC | 14:53 | |
| *** mattfarina has left #openstack-infra | 14:53 | |
| fungi | should also work in periodic, but you're right that may still be uncharted territory | 14:53 |
| *** yamamoto has quit IRC | 14:53 | |
| *** yamamoto has joined #openstack-infra | 14:54 | |
| *** yamamoto has quit IRC | 14:54 | |
| openstackgerrit | Kaitlin Farr proposed openstack-infra/project-config master: New project for Castellan UI plugin https://review.openstack.org/475835 | 14:56 |
| *** rhallisey is now known as rhallisey-brb | 14:57 | |
| *** rhallisey-brb is now known as rhallisey-away | 14:57 | |
| openstackgerrit | Arx Cruz proposed openstack-infra/devstack-gate master: Enable process_stackviz function to be optional https://review.openstack.org/475788 | 14:58 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: Use IP version appropriate dns servers https://review.openstack.org/475608 | 15:00 |
| clarkb | ^ should be much more correct now after reading https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ it still needs testing though | 15:00 |
| *** isaacb has quit IRC | 15:01 | |
| openstackgerrit | Kaitlin Farr proposed openstack-infra/project-config master: New project for Castellan UI plugin https://review.openstack.org/475835 | 15:01 |
| *** gomarivera has quit IRC | 15:01 | |
| *** gomarivera has joined #openstack-infra | 15:02 | |
| *** yamamoto has joined #openstack-infra | 15:04 | |
| *** annegentle has joined #openstack-infra | 15:05 | |
| *** hashar is now known as hasharAway | 15:05 | |
| *** gomarivera has quit IRC | 15:06 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: Disable telemetry from fakeha-caserver job https://review.openstack.org/475736 | 15:08 |
| *** sbezverk has joined #openstack-infra | 15:08 | |
| *** yamamoto has quit IRC | 15:11 | |
| *** yamamoto has joined #openstack-infra | 15:11 | |
| *** isaacb has joined #openstack-infra | 15:12 | |
| *** pblaho has joined #openstack-infra | 15:14 | |
| *** sree has quit IRC | 15:14 | |
| clarkb | fungi: my dns query has grown to (message:"failed: Temporary failure in name resolution." OR message:"Temporary failure resolving" OR message:"wget: unable to resolve" OR message:"Could not resolve host") AND NOT message:"Could not resolve host: fake" AND tags:"console" | 15:14 |
| *** sree has joined #openstack-infra | 15:15 | |
| *** yamamoto has quit IRC | 15:15 | |
| *** ykarel has joined #openstack-infra | 15:17 | |
| *** sree has quit IRC | 15:18 | |
| *** hichihara has quit IRC | 15:18 | |
| openstackgerrit | Kaitlin Farr proposed openstack-infra/project-config master: New project for Castellan UI plugin https://review.openstack.org/475835 | 15:22 |
| clarkb | fungi: its 35:5 osic:else for dns lookup failures according to logstash query above | 15:23 |
| clarkb | in last 12 hours | 15:24 |
| clarkb | the negated expression if not present shows a lot of hits because kolla tests query an NXDOMAIN | 15:24 |
| clarkb | so once you filter that out its clearly osic dominating | 15:24 |
| clarkb | I'm going to get caffeine and breakfast then work on building an image with my change above and test bootign that in osic and other clouds. My current concern is that waitng on network-online doesn't mean that our ipv6 interface will be up and routable which is how I am testing for what forwarders to use... | 15:25 |
| mnaser | is paste.openstack.org working? | 15:25 |
| mnaser | (its not loading for me :x) | 15:25 |
| clarkb | infra-root ^ If you have better ideas on how to do that I am all ears | 15:25 |
| clarkb | mnaser: I confirm it doesn't appear to work from here either | 15:26 |
| clarkb | I really need fuel but can look more closely once I've got that | 15:26 |
| fungi | mnaser: i saw something in scrollback earlier about it being down too. just got out of another meeting and about to look into it | 15:27 |
| *** claudiub has quit IRC | 15:27 | |
| *** bobh has quit IRC | 15:29 | |
| *** bobh has joined #openstack-infra | 15:29 | |
| *** claudiub has joined #openstack-infra | 15:30 | |
| mnaser | no worries, just wanted to put eyes on it! | 15:30 |
| openstackgerrit | Julien Danjou proposed openstack-infra/project-config master: ceilometer: do not install Gnocchi via devstack in master https://review.openstack.org/468876 | 15:31 |
| *** thorst has joined #openstack-infra | 15:33 | |
| *** thorst has quit IRC | 15:34 | |
| *** ltomasbo is now known as ltomasbo|away | 15:34 | |
| *** thorst has joined #openstack-infra | 15:34 | |
| *** markus_z has quit IRC | 15:36 | |
| *** bramwelt2 is now known as bramwelt | 15:36 | |
| *** bramwelt has quit IRC | 15:36 | |
| *** bramwelt has joined #openstack-infra | 15:36 | |
| *** gyee has joined #openstack-infra | 15:38 | |
| openstackgerrit | Akihiro Motoki proposed openstack-infra/project-config master: Modifies job for i18n repo to build translated doc https://review.openstack.org/471242 | 15:41 |
| fungi | #status log Restarted openstack-paste service on paste.openstack.org as lodgeit runserver process was hung and unrsponsive (required sigterm followed up sighup before it would exit) | 15:41 |
| openstackstatus | fungi: finished logging | 15:41 |
| *** dizquierdo has quit IRC | 15:42 | |
| fungi | mnaser: should be working again, thanks for mentioning it in here | 15:42 |
| *** ykarel has quit IRC | 15:43 | |
| openstackgerrit | Akihiro Motoki proposed openstack-infra/project-config master: Modifies job for i18n repo to build translated doc https://review.openstack.org/471242 | 15:43 |
| *** jcoufal has quit IRC | 15:45 | |
| mnaser | fungi np :) | 15:45 |
| fungi | also doesn't appear we're logging the lodgeit process anywhere... seems to emit log data on stdout/stderr which are being swallowed by upstart, as best i can tell | 15:46 |
| *** rkukura has quit IRC | 15:46 | |
| fungi | though also possible i just don't know where to look | 15:46 |
| clarkb | fungi: ya I think thats how it is logged | 15:46 |
| *** jpich has quit IRC | 15:47 | |
| fungi | systemd to the rescue? ;) | 15:47 |
| *** gomarivera has joined #openstack-infra | 15:47 | |
| clarkb | well it will just dump stdout to journald :) | 15:47 |
| * clarkb resurrects image build vm and realizes that this would be easier if he just built images all the time | 15:48 | |
| clarkb | my plan is to make a small but representative image using my change above that I can upload to osic and boot there to make sure that ipv6 is up by the time network-online finishes and runs unbound-config | 15:49 |
| *** palimarium has quit IRC | 15:50 | |
| *** tphummel has joined #openstack-infra | 15:50 | |
| *** tphummel has quit IRC | 15:50 | |
| *** mriedem is now known as mriedem_bowlcut | 15:51 | |
| *** LindaWang has joined #openstack-infra | 15:54 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: Use IP version appropriate dns servers https://review.openstack.org/475608 | 15:54 |
| *** andreas_s has quit IRC | 15:56 | |
| *** jcoufal has joined #openstack-infra | 15:59 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: Update EnvInject to make use of convert_mapping_to_xml https://review.openstack.org/408801 | 16:00 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: builders: do not pass empty "preload-script" https://review.openstack.org/452331 | 16:00 |
| openstackgerrit | Terry Wilson proposed openstack-infra/project-config master: Add ovsdbapp master branch periodic job https://review.openstack.org/466373 | 16:01 |
| *** eharney has quit IRC | 16:02 | |
| *** armax has quit IRC | 16:03 | |
| *** eharney has joined #openstack-infra | 16:03 | |
| *** makowals has quit IRC | 16:05 | |
| *** jascott1 has joined #openstack-infra | 16:07 | |
| *** Julien-zte has quit IRC | 16:07 | |
| *** dtantsur is now known as dtantsur|afk | 16:08 | |
| *** panda is now known as pamda|off | 16:08 | |
| *** Julien-zte has joined #openstack-infra | 16:08 | |
| *** Julien-zte has quit IRC | 16:08 | |
| *** thorst has quit IRC | 16:09 | |
| *** martinkopec has quit IRC | 16:10 | |
| *** dizquierdo has joined #openstack-infra | 16:10 | |
| *** isaacb has quit IRC | 16:10 | |
| *** Apoorva has joined #openstack-infra | 16:11 | |
| *** mhickey has quit IRC | 16:11 | |
| *** Julien-zte has joined #openstack-infra | 16:12 | |
| *** psachin has quit IRC | 16:13 | |
| *** mdrabe has quit IRC | 16:13 | |
| *** csomerville has quit IRC | 16:13 | |
| *** ltomasbo|away is now known as ltomasbo | 16:13 | |
| *** csomerville has joined #openstack-infra | 16:14 | |
| openstackgerrit | Merged openstack-infra/devstack-gate master: remove etcd3 from subnode install https://review.openstack.org/475726 | 16:14 |
| clarkb | sdague: ^ fyi | 16:15 |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Don't install tempest globally https://review.openstack.org/474036 | 16:15 |
| sdague | clarkb: great | 16:16 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: Add support for chuck_norris publisher https://review.openstack.org/474359 | 16:16 |
| *** yamamoto has joined #openstack-infra | 16:17 | |
| *** rossella_ has quit IRC | 16:19 | |
| *** mdrabe has joined #openstack-infra | 16:19 | |
| clarkb | I think ^ is the best commit message ever | 16:19 |
| *** pcaruana has quit IRC | 16:19 | |
| openstackgerrit | Merged openstack-dev/pbr master: Restore previous IP binding https://review.openstack.org/475790 | 16:19 |
| clarkb | mordred: I expect you may particularly enjoy that jjb commit title | 16:19 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: Update xcode to make use of convert_mapping_to_xml() https://review.openstack.org/475141 | 16:21 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: Update docker_build_publish to make use of convert_mapping_to_xml() https://review.openstack.org/475148 | 16:21 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: Update ssh_builder to make use of convert_mapping_to_xml() https://review.openstack.org/475146 | 16:22 |
| *** ykarel has joined #openstack-infra | 16:24 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: Update tox to make use of convert_mapping_to_xml() https://review.openstack.org/475145 | 16:25 |
| *** thorst has joined #openstack-infra | 16:25 | |
| *** rnoriega has joined #openstack-infra | 16:26 | |
| *** yamamoto has quit IRC | 16:28 | |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder master: Adding support for Manual Build Trigger https://review.openstack.org/202543 | 16:28 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: Only configure jenkins cache if cache exists https://review.openstack.org/475864 | 16:29 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add linebreak to generated passwd/group file https://review.openstack.org/475723 | 16:30 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder master: Consolidate trigger-manual and trigger-parameterized-builds https://review.openstack.org/314108 | 16:30 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix access before assignment in zuul streamer https://review.openstack.org/475741 | 16:31 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix typo in localhost check in zuul streamer https://review.openstack.org/475742 | 16:31 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: requirements: remove ordereddict https://review.openstack.org/475229 | 16:32 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Remove use of six library https://review.openstack.org/475135 | 16:33 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add redhat-rpm-config to rpm build deps https://review.openstack.org/475082 | 16:33 |
| openstackgerrit | Kaitlin Farr proposed openstack-infra/project-config master: Barbican: update tests https://review.openstack.org/469648 | 16:33 |
| *** rossella_ has joined #openstack-infra | 16:34 | |
| *** thorst has quit IRC | 16:35 | |
| mordred | clarkb: wow | 16:36 |
| mordred | infra-root: shade 1.22.1 has been released, so it's time for another nodepool bump - but I'm aboutto get on aplane - so I'll do that when I'm in the air and can watch | 16:36 |
| mmedvede | mordred: shade 1.21.1 release appears to break nodepool 1.3.1 http://paste.openstack.org/show/613192/ | 16:36 |
| clarkb | mordred: ok | 16:36 |
| fungi | thanks mordred! | 16:36 |
| clarkb | mmedvede: 1.21.1 or 1.22.1 ? | 16:37 |
| mordred | mmedvede: yes- 1.22.1 should fix that | 16:37 |
| mordred | mmedvede: and sorry- we missed a logical combo (missed a test for delete-server with delete_ips=True but server doesn't have floating ips) | 16:38 |
| mmedvede | mordred: my bad, 1.22.1 has that error, 1.21.0 is ok | 16:38 |
| *** cody-somerville has joined #openstack-infra | 16:38 | |
| *** cody-somerville has joined #openstack-infra | 16:38 | |
| mmedvede | checking 1.22.0 | 16:38 |
| mordred | mmedvede: 1.22.1 still has the error? 1.22.0 definitely should have it - it should be fixed in 1.22.1 :( | 16:39 |
| * mordred has sads - are you sure you restarted with 1.22.1 and not 1.22.0 ? | 16:39 | |
| mordred | mmedvede: I've got to AFK for about 45 minutes - will follow up as soon as I'm back online | 16:39 |
| mmedvede | mordred: let me do a double sanity check | 16:39 |
| openstackgerrit | Kaitlin Farr proposed openstack-infra/project-config master: Barbican: update tests https://review.openstack.org/469648 | 16:41 |
| *** csomerville has quit IRC | 16:41 | |
| *** ltomasbo is now known as ltomasbo|away | 16:42 | |
| *** SamYaple has quit IRC | 16:44 | |
| openstackgerrit | Tin Lam proposed openstack-infra/project-config master: Adding linter and check to openstack-helm-infra repo https://review.openstack.org/475870 | 16:44 |
| *** SamYaple has joined #openstack-infra | 16:44 | |
| *** SamYaple has quit IRC | 16:44 | |
| *** SamYaple has joined #openstack-infra | 16:44 | |
| *** sambetts is now known as sambetts|afk | 16:45 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config master: Add non-voting jobs for ARA to test the devel branch of Ansible https://review.openstack.org/475871 | 16:45 |
| dmsimard | rcarrillocruz, mordred ^ fyi | 16:46 |
| *** kfarr has joined #openstack-infra | 16:46 | |
| mmedvede | mordred: 1) pip install shade==1.21.0, restarted nodepool - it can boot VMs without errors; 2) pip install shade==1.22.1, restart nodepool - the error appears in the log | 16:47 |
| *** iyamahat has joined #openstack-infra | 16:47 | |
| mmedvede | mordred: so I think this qualifies as confirmation. Unless there is something strange with other dependencies | 16:47 |
| kfarr | sdage , re: https://review.openstack.org/#/c/475530/ , is python-barbicanclient supposed to go in lib/oslo even though it's not part of oslo? | 16:48 |
| kfarr | sdague ^^ | 16:48 |
| rcarrillocruz | dmsimard: i'm confused, from source it seems if ansible_version is set, it is replaced from requirements.txt the ansible line for ansible=={ansible_version}, which is then consumed by pip | 16:49 |
| rcarrillocruz | but pip won't know that devel is in reality a VCS editable package | 16:49 |
| dmsimard | rcarrillocruz: yeah I need to land a patch in ara's run_tests.sh still | 16:49 |
| rcarrillocruz | long-story short: i believe logic is needed on run_tests.sh | 16:49 |
| rcarrillocruz | jynx | 16:49 |
| rcarrillocruz | aye | 16:49 |
| rcarrillocruz | i know cos i'm actually debugging a weird issue with pip | 16:50 |
| rcarrillocruz | tox uses for usedevelop true 'pip install -e .' | 16:50 |
| rcarrillocruz | now | 16:50 |
| rcarrillocruz | if you put in a requirements.txt | 16:50 |
| rcarrillocruz | an editable package pointing to ansible github | 16:50 |
| rcarrillocruz | e.g. | 16:50 |
| rcarrillocruz | -e git://github.com/ansible/ansible.git@devel#egg=ansible | 16:50 |
| rcarrillocruz | it doesn't work with pip install -e . | 16:51 |
| rcarrillocruz | whereas with python setup.py develop does work | 16:51 |
| *** iyamahat has quit IRC | 16:51 | |
| dmsimard | rcarrillocruz: yeah, I need to edit requirements.txt so that tox can handle the installation | 16:51 |
| rcarrillocruz | found it cos i just added logic to your run_tests.sh which does compare ansible_version and if i pass 'devel' then i sed the editable package from above ^ | 16:51 |
| *** iyamahat has joined #openstack-infra | 16:51 | |
| *** sbezverk has quit IRC | 16:51 | |
| *** iyamahat has quit IRC | 16:51 | |
| *** iyamahat has joined #openstack-infra | 16:52 | |
| *** thorst has joined #openstack-infra | 16:52 | |
| dmsimard | pretty sure -e git should work, think I've done it before but let me look | 16:52 |
| *** yland has joined #openstack-infra | 16:54 | |
| rcarrillocruz | catch you later, kid time | 16:54 |
| *** zzzeek has quit IRC | 16:55 | |
| *** oneswig has joined #openstack-infra | 16:55 | |
| *** thorst has quit IRC | 16:55 | |
| *** derekh has quit IRC | 16:56 | |
| oneswig | Hi all, can anyone help me with planet.openstack.org? Our blog (StackHPC team blog) is not getting syndicated. In the planet page source, it's tagged with "internal server error" - is that something we can fix or the result of a transient outage, or...? | 16:57 |
| *** yamamoto has joined #openstack-infra | 16:57 | |
| openstackgerrit | Tin Lam proposed openstack-infra/project-config master: Adding linter and check to openstack-helm-infra repo https://review.openstack.org/475870 | 16:57 |
| *** makowals has joined #openstack-infra | 16:57 | |
| *** ralonsoh has quit IRC | 16:58 | |
| *** martinkopec has joined #openstack-infra | 16:59 | |
| *** thorst has joined #openstack-infra | 17:00 | |
| *** tesseract has quit IRC | 17:00 | |
| oneswig | ... seems like there are 26 blog feeds currently in this state (ours has been like it for a few weeks at least) | 17:01 |
| rcarrillocruz | -e works, but only lookups pypi | 17:02 |
| rcarrillocruz | -vvv tells me that on my current pip version | 17:03 |
| *** lucasagomes is now known as lucas-afk | 17:03 | |
| *** yamamoto has quit IRC | 17:03 | |
| *** tushar1 has joined #openstack-infra | 17:04 | |
| openstackgerrit | Ricardo Noriega proposed openstack-infra/project-config master: Enabling Tempest Devstack plugin for networking-l2gw https://review.openstack.org/475875 | 17:05 |
| *** ykarel has quit IRC | 17:05 | |
| *** makowals has quit IRC | 17:06 | |
| *** rossella__ has joined #openstack-infra | 17:07 | |
| *** JudeC has joined #openstack-infra | 17:08 | |
| *** oneswig has quit IRC | 17:08 | |
| *** dhinesh has joined #openstack-infra | 17:09 | |
| *** Swami has joined #openstack-infra | 17:09 | |
| *** rossella_ has quit IRC | 17:10 | |
| *** tosky has quit IRC | 17:11 | |
| *** bh526r has quit IRC | 17:12 | |
| clarkb | building an infra image without cache-devstack is really difficult :/ This may have gone quicker if I had just built with the cache elemtn | 17:12 |
| clarkb | here I thought I was being smart by trimming the build time | 17:12 |
| *** martinkopec has quit IRC | 17:12 | |
| *** felipemonteiro has quit IRC | 17:13 | |
| *** jkilpatr has quit IRC | 17:14 | |
| *** yolanda_ has joined #openstack-infra | 17:16 | |
| *** jkilpatr has joined #openstack-infra | 17:17 | |
| *** makowals has joined #openstack-infra | 17:17 | |
| *** yolanda_ has quit IRC | 17:21 | |
| *** iyamahat has quit IRC | 17:22 | |
| *** iyamahat_ has joined #openstack-infra | 17:23 | |
| *** amotoki is now known as amotoki_away | 17:23 | |
| tushar1 | clarkb : Hi | 17:26 |
| tushar1 | clarkb : while updating the jenkins jobs got error | 17:27 |
| tushar1 | clarkb : executed command - sudo jenkins-jobs --flush-cache update /etc/jenkins_jobs/config/ | 17:27 |
| *** LindaWang has quit IRC | 17:28 | |
| tushar1 | clarkb : error log - https://gist.github.com/anonymous/0e58578374c1dc97e49c1fccf6f7376c | 17:29 |
| *** mamitchl_ has joined #openstack-infra | 17:29 | |
| tushar1 | clarkb : Any idea about this error? | 17:29 |
| clarkb | fungi: I see you are poking planet too | 17:30 |
| clarkb | fungi: I have the crontab commented out and am going to run planet with verbose logging | 17:30 |
| clarkb | fungi: that ok with you? I don't want to ste p on toes | 17:30 |
| clarkb | fungi: the log shows that planet has 500 errors dealing with those atom files but that is all | 17:31 |
| clarkb | tushar1: that looks like a yaml error, can you share the yaml that you have that bahs script in? | 17:31 |
| fungi | clarkb: go for it. i was just looking in the log | 17:31 |
| fungi | looks like it got a 500 error trying to access https://www.stackhpc.com/feeds/stackhpc.atom.xml | 17:32 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: Use IP version appropriate dns servers https://review.openstack.org/475608 | 17:32 |
| clarkb | fungi: ya, which seems to work fine in my browser | 17:33 |
| fungi | same thing i noticed | 17:33 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder master: Consolidate pipeline and trigger-parameterized-builds https://review.openstack.org/314108 | 17:33 |
| clarkb | fungi: it is running now and dumping to the log file with -v passed in | 17:33 |
| clarkb | fungi: so you should be able to follow it too | 17:33 |
| fungi | thanks, tailing | 17:34 |
| clarkb | it doesn't look much more verbose may need to pass --debug instead | 17:34 |
| *** e0ne has joined #openstack-infra | 17:35 | |
| tushar1 | clarkb: https://gist.github.com/anonymous/45a6a64ccc1c01352d4e0c05d1681fb6 | 17:37 |
| tushar1 | ^ yaml file | 17:37 |
| fungi | based on these logs, looks like the planet config could be trimmed down/cleaned up significantly | 17:39 |
| fungi | lots of 404 and 301 | 17:39 |
| clarkb | tushar1: https://gist.github.com/anonymous/45a6a64ccc1c01352d4e0c05d1681fb6#file-job-clients-yaml-L6-L12 all of those sections need to be indented an extra two lines. | 17:39 |
| clarkb | tushar1: the - needs to be indented one level not at the same level as the parent | 17:39 |
| clarkb | tushar1: I don't know if that will fix your problem but it may | 17:39 |
| clarkb | fungi: yup | 17:40 |
| fungi | still seems to be getting a 500 error from that feed | 17:40 |
| fungi | :q | 17:41 |
| fungi | hah | 17:41 |
| clarkb | I've reupdated the crontab to not run again | 17:41 |
| clarkb | (puppet had redone what I did) | 17:41 |
| *** electrofelix has quit IRC | 17:41 | |
| clarkb | thinking try again with --debug and see if that gives us how it is requesting the url then try testing locally? | 17:41 |
| fungi | clarkb: ERROR: cannot verify www.stackhpc.com's certificate, issued by 'CN=Let\'s Encrypt Authority X3,O=Let\'s Encrypt,C=US': | 17:42 |
| fungi | Unable to locally verify the issuer's authority. | 17:42 |
| *** priteau has quit IRC | 17:42 | |
| fungi | is what i get trying to hit it with wget from there | 17:42 |
| *** bobh has quit IRC | 17:42 | |
| clarkb | yup just noticed that with curl | 17:42 |
| * clarkb checks locally | 17:42 | |
| *** priteau has joined #openstack-infra | 17:42 | |
| clarkb | same thing locally so not our deployment at fault | 17:42 |
| clarkb | oneswig is not here but that seems like a simple thing for us :) | 17:43 |
| fungi | yeah, likely needs a chain cert served up with it | 17:43 |
| *** tushar1_ has joined #openstack-infra | 17:43 | |
| clarkb | interesting that firefox doesn't care | 17:43 |
| fungi | it may have a more direct trust in its own set | 17:44 |
| *** tushar1 has quit IRC | 17:44 | |
| clarkb | oh no firefox says it is not secure | 17:44 |
| clarkb | but opens anyways without warning, interesting | 17:44 |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: place holder environment config for toci and quickstart https://review.openstack.org/475887 | 17:44 |
| tushar1_ | clarkb : okay , trying | 17:45 |
| *** claudiub has quit IRC | 17:45 | |
| clarkb | fungi: are you going to respond to the list? | 17:45 |
| fungi | sure, i can. just collecting a little more detail | 17:46 |
| clarkb | kk | 17:46 |
| * clarkb goes back to image building | 17:46 | |
| clarkb | I think I almost have an image that does what I want locally then to test in osci | 17:46 |
| fungi | clarkb: i'll link them to https://www.ssllabs.com/ssltest/analyze.html?d=www.stackhpc.com&latest | 17:47 |
| *** priteau has quit IRC | 17:47 | |
| *** zzzeek has joined #openstack-infra | 17:47 | |
| clarkb | ++ | 17:47 |
| *** iyamahat__ has joined #openstack-infra | 17:48 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add ovsdbapp master branch periodic job https://review.openstack.org/466373 | 17:48 |
| *** rbrndt has quit IRC | 17:48 | |
| *** priteau has joined #openstack-infra | 17:49 | |
| *** iyamahat_ has quit IRC | 17:50 | |
| *** eranrom has quit IRC | 17:50 | |
| openstackgerrit | Pete Birley proposed openstack-infra/system-config master: Add #openstack-helm to meetbot channels https://review.openstack.org/455742 | 17:50 |
| *** eranrom has joined #openstack-infra | 17:51 | |
| *** e0ne has quit IRC | 17:51 | |
| *** yland has quit IRC | 17:51 | |
| tushar1_ | clarkb : It didnt solve the problem | 17:52 |
| *** yland has joined #openstack-infra | 17:52 | |
| *** yland is now known as Guest72024 | 17:52 | |
| *** priteau has quit IRC | 17:53 | |
| *** priteau has joined #openstack-infra | 17:54 | |
| *** priteau has quit IRC | 17:54 | |
| *** jpena is now known as jpena|off | 17:54 | |
| *** priteau has joined #openstack-infra | 17:54 | |
| sdague | kfarr: I've been putting additional libs in there instead of making new files | 17:56 |
| sdague | kfarr: eventually we'll rename some of the files / functions | 17:56 |
| *** zzzeek has quit IRC | 17:57 | |
| *** trown is now known as trown|lunch | 17:58 | |
| *** kfarr has quit IRC | 17:58 | |
| *** mriedem_bowlcut is now known as mriedem | 17:59 | |
| *** priteau has quit IRC | 17:59 | |
| *** pvaneck has joined #openstack-infra | 17:59 | |
| *** xarses has quit IRC | 18:01 | |
| dmsimard | mordred, rcarrillocruz: does that ring you a bell ? http://paste.openstack.org/raw/613201/ | 18:01 |
| dmsimard | I don't think it's related to 1.22.0/1.22.1 because we're also seeing it with 1.21.0 but the timing is odd | 18:02 |
| *** priteau has joined #openstack-infra | 18:02 | |
| *** iyamahat__ has quit IRC | 18:02 | |
| dmsimard | something about an image having a properties property ? | 18:02 |
| *** zzzeek has joined #openstack-infra | 18:03 | |
| *** rwsu has quit IRC | 18:03 | |
| *** thorst has quit IRC | 18:05 | |
| *** mamitchl_ has quit IRC | 18:05 | |
| *** MasterOfBugs has joined #openstack-infra | 18:06 | |
| openstackgerrit | Tin Lam proposed openstack-infra/project-config master: Adding linter and check to openstack-helm-infra repo https://review.openstack.org/475870 | 18:09 |
| dmsimard | yeah.. the image in question does have a "properties" property, probably not the smartest thing to do http://paste.openstack.org/raw/613202/ | 18:09 |
| *** efoley has quit IRC | 18:11 | |
| clarkb | tushar1_: can you paste the current yaml file and the output of jjb if it changed | 18:11 |
| *** rderose has joined #openstack-infra | 18:16 | |
| clarkb | I'm uploading my unbound test image to osic now | 18:17 |
| fungi | dhellmann: did you see my questions about redirects on patchset 7 of https://review.openstack.org/472275 ? | 18:19 |
| fungi | concerned that without nailing that down we may end up stuck/scrambling | 18:19 |
| dhellmann | fungi : no, looking now | 18:19 |
| dhellmann | fungi : the docs repo was already publishing a couple of files to /developer so I put https://review.openstack.org/475572 together with a redirect | 18:20 |
| dhellmann | fungi : unstated at the end of all of this is to circle back with the infra team and remove stale content that ends up being moved | 18:20 |
| fungi | dhellmann: oh, thanks that will probably work out as long as the docs team is comfortable maintaining that and working through the merge conflicts | 18:21 |
| dhellmann | there's just 1 rule right now, so I don't expect a bunch of merge conflicts | 18:21 |
| dhellmann | I could probably refine it a bit | 18:22 |
| dhellmann | redirect the 2 html files that were published out of the docs repo to their new home first, then redirect everything else | 18:22 |
| fungi | dhellmann: oh! i see the change. so basically stop publishing new content to the old location, but don't actually redirect to the new content until someone manually deletes the old content? | 18:22 |
| *** rloo has joined #openstack-infra | 18:22 | |
| dhellmann | the idea is we'd wait to merge that until some critical mass of stuff gets published to the new location | 18:23 |
| fungi | i guess that could work as long as it's over a relatively brief period and we batch up deletion work | 18:23 |
| dhellmann | at that point, iiuc, the redirect rule would prevent serving the content from the old location? | 18:23 |
| clarkb | ok my concern that network-online would trigger before ipv6 was up and working seems to be confirmed | 18:23 |
| dhellmann | maybe that's a bad assumption | 18:23 |
| clarkb | I got ipv4 resolvers on osic | 18:23 |
| *** priteau has quit IRC | 18:23 | |
| clarkb | everything else seems fairly sound though | 18:23 |
| rloo | hi AJaeger, wrt https://review.openstack.org/#/c/475521/, if it works as folks expect, then it is fine with me. I have to be honest and don't really understand all this stuff. | 18:23 |
| *** priteau has joined #openstack-infra | 18:23 | |
| fungi | dhellmann: nevermind. i misread it as a redirect after not-found. so this is more like the suggestion in my spec comment about a "flag day" change to just redirect everyone starting at the same time? | 18:24 |
| dhellmann | yeah | 18:24 |
| dhellmann | with the criteria for that to be determined | 18:24 |
| AJaeger | rloo: just check a change and see whether everything looks as it should. It should state "non-voting" for those jobs that have -nv. | 18:24 |
| clarkb | I'm guessing that is beacuse RAs are async whereas dhcp is synchronous so ipv4 can come up first and trigger network-online | 18:24 |
| dhellmann | I keep saying "a critical mass" but I don't know how much that is | 18:24 |
| dhellmann | we could also do 1 infra patch to change all of the doc publishing jobs at once, and then land the redirect patch | 18:24 |
| clarkb | mordred: pabelanger fungi is there a good way to check this based on the contents of something glean writes out? | 18:25 |
| dhellmann | that might be smoother | 18:25 |
| fungi | dhellmann: if nothing else, it'll light a fire under the stragglers when their documentation suddenly stops being served ;) | 18:25 |
| dhellmann | yes, there's that | 18:25 |
| AJaeger | rloo: so, looking at https://review.openstack.org/#/c/470406/ , I see two "non-voting" ones - and those are the -nv jobs. So, all looks fine. | 18:25 |
| dhellmann | :-) | 18:25 |
| clarkb | oh wait | 18:25 |
| clarkb | I derped and didn't boot with config drive | 18:25 |
| clarkb | mriedem: ^ can nova make that the default already :P | 18:25 |
| fungi | dhellmann: and relatedly, what's the plan for implementing the service type redirects? | 18:26 |
| dhellmann | fungi : ugh, forgot about those | 18:26 |
| rloo | AJaeger: right, we want them to be non-voting. the question was why we had the suffix='nv' if they are nonvoting (line 5777 of https://review.openstack.org/#/c/475521/2/jenkins/jobs/projects.yaml) | 18:26 |
| dhellmann | I'll put that on my list to figure out tomorrow | 18:26 |
| *** amoralej is now known as amoralej|off | 18:27 | |
| AJaeger | rloo, the suffix -nv makes them non-voting | 18:27 |
| rloo | AJaeger: i mean, why not make it suffix=''. but i think it is cuz there are some jobs where it is non-voting | 18:27 |
| fungi | dhellmann: thanks! i'm more comfortable now that i know it's at least part of the plan | 18:27 |
| rloo | AJaeger: well, we just saw that they are voting in the link you provided | 18:27 |
| mriedem | clarkb: force_config_drive=True you mean? | 18:27 |
| AJaeger | rloo: yes, they are voting now | 18:27 |
| clarkb | mriedem: ya | 18:28 |
| rloo | AJaeger: i think i have confused me and/or you. | 18:28 |
| *** priteau has quit IRC | 18:28 | |
| mriedem | clarkb: you could propose a change... | 18:28 |
| dhellmann | fungi : thanks for the review. I'm hoping asettle and I can get the docs spec approved this week so teams can start moving content around. | 18:28 |
| rloo | AJaeger: they are voting. but that suffix='nv' you are saying means that they are non-voting. i think that's my confusion. | 18:28 |
| AJaeger | rloo: you lost me. | 18:29 |
| *** kzaitsev_mb has joined #openstack-infra | 18:29 | |
| AJaeger | rloo: if thee name of the job ends with "-nv", the job will be non-voting | 18:29 |
| AJaeger | rloo: everything seems to be working as it should | 18:30 |
| rloo | AJaeger: https://review.openstack.org/#/c/470406/. gate-grenade-dsvm-ironic-ubuntu-xenial is voting | 18:30 |
| rloo | AJaeger: at https://review.openstack.org/#/c/475521/2/jenkins/jobs/projects.yaml, suffix='nv' | 18:30 |
| rloo | AJaeger: what is that suffix='nv' meant to indicate? | 18:30 |
| rloo | AJaeger: why not do suffix=''? | 18:31 |
| clarkb | hrm so even with config drive we don't get ipv6 info from osic | 18:32 |
| clarkb | cloudnull: ^ do you know why? | 18:32 |
| dhellmann | fungi : for those service type redirects, should that be 301 or 302? | 18:33 |
| *** sbezverk has joined #openstack-infra | 18:33 | |
| fungi | dhellmann: a subtle distinction, and i don't really know which is ultimately more appropriate. do we anticipate redirecting them to different things over time? | 18:33 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Fix periodic jobs uploads https://review.openstack.org/475715 | 18:34 |
| dhellmann | fungi : there's that discussion about trove being deprecated | 18:34 |
| dhellmann | otherwise I'd say 301 | 18:34 |
| * dhellmann remembers that this case is why we have code names in the first place | 18:34 | |
| AJaeger | rloo: if you want it voting, use suffix='', if not voting, use suffix='-nv' | 18:34 |
| rloo | AJaeger: It is voting, and we have suffix='-nv'. | 18:35 |
| dhellmann | fungi : I'm going to say 302 for now | 18:35 |
| *** jcoufal has quit IRC | 18:35 | |
| *** dhinesh has quit IRC | 18:35 | |
| rloo | AJaeger: which is why i am guessing that the suffix=value may only be used at the end of the day to prohibit non-value suffixes from working. but i'm just guessing. | 18:36 |
| clarkb | fungi: I'm currently stumped on how to know better at boot if ipv6 is going to work... | 18:37 |
| tushar1_ | clarkb : current yaml file - https://gist.github.com/anonymous/a3f26baabc70ebb69ebd14df3569ce2c | 18:37 |
| AJaeger | rloo: where is the job voting with a suffix of -nv? Show me a change, please | 18:37 |
| *** watersoul_ has quit IRC | 18:37 | |
| tushar1_ | clarkb : and the error is same | 18:37 |
| mordred | dmsimard: oh awesome. do you have an opinions on how we should handle an image with a properties property? | 18:38 |
| tushar1_ | I validated the file from online yaml validtor | 18:38 |
| mordred | dmsimard: maybe move it to the properties dict with a key of properties if we find one? | 18:38 |
| rloo | AJaeger: i think we are talking in circles, we just agreed i think that gate-grenade-dsvm-ironic-ubuntu-xenial is voting, from https://review.openstack.org/#/c/470406/ | 18:38 |
| dmsimard | mordred: well first things first we got rid of that nonsense | 18:38 |
| *** salv-orlando has quit IRC | 18:38 | |
| mordred | dmsimard: awesome. I think that's a great plan :) | 18:38 |
| AJaeger | rloo: exactly, that job is voting. But it does not have suffix -nv, otherwise the job would be named gate-grenade-dsvm-ironic-ubuntu-xenial-nv | 18:39 |
| clarkb | tushar1_: http://paste.openstack.org/show/613204/ is how it should look, can you update to look like that (with all of the directives) and then see if it works | 18:39 |
| *** salv-orlando has joined #openstack-infra | 18:39 | |
| dmsimard | mordred: now, suppose an operator really does that -- I need to look at that bit of code more closely | 18:39 |
| *** eharney has quit IRC | 18:40 | |
| tushar1_ | clarkb : okk | 18:40 |
| *** watersoul has joined #openstack-infra | 18:40 | |
| rloo | AJaeger: I agree with you, if the job is nonvoting, it would be named with a suffix of '-nv' | 18:41 |
| *** pradk has joined #openstack-infra | 18:41 | |
| rloo | AJaeger: my question is what does this file do, at line 5791: https://github.com/openstack-infra/project-config/blob/d1cf20a9a6b355e56a5554615704f43242241bb7/jenkins/jobs/projects.yaml#L5786-L5791 | 18:41 |
| dmsimard | mordred: just looked, sort of sucks if you ask me -- it's freeform in glance so anyhing you pick can end up being used by an operator/user | 18:42 |
| mordred | dmsimard: yah - I'll poke at that too - I _think_ just putting it into the properties dict (which is where we put all key/value properties in shade) is the best shade can do | 18:42 |
| AJaeger | rloo: that defines two jobs with names gate-grenade-dsvm-ironic-ubuntu-trusty-nv and gate-grenade-dsvm-ironic-ubuntu-xenial-nv | 18:42 |
| dmsimard | mordred: I can file a bug. Launchpad or storyboard ? | 18:43 |
| mordred | dmsimard: storyboard please | 18:43 |
| AJaeger | rloo: those two jobs will be non-voting. You could define in addition also a job like gate-grenade-dsvm-ironic-ubuntu-xenial and have it voting. And thus have the one job in e.g. ironic as non-voting and the other in ironic-ui as voting. | 18:43 |
| rloo | AJaeger: so for the voting job that we identified above, do you think there is some other entry in that projects.yaml that defines those? | 18:43 |
| * clarkb wonders if this isn't a reason to keep nodepool ready scripts around | 18:43 | |
| *** makowals has quit IRC | 18:44 | |
| clarkb | mordred: ^ fyi we don't actually know at boot if ipv6 will work because 1) RAs are async so ipv4 configures first and 2) osic doesn't provide the ipv6 infos in condig drive | 18:44 |
| dmsimard | mordred: oh, you know what's the worst thing about this bug ? | 18:44 |
| mordred | clarkb: AWESOME | 18:44 |
| dmsimard | mordred: that wasn't even the image we were using, it's picked up when listing all images | 18:44 |
| AJaeger | rloo: yes, there is - via some magic ;) | 18:44 |
| clarkb | mordred: so the only real way to know if ipv6 should be preferred is via the boot orchestrator I think | 18:44 |
| clarkb | mordred: which seems like it may be a reason to continue to have ready scripts in nodepool | 18:44 |
| mordred | dmsimard: well, we shoujld _definitely_ make shade not error if someone does it | 18:44 |
| rloo | AJaeger: ok thx, it is the magic that I was missing :-) | 18:44 |
| *** ericyoung has joined #openstack-infra | 18:45 | |
| AJaeger | rloo: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n2497 | 18:45 |
| AJaeger | rloo: I meant http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n2465 | 18:45 |
| AJaeger | And then in projects.yaml the devstack-jobs get "instantiated" | 18:45 |
| *** eharney has joined #openstack-infra | 18:46 | |
| mordred | clarkb: we can also put this in the configure_mirror script / pre-playbook | 18:46 |
| clarkb | mordred: we could except I don't really expect zuul to know or care about these cloud/region specific deployment concerns | 18:47 |
| rloo | AJaeger: ok. so we only need to set suffix="-nv" if we want it to be non-voting. however, it seems to me that we could always set/leave suffix="-nv", because regardless of that value, we can always have a voting job. | 18:47 |
| clarkb | mordred: my first impression is that that is very much a nodepool responsibility | 18:47 |
| mordred | clarkb: no - not zuul - but the pre playbook of the base job is where configure-mirror is happening | 18:47 |
| clarkb | yes that is executed by zuul | 18:48 |
| mordred | yes | 18:48 |
| *** gardlt_ has quit IRC | 18:48 | |
| clarkb | I'm saying thats the wrong layer for this | 18:48 |
| clarkb | because you have to pass through all the weird cloud info from nodepool to zuul or you can just address in nodepool instead | 18:48 |
| mordred | we already do that | 18:48 |
| *** gardlt has joined #openstack-infra | 18:48 | |
| mordred | the pre-playbook already gets the cloud info from nodepool in the inventor vars | 18:49 |
| clarkb | but only region cloud etc not things like this cloud is derp in way X or Y or Z right? | 18:49 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Add Dockerfile https://review.openstack.org/465912 | 18:50 |
| mordred | it gets whatever we put into the playbook - it's just there's no real benefit in having two different places to add playbooks that run on nodes before the job content runs | 18:50 |
| clarkb | also if you fix it in nodepool any instance booted by nodepool gets the right config rather than just those that end up in zuul | 18:50 |
| clarkb | well ^ is a benefit | 18:50 |
| clarkb | if nothing else it allows you to run one system instead of two if you don't need zuul for some reason | 18:50 |
| mordred | well - this is a rather deep design thing - so should likely spin it up for discussion when jeblair is back online | 18:51 |
| AJaeger | rloo: you have different ways to achieve things ;) | 18:51 |
| *** e0ne has joined #openstack-infra | 18:51 | |
| rloo | AJaeger: more than one way is good :-) Thx for taking the time to explain it to me. | 18:52 |
| *** jkilpatr_ has joined #openstack-infra | 18:52 | |
| mordred | in the meantime - it's definitely functionality that can be accomplished in pre playbooks in v3 today and in ready scripts in v2 - so we shouldn't be blocked while we discuss it | 18:52 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: Increase OVN testing coverage https://review.openstack.org/475423 | 18:52 |
| clarkb | ya for 2.5 I think we just check if /etc/nodepool has ipv6 public addrs | 18:53 |
| *** jkilpatr has quit IRC | 18:53 | |
| *** gardlt has quit IRC | 18:53 | |
| clarkb | in a ready script | 18:54 |
| mordred | clarkb: well- by ready script time, it should be possible to check to see if the node has a working v6 address right? | 18:54 |
| *** hrubi has joined #openstack-infra | 18:55 | |
| fungi | clarkb: yeah, i have no good answers for at-boot ipv6. what were the concerns about switching to ipv6 after boot (say via nodepool ready-script)? | 18:56 |
| *** trown|lunch is now known as trown | 18:56 | |
| clarkb | mordred: ya actually at ready script time we have a lot more options | 18:56 |
| mordred | clarkb: netifaces.AF_INET6 in netifaces.gateways()['default'] | 18:56 |
| fungi | clarkb: oh, now that i read scrollback i see you said the same | 18:56 |
| mordred | clarkb: is what we use to detect working ipv6 on the host in shade, fwiw | 18:56 |
| clarkb | fungi: my biggest concern with it is that we don't have a way to solve this problem without zuul in zuulv3 land | 18:56 |
| *** rbrndt has joined #openstack-infra | 18:56 | |
| clarkb | fungi: whereas having the image figure it out properly on boot would be more universal except clouds | 18:57 |
| mordred | clarkb: yah - but we don't have any use-cases without zuul in zuulv3 land atm - so I'd like to be careful that we don't add complexity in support of a non-thing | 18:57 |
| *** gardlt has joined #openstack-infra | 18:57 | |
| *** thorst has joined #openstack-infra | 18:58 | |
| clarkb | mordred: but we dio | 18:58 |
| *** zara_the1lemur__ is now known as zara_the_lemur__ | 18:58 | |
| clarkb | mordred: many people have expressed their desire to use nodepool without zuul | 18:58 |
| clarkb | whether that is just for image curation or maintaing a stable of instances that are ready to be used when needed | 18:59 |
| fungi | i feel similarly that solving it with zuulv3 when we switch to zuulv3 is fine, and if enough people want to use nodepool without zuul and need to have post-boot orchestration then we can revisit adding ready-scripts back in for them or encourage doing that with their own bespoke post-boot configuration management (whatever that happens to be) | 18:59 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix clone for non-existing cache dir https://review.openstack.org/475900 | 18:59 |
| mordred | sure. have those people also expressed the desire to run job setup code using somethign other than their job execution system? | 18:59 |
| clarkb | and dns is fundamental to having a working instance not jsut related to performing ci | 18:59 |
| *** mdrabe_ has joined #openstack-infra | 18:59 | |
| clarkb | mordred: I'm not sure I understand that question | 18:59 |
| fungi | well, dns is working, it's just that after the system is up and reachable we can make _better_ decisions about how to configure it for performance in specific places it gets booted | 19:00 |
| fungi | anyway | 19:00 |
| mordred | I believe the problem si that I view this as a problem that is related to the fact that we use these nodes for CI | 19:00 |
| *** mdrabe has quit IRC | 19:00 | |
| mordred | as fungi mentoins - they do boot with working dns | 19:00 |
| fungi | it's that (weekly infra team meeting) time again! find us in #openstack-meeting for the next hour | 19:00 |
| mordred | it'sjust not theBEST dns they could have | 19:00 |
| clarkb | I don't think thats the case consider rax where you don't get working dns at all by default | 19:00 |
| clarkb | (because of blacklists) | 19:00 |
| fungi | clarkb: if you really want it "at boot" (for fuzzy values of what that means) then udev rules maybe? | 19:01 |
| mordred | and the cost of less good DNS is most heavily impacting to ci jobs - if someone wants to use nodepool to manage nodes for non-ci uses, that's neat, but they're going to manage the content on those nodes in some manner (config management perhaps?) | 19:01 |
| *** gardlt has quit IRC | 19:01 | |
| mordred | so I'm mostly suggesting that the thing that is responsible for using the nodes be the thing that decides whether or not it wants to alter aspects of how the node is configured | 19:02 |
| *** rockyg has joined #openstack-infra | 19:02 | |
| clarkb | right and I'm saying isn't that literally nodepool? | 19:04 |
| clarkb | it has all of the info necessary and controls the boot delete process | 19:04 |
| Shrews | mordred: clarkb: so much scrollback, but is the tl;dr here determining which ip is the interface ip for a nodepool provided node? | 19:04 |
| clarkb | Shrews: more specifically what version of IP is available | 19:04 |
| clarkb | we don't necessarily care about the specific address | 19:04 |
| Shrews | nodepool records both ipv4 & ipv6 & the shade-determined interface ip in zk | 19:05 |
| *** xarses has joined #openstack-infra | 19:05 | |
| tushar1_ | clarkb : still same issue | 19:05 |
| tushar1_ | clarkb : current file - https://gist.github.com/anonymous/05114618b171f188a72ceb19e8eb6ca3 | 19:05 |
| clarkb | Shrews: yup nodepool has the info | 19:06 |
| clarkb | Shrews: talking about how best to expose that in the instance to make decisions | 19:06 |
| mordred | clarkb: to me that's not nodepool - nodepool's job is to get nodes, not to execute content on them | 19:06 |
| mordred | clarkb: it is a service that provides nodes to things that do want to execute content on them | 19:06 |
| mordred | if we also have nodepool execute content on the nodes, then we have 2 systems for executing content on nodes that are provided by nodepool | 19:07 |
| *** xarses_ has joined #openstack-infra | 19:07 | |
| *** e0ne has quit IRC | 19:08 | |
| clarkb | I mostly agree except that working networking is a fundamental prerequisite, ideally clouds would solve/address this for us but reality is they dont | 19:08 |
| *** gardlt has joined #openstack-infra | 19:08 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at using CI mirrors for fedora/centos https://review.openstack.org/475607 | 19:08 |
| Shrews | why can't the non-zuul thing wanting to execute the content make the decision based on the info recorded in ZK? | 19:09 |
| mordred | clarkb: sure - and I agree that working networking is a fundamental prerequisite ... I just don't agree that optimized dns == working networking | 19:09 |
| clarkb | mordred: see I don't see it as optimized dns because of the rax issue | 19:09 |
| *** xarses has quit IRC | 19:09 | |
| clarkb | you don't get working dns in rax if you use the cloud provided answer to dns | 19:09 |
| mordred | clarkb: right. which we solve with unbound in the image content, no? | 19:10 |
| *** Guest72024 has quit IRC | 19:10 | |
| clarkb | mordred: sort of, we don't configure unbound properly because image build is too early to do that | 19:10 |
| clarkb | and this is the problem I am trying to sovle | 19:10 |
| ianw | (is this not configured properly ... or a workaround to empirical evidence of an issue, but is probably a bug? ... in my mind it's different) | 19:11 |
| mordred | right. so we have "working" but less optimal unbound config from the image | 19:11 |
| *** xarses_ has quit IRC | 19:12 | |
| mordred | clarkb: I do believe we at least know precisely where the disagreement is - so discussing it in detail shouldnt' be terribly hard :) | 19:12 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Install additional required packages for DLRN https://review.openstack.org/475818 | 19:13 |
| *** e0ne has joined #openstack-infra | 19:14 | |
| *** bhavik1 has joined #openstack-infra | 19:14 | |
| *** eranrom has quit IRC | 19:14 | |
| *** thorst has quit IRC | 19:15 | |
| *** bhavik1 has quit IRC | 19:15 | |
| *** thorst has joined #openstack-infra | 19:15 | |
| *** xarses has joined #openstack-infra | 19:16 | |
| *** rhallisey-away is now known as rhallisey | 19:16 | |
| *** xarses_ has joined #openstack-infra | 19:18 | |
| *** e0ne has quit IRC | 19:19 | |
| *** hasharAway is now known as hashar | 19:19 | |
| *** makowals has joined #openstack-infra | 19:20 | |
| *** thorst has quit IRC | 19:20 | |
| *** xarses has quit IRC | 19:21 | |
| *** nicolasbock has quit IRC | 19:21 | |
| *** xarses has joined #openstack-infra | 19:22 | |
| *** dhinesh has joined #openstack-infra | 19:23 | |
| *** xarses_ has quit IRC | 19:23 | |
| *** tushar1_ has quit IRC | 19:23 | |
| openstackgerrit | Kirill Zaitsev proposed openstack-infra/project-config master: Add voting docs jobs to kuryr-tempest-plugin https://review.openstack.org/475904 | 19:23 |
| mordred | mmedvede: AHA! I have figured out what your issue is | 19:25 |
| *** priteau has joined #openstack-infra | 19:25 | |
| *** priteau has quit IRC | 19:25 | |
| mordred | Shrews: ^^ the thing mmedvede reported back in scrollback about deleteing floating ips ... there is another case | 19:26 |
| *** priteau has joined #openstack-infra | 19:26 | |
| Shrews | mordred: do tell | 19:28 |
| *** e0ne has joined #openstack-infra | 19:29 | |
| *** ericyoung has quit IRC | 19:29 | |
| *** rderose has quit IRC | 19:33 | |
| *** gomarivera has quit IRC | 19:36 | |
| *** gomarivera has joined #openstack-infra | 19:36 | |
| *** e0ne_ has joined #openstack-infra | 19:37 | |
| *** e0ne has quit IRC | 19:39 | |
| *** dimtruck is now known as zz_dimtruck | 19:39 | |
| mordred | Shrews: we throw an exception in list_floating_ips for neutron if there is a filter pushdown condition, rather than just returning a [] | 19:40 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Switch apt-sources test to ubuntu-minimal https://review.openstack.org/475907 | 19:40 |
| *** e0ne has joined #openstack-infra | 19:40 | |
| mordred | Shrews: I'm going to push up two different possible fixes and see what you think | 19:40 |
| *** gomarivera has quit IRC | 19:41 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Don't fail hard on 404 from neutron FIP listing https://review.openstack.org/475908 | 19:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Switch apt-sources test to ubuntu-minimal https://review.openstack.org/475907 | 19:44 |
| *** e0ne_ has quit IRC | 19:44 | |
| *** mgkwill_ is now known as mgkwill | 19:46 | |
| *** florianf has quit IRC | 19:53 | |
| *** gomarivera has joined #openstack-infra | 19:53 | |
| *** gomarivera has quit IRC | 19:53 | |
| *** zzzeek has quit IRC | 19:53 | |
| *** gomarivera has joined #openstack-infra | 19:54 | |
| ianw | E: Failed getting release signature file http://mirror.sto2.citycloud.openstack.org/ubuntu/dists/xenial/Release.gpg <-- i feel like i know why this is, but can't remember atm ... sure i've discussed it with pabelanger | 19:54 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Only search for floating ips if the server has them https://review.openstack.org/475909 | 19:54 |
| openstackgerrit | Tom Barron proposed openstack-infra/project-config master: Experimental py35 dsvm for ceph devstack plugin https://review.openstack.org/454890 | 19:54 |
| clarkb | ianw: we don't sign our reprepro built mirrors | 19:55 |
| clarkb | ianw: and since reprepro is building from scratch it can't just mirror the upstream | 19:56 |
| *** e0ne has quit IRC | 19:57 | |
| fungi | also, not allowing secure apt discourages people from trying to use our mirrors outside our ci environment | 19:58 |
| fungi | which i count as a "feature" | 19:58 |
| ianw | true ... it throws somewhat of a spanner in the works for dib using those mirrors | 19:58 |
| *** JudeC has quit IRC | 20:00 | |
| *** Guest72024 has joined #openstack-infra | 20:03 | |
| *** Apoorva_ has joined #openstack-infra | 20:05 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: Use ipv6 resolvers only with ipv6 networking https://review.openstack.org/475910 | 20:06 |
| clarkb | fungi: mordred ianw ^ thats the raedy script change | 20:06 |
| fungi | thanks! | 20:07 |
| clarkb | I'm going to pop out now before fungi and my scheduled review-dev fun | 20:07 |
| clarkb | will be back for that | 20:07 |
| fungi | i'll be amused to see if any jobs break ipv6 somehow and will suddenly start failing dns resolution when that goes in | 20:08 |
| *** Apoorva has quit IRC | 20:08 | |
| clarkb | I think that that would break ssh to the nodes too | 20:09 |
| clarkb | so shouldn't happen without already being a problem | 20:09 |
| clarkb | though if established connections are unnaffected it might | 20:09 |
| *** jkilpatr_ has quit IRC | 20:10 | |
| *** JudeC has joined #openstack-infra | 20:12 | |
| *** jascott1 has quit IRC | 20:15 | |
| *** jascott1 has joined #openstack-infra | 20:16 | |
| *** jascott1 has quit IRC | 20:17 | |
| openstackgerrit | Arx Cruz proposed openstack-infra/devstack-gate master: Enable process_stackviz function to be optional https://review.openstack.org/475788 | 20:18 |
| *** mdrabe_ is now known as mdrabe | 20:18 | |
| *** e0ne has joined #openstack-infra | 20:19 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Switch apt-sources test to ubuntu-minimal https://review.openstack.org/475907 | 20:19 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Remove potentially compromised bkero key https://review.openstack.org/475914 | 20:21 |
| clarkb | fungi: ^ | 20:21 |
| fungi | thanks! +3 | 20:22 |
| *** amotoki_away is now known as amotoki | 20:24 | |
| clarkb | fungi: I also updated the gerrit upgrade etherpad with a link to the cahnge that will bump gerrit war on review-dev | 20:25 |
| clarkb | so I think we are fairly well set to get started on that once not in meetings | 20:25 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Switch apt-sources test to ubuntu-minimal https://review.openstack.org/475907 | 20:26 |
| clarkb | ianw: I noticed that my python-devel fix for suse is failing a lot | 20:26 |
| fungi | clarkb: thanks! maybe i can find bandwidth to review it now | 20:26 |
| clarkb | ianw: looking at logs it looks like mount is trying to mount something not ready? | 20:26 |
| clarkb | ianw: or there is some race in mount? is that a known issue? | 20:26 |
| ianw | clarkb: yes ... there's tons of different errors atm. sometimes it's the dns issues, other times it's mirror failures. i think the error you're seeing is on the failure path, which is a bit of a red herring | 20:27 |
| clarkb | ah ok | 20:27 |
| ianw | hence me trying to get things using mirrors more ... it's a lot harder than you might think :( | 20:27 |
| clarkb | ianw: :( | 20:27 |
| *** kzaitsev_mb has quit IRC | 20:28 | |
| clarkb | ianw: hopefuilly we are close to resolving these dns issues at least | 20:30 |
| *** kzaitsev_mb has joined #openstack-infra | 20:31 | |
| *** amotoki is now known as amotoki_away | 20:33 | |
| *** xyang1 has joined #openstack-infra | 20:33 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Switch apt-sources test to ubuntu-minimal https://review.openstack.org/475907 | 20:36 |
| *** kgiusti has left #openstack-infra | 20:36 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Switch apt-sources test to ubuntu-minimal https://review.openstack.org/475907 | 20:39 |
| *** shardy has quit IRC | 20:40 | |
| *** Goneri has quit IRC | 20:40 | |
| *** zzzeek has joined #openstack-infra | 20:42 | |
| *** ijw has joined #openstack-infra | 20:45 | |
| *** eharney has quit IRC | 20:46 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Switch apt-sources test to ubuntu-minimal https://review.openstack.org/475907 | 20:47 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Switch apt-sources test to ubuntu-minimal https://review.openstack.org/475907 | 20:51 |
| *** makowals has quit IRC | 20:52 | |
| mmedvede | mordred: would you like me to test your shade patches? | 20:53 |
| fungi | the ericcson platinum member application presentation included a great shout out to "openstack ci/cd and development workflows" which they're applying internally | 20:54 |
| mordred | mmedvede: yes please! I believe I found the issue this time for sure though :) | 20:54 |
| mmedvede | ok ok :) | 20:54 |
| *** e0ne has quit IRC | 20:55 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Switch apt-sources test to ubuntu-minimal https://review.openstack.org/475907 | 20:58 |
| *** camunoz has quit IRC | 21:00 | |
| fungi | clarkb: should openstack-infra/logstash-filters be an infra deliverable? i think we probably just missed adding it in governance | 21:02 |
| *** kzaitsev_mb has quit IRC | 21:04 | |
| *** abelur has quit IRC | 21:04 | |
| clarkb | fungi: yes it should | 21:04 |
| clarkb | fungi: re ericcson nice | 21:04 |
| fungi | clarkb: did you see zaro's comment on the review-dev upgrade change? | 21:05 |
| fungi | looks like he already had one posted from ~1.5 months ago | 21:05 |
| fungi | er, ~2.5 months ago i mean | 21:05 |
| clarkb | I didn't we can use that one | 21:05 |
| clarkb | also I want to add a db backup step | 21:06 |
| fungi | yeah, let's go with the older one, and db backup sounds like a great idea | 21:06 |
| rockyg | hey guys. | 21:06 |
| mmedvede | mordred: neither patch had an effect :(. To made sure nodepool is using the new code by adding some custom logging to it | 21:07 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: upgrade review-dev.o.o with gerrit v2.13.7 https://review.openstack.org/453323 | 21:09 |
| rockyg | oops..wanted to be in dev. just passing thru. see ya! | 21:09 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Don't fail hard on 404 from neutron FIP listing https://review.openstack.org/475908 | 21:09 |
| clarkb | fungi: ^ rebased to make parent current | 21:09 |
| fungi | thanks | 21:09 |
| clarkb | and updated etherpad | 21:09 |
| *** trown is now known as trown|outtypewww | 21:09 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Catch URI Not found on floating ip search https://review.openstack.org/475927 | 21:10 |
| mordred | mmedvede: try that one ^^ | 21:10 |
| *** rhallisey has quit IRC | 21:10 | |
| openstackgerrit | Merged openstack-infra/project-config master: Use ipv6 resolvers only with ipv6 networking https://review.openstack.org/475910 | 21:12 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add configuration documentation https://review.openstack.org/463328 | 21:12 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP: reorganize docs into user/admin guide https://review.openstack.org/475928 | 21:12 |
| *** makowals has joined #openstack-infra | 21:13 | |
| *** kzaitsev_mb has joined #openstack-infra | 21:13 | |
| clarkb | fungi: I have made a root screen on review-dev | 21:13 |
| clarkb | even made my terminal tiny for you :) | 21:14 |
| clarkb | I am going to stop puppet now by editing the emergency file | 21:14 |
| *** camunoz has joined #openstack-infra | 21:14 | |
| *** dprince has quit IRC | 21:14 | |
| mmedvede | mordred: appears the same http://paste.openstack.org/show/613219/ | 21:15 |
| fungi | clarkb: thanks! i'm attached to your (tiny) screen now | 21:16 |
| *** jklare has quit IRC | 21:16 | |
| *** jkilpatr_ has joined #openstack-infra | 21:16 | |
| clarkb | fungi: should I just continue with the plan on the etherpad? | 21:16 |
| *** JudeC has quit IRC | 21:16 | |
| *** salv-orl_ has joined #openstack-infra | 21:17 | |
| fungi | clarkb: yep, let's go for it | 21:17 |
| clarkb | ok stopping gerrit now and will continue through. Are people in channel interested in updates? | 21:17 |
| fungi | did you confirm puppet has actually stopped running against review-dev? | 21:17 |
| fungi | as in not still in the middle of whatever run it started before you updated the emergency list | 21:17 |
| clarkb | fungi: I haven't I guess if it was already running against it then it could sneak in? let me check | 21:17 |
| *** JudeC has joined #openstack-infra | 21:18 | |
| clarkb | it is indeed in the middle of an update pass so maybe we wait for that to finish | 21:18 |
| fungi | i usually check that because it's caught me unawares more times than i can count | 21:18 |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: update toci to use --environment and --nodes options in quickstart https://review.openstack.org/475930 | 21:18 |
| *** jbadiapa_ has quit IRC | 21:19 | |
| fungi | #status log The Gerrit service on review-dev.openstack.org is being taken offline for an upgrade to 2.13.7.4.988b40f | 21:19 |
| openstackstatus | fungi: finished logging | 21:19 |
| clarkb | ansible is doing the things so ya just waiting I guess | 21:19 |
| *** salv-orlando has quit IRC | 21:20 | |
| *** ldnunes has quit IRC | 21:20 | |
| *** jklare has joined #openstack-infra | 21:20 | |
| fungi | started at 21:00 and the emergency file was last modified at 21:14 so yeah, i would wait | 21:20 |
| fungi | clarkb: Jun 20 21:12:40 review-dev puppet-user[23871]: Finished catalog run in 12.30 sec | 21:21 |
| fungi | so probably actually safe to proceed?> | 21:22 |
| *** armax has joined #openstack-infra | 21:22 | |
| clarkb | 2017-06-20 21:12:58,915 p=17490 u=root | changed: [review-dev.openstack.org] from the puppetmaster log | 21:22 |
| clarkb | I agree should be safe to proceed | 21:22 |
| *** kzaitsev_mb has quit IRC | 21:24 | |
| clarkb | fungi: that sha1sum lgtm | 21:25 |
| fungi | i concur | 21:25 |
| clarkb | fungi: I want to double check really quickly that we don't need to do anything with gerrit.sh | 21:26 |
| openstackgerrit | Merged openstack-infra/shade master: Only search for floating ips if the server has them https://review.openstack.org/475909 | 21:26 |
| clarkb | fungi: looks like that comes out of the initial gerrit init. So likely we don't need to do anything to it | 21:27 |
| * clarkb continues with plan | 21:27 | |
| fungi | k | 21:27 |
| *** abelur has joined #openstack-infra | 21:28 | |
| clarkb | fungi: ready to run the init? | 21:28 |
| fungi | yep, all set i guess | 21:28 |
| *** abelur has quit IRC | 21:29 | |
| fungi | ugh | 21:29 |
| clarkb | wee :) | 21:29 |
| clarkb | I think puppet-gerrit may actualyl do this for us | 21:30 |
| fungi | oh, should we be not root here? | 21:30 |
| clarkb | but since we are doing manual to watch the reindex... | 21:30 |
| *** bkero has joined #openstack-infra | 21:30 | |
| *** bkero has quit IRC | 21:30 | |
| *** bkero has joined #openstack-infra | 21:30 | |
| clarkb | fungi: they are all root and old, I think with puppet we just force it to be right with puppet | 21:30 |
| *** abelur has joined #openstack-infra | 21:31 | |
| clarkb | fungi: rather than have the init command do it, so we want to mimic that here and I just derped it | 21:31 |
| fungi | i mean, yes, puppet gerrit would normally take care of it, but file permissions aside we should be doing this as gerrit2 instead normally? | 21:31 |
| *** jkilpatr_ has quit IRC | 21:31 | |
| *** jkilpatr has joined #openstack-infra | 21:31 | |
| *** gouthamr has quit IRC | 21:31 | |
| fungi | oh, i guess it doesn't really create new files which gerrit2 will need to modify later | 21:32 |
| fungi | and we have a chown further down | 21:32 |
| fungi | okay | 21:32 |
| clarkb | ya though we do run as user gerrit2 in puppet | 21:32 |
| *** thorst has joined #openstack-infra | 21:32 | |
| clarkb | anyways I need to edit things to copy the libs and tidy the libs as puppet would :/ | 21:33 |
| clarkb | so this will take a few minutes to get straight | 21:33 |
| fungi | sure | 21:33 |
| fungi | i also need to grab some food in a few minutes, but still definitely around | 21:33 |
| *** camunoz has quit IRC | 21:34 | |
| clarkb | ok | 21:34 |
| *** gardlt has quit IRC | 21:34 | |
| *** gardlt has joined #openstack-infra | 21:34 | |
| *** dave-mccowan has quit IRC | 21:35 | |
| clarkb | ooh I think Isee a bug in the puppet | 21:35 |
| *** dhajare has quit IRC | 21:36 | |
| *** pramodrj07 has joined #openstack-infra | 21:37 | |
| clarkb | oh nevermind it handles that case | 21:38 |
| *** baoli has quit IRC | 21:40 | |
| *** MasterOfBugs has quit IRC | 21:40 | |
| *** jklare has quit IRC | 21:41 | |
| *** Guest72024 has quit IRC | 21:42 | |
| *** rfolco has quit IRC | 21:42 | |
| *** Guest72024 has joined #openstack-infra | 21:42 | |
| *** jklare has joined #openstack-infra | 21:43 | |
| clarkb | fungi: how does that look? ready to run init again you think? | 21:45 |
| * clarkb goes for it | 21:46 | |
| *** gouthamr has joined #openstack-infra | 21:47 | |
| *** thorst has quit IRC | 21:47 | |
| clarkb | fungi: it would apepar this is why it is important to run as gerrit2 not root | 21:47 |
| fungi | clarkb: yep | 21:48 |
| fungi | sorry, got sidetracked by food for a sec | 21:48 |
| clarkb | fungi: I'm going to do a chown -R gerrit2:gerrit2 /home/gerrit2/review_site that shouldn't be necessary in production now that we've documented running the init as gerrit2 | 21:48 |
| fungi | yeah that sounds better | 21:49 |
| clarkb | fungi: also the puppet has a step to delete those jars it just downloaded so I'll add that | 21:50 |
| clarkb | this db migration isn't fast | 21:50 |
| clarkb | we may want to perform it on a backup of the production db | 21:51 |
| clarkb | to time it | 21:51 |
| fungi | i wonder how easy that'll be without a copy of the git repos, but in theory it ought to work | 21:53 |
| *** hashar has quit IRC | 21:53 | |
| *** armax has quit IRC | 21:54 | |
| fungi | latest gzip mysqldumps there are 1gb | 21:55 |
| clarkb | fungi: so that init removed our libs | 21:55 |
| fungi | hah | 21:56 |
| clarkb | looking at puppet I don't know that we garunteee that our lib install will happen after init | 21:56 |
| clarkb | also we will delete the libs that gerrit installed :/ | 21:56 |
| *** markvoelker has quit IRC | 21:56 | |
| fungi | we've gone back and forth on that. for a while we were symlinking to the distro packaged versions when they were sufficiently new | 21:57 |
| *** jklare has quit IRC | 21:57 | |
| *** rockyg has quit IRC | 21:57 | |
| *** thorst has joined #openstack-infra | 21:57 | |
| clarkb | fungi: I'm going to test it as if puppet were running, remove the gerrit install libs and install our own libs | 21:58 |
| clarkb | since that seems to be the intent of what puppet is doing | 21:58 |
| fungi | go for it | 21:58 |
| *** jklare has joined #openstack-infra | 21:59 | |
| fungi | and yeah, i wouldn't be surprised by holes in the upgrade logic in puppet at this point. i don't think we've effectively exercised it between anything other than patch upgrades (and maybe not even that) in a while | 21:59 |
| *** priteau has quit IRC | 22:00 | |
| fungi | mainly owing to needing hours for offline reindexing | 22:00 |
| *** wolverineav has quit IRC | 22:00 | |
| *** wolverineav has joined #openstack-infra | 22:01 | |
| clarkb | ya | 22:01 |
| openstackgerrit | Yolande Amate proposed openstack-infra/jenkins-job-builder master: Update description_setter to make use of convert_mapping_to_xml() https://review.openstack.org/475147 | 22:02 |
| *** gardlt has quit IRC | 22:02 | |
| mmedvede | mordred: adding explicit config-drive setting to nodepool config fixes the shade error. I guess default needs to change from 'None' to 'false', or shade needs to convert 'None' into 'false'. I did not dive deep enough to understand why it was not happening for shade 1.21.0 | 22:03 |
| *** armax has joined #openstack-infra | 22:03 | |
| clarkb | fungi: I worry though that this means we can't even do point release upgrades automatically | 22:03 |
| clarkb | fungi: beacuse the init will blow away our libs | 22:03 |
| fungi | i agree entirely, that's a challenge | 22:04 |
| *** markvoelker has joined #openstack-infra | 22:04 | |
| clarkb | though I guess with proper ordering in the puppet we can make that work | 22:04 |
| *** thorst has quit IRC | 22:05 | |
| clarkb | big issue right now is we can theoretically decide in puppet no need to download libs because there, run init, they are now gone | 22:05 |
| *** thorst has joined #openstack-infra | 22:05 | |
| *** wolverineav has quit IRC | 22:05 | |
| clarkb | I guess we can test that when we go to .8 :) | 22:06 |
| *** mdrabe has quit IRC | 22:06 | |
| fungi | right now i think our puppet still assumes offline reindexing on every upgrade too, right? | 22:06 |
| clarkb | no it by default does online iirc | 22:06 |
| clarkb | I had a change up to make it do offline for this upgrade if we wanted to try having it be automated | 22:06 |
| *** thorst has quit IRC | 22:07 | |
| *** jamesmcarthur has quit IRC | 22:07 | |
| fungi | ahh, okay, i didn't realize we'd changed that already. good to know | 22:07 |
| *** esberglu has quit IRC | 22:07 | |
| *** esberglu has joined #openstack-infra | 22:08 | |
| fungi | but also running offline reindex under puppet for hours is not so great. you lose all the status feedback | 22:08 |
| *** gardlt has joined #openstack-infra | 22:08 | |
| clarkb | yup I think offline reindex under puppet is sadness | 22:09 |
| clarkb | so having puppet only do online then doing offline like this is likely best | 22:09 |
| *** yee37923 has joined #openstack-infra | 22:10 | |
| *** yee37923 has quit IRC | 22:10 | |
| *** openstack has joined #openstack-infra | 22:15 | |
| *** puiterwi1 has joined #openstack-infra | 22:16 | |
| *** mrunge has joined #openstack-infra | 22:16 | |
| *** jrist has joined #openstack-infra | 22:16 | |
| *** jgriffith has joined #openstack-infra | 22:16 | |
| *** cburgess has joined #openstack-infra | 22:16 | |
| *** xarses_ has joined #openstack-infra | 22:16 | |
| *** Apoorva_ has quit IRC | 22:17 | |
| *** dansmith_ is now known as dansmith | 22:17 | |
| *** oomichi has quit IRC | 22:17 | |
| *** Apoorva has joined #openstack-infra | 22:17 | |
| clarkb | fungi: this is going to be so slow on the production site | 22:17 |
| *** dansmith is now known as Guest96259 | 22:17 | |
| *** gardlt has quit IRC | 22:17 | |
| *** jklare has quit IRC | 22:17 | |
| *** hrubi has quit IRC | 22:17 | |
| fungi | i am painfully aware | 22:18 |
| fungi | it's been slow in the past too | 22:18 |
| inc0 | hey, does zuul cloner work with cross-project deps? | 22:18 |
| *** dhellmann has quit IRC | 22:18 | |
| fungi | clarkb: i'm expecting at least a four-hour outage for everything, maybe longer | 22:18 |
| fungi | inc0: yes | 22:18 |
| *** jrist has quit IRC | 22:18 | |
| *** xarses has quit IRC | 22:19 | |
| *** dhellmann has joined #openstack-infra | 22:19 | |
| *** xarses_ has quit IRC | 22:19 | |
| fungi | inc0: as long as everything the job needs that might have a depends-on declared is also obtained via zuul-cloner rather than from pypi or something | 22:19 |
| inc0 | yeah should be | 22:19 |
| *** oomichi has joined #openstack-infra | 22:19 | |
| inc0 | do you have docs for cross project deps handy? | 22:20 |
| *** dhellmann has quit IRC | 22:20 | |
| *** yee379 has quit IRC | 22:20 | |
| *** xarses has joined #openstack-infra | 22:20 | |
| *** dosaboy has joined #openstack-infra | 22:20 | |
| *** hrubi has joined #openstack-infra | 22:20 | |
| *** Adri2000 has joined #openstack-infra | 22:20 | |
| *** xarses has quit IRC | 22:20 | |
| *** xarses_ has joined #openstack-infra | 22:20 | |
| *** dhellmann has joined #openstack-infra | 22:20 | |
| *** neiljerram has quit IRC | 22:20 | |
| *** yee379 has joined #openstack-infra | 22:20 | |
| *** jascott1 has joined #openstack-infra | 22:20 | |
| *** andreww has joined #openstack-infra | 22:21 | |
| *** andreww has quit IRC | 22:21 | |
| fungi | inc0: somewhere... lemme get you links, just a sec | 22:21 |
| inc0 | thank you very much fungi, I have problems finding it | 22:21 |
| fungi | inc0: https://docs.openstack.org/infra/manual/developers.html#cross-repository-dependencies is one place we go over them | 22:21 |
| *** gtmanfred has quit IRC | 22:21 | |
| fungi | inc0: also https://docs.openstack.org/infra/zuul/gating.html#cross-repository-dependencies | 22:22 |
| clarkb | fungi: 2.13.8 looks like a pita https://www.gerritcodereview.com/releases/2.13.md#Manual-Schema-Update-for-Reviewed-Flags | 22:22 |
| *** andreww has joined #openstack-infra | 22:23 | |
| clarkb | there are not enough words around ^ to understand what they mean | 22:23 |
| *** puiterwi1 is now known as puiterwijk | 22:23 | |
| clarkb | we use mysql but gerrit added some h2 stuff in 2.13? | 22:23 |
| *** gomarivera has joined #openstack-infra | 22:23 | |
| fungi | inc0: the section just before that one in the zuul docs about cross-project testing is probably also useful to provide some context | 22:23 |
| *** gardlt has joined #openstack-infra | 22:24 | |
| *** jklare has joined #openstack-infra | 22:24 | |
| fungi | clarkb: huh... wow? | 22:24 |
| *** gtmanfred has joined #openstack-infra | 22:24 | |
| clarkb | ugh this makes me wish we didn't do .7 now | 22:24 |
| clarkb | zaro: ^ can you clarify that? | 22:24 |
| *** mika has joined #openstack-infra | 22:24 | |
| *** mika has joined #openstack-infra | 22:24 | |
| *** gordc has quit IRC | 22:24 | |
| *** StevenK has joined #openstack-infra | 22:25 | |
| clarkb | fungi: ya I think they put some things in h2 regardless of what other db you use | 22:25 |
| clarkb | fungi: then with 2.13.8 they have you move to mysql | 22:25 |
| fungi | looks that way | 22:26 |
| clarkb | it is confusing because then after they say use mysql they have you run commands for h2 to increase the file_name column width | 22:26 |
| fungi | maybe we should revise our plan, restore from db backup after we're done testing 2.13.7 and reupgrade from 2.11 to 2.13.8 directly? | 22:27 |
| fungi | since it sounds like we don't need to perform those steps when going from <2.13 to 2.13.8 | 22:27 |
| clarkb | fungi: its tempting though that may make review-dev unhappy if db and git repos don't line up | 22:27 |
| clarkb | and yes looks like we don't need these steps if going straight to 2.13.8 | 22:27 |
| fungi | hrm, i guess we can just restore the db and redowngrade immediately after testing the upgrade process, alternatively | 22:28 |
| *** gardlt has quit IRC | 22:28 | |
| clarkb | or just plan to perform those steps? | 22:28 |
| *** apuimedo has joined #openstack-infra | 22:28 | |
| clarkb | I really wish the docs were more complete here | 22:28 |
| clarkb | I'm still confused as to when this is necessary and why h2 shows up for file_name column if I am moving off h2 | 22:29 |
| fungi | but regardless, testing an upgrade from 2.11 to 2.13.8 now seems more important since it would be good to test skipping that step | 22:29 |
| *** gomarivera has quit IRC | 22:29 | |
| apuimedo | Hi, could somebody add me to the kuryr-release team? It's still only populated by the previous PTL | 22:29 |
| clarkb | fungi: ya | 22:29 |
| openstackgerrit | Merged openstack-infra/shade master: Don't fail hard on 404 from neutron FIP listing https://review.openstack.org/475908 | 22:30 |
| *** jascott1 has quit IRC | 22:31 | |
| clarkb | fungi: looks like you may haev to configure 2.13.8 to get it in the non default location regardless | 22:31 |
| *** esberglu has joined #openstack-infra | 22:32 | |
| clarkb | fungi: anyways let me finish up with 2.13.7 then we can see what zaro and/or #gerrit say about 2.13.8 | 22:32 |
| fungi | k | 22:32 |
| *** Apoorva_ has joined #openstack-infra | 22:33 | |
| *** Apoorva has quit IRC | 22:33 | |
| clarkb | hrm looks like we have 3 additional non default plugins to sort out /me digs into that | 22:34 |
| fungi | if one of them is the delete project plugin, we never ended up using it and can remove it as far as i'm concerned | 22:35 |
| *** rbrndt has quit IRC | 22:36 | |
| clarkb | javamelody and its-storyboard | 22:36 |
| clarkb | (and delete but I guess we can ignore that for now) | 22:36 |
| clarkb | it doesn't look like we have a 2.13.7 javamelody | 22:36 |
| *** esberglu has quit IRC | 22:37 | |
| clarkb | http://tarballs.openstack.org/ci/gerrit/plugins/javamelody/ | 22:37 |
| * clarkb figures out how to make one of those | 22:37 | |
| *** wolverineav has joined #openstack-infra | 22:37 | |
| *** rossella__ has quit IRC | 22:38 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at using CI mirrors for fedora/centos https://review.openstack.org/475607 | 22:38 |
| clarkb | we are supposed to periodically build them, guessing they don't build for some reason | 22:38 |
| clarkb | http://logs.openstack.org/periodic/javamelody-gerrit-plugin-stable-2.13/078bd32/console.html#_2017-06-20_06_07_04_615973 | 22:39 |
| clarkb | fungi: given that ^ needs fixing. We could rollback and instead plan for 2.13.8 | 22:40 |
| openstackgerrit | John L. Villalovos proposed openstack-dev/hacking master: Add a check to enforce use of 'autospec' in mock.patch https://review.openstack.org/447252 | 22:40 |
| * clarkb is hitting point of upgrade where everything is far more complicated than it used to be and getting frustrated | 22:41 | |
| fungi | er, yeah | 22:41 |
| fungi | let's do that | 22:42 |
| *** sbezverk has quit IRC | 22:42 | |
| fungi | we've gotten some very good data out of this run | 22:42 |
| clarkb | yup defiitely useful if not happy making :) | 22:42 |
| fungi | in a better position to gauge what's involved than we were, for sure | 22:42 |
| clarkb | fungi: ok I've written down a list of steps for a rollback | 22:43 |
| clarkb | want to give them a quick look over? | 22:44 |
| clarkb | fungi: are you able to run the db restore? | 22:44 |
| clarkb | I can sort out the on disk mess | 22:44 |
| fungi | clarkb: yeah, lgtm. and good that we have it in case we need to do another rollback in the future. i'll start on the db restore in a separate terminal session | 22:45 |
| clarkb | fungi: ok I am going to start moving wars and libs around and the secondary index, this shouldn't affect your work with the db | 22:46 |
| *** jascott1 has joined #openstack-infra | 22:46 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/devstack-gate master: Remove setting IRONIC_VM_SPECS_RAM in devstack-gate https://review.openstack.org/422911 | 22:46 |
| *** mriedem1 has joined #openstack-infra | 22:48 | |
| fungi | database restore complete (i think) | 22:48 |
| fungi | i did `zcat /home/gerrit2/mysql_backups/2.11-backup.sql.gz | sudo -H mysql` | 22:48 |
| clarkb | fungi: I don't think sudo -H mysql will use the remote db | 22:49 |
| clarkb | since the creds are in a non default file | 22:49 |
| clarkb | that may hit the lcoal db if it is still floating around | 22:49 |
| fungi | it has a host=<REDACTED>.rackspaceclouddb.com in it | 22:49 |
| mordred | mmedvede: awesome - I can fix that too | 22:49 |
| fungi | clarkb: in the ~root/.my.cnf i mean | 22:49 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Send config_drive as False, not as None https://review.openstack.org/475950 | 22:50 |
| clarkb | fungi: oh it does? check it against the other .cnf | 22:50 |
| *** mriedem has quit IRC | 22:50 | |
| clarkb | fungi: its possible we fixed that and made it default too | 22:50 |
| fungi | clarkb: just confirmed, its contents match .gerrit_db.cnf | 22:50 |
| clarkb | cool | 22:50 |
| fungi | i can drop the db and restore again just to be sure | 22:51 |
| clarkb | up to you | 22:51 |
| fungi | the earlier command provided far less feedback than i had hoped, so i don't actually know if it did anything | 22:51 |
| fungi | clarkb: yeah, seems to have worked as intended | 22:52 |
| *** pramodrj07 has quit IRC | 22:52 | |
| *** pramodrj07 has joined #openstack-infra | 22:53 | |
| *** xyang1 has quit IRC | 22:55 | |
| clarkb | fungi: ok I think everything before the init is done. Question is do we need to run the init? if we've restored the db and the index and the plugins and libs | 22:56 |
| clarkb | fungi: I'm guessing we want to run the init and the reindex just to be safe? | 22:56 |
| fungi | yes, i believe so. and expect it to be a no-op | 22:56 |
| clarkb | ok will run init momentarily | 22:56 |
| *** Guest96259 is now known as dansmith | 22:56 | |
| fungi | well, the renidex will take just as long as ever, but ideally won't actually result in different index contents | 22:56 |
| mordred | fungi, clarkb looks like ya'll are having fun - let me knw if I can be helpful - I'm in the air for the next 6.75 hours | 22:57 |
| *** aeng has joined #openstack-infra | 22:57 | |
| clarkb | fungi: it derped the libs again as expected, correcting them now, but otherwise apepars to have been a noop | 22:57 |
| clarkb | mordred: eh I think its mostly taken care of but will let you know if this rollback goes sideways | 22:57 |
| clarkb | mordred: tldr is upgrading gerrit is stupid complicated now | 22:57 |
| fungi | mordred: i think we've mostly worn out upgrade testing inertia for the evening. short story is that 2.13 before 2.13.8 is not recommended happymaking | 22:57 |
| mordred | oh that sounds wonderful | 22:58 |
| fungi | a good thing to learn | 22:58 |
| *** annegentle has quit IRC | 22:58 | |
| *** annegentle has joined #openstack-infra | 22:58 | |
| fungi | and clarkb also discovered that something's gone sour with our javamelofy plugin builds | 22:58 |
| fungi | javamelody | 22:58 |
| mordred | even better | 22:58 |
| clarkb | fungi: you know I wonder if we used matching lib names if gerrit init would stop deleting ours | 22:59 |
| *** salv-orl_ has quit IRC | 22:59 | |
| fungi | which will need to be sorted out before we try a 2.11->2.13.8 upgrade | 22:59 |
| clarkb | fungi: something to test next time maybe | 22:59 |
| fungi | clarkb: not a bad idea at all | 22:59 |
| *** salv-orlando has joined #openstack-infra | 22:59 | |
| clarkb | fungi: similar to the init we should reindex ya? | 23:01 |
| fungi | i think so, yes | 23:01 |
| clarkb | ok | 23:01 |
| *** Apoorva has joined #openstack-infra | 23:02 | |
| clarkb | in progress | 23:02 |
| *** annegentle has quit IRC | 23:03 | |
| clarkb | mordred: 2.13 added an h2 db for accountpatchreviews in 2.13.8 they are recommending you migrate that to mysql additionally there is a manual db migration on that db that you have to perform if you already went to 2.13 | 23:03 |
| clarkb | mordred: my reading of it is that if you just go to 2.13.8 you skip all that trouble entirely and life is better | 23:03 |
| *** gardlt has joined #openstack-infra | 23:03 | |
| *** salv-orlando has quit IRC | 23:03 | |
| clarkb | mordred: and considering that we need to get plugin builds working anyways may as well spin that all up properly for 2.13.8 | 23:03 |
| *** gomarivera has joined #openstack-infra | 23:03 | |
| mordred | ++ | 23:04 |
| fungi | also, looks like ubuntu's kernel updates didn't happen until today so should probably arrange some reboots tomorrow: https://lists.ubuntu.com/archives/ubuntu-security-announce/2017-June/date.html | 23:04 |
| mordred | aren't they trying to move away from databases? | 23:04 |
| clarkb | mordred: yes I think this was a halfway step to that by using embedded sql db but must've been bad at performance? | 23:04 |
| clarkb | fungi: see those indexing timeout errors? | 23:04 |
| *** pvaneck has quit IRC | 23:04 | |
| mordred | ah | 23:04 |
| *** Apoorva_ has quit IRC | 23:05 | |
| clarkb | fungi: they don't appear to be related to the data itself, just db connections | 23:05 |
| fungi | clarkb: yeah, that's odd | 23:05 |
| clarkb | so hopefully its ok | 23:05 |
| fungi | finished way faster | 23:05 |
| clarkb | hrm and it only got to 4% | 23:05 |
| fungi | er, no, didn't finish | 23:05 |
| fungi | right | 23:05 |
| openstackgerrit | Yolande Amate proposed openstack-infra/jenkins-job-builder master: Update shell and python plugins to make use of convert_mapping_to_xml https://review.openstack.org/473051 | 23:06 |
| fungi | clarkb: good thinking on the h2 db | 23:06 |
| clarkb | fungi: oh looks like I ran that as root again like a derp | 23:07 |
| fungi | heh | 23:07 |
| fungi | so maybe that was it | 23:08 |
| *** gardlt has quit IRC | 23:08 | |
| clarkb | fungi: how does that look? | 23:08 |
| *** gomarivera has quit IRC | 23:09 | |
| fungi | much better. not root now! | 23:09 |
| *** andreww has quit IRC | 23:10 | |
| clarkb | still happening though | 23:10 |
| fungi | still strange on the db errors | 23:10 |
| fungi | and at a different point too | 23:10 |
| clarkb | I wonder if gerrit changed its own config on 2.13? | 23:10 |
| fungi | oh, ew if so | 23:10 |
| fungi | i guess we could just unleash the puppet | 23:11 |
| fungi | and let it try to set stuff back again | 23:11 |
| clarkb | just skimming the db config it looks sane | 23:12 |
| *** dangers_away has quit IRC | 23:12 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Remove unnecessary settins of undercloud registry https://review.openstack.org/473158 | 23:12 |
| fungi | seems sorta like our trove instance decided to start having connectivity/performance issues | 23:12 |
| clarkb | ya maybe reduce to --threads 1 | 23:13 |
| fungi | worth a try | 23:13 |
| clarkb | nope | 23:14 |
| *** Julien-zte has joined #openstack-infra | 23:17 | |
| clarkb | https://bugs.chromium.org/p/gerrit/issues/detail?id=3634 apparently a known issue | 23:17 |
| clarkb | it removed our poolLimit looks like | 23:18 |
| * clarkb checks if it was ever puppeted | 23:18 | |
| *** dangers has joined #openstack-infra | 23:18 | |
| fungi | gah | 23:18 |
| clarkb | doesn't look like we set it on review-dev | 23:19 |
| *** harlowja has quit IRC | 23:19 | |
| clarkb | I'm going to set it to 150, run reindexing, then we can remove the line again | 23:19 |
| fungi | okay | 23:19 |
| *** annegentle has joined #openstack-infra | 23:19 | |
| clarkb | that seems much better | 23:20 |
| *** makowals has joined #openstack-infra | 23:20 | |
| clarkb | thats :( | 23:21 |
| *** annegentle has quit IRC | 23:22 | |
| clarkb | I've written down the 3 big things we have laerned from this in the ehterpad | 23:22 |
| clarkb | fungi: please add anything else that you can think of | 23:22 |
| *** dangers has quit IRC | 23:23 | |
| *** Apoorva has quit IRC | 23:24 | |
| clarkb | fungi: after this reindex is done anything else you want to do before starting the service back up again | 23:26 |
| fungi | nothing comes to mind | 23:26 |
| fungi | i say go ahead and bring it back online | 23:26 |
| fungi | once the reindex completes that is | 23:26 |
| *** armax has quit IRC | 23:26 | |
| *** gomarivera has joined #openstack-infra | 23:27 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use local mirror for ubuntu-minimal jobs https://review.openstack.org/475907 | 23:29 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at using CI mirrors for fedora/centos https://review.openstack.org/475607 | 23:29 |
| *** mriedem1 is now known as mriedem | 23:31 | |
| openstackgerrit | Yolande Amate proposed openstack-infra/jenkins-job-builder master: Update artifact_resolver to make use of convert_mapping_to_xml() https://review.openstack.org/473994 | 23:31 |
| clarkb | kevinbenton: is the neutron-vpnaas-dashboard project part of neutron? | 23:31 |
| clarkb | kornicameister: the neutron-fwaas-dashboard project is | 23:31 |
| *** stakeda has joined #openstack-infra | 23:31 | |
| clarkb | er kevinbenton ^ | 23:32 |
| *** dangers has joined #openstack-infra | 23:32 | |
| openstackgerrit | Yolande Amate proposed openstack-infra/jenkins-job-builder master: Update trigger_remote to make use of convert_mapping_to_xml https://review.openstack.org/473965 | 23:33 |
| clarkb | oh looks like vpnaas was retired from neutron late last year | 23:33 |
| *** ijw has quit IRC | 23:33 | |
| *** Guest72024 is now known as yland | 23:33 | |
| *** JudeC has quit IRC | 23:34 | |
| clarkb | ok reindex is done, starting gerrit now | 23:35 |
| clarkb | its up I can navigate changes | 23:36 |
| clarkb | going to try pushing one | 23:36 |
| *** jrist has joined #openstack-infra | 23:37 | |
| clarkb | ok I can push and update changes | 23:38 |
| clarkb | and I can search on them | 23:38 |
| clarkb | I think that is that | 23:38 |
| * clarkb is going to go outside now and do something not involving computers | 23:39 | |
| *** Julien-zte has quit IRC | 23:40 | |
| *** annegentle has joined #openstack-infra | 23:42 | |
| fungi | apuimedo: i've added you to https://review.openstack.org/#/admin/groups/997,members just now | 23:43 |
| fungi | clarkb: good call | 23:43 |
| openstackgerrit | Merged openstack-infra/system-config master: Remove potentially compromised bkero key https://review.openstack.org/475914 | 23:44 |
| *** sbezverk has joined #openstack-infra | 23:44 | |
| bkero | Thanks for quick turnaround. | 23:45 |
| *** gomarivera has quit IRC | 23:46 | |
| *** dangers has quit IRC | 23:46 | |
| fungi | bkero: you probably already have but don't forget to reset the ssh key on your gerrit account too? | 23:47 |
| bkero | fungi: I did that early on, yeah | 23:47 |
| fungi | excellent | 23:47 |
| *** annegentle has quit IRC | 23:47 | |
| kevinbenton | clarkb: yeah, we're in the process of bringing vpnaas back (we got some new contributors) but it's not quite there yet | 23:49 |
| *** dangers has joined #openstack-infra | 23:50 | |
| *** wolverineav has quit IRC | 23:50 | |
| *** bobh has joined #openstack-infra | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!