| *** slaweq has quit IRC | 00:10 | |
| *** slaweq has joined #openstack-infra | 00:11 | |
| *** dingyichen has joined #openstack-infra | 00:11 | |
| *** spzala has quit IRC | 00:24 | |
| SamYaple | ianw: did the sync for the docker repo work? havent seen it propogate | 00:24 |
|---|---|---|
| SamYaple | want to make sure i didnt mess up the patch again | 00:25 |
| *** baoli has joined #openstack-infra | 00:25 | |
| *** baoli_ has joined #openstack-infra | 00:26 | |
| *** baoli has quit IRC | 00:26 | |
| *** baoli has joined #openstack-infra | 00:29 | |
| *** baoli_ has quit IRC | 00:29 | |
| ianw | SamYaple: no, it seems the key has changed? | 00:30 |
| ianw | i think we want http://pgp.mit.edu/pks/lookup?search=0x2C52609D&op=index | 00:30 |
| ianw | but double checking | 00:30 |
| SamYaple | key probably did change | 00:30 |
| SamYaple | i can verify | 00:30 |
| SamYaple | ianw: oh yea the key did change when the repos changed | 00:31 |
| SamYaple | this was ~6 months ago | 00:31 |
| SamYaple | ill send an update | 00:31 |
| *** bobh has joined #openstack-infra | 00:32 | |
| ianw | https://download.docker.com/linux/ubuntu/gpg is "Docker Release (CE deb)" | 00:34 |
| SamYaple | yup | 00:34 |
| openstackgerrit | Sam Yaple proposed openstack-infra/system-config master: Updated docker asc https://review.openstack.org/510385 | 00:35 |
| *** dave-mccowan has joined #openstack-infra | 00:35 | |
| ianw | SamYaple: why is that "ceph-mirror..." ? | 00:36 |
| SamYaple | ianw: cause i was working on ceph too and i made a mistake :) | 00:36 |
| openstackgerrit | Sam Yaple proposed openstack-infra/system-config master: Updated docker asc https://review.openstack.org/510385 | 00:36 |
| *** baoli has quit IRC | 00:38 | |
| ianw | so we had the right fingerprint for the wrong key? | 00:38 |
| SamYaple | docker changed repos when it when to docker-ce from docker-engine a while back | 00:39 |
| SamYaple | i submitted a patch to updated the url | 00:39 |
| SamYaple | didnt see the dist changes (that was the patch you merged) | 00:39 |
| SamYaple | and it also has a new key | 00:39 |
| SamYaple | this patch | 00:39 |
| SamYaple | thats three patches now just to get up to date :/ | 00:39 |
| ianw | the key in modules/openstack_project/manifests/mirror_update.pp is 0EBFCD88 | 00:40 |
| ianw | but ./modules/openstack_project/files/reprepro/docker-mirror-gpg-key.asc is currently for "pub 4096R/2C52609D 2015-07-14 Docker Release Tool (releasedocker) <docker@docker.com>" | 00:41 |
| ianw | how did that ever work? | 00:43 |
| SamYaple | the changes were from like 2 days ago | 00:43 |
| SamYaple | it didnt work | 00:43 |
| SamYaple | the repo is 5 months out of date, thats what im trying to fix | 00:44 |
| *** slaweq has quit IRC | 00:44 | |
| SamYaple | see https://review.openstack.org/#/c/510314/ | 00:44 |
| *** LindaWang has joined #openstack-infra | 00:44 | |
| SamYaple | docker stopped publishing to the old address | 00:44 |
| SamYaple | its basically a new repo | 00:46 |
| *** gouthamr has quit IRC | 00:47 | |
| *** slaweq has joined #openstack-infra | 00:48 | |
| ianw | https://review.openstack.org/#/c/453304/ committed .asc for key 2c52609d but the "docker archive key" in puppet is for 0ebfcd88 | 00:48 |
| *** baoli has joined #openstack-infra | 00:48 | |
| SamYaple | oh i see your point | 00:48 |
| SamYaple | the 0ebfcd88 is the new key. so there was probably some confusion there | 00:49 |
| SamYaple | my guess is VerifyRelease is being superceeded by the asc? | 00:49 |
| SamYaple | i dont understand the mirroring process for infra | 00:49 |
| *** kiennt26 has joined #openstack-infra | 00:49 | |
| SamYaple | either way, 0ebfcd88 is the correct key, 'ubuntu' is hte correct dist, and the url is now correct | 00:50 |
| SamYaple | maybe it was a manual sync before? because the repo hasnt been updated since that patch merged | 00:51 |
| SamYaple | it also hasnt been published too since that patch merged | 00:52 |
| SamYaple | in fact, the new repo was always up when that patch was purposed | 00:52 |
| SamYaple | i blame docker, they have had like 4 repo changes in 2 years | 00:52 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Updated docker asc https://review.openstack.org/510385 | 00:57 |
| *** hongbin has joined #openstack-infra | 00:58 | |
| ianw | ok, just added some breadcrumbs in the comment. i'll merge and make sure it updates | 00:58 |
| SamYaple | ianw: thanks for the help here, i dont know where to find the mirror logs (if i even have access) to check these things | 00:59 |
| ianw | SamYaple: they're not exposed, only on mirror-update. that might all change in the zuulv3 world though | 01:01 |
| SamYaple | cool. | 01:02 |
| SamYaple | well it looks like it might be sorted now :) | 01:02 |
| SamYaple | with this patch the loci jobs pulled 0 outside files, all infra mirrors stuff showing it can be done in an airgaped environment | 01:02 |
| *** xinliang has joined #openstack-infra | 01:05 | |
| *** zhenguo has joined #openstack-infra | 01:06 | |
| openstackgerrit | Merged openstack-infra/system-config master: Updated docker asc https://review.openstack.org/510385 | 01:17 |
| *** psachin has joined #openstack-infra | 01:19 | |
| *** slaweq has quit IRC | 01:21 | |
| *** cuongnv has joined #openstack-infra | 01:22 | |
| *** edmondsw has joined #openstack-infra | 01:24 | |
| *** liusheng has joined #openstack-infra | 01:27 | |
| *** edmondsw has quit IRC | 01:29 | |
| *** gongysh has joined #openstack-infra | 01:29 | |
| *** slaweq has joined #openstack-infra | 01:33 | |
| *** dave-mccowan has quit IRC | 01:49 | |
| *** yamamoto has joined #openstack-infra | 01:51 | |
| *** markvoelker has quit IRC | 01:54 | |
| *** liujiong has joined #openstack-infra | 01:59 | |
| openstackgerrit | ChangBo Guo(gcb) proposed openstack-infra/project-config master: End gate of openstack/pylockfile https://review.openstack.org/508650 | 02:02 |
| openstackgerrit | Nam Nguyen Hoai proposed openstack-infra/project-config master: Remove legacy jobs from Barbican https://review.openstack.org/510390 | 02:03 |
| *** slaweq has quit IRC | 02:03 | |
| *** yamamoto has quit IRC | 02:05 | |
| openstackgerrit | Nam Nguyen Hoai proposed openstack-infra/project-config master: Remove legacy jobs from Barbican https://review.openstack.org/510390 | 02:06 |
| *** slaweq has joined #openstack-infra | 02:11 | |
| *** askb_ has joined #openstack-infra | 02:12 | |
| openstackgerrit | ChangBo Guo(gcb) proposed openstack-infra/project-config master: Remove project pylockfile from infrastructure systems https://review.openstack.org/508656 | 02:12 |
| ianw | i wonder if it's subkey signing messing this up | 02:16 |
| ianw | it's still not working | 02:16 |
| ianw | yeah it is | 02:19 |
| ianw | and also it's moved to "stable" | 02:19 |
| *** ijw_ has joined #openstack-infra | 02:22 | |
| *** ijw has quit IRC | 02:22 | |
| *** namnh has joined #openstack-infra | 02:24 | |
| *** yamamoto has joined #openstack-infra | 02:24 | |
| *** askb has quit IRC | 02:28 | |
| *** askb_ has quit IRC | 02:28 | |
| *** askb has joined #openstack-infra | 02:28 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Further docker/deb updates https://review.openstack.org/510393 | 02:29 |
| ianw | SamYaple: ^ hopefully the end of this! | 02:29 |
| *** yamamoto has quit IRC | 02:30 | |
| openstackgerrit | Duong Ha-Quang proposed openstack-infra/openstack-zuul-jobs master: Remove legacy jobs in Tacker https://review.openstack.org/510394 | 02:33 |
| openstackgerrit | Duong Ha-Quang proposed openstack-infra/project-config master: Remove legacy jobs in Tacker https://review.openstack.org/510395 | 02:33 |
| *** yamamoto has joined #openstack-infra | 02:41 | |
| *** slaweq has quit IRC | 02:42 | |
| *** bhavik1 has joined #openstack-infra | 02:45 | |
| *** slaweq has joined #openstack-infra | 02:50 | |
| SamYaple | ianw: wow. incredible. it shouldnt be this difficult :) | 02:53 |
| *** bhavik1 has quit IRC | 02:54 | |
| *** liusheng has quit IRC | 02:58 | |
| openstackgerrit | Merged openstack-infra/system-config master: Further docker/deb updates https://review.openstack.org/510393 | 03:00 |
| rakhmerov | hi, we have TripleO gate in Mistral that has been very unstable in the recent weeks. How can we make it non-voting for now given that the yaml files in project-config are now frozen? | 03:03 |
| rakhmerov | i.e. layout.yaml | 03:03 |
| rakhmerov | any hints? | 03:04 |
| *** liusheng has joined #openstack-infra | 03:05 | |
| rakhmerov | AJaeger: ^ | 03:06 |
| *** yamamoto has quit IRC | 03:18 | |
| SamYaple | ianw: looks like it finally worked! one small problem that isnt a deal breaker. we seem to only be mirroring the latest package | 03:21 |
| *** slaweq has quit IRC | 03:21 | |
| SamYaple | https://download.docker.com/linux/ubuntu/dists/xenial/pool/stable/amd64/ vs http://mirror.gra1.ovh.openstack.org/deb-docker/pool/stable/d/docker-ce/ | 03:22 |
| *** slaweq has joined #openstack-infra | 03:23 | |
| *** yamamoto has joined #openstack-infra | 03:31 | |
| *** bobh has quit IRC | 03:33 | |
| *** bobh has joined #openstack-infra | 03:37 | |
| *** udesale has joined #openstack-infra | 03:40 | |
| *** yamamoto has quit IRC | 03:42 | |
| ianw | SamYaple: i think that's right? don't think reprepro knows about old ones | 03:46 |
| *** yamamoto has joined #openstack-infra | 03:47 | |
| *** bobh has quit IRC | 03:47 | |
| SamYaple | ianw: but the repo knows about them https://download.docker.com/linux/ubuntu/dists/xenial/stable/binary-amd64/Packages | 03:48 |
| SamYaple | its not just stale copies, its atually in the manifest, signed and all | 03:48 |
| *** links has joined #openstack-infra | 03:50 | |
| *** yamahata has joined #openstack-infra | 03:50 | |
| *** yamamoto has quit IRC | 03:51 | |
| *** hongbin has quit IRC | 03:54 | |
| *** markvoelker has joined #openstack-infra | 03:55 | |
| *** slaweq has quit IRC | 03:56 | |
| *** gouthamr has joined #openstack-infra | 03:58 | |
| *** ramishra has joined #openstack-infra | 03:58 | |
| *** yamamoto has joined #openstack-infra | 04:00 | |
| *** slaweq has joined #openstack-infra | 04:06 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Fix docker-deb components https://review.openstack.org/510401 | 04:06 |
| *** jaosorior has joined #openstack-infra | 04:07 | |
| ianw | SamYaple: hmm, i dunno. reprepro has no shortage of options; if there's one to turn on keep old packages or something, i don't know what it is | 04:10 |
| *** liujiong has quit IRC | 04:11 | |
| *** felipemonteiro_ has joined #openstack-infra | 04:12 | |
| *** claudiub has joined #openstack-infra | 04:13 | |
| *** felipemonteiro__ has joined #openstack-infra | 04:13 | |
| *** liujiong has joined #openstack-infra | 04:15 | |
| *** eumel8 has joined #openstack-infra | 04:17 | |
| *** felipemonteiro_ has quit IRC | 04:17 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add Ceph Luminous Debian mirrors https://review.openstack.org/507864 | 04:18 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: Don't try to update cloud-init anymore https://review.openstack.org/504850 | 04:23 |
| AJaeger | rakhmerov: we accept changes that change both zuul v2 *and* v3 together | 04:25 |
| rakhmerov | ok | 04:25 |
| AJaeger | ianw, could you review these changes for periodic jobs: https://review.openstack.org/510297 and https://review.openstack.org/510298 , please? Like to see those run today | 04:25 |
| rakhmerov | AJaeger: in gerrit I see two sections "zuul" and "jenkins". Do I understand right that if "Jenkins" part is green then we can still merge a patch? | 04:26 |
| *** liujiong has quit IRC | 04:26 | |
| rakhmerov | even if some of the "zuul"s jobs are failing | 04:26 |
| rakhmerov | so "zuul" is for the new v3 stuff and "jenkins" is for v2? | 04:27 |
| *** liujiong has joined #openstack-infra | 04:28 | |
| AJaeger | rakhmerov: yes, a +1 by Jenkins allows you to merge - independent of Zuul votes (for now!) | 04:28 |
| *** felipemonteiro__ has quit IRC | 04:28 | |
| *** kiennt26 has quit IRC | 04:28 | |
| *** markvoelker has quit IRC | 04:28 | |
| *** felipemonteiro__ has joined #openstack-infra | 04:29 | |
| AJaeger | So, zuul == v3; Jenkins user = v2 | 04:29 |
| rakhmerov | ok, awesome, then we're fine now ) | 04:29 |
| AJaeger | ;) | 04:29 |
| rakhmerov | yep, thanks AJaeger | 04:29 |
| AJaeger | rakhmerov: then talk with tripleo folks to get this fixed once we move over to v3 ;) | 04:29 |
| AJaeger | bbl | 04:29 |
| rakhmerov | yes, we talk all the time | 04:29 |
| *** slaweq has quit IRC | 04:38 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Add requirements repo to some periodic jobs https://review.openstack.org/510297 | 04:38 |
| *** felipemonteiro__ has quit IRC | 04:38 | |
| openstackgerrit | ChangBo Guo(gcb) proposed openstack-infra/project-config master: End gate of openstack/pylockfile https://review.openstack.org/508650 | 04:39 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Use v3 translation jobs for doc projects https://review.openstack.org/510298 | 04:39 |
| *** pratik has joined #openstack-infra | 04:40 | |
| pratik | hi yolanda, can you please review https://review.openstack.org/#/c/510015/ when you get a chance? | 04:41 |
| *** ethfci has quit IRC | 04:41 | |
| *** ykarel_ has joined #openstack-infra | 04:42 | |
| *** gongysh has quit IRC | 04:42 | |
| *** coolsvap has joined #openstack-infra | 04:42 | |
| *** ykarel_ is now known as ykarel | 04:42 | |
| *** ethfci has joined #openstack-infra | 04:44 | |
| *** pratik_ has joined #openstack-infra | 04:47 | |
| *** pratik has quit IRC | 04:47 | |
| *** bobh has joined #openstack-infra | 04:48 | |
| *** pratik_ has quit IRC | 04:48 | |
| *** Dinesh_Bhor has joined #openstack-infra | 04:48 | |
| *** pratik33 has joined #openstack-infra | 04:48 | |
| *** slaweq has joined #openstack-infra | 04:50 | |
| *** bobh has quit IRC | 04:52 | |
| *** gongysh has joined #openstack-infra | 05:00 | |
| *** edmondsw has joined #openstack-infra | 05:00 | |
| *** yamamoto has quit IRC | 05:02 | |
| *** yamamoto has joined #openstack-infra | 05:03 | |
| *** mikal has quit IRC | 05:04 | |
| *** edmondsw has quit IRC | 05:05 | |
| *** yamamoto has quit IRC | 05:08 | |
| *** sree has joined #openstack-infra | 05:14 | |
| *** jascott1 has quit IRC | 05:15 | |
| *** jascott1 has joined #openstack-infra | 05:16 | |
| *** jascott1 has quit IRC | 05:20 | |
| *** liujiong has quit IRC | 05:20 | |
| SamYaple | ianw: ok. we were pinning to a specific version. but ill keep an eye on it, if it becomes a problem ill figure out how to get the older versions to stick around. thansk for the help again. i wasn't going to figure out that key issue any time soon | 05:21 |
| *** wolverineav has joined #openstack-infra | 05:22 | |
| *** slaweq has quit IRC | 05:23 | |
| *** markvoelker has joined #openstack-infra | 05:26 | |
| *** wolverineav has quit IRC | 05:27 | |
| *** gouthamr has quit IRC | 05:27 | |
| *** slaweq has joined #openstack-infra | 05:28 | |
| *** dhajare has joined #openstack-infra | 05:30 | |
| *** jascott1 has joined #openstack-infra | 05:31 | |
| *** cshastri has joined #openstack-infra | 05:36 | |
| *** gongysh has quit IRC | 05:39 | |
| *** yamamoto has joined #openstack-infra | 05:39 | |
| *** yamamoto has quit IRC | 05:41 | |
| *** yamamoto has joined #openstack-infra | 05:45 | |
| *** liujiong has joined #openstack-infra | 05:48 | |
| *** yamamoto has quit IRC | 05:48 | |
| *** bobh has joined #openstack-infra | 05:49 | |
| *** bobh has quit IRC | 05:53 | |
| *** spectr has joined #openstack-infra | 05:54 | |
| *** jtomasek has joined #openstack-infra | 05:56 | |
| *** markvoelker has quit IRC | 05:58 | |
| *** slaweq has quit IRC | 06:00 | |
| *** udesale__ has joined #openstack-infra | 06:03 | |
| *** liujiong has quit IRC | 06:04 | |
| *** liujiong has joined #openstack-infra | 06:04 | |
| *** cody-somerville has joined #openstack-infra | 06:05 | |
| *** cody-somerville has quit IRC | 06:05 | |
| *** cody-somerville has joined #openstack-infra | 06:05 | |
| *** udesale has quit IRC | 06:05 | |
| *** slaweq has joined #openstack-infra | 06:06 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy jobs of Zun https://review.openstack.org/509970 | 06:09 |
| *** vsaienk0 has joined #openstack-infra | 06:14 | |
| openstackgerrit | Nam Nguyen Hoai proposed openstack-infra/openstack-zuul-jobs master: Remove Barbican legacy jobs https://review.openstack.org/510414 | 06:14 |
| *** hrubi has quit IRC | 06:15 | |
| *** hrubi has joined #openstack-infra | 06:15 | |
| ianw | infra-root: it's not really desperation point, but afs01 : /dev/mapper/main-vicepa 3.0T 3.0T 45G 99% /vicepa ... | 06:15 |
| *** LindaWang has quit IRC | 06:16 | |
| *** LindaWang has joined #openstack-infra | 06:17 | |
| *** aeng has quit IRC | 06:17 | |
| openstackgerrit | Nam Nguyen Hoai proposed openstack-infra/project-config master: Remove legacy jobs from Barbican https://review.openstack.org/510390 | 06:17 |
| openstackgerrit | Duong Ha-Quang proposed openstack-infra/project-config master: Remove legacy jobs in Tacker https://review.openstack.org/510395 | 06:19 |
| *** isaacb has joined #openstack-infra | 06:26 | |
| *** ykarel_ has joined #openstack-infra | 06:28 | |
| *** cody-somerville has quit IRC | 06:28 | |
| *** cody-somerville has joined #openstack-infra | 06:28 | |
| *** cody-somerville has joined #openstack-infra | 06:28 | |
| *** jogo has quit IRC | 06:29 | |
| *** ykarel has quit IRC | 06:31 | |
| *** kiennt26 has joined #openstack-infra | 06:34 | |
| *** ankkumar has joined #openstack-infra | 06:36 | |
| *** slaweq has quit IRC | 06:36 | |
| *** threestrands has quit IRC | 06:37 | |
| *** rcernin has joined #openstack-infra | 06:40 | |
| *** slaweq has joined #openstack-infra | 06:41 | |
| *** igordc has quit IRC | 06:48 | |
| *** yamamoto has joined #openstack-infra | 06:49 | |
| *** igordc has joined #openstack-infra | 06:50 | |
| *** andreas_s has joined #openstack-infra | 06:52 | |
| *** ccamacho has joined #openstack-infra | 06:52 | |
| *** gcb has joined #openstack-infra | 06:55 | |
| *** markvoelker has joined #openstack-infra | 06:55 | |
| *** yamamoto has quit IRC | 06:58 | |
| *** baoli has quit IRC | 06:59 | |
| *** armaan has joined #openstack-infra | 06:59 | |
| *** udesale has joined #openstack-infra | 06:59 | |
| *** udesale has quit IRC | 06:59 | |
| *** udesale has joined #openstack-infra | 06:59 | |
| *** udesale has quit IRC | 07:00 | |
| *** udesale__ has quit IRC | 07:01 | |
| *** udesale has joined #openstack-infra | 07:01 | |
| *** gildub has quit IRC | 07:05 | |
| *** pcaruana has joined #openstack-infra | 07:08 | |
| ankkumar | Hi AJaeger | 07:12 |
| ankkumar | AJaeger, Regarding the patch https://review.openstack.org/#/c/462695. Is the migration to zuul v3 is done or still WIP? Actually i want to confirm that where we should put the new gate, does it has to put in zuul-legacy-jobs.yaml? | 07:12 |
| chandankumar | AJaeger: hello | 07:12 |
| chandankumar | AJaeger: thanks for fixing the patch of python-tempestconf | 07:13 |
| chandankumar | AJaeger: what is noop-jobs here https://review.openstack.org/#/c/508502/12/zuul/layout.yaml@16461 ? | 07:13 |
| *** slaweq has quit IRC | 07:14 | |
| *** edmondsw has joined #openstack-infra | 07:18 | |
| AJaeger | ankkumar: http://lists.openstack.org/pipermail/openstack-dev/2017-October/123049.html | 07:19 |
| AJaeger | ankkumar: for v3, you would touch zuul.d/projects.yaml - and openstack-zuul-jobs. | 07:20 |
| AJaeger | chandankumar: noop-job is a job that is needed if no other exists. | 07:21 |
| AJaeger | chandankumar: you can add others later - or add them to v3 only... | 07:21 |
| *** edmondsw has quit IRC | 07:22 | |
| openstackgerrit | Merged openstack-infra/system-config master: Fix docker-deb components https://review.openstack.org/510401 | 07:23 |
| *** pratik33 has quit IRC | 07:23 | |
| *** alexchadin has joined #openstack-infra | 07:25 | |
| ankkumar | AJaeger, thanks :) | 07:25 |
| openstackgerrit | Kien Nguyen proposed openstack-infra/project-config master: Remove Zunclient-legacy jobs https://review.openstack.org/510422 | 07:25 |
| openstackgerrit | Kien Nguyen proposed openstack-infra/openstack-zuul-jobs master: Remove Zunclient legacy jobs https://review.openstack.org/510423 | 07:26 |
| chandankumar | AJaeger: i will fix other tempest plugin repo creation jobs similar to that | 07:27 |
| openstackgerrit | Kien Nguyen proposed openstack-infra/project-config master: Remove Zun-client legacy jobs https://review.openstack.org/510422 | 07:28 |
| *** markvoelker has quit IRC | 07:29 | |
| openstackgerrit | Kien Nguyen proposed openstack-infra/openstack-zuul-jobs master: Remove Zun-client legacy jobs https://review.openstack.org/510423 | 07:29 |
| *** Hal has joined #openstack-infra | 07:31 | |
| *** Hal is now known as Guest53610 | 07:32 | |
| *** MasterPiece2 has joined #openstack-infra | 07:34 | |
| *** jpena|off is now known as jpena | 07:39 | |
| *** egonzalez has joined #openstack-infra | 07:41 | |
| *** slaweq has joined #openstack-infra | 07:45 | |
| *** ykarel_ is now known as ykarel | 07:45 | |
| openstackgerrit | Duong Ha-Quang proposed openstack-infra/openstack-zuul-jobs master: Remove legacy jobs in Vitrage https://review.openstack.org/510431 | 07:47 |
| openstackgerrit | Duong Ha-Quang proposed openstack-infra/project-config master: Remove legacy jobs in Vitrage https://review.openstack.org/510432 | 07:47 |
| *** tmorin has joined #openstack-infra | 07:50 | |
| *** sree_ has joined #openstack-infra | 07:50 | |
| *** sree has quit IRC | 07:50 | |
| *** sree_ is now known as Guest33221 | 07:51 | |
| *** sanket_ has joined #openstack-infra | 07:52 | |
| *** gildub has joined #openstack-infra | 07:52 | |
| jianghuaw | hi, legacy-grenade-dsvm-neutron-multinode failed with "RETRY_LIMIT". Is it a known issue? Same error on several patches. | 07:55 |
| jianghuaw | 2017-10-08 10:02:10.267860 | secondary | "msg": "Destination /etc/nodepool/sub_nodes does not exist !", | 07:55 |
| jianghuaw | http://logs.openstack.org/04/506204/3/check/legacy-grenade-dsvm-neutron-multinode/df3914d/job-output.txt.gz | 07:55 |
| jianghuaw | http://logs.openstack.org/06/508906/6/check/legacy-grenade-dsvm-neutron-multinode/49d1e63/job-output.txt.gz | 07:55 |
| *** shardy has joined #openstack-infra | 07:55 | |
| *** ianychoi has joined #openstack-infra | 07:59 | |
| *** pratik has joined #openstack-infra | 07:59 | |
| *** jpich has joined #openstack-infra | 08:02 | |
| *** sshnaidm is now known as sshnaidm_ | 08:03 | |
| *** sshnaidm_ is now known as sshnaidm | 08:03 | |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config master: adds missing required-projects to n8g-(bagpipe|bgppvn) https://review.openstack.org/510435 | 08:05 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: /etc/machine-id detection improvements https://review.openstack.org/510312 | 08:06 |
| *** jascott1 has quit IRC | 08:07 | |
| tmorin | hi infraroot (yolanda? AJaeger?) : https://review.openstack.org/510435 adds a few required-projets that were still missing for networking-bgpvpn and networking-bagpipe can work... would you have some time to review ? | 08:07 |
| *** jascott1 has joined #openstack-infra | 08:08 | |
| *** jascott1 has quit IRC | 08:12 | |
| *** alexchadin has quit IRC | 08:14 | |
| *** ykarel_ has joined #openstack-infra | 08:14 | |
| *** alexchadin has joined #openstack-infra | 08:14 | |
| *** sanket_ has quit IRC | 08:15 | |
| *** pratik has quit IRC | 08:16 | |
| *** ykarel_ has quit IRC | 08:16 | |
| *** gcb has quit IRC | 08:17 | |
| openstackgerrit | Nam Nguyen Hoai proposed openstack-infra/project-config master: Remove legacy jobs from Congress https://review.openstack.org/510437 | 08:17 |
| *** gcb has joined #openstack-infra | 08:17 | |
| *** alexchadin has quit IRC | 08:18 | |
| *** liusheng has quit IRC | 08:18 | |
| *** slaweq has quit IRC | 08:18 | |
| *** sanket_ has joined #openstack-infra | 08:20 | |
| *** bauwser is now known as bauzas | 08:20 | |
| *** alexchadin has joined #openstack-infra | 08:21 | |
| *** gongysh has joined #openstack-infra | 08:21 | |
| openstackgerrit | Emma Foley proposed openstack-infra/project-config master: Removing collectd-ceilometer-plugin definition from zuul.d/projects.yaml https://review.openstack.org/510086 | 08:23 |
| *** slaweq has joined #openstack-infra | 08:23 | |
| *** lucas-afk is now known as lucasagomes | 08:24 | |
| openstackgerrit | Chandan Kumar proposed openstack-infra/project-config master: Add new repo for magnum tempest plugin https://review.openstack.org/501231 | 08:25 |
| *** ianychoi has quit IRC | 08:26 | |
| *** markvoelker has joined #openstack-infra | 08:26 | |
| openstackgerrit | Nam Nguyen Hoai proposed openstack-infra/openstack-zuul-jobs master: Remove Congress legacy jobs https://review.openstack.org/510440 | 08:26 |
| *** vsaienk0 has quit IRC | 08:27 | |
| chandankumar | AJaeger: "add them to v3 only" means once project is imported to openstack namespace add respective jobs with in the project using .zuul.yaml? | 08:27 |
| *** martinkopec has joined #openstack-infra | 08:27 | |
| *** dizquierdo has joined #openstack-infra | 08:29 | |
| *** hashar has joined #openstack-infra | 08:30 | |
| *** d0ugal_ has quit IRC | 08:31 | |
| *** d0ugal has joined #openstack-infra | 08:31 | |
| *** d0ugal has quit IRC | 08:31 | |
| *** d0ugal has joined #openstack-infra | 08:31 | |
| *** vsaienk0 has joined #openstack-infra | 08:31 | |
| *** derekh has joined #openstack-infra | 08:32 | |
| *** e0ne has joined #openstack-infra | 08:35 | |
| *** sanket_ has quit IRC | 08:36 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Don't try to update cloud-init anymore https://review.openstack.org/504850 | 08:38 |
| *** edmondsw has joined #openstack-infra | 08:39 | |
| *** vsaienk0 has quit IRC | 08:41 | |
| *** edmondsw has quit IRC | 08:43 | |
| *** jamesdenton has quit IRC | 08:44 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Fix TripleO CI jobs https://review.openstack.org/508660 | 08:46 |
| *** neon_icon has joined #openstack-infra | 08:48 | |
| *** slaweq_ has joined #openstack-infra | 08:48 | |
| *** bobh has joined #openstack-infra | 08:51 | |
| *** dingyichen has quit IRC | 08:51 | |
| *** dizquierdo has quit IRC | 08:54 | |
| *** vsaienk0 has joined #openstack-infra | 08:54 | |
| *** finucannot is now known as stephenfin | 08:54 | |
| *** bobh has quit IRC | 08:56 | |
| *** bhavik1 has joined #openstack-infra | 08:56 | |
| *** gildub has quit IRC | 08:57 | |
| *** slaweq has quit IRC | 08:57 | |
| *** markvoelker has quit IRC | 08:59 | |
| *** electrofelix has joined #openstack-infra | 09:01 | |
| *** kjackal_ has joined #openstack-infra | 09:01 | |
| *** kiennt26 has quit IRC | 09:04 | |
| *** ykarel is now known as ykarel|lunch | 09:05 | |
| *** slaweq has joined #openstack-infra | 09:06 | |
| *** gcb has quit IRC | 09:06 | |
| *** kaisers1 has quit IRC | 09:06 | |
| openstackgerrit | Raissa Sarmento proposed openstack-infra/project-config master: Add new repo for manila tempest plugin https://review.openstack.org/507905 | 09:14 |
| *** tosky has joined #openstack-infra | 09:15 | |
| *** gcb has joined #openstack-infra | 09:19 | |
| *** psachin has quit IRC | 09:20 | |
| *** yamahata has quit IRC | 09:22 | |
| *** psachin has joined #openstack-infra | 09:22 | |
| *** kaisers has joined #openstack-infra | 09:24 | |
| *** kiennt26 has joined #openstack-infra | 09:24 | |
| openstackgerrit | Raissa Sarmento proposed openstack-infra/project-config master: Add new repo for manila tempest plugin https://review.openstack.org/507905 | 09:27 |
| *** yamamoto has joined #openstack-infra | 09:29 | |
| *** gongysh has quit IRC | 09:30 | |
| *** bhavik1 has quit IRC | 09:30 | |
| *** liusheng has joined #openstack-infra | 09:31 | |
| *** ociuhandu has joined #openstack-infra | 09:35 | |
| *** gildub has joined #openstack-infra | 09:35 | |
| *** abelur_ has joined #openstack-infra | 09:36 | |
| *** slaweq has quit IRC | 09:38 | |
| *** ccamacho has quit IRC | 09:42 | |
| *** ccamacho has joined #openstack-infra | 09:43 | |
| openstackgerrit | Raissa Sarmento proposed openstack-infra/project-config master: Add new repo for manila tempest plugin https://review.openstack.org/507905 | 09:43 |
| *** raissa_ is now known as raissa | 09:46 | |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Remove support for command hooks https://review.openstack.org/510163 | 09:48 |
| chandankumar | AJaeger: is this review https://review.openstack.org/#/c/501231/ ok? | 09:49 |
| *** slaweq has joined #openstack-infra | 09:49 | |
| *** egonzalez has quit IRC | 09:50 | |
| *** bobh has joined #openstack-infra | 09:52 | |
| *** ociuhandu has quit IRC | 09:55 | |
| *** dbecker has joined #openstack-infra | 09:56 | |
| *** bobh has quit IRC | 09:56 | |
| *** markvoelker has joined #openstack-infra | 09:56 | |
| openstackgerrit | Ricardo Noriega proposed openstack-infra/project-config master: Add new repo for Neutron L2GW Tempest Plugin https://review.openstack.org/509823 | 09:57 |
| *** liujiong has quit IRC | 09:58 | |
| openstackgerrit | Raissa Sarmento proposed openstack-infra/project-config master: Add new repo for manila tempest plugin https://review.openstack.org/507905 | 09:58 |
| *** andreas_s has quit IRC | 09:58 | |
| *** gcb has quit IRC | 09:59 | |
| *** kjackal__ has joined #openstack-infra | 10:02 | |
| *** egonzalez has joined #openstack-infra | 10:03 | |
| *** Guest33221 has quit IRC | 10:03 | |
| openstackgerrit | Nam Nguyen Hoai proposed openstack-infra/project-config master: Remove legacy jobs from Ceilometer https://review.openstack.org/510459 | 10:03 |
| *** sree has joined #openstack-infra | 10:03 | |
| *** sree has quit IRC | 10:04 | |
| *** kjackal_ has quit IRC | 10:05 | |
| *** psachin has quit IRC | 10:05 | |
| *** andreas_s has joined #openstack-infra | 10:06 | |
| *** dizquierdo has joined #openstack-infra | 10:06 | |
| *** ykarel|lunch is now known as ykarel | 10:07 | |
| *** sree has joined #openstack-infra | 10:08 | |
| *** pbourke has quit IRC | 10:08 | |
| *** jascott1 has joined #openstack-infra | 10:09 | |
| *** cuongnv has quit IRC | 10:09 | |
| *** pbourke has joined #openstack-infra | 10:10 | |
| *** sree has quit IRC | 10:12 | |
| *** andreas_s has quit IRC | 10:14 | |
| *** gongysh has joined #openstack-infra | 10:15 | |
| *** andreas_s has joined #openstack-infra | 10:16 | |
| *** thorre has quit IRC | 10:17 | |
| *** psachin has joined #openstack-infra | 10:20 | |
| *** andreas_s has quit IRC | 10:20 | |
| *** slaweq has quit IRC | 10:23 | |
| *** udesale has quit IRC | 10:25 | |
| *** pblaho has joined #openstack-infra | 10:26 | |
| *** neon_iconstar has joined #openstack-infra | 10:27 | |
| frickler | looks like there is a regression for v3 devstack multinode jobs, added some info to etherpad | 10:27 |
| *** edmondsw has joined #openstack-infra | 10:28 | |
| frickler | jianghuaw: seems to be a new issue, those jobs were passing last week | 10:28 |
| *** LindaWang has quit IRC | 10:28 | |
| *** slaweq has joined #openstack-infra | 10:29 | |
| *** namnh has quit IRC | 10:30 | |
| *** markvoelker has quit IRC | 10:30 | |
| *** edmondsw has quit IRC | 10:32 | |
| jianghuaw | frickler: Thanks for the response. True. that was good in last week. But it has been failed with RETRY_LIMI in the last two days. | 10:34 |
| *** andreas_s has joined #openstack-infra | 10:34 | |
| *** kiennt26 has quit IRC | 10:34 | |
| jianghuaw | Is anyone looking at this issue? | 10:34 |
| AJaeger | chandankumar: see also https://review.openstack.org/#/c/509939/ | 10:36 |
| *** neon_icon has quit IRC | 10:36 | |
| *** neon_iconstar has quit IRC | 10:36 | |
| AJaeger | chandankumar: yes, it's ok - but needs a followup patch for merge-check addition. And then you can add the jobs to your .zuul.yaml or to the global list in project-config - see also https://docs.openstack.org/infra/manual/zuulv3.html#what-not-to-convert | 10:37 |
| *** andreas_s has quit IRC | 10:39 | |
| *** andreas_s has joined #openstack-infra | 10:40 | |
| *** gongysh has quit IRC | 10:41 | |
| *** sdague has joined #openstack-infra | 10:41 | |
| *** andreas_s has quit IRC | 10:42 | |
| *** jkilpatr has joined #openstack-infra | 10:43 | |
| *** andreas_s has joined #openstack-infra | 10:43 | |
| chandankumar | AJaeger: or will i directly update the same patch for adding an entry for merge-check in zuul.d/projects.yaml? | 10:50 |
| *** jkilpatr has quit IRC | 10:51 | |
| openstackgerrit | Chandan Kumar proposed openstack-infra/project-config master: Add new repo for magnum tempest plugin https://review.openstack.org/501231 | 10:53 |
| chandankumar | AJaeger: ^^ | 10:53 |
| *** neon_iconstar has joined #openstack-infra | 10:53 | |
| ssbarnea | any pylint guru around? I wanted to question the "grand" design of lintstack.py ;) | 10:53 |
| *** andreas_s has quit IRC | 10:54 | |
| chandankumar | AJaeger: sorry got that | 10:54 |
| *** dtantsur|afk is now known as dtantsur | 10:55 | |
| *** Goneri has joined #openstack-infra | 10:56 | |
| AJaeger | chandankumar: So, separate patch. Will fail the first time - once the first is merged, we recheck the second | 10:57 |
| openstackgerrit | Chandan Kumar proposed openstack-infra/project-config master: Add new repo for magnum tempest plugin https://review.openstack.org/501231 | 10:57 |
| *** alexchadin has quit IRC | 10:57 | |
| *** LindaWang has joined #openstack-infra | 10:59 | |
| openstackgerrit | Chandan Kumar proposed openstack-infra/project-config master: Added merge-check job template for magnum-tempest-plugin https://review.openstack.org/510529 | 10:59 |
| chandankumar | AJaeger: done | 10:59 |
| *** rgerganov has joined #openstack-infra | 11:00 | |
| rgerganov | Does anyone know why Zuul gives -1 on a patch which can be clearly applied on master? https://review.openstack.org/#/c/509417/ | 11:00 |
| *** nicolasbock has joined #openstack-infra | 11:01 | |
| *** yamamoto has quit IRC | 11:03 | |
| *** slaweq has quit IRC | 11:03 | |
| *** jaosorior has quit IRC | 11:05 | |
| *** rhallisey_ has joined #openstack-infra | 11:05 | |
| *** jkilpatr has joined #openstack-infra | 11:05 | |
| *** jaosorior has joined #openstack-infra | 11:05 | |
| frickler | rgerganov: I think there may have been an issue with zuul at the time, you can do a recheck if you want | 11:05 |
| frickler | jianghuaw: yes, I'm taking a look, also added to etherpad | 11:08 |
| *** edmondsw has joined #openstack-infra | 11:10 | |
| *** edmondsw has quit IRC | 11:10 | |
| *** nicolasbock_ has joined #openstack-infra | 11:13 | |
| openstackgerrit | Shachar Snapiri proposed openstack-infra/openstack-zuul-jobs master: Add publish-to-pypi-neutron template https://review.openstack.org/510531 | 11:14 |
| *** andreas_s has joined #openstack-infra | 11:14 | |
| openstackgerrit | Shachar Snapiri proposed openstack-infra/project-config master: Add neutron to the project dependecy list for zuul https://review.openstack.org/510347 | 11:16 |
| *** nicolasbock has quit IRC | 11:16 | |
| *** andreas_s has quit IRC | 11:18 | |
| *** andreas_s has joined #openstack-infra | 11:18 | |
| *** egonzalez has quit IRC | 11:19 | |
| *** pblaho has quit IRC | 11:20 | |
| *** tmorin has quit IRC | 11:23 | |
| *** gildub has quit IRC | 11:25 | |
| *** egonzalez has joined #openstack-infra | 11:26 | |
| *** lucasagomes is now known as lucas-hungry | 11:27 | |
| *** markvoelker has joined #openstack-infra | 11:27 | |
| *** Goneri has quit IRC | 11:28 | |
| *** snapiri has joined #openstack-infra | 11:29 | |
| *** andreas_s has quit IRC | 11:30 | |
| snapiri | AJaeger: could you please review https://review.openstack.org/#/c/510531/ zuul failed it and I am not sure I understand why... | 11:30 |
| *** ankkumar has quit IRC | 11:33 | |
| *** tushar has quit IRC | 11:33 | |
| *** yamamoto has joined #openstack-infra | 11:34 | |
| *** neon_iconstar has quit IRC | 11:35 | |
| *** coolsvap has quit IRC | 11:35 | |
| *** slaweq has joined #openstack-infra | 11:35 | |
| *** neon_icon has joined #openstack-infra | 11:36 | |
| AJaeger | snapiri: check http://logs.openstack.org/31/510531/1/infra-check/multinode-integration-fedora-26/34df70b/ara/ - the ara interface helps you to find the bug. It's a dnf mirror problem, I suggest to recheck | 11:37 |
| openstackgerrit | Ricardo Noriega proposed openstack-infra/project-config master: Add new repo for Neutron L2GW Tempest Plugin https://review.openstack.org/509823 | 11:37 |
| frickler | mordred: I think https://review.openstack.org/510213 is broken, seems like you need to create /etc/nodepool/sub_nodes too, before you do lineinfile on it. see what jianghuaw posted earlier | 11:38 |
| *** martinkopec has quit IRC | 11:38 | |
| *** andreas_s has joined #openstack-infra | 11:38 | |
| snapiri | AJaeger: thanks | 11:38 |
| *** yamamoto has quit IRC | 11:39 | |
| *** markus_z has joined #openstack-infra | 11:39 | |
| *** shardy is now known as shardy_lunch | 11:40 | |
| markus_z | zuul: How do I verify that the skip-if statements I added to my layout get recognized? I don't find a hint in the zuul logs. | 11:41 |
| sshnaidm | hi, just wanted to know if "retry_limit" errors are known issue..? | 11:41 |
| *** yongwc has joined #openstack-infra | 11:41 | |
| openstackgerrit | Jens Harbott (frickler) proposed openstack-infra/openstack-zuul-jobs master: Create sub_nodes file before trying to modify it https://review.openstack.org/510540 | 11:43 |
| *** sambetts|afk is now known as sambetts | 11:44 | |
| frickler | sshnaidm: for devstack on multinode jobs? see ^^ | 11:44 |
| frickler | mordred: jianghuaw: ^^ | 11:44 |
| sshnaidm | frickler, for multinode in my case | 11:45 |
| *** ansiwen[q] has joined #openstack-infra | 11:46 | |
| *** leyal has quit IRC | 11:47 | |
| *** tushar has joined #openstack-infra | 11:47 | |
| *** lihi has quit IRC | 11:47 | |
| hwoarang | hello. I think it's confusing that in the gerrit dashboard we see a -1 when zuulv3 failed but Jenkins passed(+1). Since jenkins is used for gates right now, shouldn't gerrit dashboard show the jenkins voting score instead? | 11:48 |
| yongwc | hi team, we propose to commit a cinder driver to the cinder project, and we setup a local CI environment, however, our local CI only report comment in the history portion of the review page, not like other CI which also appear at the top of the page. anyone can help me? | 11:48 |
| *** neon_icon has quit IRC | 11:48 | |
| *** ldnunes has joined #openstack-infra | 11:49 | |
| *** lihi has joined #openstack-infra | 11:49 | |
| *** leyal has joined #openstack-infra | 11:49 | |
| frickler | yongwc: what is the name of your CI? | 11:52 |
| *** tpsilva has joined #openstack-infra | 11:52 | |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config master: adds missing required-projects to n8g-(bagpipe|bgppvn) https://review.openstack.org/510435 | 11:52 |
| dtantsur | folks, have you seen an error "change is new" when trying to restore a change? | 11:52 |
| dtantsur | well, at least it did restore it :) | 11:52 |
| *** dave-mccowan has joined #openstack-infra | 11:53 | |
| yongwc | frickler: INSPUR-CI, and the review page is https://review.openstack.org/#/c/502455/. | 11:53 |
| *** bobh has joined #openstack-infra | 11:54 | |
| frickler | yongwc: make the name "INSPUR CI", see item 2. at https://docs.openstack.org/infra/system-config/third_party.html#creating-a-service-account | 11:54 |
| *** dave-mcc_ has joined #openstack-infra | 11:56 | |
| *** jaypipes has joined #openstack-infra | 11:56 | |
| yongwc | frickler: thanks, I will change the name and try again. | 11:56 |
| *** eumel8 has quit IRC | 11:57 | |
| *** dave-mccowan has quit IRC | 11:58 | |
| *** bobh has quit IRC | 11:58 | |
| *** trown|outtypewww is now known as trown | 11:59 | |
| *** markvoelker has quit IRC | 12:00 | |
| *** gouthamr has joined #openstack-infra | 12:01 | |
| *** tmorin has joined #openstack-infra | 12:01 | |
| *** Guest53610 has quit IRC | 12:02 | |
| *** edmondsw has joined #openstack-infra | 12:05 | |
| openstackgerrit | Raissa Sarmento proposed openstack-infra/project-config master: Add merge-check for manila-tempest-plugin https://review.openstack.org/510548 | 12:06 |
| *** Hal has joined #openstack-infra | 12:08 | |
| *** Hal is now known as Guest94492 | 12:09 | |
| *** tushar has quit IRC | 12:09 | |
| *** tushar has joined #openstack-infra | 12:09 | |
| *** leyal has quit IRC | 12:11 | |
| *** leyal has joined #openstack-infra | 12:11 | |
| *** tushar has quit IRC | 12:12 | |
| *** tumbarka has joined #openstack-infra | 12:13 | |
| *** yongwc has quit IRC | 12:13 | |
| *** udesale has joined #openstack-infra | 12:13 | |
| *** bobh has joined #openstack-infra | 12:13 | |
| *** alexchadin has joined #openstack-infra | 12:21 | |
| *** LindaWang has quit IRC | 12:21 | |
| *** markvoelker has joined #openstack-infra | 12:21 | |
| *** yamamoto has joined #openstack-infra | 12:23 | |
| *** lucas-hungry is now known as lucasagomes | 12:24 | |
| *** bobh has quit IRC | 12:28 | |
| openstackgerrit | Bernard Cafarelli proposed openstack-infra/project-config master: Add new repo for networking-sfc tempest plugin https://review.openstack.org/510553 | 12:28 |
| *** pblaho has joined #openstack-infra | 12:29 | |
| AJaeger | jeblair, mordred , fungi, pabelanger , everybody is pushing out new repos today - could you review and merge an infra-manual update, please? https://review.openstack.org/#/c/509939/ - I updated mordred's patch over the weekend... | 12:34 |
| *** rlandy has joined #openstack-infra | 12:34 | |
| AJaeger | frickler, yongwc, see also the thread at http://lists.openstack.org/pipermail/openstack-dev/2017-October/123251.html | 12:35 |
| *** jcoufal has joined #openstack-infra | 12:36 | |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/openstack-zuul-jobs master: [rally] Add openstack/dib-utils to jobs requirements https://review.openstack.org/510555 | 12:37 |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/openstack-zuul-jobs master: WIP: [rally] Disable sahara for *-verify-* jobs https://review.openstack.org/510556 | 12:37 |
| *** chlong has quit IRC | 12:37 | |
| *** gouthamr has quit IRC | 12:38 | |
| *** baoli has joined #openstack-infra | 12:39 | |
| *** slaweq has quit IRC | 12:39 | |
| openstackgerrit | Shachar Snapiri proposed openstack-infra/project-config master: Add neutron as required project to legacy-dragonflow-dsvm-fullstack https://review.openstack.org/510557 | 12:40 |
| *** mriedem has joined #openstack-infra | 12:40 | |
| *** shardy_lunch is now known as shardy | 12:40 | |
| *** bramleya_ has joined #openstack-infra | 12:42 | |
| markus_z | When I use zuul in my 3rd-party-CI, how do I verify that the skip-if statements I added to my layout get recognized? I don't find a hint in the zuul logs. | 12:43 |
| AJaeger | the layout.yaml files expands it, let me give you an example... | 12:43 |
| AJaeger | markus_z: http://logs.openstack.org/35/510435/2/check/gate-project-config-layout/adb2d5d/console.html#_2017-10-09_12_04_22_216612 | 12:44 |
| AJaeger | These MatchAny lines come from skip-if | 12:44 |
| *** thorre has joined #openstack-infra | 12:45 | |
| *** camunoz has joined #openstack-infra | 12:47 | |
| markus_z | Is that the normal "console.log" file which gets created when devstack is running? | 12:48 |
| markus_z | I don't have such lines in my "console.log" | 12:48 |
| markus_z | http://paste.openstack.org/show/623100/ is our layout.yml file. Did I put the skip-if at the wrong place? | 12:50 |
| *** slaweq has joined #openstack-infra | 12:51 | |
| markus_z | A few log lines at https://github.com/openstack-infra/zuul/blob/71d98174067951bd2dbe24b57747159e24a0a42f/zuul/model.py#L516-L538 would have been nice. | 12:52 |
| markus_z | That's the area where the skip-if magic happens, right? | 12:53 |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/openstack-zuul-jobs master: [rally] Disable sahara for *-verify-* jobs https://review.openstack.org/510556 | 12:54 |
| AJaeger | markus_z: that's the project-config layout check (run tox -e zuul in project-config to get it) | 12:57 |
| *** wolverineav has joined #openstack-infra | 12:57 | |
| markus_z | AJaeger: sorry, I didn't get that | 12:58 |
| *** jpena is now known as jpena|lunch | 12:58 | |
| AJaeger | markus_z: change "name: check-tempest-dsvm-neutron-full.*$" to "name: ^check-tempest-dsvm-neutron-full.*$" | 12:58 |
| AJaeger | otherwise it's not treated as regex | 12:58 |
| AJaeger | markus_z: the link I gave is the output of tox -e zuul - it has nothing to do with devstack | 12:59 |
| markus_z | OK, I understand the regex thing with "^", I didn't know that. | 13:01 |
| mordred | morning AJaeger! | 13:01 |
| markus_z | I'm still not sure how to use tox in my "3rd party CI staging environment". I don't want to add things to the zuul code base (where I assume that tox gets used). | 13:01 |
| tosky | question for zuulv3 maintainers: would it make sense to have a shared pylint job in openstack-zuul-jobs ? | 13:02 |
| mordred | AJaeger: yah - let's go ahead and land that infra-manual patch - your updates look good to me | 13:02 |
| *** wolverineav has quit IRC | 13:02 | |
| tosky | there are tons of project using it and the converted legacy jobs all use /usr/local/jenkins/slave_scripts/run-pylint.sh pylint | 13:02 |
| tosky | legacy projects | 13:02 |
| mordred | tosky: yes | 13:02 |
| mordred | tosky: I think that would be a great idea | 13:03 |
| tosky | mordred: it is something in the queue, or something that will wait until someone writes a patch for it? | 13:03 |
| tosky | setting expectation of whether I should start writing it or wait | 13:03 |
| mordred | tosky: it's not in the queue yet - so a patch would be welcome if you have time | 13:04 |
| tosky | ack, thanks | 13:05 |
| mordred | tosky: I *think* it should just be a job with openstack-tox as parent and tox_envlist: pylint ... it doesn't look like /usr/local/jenkins/slave_scripts/run-pylint.sh actually does anything particularly useful | 13:06 |
| *** pblaho has quit IRC | 13:06 | |
| *** wolverineav has joined #openstack-infra | 13:06 | |
| tosky | yep, looks like that | 13:07 |
| tosky | then I would need some job in a local repository to test it | 13:07 |
| tosky | ok, let's see | 13:07 |
| tosky | I need to start with a basic job for all the jobs in the sahara repositories, and instead of converting the tempest one I wanted to try a simple one anyway | 13:08 |
| AJaeger | mordred: requirements check is broken, see http://logs.openstack.org/01/509401/1/check/requirements-check/86853eb/ - I'll add to etherpad | 13:10 |
| *** dprince has joined #openstack-infra | 13:10 | |
| *** pblaho has joined #openstack-infra | 13:11 | |
| openstackgerrit | Bernard Cafarelli proposed openstack-infra/project-config master: Add new repo for networking-sfc tempest plugin https://review.openstack.org/510553 | 13:12 |
| AJaeger | markus_z: the log files I referenced comes from http://git.openstack.org/cgit/openstack-infra/project-config/tree/tox.ini#n78 - this is somethign you can add to your instance if you want to learn what's going on. | 13:13 |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Make sure releasenotes always build on master https://review.openstack.org/510564 | 13:14 |
| *** liusheng has quit IRC | 13:15 | |
| mordred | AJaeger: that ^^ should fix the releasenotes thing | 13:15 |
| *** liusheng has joined #openstack-infra | 13:16 | |
| markus_z | AJaeger: You mean this line? zuul-server -c tools/zuul.conf-sample -l zuul/layout.yaml -t {envdir}/tmp/job-list.txt | 13:16 |
| *** gcb has joined #openstack-infra | 13:17 | |
| mordred | AJaeger: looking at check-reqiurements now | 13:17 |
| *** eharney has joined #openstack-infra | 13:19 | |
| *** kgiusti has joined #openstack-infra | 13:19 | |
| *** slaweq has quit IRC | 13:25 | |
| *** lbragstad has joined #openstack-infra | 13:26 | |
| dirk | so zuul and jenkins are running in parallel now and can't agree ? | 13:26 |
| dirk | https://review.openstack.org/#/c/510376/1 | 13:26 |
| *** LindaWang has joined #openstack-infra | 13:26 | |
| markus_z | AJaeger: Thanks, I think adding the ^ enabled it for me. I hacked some log statements into the code which showed me that the skip-if gets applied. | 13:26 |
| AJaeger | markus_z: great! | 13:27 |
| *** bobh has joined #openstack-infra | 13:27 | |
| *** sree has joined #openstack-infra | 13:27 | |
| AJaeger | mordred: thanks. Could you review https://review.openstack.org/510306 , please? Hope I done it right. As a followup we need to do publishing to docs.o.o and developer.o.o. | 13:30 |
| AJaeger | mordred: question on 510564 | 13:30 |
| *** sree has quit IRC | 13:32 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add base job and roles for javascript https://review.openstack.org/510236 | 13:32 |
| mordred | AJaeger: reviewing/looking | 13:33 |
| AJaeger | mordred: take your time ;) | 13:34 |
| *** jstockto2000 has joined #openstack-infra | 13:35 | |
| *** jstockto2000 has quit IRC | 13:35 | |
| mordred | AJaeger: yes - I think that looks right. HOWEVER - if you want to put those into openstack-manuals I think that's totally fine | 13:36 |
| *** slaweq has joined #openstack-infra | 13:37 | |
| mordred | AJaeger: the publish job in project-config won't actually use these jobs as a parent - they will just have a similar playbook (there is a piece of copy-pasta required related to publish jobs right now) | 13:37 |
| mordred | AJaeger: so you'll essentially copy https://review.openstack.org/#/c/510306/2/playbooks/build-manuals-tox/post.yaml into a project-config job | 13:38 |
| mordred | AJaeger: anyway - it's your call as to where you want to keep the build job and project-template | 13:38 |
| mordred | but I'd suggest openstack-manuals just so the doc team can own it and have +A on it | 13:38 |
| AJaeger | I can move them - I was thinking it would be beneficial for the publish job to have them in one place | 13:39 |
| *** bnemec has joined #openstack-infra | 13:40 | |
| *** gouthamr has joined #openstack-infra | 13:40 | |
| *** gouthamr has quit IRC | 13:41 | |
| AJaeger | mordred: will think about it and get back on these - thanks | 13:41 |
| *** psachin has quit IRC | 13:48 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Make sure releasenotes always build on master https://review.openstack.org/510564 | 13:48 |
| mordred | AJaeger: good catch ^^ | 13:49 |
| *** gouthamr has joined #openstack-infra | 13:49 | |
| mordred | fungi, yolanda: feel like an easy +A? https://review.openstack.org/#/c/510540 | 13:49 |
| AJaeger | mordred: +2a on 510540 | 13:50 |
| openstackgerrit | Merged openstack-infra/project-config master: Publish docs for storyboard in post https://review.openstack.org/509888 | 13:51 |
| openstackgerrit | Merged openstack-infra/project-config master: Add glance to integrated gate https://review.openstack.org/510305 | 13:51 |
| openstackgerrit | Merged openstack-infra/project-config master: Add more neutron references to neutron-dynamic-routing https://review.openstack.org/510355 | 13:51 |
| openstackgerrit | Bernard Cafarelli proposed openstack-infra/project-config master: Add new repo for networking-sfc tempest plugin https://review.openstack.org/510553 | 13:52 |
| *** sree has joined #openstack-infra | 13:52 | |
| mordred | AJaeger: oh - however, if we do keep those manuals jobs in o-z-j- - we should move the project-templates out of zuul-legacy-project-tempaltes | 13:52 |
| AJaeger | mordred: yes, we should - followup does this | 13:53 |
| mordred | cool | 13:53 |
| AJaeger | mordred: https://review.openstack.org/#/c/510172/ | 13:53 |
| AJaeger | mordred: but first your change https://review.openstack.org/510564 should merge, then I can recheck the openstack-manauls to see that the release notes build - and then +A and we can merge 510172 (alternative: Push a test change) | 13:55 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove mistral legacy jobs https://review.openstack.org/510009 | 13:56 |
| *** jpena|lunch is now known as jpena | 13:56 | |
| openstackgerrit | Luigi Toscano proposed openstack-infra/openstack-zuul-jobs master: Add a shared openstack-tox-pylint job https://review.openstack.org/510578 | 13:57 |
| * AJaeger will do some cycling now, bbl | 13:57 | |
| *** isaacb has quit IRC | 13:58 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Don't modify project-templates to add job name attributes https://review.openstack.org/510304 | 13:58 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Don't use cached config when deleting files https://review.openstack.org/510317 | 13:58 |
| *** jdandrea_ has joined #openstack-infra | 13:59 | |
| *** links has quit IRC | 14:03 | |
| *** felipemonteiro__ has joined #openstack-infra | 14:03 | |
| *** bramleya_ has quit IRC | 14:03 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Update statsd output for tenants https://review.openstack.org/510580 | 14:03 |
| *** felipemonteiro_ has joined #openstack-infra | 14:04 | |
| *** ykarel is now known as ykarel|away | 14:07 | |
| *** zaneb has joined #openstack-infra | 14:08 | |
| *** chlong has joined #openstack-infra | 14:08 | |
| *** felipemonteiro__ has quit IRC | 14:08 | |
| *** david-lyle has joined #openstack-infra | 14:08 | |
| *** alexchadin has quit IRC | 14:10 | |
| openstackgerrit | Merged openstack-infra/infra-manual master: Be a bit more prescriptive about getting started https://review.openstack.org/509939 | 14:15 |
| *** slaweq has quit IRC | 14:16 | |
| *** panda|bbl is now known as panda | 14:16 | |
| *** esberglu has joined #openstack-infra | 14:19 | |
| *** chlong_ has joined #openstack-infra | 14:21 | |
| *** gouthamr has quit IRC | 14:23 | |
| *** hashar is now known as hasharAway | 14:27 | |
| *** MasterPiece2 has quit IRC | 14:29 | |
| *** spectr has quit IRC | 14:30 | |
| *** markus_z has quit IRC | 14:32 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Mirror images from RDO server https://review.openstack.org/510362 | 14:34 |
| *** kiennt26 has joined #openstack-infra | 14:37 | |
| *** gouthamr has joined #openstack-infra | 14:37 | |
| *** slaweq has joined #openstack-infra | 14:40 | |
| jeblair | fungi: can you +3 510198 pls? | 14:44 |
| *** hongbin has joined #openstack-infra | 14:45 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Improve exception handling around lost requests https://review.openstack.org/510594 | 14:47 |
| *** thorst has joined #openstack-infra | 14:49 | |
| jeblair | dmsimard: question on 504788 | 14:51 |
| *** cshastri has quit IRC | 14:53 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add base job and roles for javascript https://review.openstack.org/510236 | 14:55 |
| *** wolverineav has quit IRC | 14:55 | |
| *** links has joined #openstack-infra | 14:57 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Improve exception handling around lost requests https://review.openstack.org/510594 | 14:57 |
| mriedem | looks like http://status.openstack.org/elastic-recheck/index.html is busted | 14:58 |
| mriedem | do we have logs that show any failures? | 14:58 |
| mriedem | e.g. it's not showing this newish query https://review.openstack.org/#/c/509899/ | 14:58 |
| *** wolverineav has joined #openstack-infra | 14:58 | |
| mordred | AJaeger: cool - https://review.openstack.org/#/c/510172/ lgtm - same question though about whether that should perhaps be in the openstack-manuals repo | 15:01 |
| tosky | for the sake of logs, here in the right channel: just to be clear: you mean convert the legacy pylint to use the new job? I thought that the idea was to not change those jobs - apart from bug fixes | 15:02 |
| jeblair | mriedem: i went to check, but status.openstack.org's host key has changed since the last time i logged into it. it's probably nothing, but i don't remember us rebuilding that recently, so i'd like to wait for another infra-root to help. | 15:02 |
| mordred | jeblair: I also do not remember rebuilding it recently - but I also agree it's probably nothing and would like to wait for another infra-root | 15:02 |
| tosky | AJaeger: and I know you answered "yes" (elsewhere), so that the jobs can be cleaned in one shot | 15:03 |
| mordred | tosky: I don't think you want to convert the legacy job - I think you just want to make a nice new openstack-tox-pylint job in openstack-zuul-jobs in zuul.d/jobs.yaml ... | 15:03 |
| AJaeger | mordred: let me check how the publish job looks like and decide afterwards... moving is easy, so I'm hppy to see these two merge for now | 15:03 |
| mordred | AJaeger: well..... moving is actually a multi-patch dance is why I'm asking | 15:04 |
| tosky | mordred: yes, that's the thing, but - ok, please check the comments on https://review.openstack.org/#/c/510578/ | 15:04 |
| mordred | tosky: then you want to update any uses of legacy-.*-pylint to ... I will go read before I keep typing :) | 15:04 |
| AJaeger | mordred: my suggestion to tosky was to remove the legacy jobs everywhere and use the new job everywhere - so that we have one change to update them instead of each project doing it | 15:04 |
| AJaeger | mordred: exactly what you said | 15:05 |
| AJaeger | mordred: let me WIP my change, work on the publish one - and then discuss again | 15:05 |
| mordred | AJaeger: yes! | 15:05 |
| tosky | and then projects, when creating their .zuul.yaml, will just notice "oh, a new job" and use it instead of having to create one | 15:05 |
| tosky | is it correct? | 15:05 |
| mordred | tosky: yes, that's is correct | 15:05 |
| tosky | ack | 15:05 |
| stephenfin | dhellmann: Want to take a look at https://review.openstack.org/#/c/510163/ today? :) | 15:06 |
| mordred | tosky: so next step (once you're happy that the job works right) is a patch to project-config as AJaeger mentioned that replaces legacy.*pylint in zuul.d/projects.yaml with the new openstack-tox-pylint | 15:06 |
| tosky | perfect | 15:06 |
| jeblair | mordred: has status.o.o key changed for you recently? | 15:06 |
| mnaser | infra-root: not sure but it might help to find out about the status.o.o rebuild with nova instance-action-list <instance-uuid> will show if it was rebuilt (and if so when) | 15:06 |
| mordred | tosky: then you can do one more patch to o-z-j that removes all the legacy jobs and their playbooks | 15:07 |
| tosky | when everything works | 15:07 |
| mordred | jeblair: I was just able to ssh in to it and did not have any errors | 15:07 |
| mordred | jeblair: what's the right way for me to produce the host fingerprint? | 15:08 |
| jeblair | mordred: 'ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key.pub' and i just did that via ansible from puppetmaster and get a matching hash, so i think i'm good | 15:08 |
| jeblair | mordred: it looks like i simply have not logged into that host in a year | 15:09 |
| jeblair | which, you know, is actually the way this is supposed to work, so yay :) | 15:09 |
| mordred | jeblair: \o/ success | 15:10 |
| *** trown is now known as trown|outtypewww | 15:12 | |
| openstackgerrit | Merged openstack-infra/project-config master: End gate of openstack/pylockfile https://review.openstack.org/508650 | 15:14 |
| openstackgerrit | Merged openstack-infra/project-config master: Add required-projects to the blazar-dashboard jobs https://review.openstack.org/509078 | 15:14 |
| *** slaweq has quit IRC | 15:14 | |
| *** kjackal__ has quit IRC | 15:16 | |
| *** slaweq has joined #openstack-infra | 15:18 | |
| *** vsaienk0 has quit IRC | 15:19 | |
| mordred | tosky: thanks for working on that btw! | 15:20 |
| *** gcb has quit IRC | 15:20 | |
| *** chlong has quit IRC | 15:22 | |
| tosky | mordred: thanks, but it's just an instance of https://imgur.com/gallery/rQIb4Vw :) | 15:22 |
| *** chlong_ has quit IRC | 15:22 | |
| mordred | tosky: :) ... but have you seen how many lightbulbs we need to replace? | 15:22 |
| fungi | mordred: jeblair: status.o.o's host key is one i have cached already (not sure how long that's been) | 15:23 |
| jeblair | fungi: cool, thanks | 15:23 |
| tosky | lightbulbs, lightbulbs everywhere | 15:23 |
| fungi | looks like those were generated on may 25 of 2016 | 15:23 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: WIP: Publish manuals https://review.openstack.org/510604 | 15:23 |
| AJaeger | mordred: could you review the above and tell me: a) what to change to publish to developer.o.o; b) whether this is correct? ^ | 15:23 |
| *** andreas_s has quit IRC | 15:26 | |
| *** andreas_s has joined #openstack-infra | 15:30 | |
| *** LindaWang has quit IRC | 15:31 | |
| AJaeger | mordred: let me rework a bit, I have some ideas now... | 15:33 |
| *** hongbin_ has joined #openstack-infra | 15:34 | |
| dhellmann | stephenfin : looking | 15:36 |
| *** rhallisey_ has quit IRC | 15:36 | |
| *** hongbin has quit IRC | 15:36 | |
| dhellmann | stephenfin : oh, is pbr itself causing the cyclical issue with the reno setuptools plugin? | 15:37 |
| *** ijw has joined #openstack-infra | 15:37 | |
| stephenfin | dhellmann: It sure is. I should have checked the logs | 15:37 |
| dhellmann | stephenfin : in your steps for reproducing the problem, do you run "tox -r" for step 5? because I was seeing repeated runs of tox work for alternating runs when I encountered the cycle thing. | 15:38 |
| dhellmann | I guess maybe you can't do that | 15:38 |
| stephenfin | dhellmann: Oh, yeah - sorry. 'tox --recreate' | 15:39 |
| *** tbarron is now known as tbarron|PTO | 15:39 | |
| stephenfin | The modifications I made were to tox in the .venv virtualenv | 15:39 |
| dhellmann | aha, right | 15:39 |
| stephenfin | So you can recreate the .tox virtualenvs as you want | 15:39 |
| *** pblaho has quit IRC | 15:39 | |
| dhellmann | tricky | 15:39 |
| *** andreas_s has quit IRC | 15:39 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: WIP: Publish manuals https://review.openstack.org/510604 | 15:39 |
| stephenfin | Aye, don't judge me for that one :) | 15:39 |
| AJaeger | mordred: gave 510604 another try and would love your expert review on that, please | 15:40 |
| dhellmann | stephenfin : cleverness leads to the dark side | 15:40 |
| *** ijw_ has quit IRC | 15:40 | |
| openstackgerrit | Merged openstack-infra/gear master: Add a send lock to the base Connection class https://review.openstack.org/508698 | 15:40 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: WIP: Publish manuals https://review.openstack.org/510604 | 15:40 |
| stephenfin | dhellmann: I've been told they have cookies though, so... | 15:41 |
| *** jgriffith_ is now known as jgriffith | 15:41 | |
| dhellmann | yes, we do | 15:41 |
| dhellmann | also pie | 15:41 |
| *** ijw_ has joined #openstack-infra | 15:41 | |
| asettle | dhellmann: \o/ pie! | 15:41 |
| jeblair | mriedem: i don't see any error messages; however, i'm not certain the logging configuration for the periodic graph setup is correct. | 15:42 |
| mordred | AJaeger: I just left a bunch of comments on PS1 ... | 15:42 |
| *** vhosakot has joined #openstack-infra | 15:42 | |
| mordred | AJaeger: but lemme look again | 15:42 |
| tosky | mordred, AJaeger: trying to convert pylint jobs, and in some cases the same job is run also on trusty; can the same job be used multiple times in the same project.check.jobs section? | 15:43 |
| *** priteau has joined #openstack-infra | 15:43 | |
| AJaeger | tosky: kill all those trusty ones, we're retiring newton in two weeks and then trusty is dead for these | 15:43 |
| dhellmann | mordred : I think stephenfin's patch https://review.openstack.org/#/c/510163/ is ready for you to +2a (you were +2 on the last draft and he had a pep8 fix) | 15:44 |
| tosky | AJaeger: ooook | 15:44 |
| AJaeger | tosky: but otherwise, you can! | 15:44 |
| tosky | ah, oki | 15:44 |
| *** dizquierdo has quit IRC | 15:44 | |
| *** ijw has quit IRC | 15:44 | |
| tosky | AJaeger: so is it just a matter of having multiple jobs with the same name in the results? | 15:44 |
| stephenfin | dhellmann: If we're going to bump the version, we might want to get this in too https://review.openstack.org/#/q/topic:bug/1702872+(status:open+OR+status:merged) | 15:44 |
| stephenfin | mordred and bnemec too ^ (dhellmann is +2 on one of them already) | 15:44 |
| dhellmann | stephenfin : makes sense | 15:44 |
| *** jpena is now known as jpena|brb | 15:45 | |
| tosky | AJaeger: also, those trusty jobs are used in some fuel-* projects; can I retire them also for fuel? | 15:45 |
| * stephenfin notes that virtually all of his pbr patches _remove_ code 🙈 | 15:45 | |
| AJaeger | mordred: I'll go over your comments and make changes as appropriate | 15:45 |
| dhellmann | stephenfin : we're already at sphinx >= 1.6.2 in g-r. | 15:46 |
| AJaeger | mordred, jeblair, clarkb, what do you think about tosky's question on fuel? I would say go ahead... | 15:46 |
| stephenfin | dhellmann: Actually, yeah - I was trying to not break existing non-OpenStack users though. Perhaps if we're doing a MAJOR bump I can simply drop it? | 15:47 |
| dhellmann | stephenfin: probably. or we could just break one thing at a time, I guess. | 15:47 |
| *** andreas_s has joined #openstack-infra | 15:49 | |
| tosky | more precisely, it's about openstack/fuel-qa (and fuel-devops (legacy-fuel-devops-pylint-ubuntu-trusty) and fuel-qa (two instances of legacy-fuel-qa-pylint-ubuntu-trusty); both projects have the pylint job for xenial too | 15:49 |
| mordred | AJaeger: ok - have commented on the new patch - which looks great! | 15:49 |
| *** jdandrea_ has left #openstack-infra | 15:49 | |
| AJaeger | mordred: cool... next iteration will then be perfect ;) | 15:49 |
| openstackgerrit | Merged openstack-infra/system-config master: Remove cloud-init when we set hostnames https://review.openstack.org/510198 | 15:50 |
| *** Guest94492 has quit IRC | 15:50 | |
| mordred | dhellmann, stephenfin: we should add some reno releasenotes to the pbr dir at some point ... | 15:50 |
| *** kiennt26 has quit IRC | 15:51 | |
| *** slaweq has quit IRC | 15:51 | |
| stephenfin | dhellmann: Your call. I'm personally for removing it if we have the chance (I really want to remove that sphinx_build thing eventually), but I can be swayed | 15:51 |
| stephenfin | mordred: Good idea. I'll go set that up | 15:52 |
| dhellmann | stephenfin : maybe set up the patch | 15:52 |
| dhellmann | (for removing the feature, that is) | 15:52 |
| stephenfin | (y) | 15:52 |
| *** ihrachys has joined #openstack-infra | 15:52 | |
| mordred | dhellmann, stephenfin: I'm good either way re: removing the thing - but I think the deprecation patch is safe so I'll go ahead and land that | 15:53 |
| stephenfin | Sounds good to me. Thanks, mordred | 15:53 |
| *** tmorin has left #openstack-infra | 15:54 | |
| stephenfin | dhellmann: fwiw, I also want to remove the apidoc integration from pbr https://github.com/sphinx-doc/sphinx/pull/4101 | 15:55 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: WIP: Publish manuals https://review.openstack.org/510604 | 15:55 |
| AJaeger | mordred: next try ^ | 15:55 |
| stephenfin | (just as an FYI) | 15:55 |
| AJaeger | mordred: I now see that there's no duplication with the other change so can move those over to openstack-manuals - and reuse for those three repos then... | 15:55 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Filter parsing issue https://review.openstack.org/510613 | 15:56 |
| openstackgerrit | Merged openstack/diskimage-builder master: Add timestamp output filter https://review.openstack.org/474830 | 15:56 |
| dhellmann | stephenfin : we'll need to be more careful about the api-doc change and make sure no projects are using it. | 15:57 |
| stephenfin | dhellmann: Yup. Think big "this is going away in 5.0 (?) warning" before I do it | 15:57 |
| dhellmann | yeah | 15:57 |
| *** andreas_s has quit IRC | 15:57 | |
| stephenfin | and that assumes it even lands upstream. I'm having difficulty articulating the difference between sphinx-apidoc and that 'autosummary' directive 🙈 | 15:58 |
| *** slaweq has joined #openstack-infra | 15:58 | |
| *** egonzalez has quit IRC | 15:58 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Filter parsing issue https://review.openstack.org/510613 | 15:59 |
| *** e0ne has quit IRC | 16:01 | |
| *** armax has joined #openstack-infra | 16:02 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Marketplace API https://review.openstack.org/498102 | 16:03 |
| mordred | dhellmann: also - sometime when we've each got a little more time, I think AJaeger and I would like to bend your ear about potentially making a sphinx plugin that does the incorporating of translations that the releasenotes job currently does | 16:03 |
| mordred | dhellmann: I don't know if it's logic that should be added to the existing reno plugin - or should be its own plugin | 16:03 |
| dhellmann | sure, I'm interested in hearing more about that, too | 16:04 |
| dhellmann | I would have thought sphinx's existing translation support would be enough, but I have so far avoided learning how any of that works so I'm just guessing | 16:04 |
| mhayden | i have a repo that isn't running anything in zuulv3 -- not even legacy jobs. am i missing a step? | 16:05 |
| *** jaianshu has joined #openstack-infra | 16:05 | |
| mordred | dhellmann: http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/roles/build-releasenotes/tasks/main.yaml is what we're doing for releasenotes translations currently | 16:05 |
| *** jaianshu has left #openstack-infra | 16:07 | |
| dhellmann | mordred : I'll bet we could do something with a domain index to build the list of available translations | 16:07 |
| *** andreas_s has joined #openstack-infra | 16:07 | |
| dhellmann | mordred : although the builds of the translated documents would be harder to do as a plugin | 16:07 |
| *** jascott1 has quit IRC | 16:09 | |
| mnaser | mhayden could it be not added in projects.yaml by any chance? | 16:10 |
| mnaser | (for zuulv3) | 16:10 |
| AJaeger | mhayden: zuul v3 is really busy right now, did you check the status page? | 16:11 |
| fungi | i notice on https://review.openstack.org/498848 (a jjb change) we ran openstack-tox-cover but not the openstack- prefixed unit test jobs. is that a mistake? | 16:11 |
| mhayden | AJaeger: does it use the existing status pages? i hunted around for my jobs in zuulv3 but couldn't find any | 16:11 |
| AJaeger | fungi, mordred changed it - we don't need requirements | 16:11 |
| AJaeger | mhayden: http://zuulv3.openstack.org/ | 16:11 |
| AJaeger | bbl | 16:12 |
| * mhayden puts on the dunce hat :| | 16:12 | |
| fungi | AJaeger: oh, so we're running regular tox-cover on jjb now and that change just needs a recheck? | 16:12 |
| * fungi checks | 16:12 | |
| *** dave-mcc_ is now known as dave-mccowan | 16:13 | |
| fungi | looks like we still explicitly add openstack-tox-cover in check and post (though not gate, amusingly enough) | 16:14 |
| fungi | mordred: ^ AJaeger says you changed that... is there a pending fix up for review i should be looking at? | 16:14 |
| *** jpena|brb is now known as jpena | 16:15 | |
| *** udesale has quit IRC | 16:16 | |
| AJaeger | fungi: change is I914f660dbcd9680f872ed99f380bda049e1e0d2c | 16:16 |
| AJaeger | (merged, might not include tox-cover) | 16:16 |
| *** chlong_ has joined #openstack-infra | 16:16 | |
| mhayden | mnaser: it's in projects.yaml -- the project is 'ansible-hardening' | 16:16 |
| *** chlong has joined #openstack-infra | 16:16 | |
| AJaeger | fungi: sorry, can't dig into this now | 16:17 |
| mhayden | mnaser: actually nevermind -- i think i am finding theissue | 16:17 |
| jaosorior | rcarrillocruz, mordred: hey, do you happen to have a doc or blog that shows how to use the openstack ansible modules and some good practices? | 16:18 |
| *** lucasagomes is now known as lucas-afk | 16:21 | |
| *** eharney has quit IRC | 16:21 | |
| rcarrillocruz | I haven't written any, but if you look at more detailed guides you could check it out books from Jesse Keating (jlk) or Walter Bentley | 16:21 |
| rcarrillocruz | Something I wrote and you could find useful is ansible-role-cloud-launcher | 16:22 |
| fungi | AJaeger: no need to dig into it right now. i can push up a patch | 16:22 |
| *** ldnunes has quit IRC | 16:23 | |
| rcarrillocruz | You feed a topology of resources and the role will create them in the right order | 16:23 |
| *** ldnunes has joined #openstack-infra | 16:23 | |
| *** andreas_s has quit IRC | 16:25 | |
| *** ykarel|away is now known as ykarel | 16:26 | |
| *** jogo has joined #openstack-infra | 16:28 | |
| *** jogo has quit IRC | 16:28 | |
| *** jogo has joined #openstack-infra | 16:28 | |
| *** vsaienk0 has joined #openstack-infra | 16:30 | |
| *** pcaruana has quit IRC | 16:32 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config master: Use tox-cover on non-constraints projects https://review.openstack.org/510620 | 16:32 |
| *** jaosorior has quit IRC | 16:33 | |
| openstackgerrit | Major Hayden proposed openstack-infra/project-config master: Migrate ansible-hardening to zuulv3 https://review.openstack.org/510622 | 16:33 |
| *** jaosorior has joined #openstack-infra | 16:33 | |
| *** jpich has quit IRC | 16:34 | |
| *** jascott1 has joined #openstack-infra | 16:34 | |
| *** andreas_s has joined #openstack-infra | 16:34 | |
| openstackgerrit | Major Hayden proposed openstack-infra/openstack-zuul-jobs master: Migrate ansible-hardening to zuulv3 https://review.openstack.org/510623 | 16:36 |
| openstackgerrit | Major Hayden proposed openstack-infra/project-config master: Migrate ansible-hardening to zuulv3 https://review.openstack.org/510622 | 16:37 |
| *** jaosorior has quit IRC | 16:38 | |
| *** vsaienk0 has quit IRC | 16:40 | |
| *** dizquierdo has joined #openstack-infra | 16:40 | |
| *** jaosorior has joined #openstack-infra | 16:40 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Mirror images from RDO server https://review.openstack.org/510362 | 16:41 |
| *** zaneb has quit IRC | 16:45 | |
| *** rcernin has quit IRC | 16:49 | |
| *** andreas_s has quit IRC | 16:52 | |
| *** csomerville has joined #openstack-infra | 16:53 | |
| *** jaosorior has quit IRC | 16:53 | |
| *** cody-somerville has quit IRC | 16:56 | |
| *** andreas_s has joined #openstack-infra | 16:57 | |
| *** Apoorva has joined #openstack-infra | 16:58 | |
| *** links has quit IRC | 16:58 | |
| *** Apoorva has quit IRC | 16:58 | |
| *** Apoorva has joined #openstack-infra | 16:58 | |
| *** derekh has quit IRC | 17:01 | |
| *** caphrim007 has joined #openstack-infra | 17:01 | |
| *** harlowja has joined #openstack-infra | 17:01 | |
| *** hongbin_ has quit IRC | 17:02 | |
| *** slaweq has quit IRC | 17:02 | |
| *** slaweq has joined #openstack-infra | 17:04 | |
| *** hongbin has joined #openstack-infra | 17:04 | |
| *** ijw_ has quit IRC | 17:07 | |
| *** hongbin has quit IRC | 17:09 | |
| *** Swami has joined #openstack-infra | 17:10 | |
| *** sshnaidm is now known as sshnaidm|off | 17:10 | |
| *** xarses has joined #openstack-infra | 17:10 | |
| *** camunoz has quit IRC | 17:10 | |
| *** lin_yang has joined #openstack-infra | 17:11 | |
| EmilienM | jeblair, mordred, clarkb: it sounds like https://review.openstack.org/#/c/508660/ was the last blocker for us to run zuul v3 legacy tripleo jobs | 17:12 |
| *** hongbin has joined #openstack-infra | 17:12 | |
| openstackgerrit | Merged openstack-dev/pbr master: builddoc: Use '[sphinx_build] builders' with Sphinx < 1.6 https://review.openstack.org/481620 | 17:12 |
| *** krtaylor_ has joined #openstack-infra | 17:13 | |
| *** ijw has joined #openstack-infra | 17:13 | |
| *** krtaylor has quit IRC | 17:14 | |
| *** armaan has quit IRC | 17:15 | |
| *** andreas_s has quit IRC | 17:15 | |
| openstackgerrit | Merged openstack-infra/project-config master: Use tox-cover on non-constraints projects https://review.openstack.org/510620 | 17:15 |
| *** armaan has joined #openstack-infra | 17:15 | |
| *** ijw has quit IRC | 17:16 | |
| *** krtaylor_ has quit IRC | 17:17 | |
| *** krtaylor has joined #openstack-infra | 17:17 | |
| AJaeger | mordred: do we still need https://review.openstack.org/495426 ? Then it needs a rebase to resolve conflicts with the split files | 17:17 |
| *** jpena is now known as jpena|away | 17:17 | |
| *** shardy has quit IRC | 17:18 | |
| *** zaneb has joined #openstack-infra | 17:22 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Publish manuals https://review.openstack.org/510604 | 17:22 |
| openstackgerrit | Luigi Toscano proposed openstack-infra/project-config master: Use the shared openstack-tox-pylint jobs https://review.openstack.org/510636 | 17:23 |
| openstackgerrit | Alex Schultz proposed openstack-infra/openstack-zuul-jobs master: Switch TripleO scenario upgrade jobs to nv https://review.openstack.org/510637 | 17:23 |
| *** slaweq has quit IRC | 17:23 | |
| *** slaweq has joined #openstack-infra | 17:24 | |
| *** lbragstad has quit IRC | 17:24 | |
| *** camunoz has joined #openstack-infra | 17:26 | |
| *** electrofelix has quit IRC | 17:26 | |
| *** dbecker has quit IRC | 17:27 | |
| AJaeger | zuul v3 is unhappy with change 510540; it's since 3:35 h in the gate and waiting for a node - the change after it is tested; the following changes have similar problems. Any Zuul v3 expert interested to check? | 17:28 |
| openstackgerrit | Luigi Toscano proposed openstack-infra/project-config master: Use the shared openstack-tox-pylint jobs https://review.openstack.org/510636 | 17:28 |
| AJaeger | jeblair, mordred, pabelanger, see my comment above ^ | 17:28 |
| openstackgerrit | Alex Schultz proposed openstack-infra/project-config master: Switch TripleO scenario upgrade jobs to nv https://review.openstack.org/510638 | 17:28 |
| *** jtomasek has quit IRC | 17:30 | |
| AJaeger | tosky: you have that extra ":" at least in one more place, best doublecheck your diff... | 17:30 |
| *** slaweq has quit IRC | 17:31 | |
| *** jtomasek has joined #openstack-infra | 17:31 | |
| *** slaweq has joined #openstack-infra | 17:31 | |
| openstackgerrit | Merged openstack-dev/pbr master: Remove support for command hooks https://review.openstack.org/510163 | 17:31 |
| *** slaweq has quit IRC | 17:32 | |
| *** slaweq has joined #openstack-infra | 17:32 | |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add a generic stage-artifacts role https://review.openstack.org/509233 | 17:33 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add compress capabilities to stage artifacts https://review.openstack.org/509234 | 17:33 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add a generic process-test-results role https://review.openstack.org/509459 | 17:33 |
| openstackgerrit | Merged openstack-dev/pbr master: Deprecate support for Sphinx < 1.6 https://review.openstack.org/481621 | 17:33 |
| *** sree has quit IRC | 17:39 | |
| *** sree has joined #openstack-infra | 17:40 | |
| *** slaweq has quit IRC | 17:40 | |
| *** slaweq has joined #openstack-infra | 17:41 | |
| *** lbragstad has joined #openstack-infra | 17:42 | |
| *** ijw has joined #openstack-infra | 17:43 | |
| *** sree has quit IRC | 17:44 | |
| *** sambetts is now known as sambetts|afk | 17:45 | |
| *** slaweq has quit IRC | 17:45 | |
| *** armaan has quit IRC | 17:45 | |
| *** armaan has joined #openstack-infra | 17:45 | |
| *** ijw has quit IRC | 17:47 | |
| *** e0ne has joined #openstack-infra | 17:51 | |
| *** thorst has quit IRC | 17:51 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/openstack-zuul-jobs master: Switch TripleO scenario upgrade jobs to nv https://review.openstack.org/510637 | 17:54 |
| *** hongbin has quit IRC | 17:54 | |
| *** dhinesh has joined #openstack-infra | 17:55 | |
| *** andreas_s has joined #openstack-infra | 17:56 | |
| *** mriedem1 has joined #openstack-infra | 17:58 | |
| *** mriedem has quit IRC | 17:59 | |
| *** edmondsw has quit IRC | 18:00 | |
| *** edmondsw has joined #openstack-infra | 18:02 | |
| *** armaan_ has joined #openstack-infra | 18:02 | |
| *** thorst has joined #openstack-infra | 18:04 | |
| *** armaan has quit IRC | 18:05 | |
| *** tosky__ has joined #openstack-infra | 18:05 | |
| *** dtantsur is now known as dtantsur|afk | 18:06 | |
| *** edmondsw has quit IRC | 18:06 | |
| *** ganso has joined #openstack-infra | 18:07 | |
| *** slaweq has joined #openstack-infra | 18:07 | |
| ganso | hello guys. I am facing something similar to https://bugs.launchpad.net/glance/+bug/1695299 | 18:07 |
| openstack | Launchpad bug 1695299 in Glance queens "Glance installation fails if password contains '@' symbol" [High,In progress] - Assigned to Aizuddin Zali (mymzbe) | 18:07 |
| AJaeger | ganso: best to ask the glance developer - #openstack-glance | 18:07 |
| ganso | AJaeger: thanks, I'll do that | 18:07 |
| *** Apoorva_ has joined #openstack-infra | 18:08 | |
| thingee | AJaeger: any suggestions of what project gets this? https://review.openstack.org/#/c/509943/ | 18:08 |
| *** armaan_ has quit IRC | 18:09 | |
| *** armaan has joined #openstack-infra | 18:09 | |
| *** andreas_s has quit IRC | 18:09 | |
| ganso | AJaeger: the glance developer is not online. I am curious if there is any way to escape characters such as "@" or "(" in devstack. It errors-out when generating the .localrcauto.conf file | 18:09 |
| fungi | thingee: if you don't know for sure what team it's under, then it might be something the tc "owns" or it might belong to a sig or it might warrant a new team, or... | 18:09 |
| fungi | ganso: oh, you didn't mention devstack... you might want to ask around in #openstack-qa in that case (devstack is a qa project) | 18:10 |
| ganso | fungi: thanks, going to check over there. Thanks again | 18:10 |
| *** pcaruana has joined #openstack-infra | 18:11 | |
| fungi | ganso: or failing that, asl in #openstack-dev | 18:11 |
| *** Apoorva has quit IRC | 18:11 | |
| mnaser | given the removal/transition out of zuul-cloner, what's the ideal best way to determine if we are running inside the gate? (in puppet, we used to check if zuul cloner exited on the machine we're on) | 18:11 |
| *** mriedem1 is now known as mriedem | 18:11 | |
| fungi | ganso: point being, #openstack-infra isn't really a channel about running openstack, it's a channel for the various bits of infrastructure our community relies on in the day-to-day production of openstack software | 18:11 |
| *** slaweq has quit IRC | 18:12 | |
| *** edmondsw has joined #openstack-infra | 18:12 | |
| mnaser | i was thinking to check if the zuul variable is defined | 18:12 |
| fungi | mnaser: yeah, i would pick an environment variable which is guaranteed to be set by zuul | 18:13 |
| *** ykarel has quit IRC | 18:14 | |
| *** andreas_s has joined #openstack-infra | 18:14 | |
| fungi | mnaser: though ideally, you would not lbyl and instead just test the things you expect to work under ci but engineer graceful fallbacks when they don't exist | 18:15 |
| fungi | that tends to lead to a superior experience for local dev environments by keeping behavior as close to ci as possible | 18:16 |
| mnaser | fungi we had a fallback to use git clone if we were running inside the gate, but im thinking we could do a git clone if the repo doesnt exist locally in the expected path | 18:16 |
| fungi | mnaser: yep, that's what devstack does, for example | 18:16 |
| mnaser | i think it'd be a good exercise to have a look at devstack-gate as i refactor this giant run_tests.sh | 18:16 |
| *** dprince has quit IRC | 18:16 | |
| fungi | well, i'm talking about devstack itself in this case | 18:16 |
| mnaser | oh | 18:17 |
| mnaser | gotcha | 18:17 |
| *** edmondsw has quit IRC | 18:17 | |
| fungi | it looks in your stack basedir for the source code it expects, and then clones it if not present | 18:17 |
| *** sambetts|afk has quit IRC | 18:17 | |
| fungi | and that's how it's operated since as long as i can recall at any rate, so must be a fairly successful approach | 18:17 |
| mnaser | fungi cool, i'll check it out, thank you! | 18:18 |
| fungi | and has useful knobs for things like forcing a reclone, et cetera | 18:18 |
| *** neon_icon has joined #openstack-infra | 18:19 | |
| openstackgerrit | Major Hayden proposed openstack-infra/project-config master: Migrate ansible-hardening to zuulv3 https://review.openstack.org/510622 | 18:19 |
| mnaser | i'll refactor things in repo but i think we'll end up with a lot of reusable roles for infra to use (ex: to test your puppet modules too :)) | 18:19 |
| mnaser | things like getting a specific version of puppet installed, puppetfile/r10k management, etc | 18:19 |
| *** sambetts_ has joined #openstack-infra | 18:20 | |
| *** slaweq has joined #openstack-infra | 18:24 | |
| *** eharney has joined #openstack-infra | 18:27 | |
| *** slaweq has quit IRC | 18:28 | |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: Add in merge-check templates for LOCI https://review.openstack.org/510649 | 18:29 |
| *** vsaienk0 has joined #openstack-infra | 18:29 | |
| *** andreas_s has quit IRC | 18:31 | |
| *** hasharAway is now known as hashar | 18:36 | |
| *** thorst_ has joined #openstack-infra | 18:36 | |
| *** thorst has quit IRC | 18:37 | |
| openstackgerrit | Mike Perez proposed openstack-infra/project-config master: Adding the Contributor Guide Project https://review.openstack.org/509943 | 18:37 |
| openstackgerrit | Mike Perez proposed openstack-infra/project-config master: Adding check/gate jobs to Contributor Guide https://review.openstack.org/509937 | 18:37 |
| *** e0ne has quit IRC | 18:39 | |
| *** vsaienk0 has quit IRC | 18:39 | |
| jeblair | AJaeger, mordred, pabelanger: 510540 is the change i mentioned in #zuul i'm looking into. i'd like to leave it there for a bit longer while i continue to investigate, but i'll need a couple of hours. | 18:41 |
| AJaeger | fungi: ^ | 18:42 |
| fungi | jeblair: oh, i missed you were already digging into that one | 18:43 |
| AJaeger | neither did I ;( | 18:43 |
| jeblair | well, i only mentioned the build id, not the change number. :) | 18:44 |
| *** dprince has joined #openstack-infra | 18:49 | |
| openstackgerrit | Mike Perez proposed openstack-infra/project-config master: Adding check/gate jobs to Contributor Guide https://review.openstack.org/509937 | 18:51 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Consolidate old database migrations https://review.openstack.org/510370 | 18:58 |
| openstackgerrit | Major Hayden proposed openstack-infra/project-config master: Migrate ansible-hardening to zuulv3 https://review.openstack.org/510622 | 19:00 |
| mhayden | does anyone know if i should remove the release announcement job from projects.yaml for zuulv3? https://review.openstack.org/#/c/510622/4/zuul.d/projects.yaml ( AJaeger susgested to ask here ) | 19:01 |
| AJaeger | question is whether that needs any secrets and therefore stay in the infra repos | 19:01 |
| jeblair | AJaeger, mhayden: the use of secrets is bound to playbooks associated with the job definition, so it's okay to have that job attached to the pipeline in either repo -- the secrets will only be available to the playbook attached to that job in project-config in either case. | 19:04 |
| openstackgerrit | Mike Perez proposed openstack-infra/project-config master: Adding the Contributor Guide Project https://review.openstack.org/509943 | 19:05 |
| openstackgerrit | Mike Perez proposed openstack-infra/project-config master: Adding check/gate jobs to Contributor Guide https://review.openstack.org/509937 | 19:05 |
| *** ijw has joined #openstack-infra | 19:06 | |
| jeblair | AJaeger, mhayden, mordred: that makes it more of a policy question about whether that's something that should be managed centrally for official openstack projects. i don't have an opinion on this one; i'd leave it to mordred+AJaeger to decide. | 19:06 |
| *** dhajare has quit IRC | 19:07 | |
| *** jrist has quit IRC | 19:07 | |
| openstackgerrit | Ken Giusti proposed openstack-infra/project-config master: Remove the legacy oslo.messaging project jobs https://review.openstack.org/510659 | 19:07 |
| *** slaweq has joined #openstack-infra | 19:08 | |
| *** eharney has quit IRC | 19:08 | |
| mnaser | this might be a nit but would it be possible to not make the -1 left by zuul display in the "verified" column in the list of reviews | 19:09 |
| mnaser | it is pretty hard figuring out which jobs actually passed CI and which ones failed inside zuul | 19:09 |
| *** edmondsw has joined #openstack-infra | 19:09 | |
| jeblair | mnaser: i don't know of a way, but i think we're close to having it "matter" anyway by switching back to v3. | 19:10 |
| mnaser | jeblair: i thought maybe it could be setup as third party ci .. because i noticed a -1 from that would usually not show up as an actual -1 in the list of reviews. but yeah, as switch is imminent it should be fine | 19:11 |
| openstackgerrit | Ken Giusti proposed openstack-infra/openstack-zuul-jobs master: Remove most of the oslo.messaging legacy tests https://review.openstack.org/510661 | 19:12 |
| *** jtomasek has quit IRC | 19:13 | |
| *** camunoz has quit IRC | 19:15 | |
| *** camunoz has joined #openstack-infra | 19:16 | |
| *** tosky has quit IRC | 19:20 | |
| *** camunoz has quit IRC | 19:20 | |
| *** jrist has joined #openstack-infra | 19:20 | |
| *** tosky__ is now known as tosky | 19:21 | |
| openstackgerrit | Mike Perez proposed openstack-infra/project-config master: Adding the Contributor Guide Project https://review.openstack.org/509943 | 19:23 |
| openstackgerrit | Mike Perez proposed openstack-infra/project-config master: Adding check/gate jobs to Contributor Guide https://review.openstack.org/509937 | 19:23 |
| *** esberglu has quit IRC | 19:23 | |
| *** neon_icon has quit IRC | 19:23 | |
| *** csomerville has quit IRC | 19:29 | |
| *** cody-somerville has joined #openstack-infra | 19:30 | |
| *** cody-somerville has joined #openstack-infra | 19:30 | |
| openstackgerrit | Luigi Toscano proposed openstack-infra/project-config master: Use the shared openstack-tox-pylint jobs https://review.openstack.org/510636 | 19:32 |
| *** srobert has joined #openstack-infra | 19:33 | |
| tosky | and this time hopefully for good | 19:36 |
| *** hemna_ has joined #openstack-infra | 19:36 | |
| *** esberglu has joined #openstack-infra | 19:37 | |
| *** thorst_ has quit IRC | 19:38 | |
| *** thorst has joined #openstack-infra | 19:39 | |
| *** esberglu has quit IRC | 19:39 | |
| *** esberglu has joined #openstack-infra | 19:39 | |
| *** esberglu has quit IRC | 19:40 | |
| *** esberglu has joined #openstack-infra | 19:40 | |
| *** esberglu has quit IRC | 19:40 | |
| *** esberglu has joined #openstack-infra | 19:40 | |
| *** srobert has quit IRC | 19:41 | |
| AJaeger | tosky: +2, thanks. | 19:43 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Add documentation on manually upgrading a StoryBoard instance https://review.openstack.org/510669 | 19:43 |
| tosky | sorry the errors, I should have checked better | 19:44 |
| *** mikal has joined #openstack-infra | 19:44 | |
| *** thorst has quit IRC | 19:44 | |
| *** thorst has joined #openstack-infra | 19:45 | |
| *** eharney has joined #openstack-infra | 19:45 | |
| *** camunoz has joined #openstack-infra | 19:46 | |
| *** mat128 has joined #openstack-infra | 19:46 | |
| *** Apoorva_ has quit IRC | 19:47 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/project-config master: Remove cron entries from nodepool launcher configs https://review.openstack.org/510676 | 19:55 |
| Shrews | pabelanger: ^^^ | 19:55 |
| mordred | AJaeger, mhayden: first of all, the announce-release job should be able to be used by that job - no need for its own job in any case | 19:56 |
| *** Apoorva has joined #openstack-infra | 19:57 | |
| mhayden | mordred: ah okay, so s/legacy-ansible-hardening-announce-release/announce-release/ there? | 19:57 |
| pabelanger | o/ | 19:57 |
| *** lukebrowning has quit IRC | 19:57 | |
| mordred | mhayden: yah. now- as for whether or not announce-release should be in the release pipeline in project-config or inside of ansible-hardening is a great question - and one I think we're still figuring out | 19:58 |
| pabelanger | Just between family activities today, have a few minutes to catch up on backscroll | 19:58 |
| *** lukebrowning has joined #openstack-infra | 19:58 | |
| mhayden | mordred: i'll leave it in projects.yaml in infra until it makes sense to yank it | 19:58 |
| mordred | cool | 19:58 |
| openstackgerrit | Major Hayden proposed openstack-infra/project-config master: Migrate ansible-hardening to zuulv3 https://review.openstack.org/510622 | 19:58 |
| mhayden | mordred: thanks for the assist ;) | 19:58 |
| fungi | mhayden: one of the things the release team is still trying to work out is how to validate whether or not official deliverables have the jobs mandated by the governance tags they're choosing to apply | 19:59 |
| fungi | it's possible keeping those centralized could help both for discoverability and also to help prevent accidental removals | 20:00 |
| mordred | mhayden, AJaeger: general working theory is that project-config should contain only the things that official projects are 'supposed' to run - but since we just grew that capability, I don't know that we've fully figured out the difference between "must" and "may" | 20:00 |
| mordred | fungi: ++ | 20:00 |
| *** lukebrow_ has joined #openstack-infra | 20:01 | |
| *** jpena|away is now known as jpena|off | 20:01 | |
| AJaeger | I suggest to leave the announce job for now in project-config. We can migrate later if needed... | 20:01 |
| mhayden | AJaeger: done | 20:01 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Properly populate Worklist.items in automatic worklists https://review.openstack.org/307712 | 20:01 |
| AJaeger | mhayden: thanks | 20:02 |
| AJaeger | mordred, fungi, could you review https://review.openstack.org/#/c/510604 , please? - hope this is fine now | 20:02 |
| mhayden | AJaeger: nah, thank you for helping me make my way through this ;) | 20:02 |
| *** lukebrowning has quit IRC | 20:03 | |
| *** ijw has quit IRC | 20:03 | |
| mordred | AJaeger: looks great! | 20:06 |
| AJaeger | Yeah ;) | 20:06 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Add reno for release notes management https://review.openstack.org/510679 | 20:06 |
| *** ijw has joined #openstack-infra | 20:07 | |
| AJaeger | thanks, mordred and fungi | 20:13 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: use announce-release everywhere https://review.openstack.org/510681 | 20:17 |
| AJaeger | mhayden, mordred, I cleaned up the announce-release on top of mhayden's change ^ | 20:17 |
| *** pcaruana has quit IRC | 20:18 | |
| mhayden | nice! | 20:18 |
| *** boris_42_ has joined #openstack-infra | 20:18 | |
| *** xarses has quit IRC | 20:18 | |
| *** dprince has quit IRC | 20:19 | |
| *** xarses has joined #openstack-infra | 20:19 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove legacy announce-release jobs https://review.openstack.org/510683 | 20:21 |
| AJaeger | and remove them all - mhayden that includes your job ^ | 20:22 |
| *** andreww has joined #openstack-infra | 20:22 | |
| openstackgerrit | Merged openstack-infra/project-config master: Publish manuals https://review.openstack.org/510604 | 20:22 |
| mhayden | AJaeger: ah okay, i guess i should back that change out of mine? | 20:23 |
| *** xarses has quit IRC | 20:23 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove legacy announce-release jobs https://review.openstack.org/510683 | 20:24 |
| AJaeger | mhayden: I can resolve a merge conflict if needed as well ;) | 20:24 |
| mhayden | sounds good | 20:24 |
| *** armaan has quit IRC | 20:24 | |
| * mhayden wants to be on the new stuff ;) | 20:24 | |
| *** armaan has joined #openstack-infra | 20:25 | |
| * AJaeger wants to cleanup ;) | 20:25 | |
| mhayden | that too | 20:26 |
| AJaeger | so, any takers to help mhayden and myself, please? https://review.openstack.org/510622 and https://review.openstack.org/510681 - and then we need rechecks and can merge https://review.openstack.org/#/c/510623/ | 20:26 |
| *** armax has quit IRC | 20:26 | |
| AJaeger | mhayden: I'll base mine on top of yours | 20:26 |
| mhayden | how kind! :) | 20:27 |
| mhayden | thanks! | 20:27 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove legacy announce-release jobs https://review.openstack.org/510683 | 20:28 |
| AJaeger | nice, removes 5700 lines ;) | 20:30 |
| * AJaeger waves good night | 20:30 | |
| *** kgiusti has left #openstack-infra | 20:30 | |
| fungi | have a good night AJaeger, and thanks for all the help! | 20:30 |
| andreaf | dmsimard: I hope I addressed your comments on https://review.openstack.org/#/q/status:open+project:openstack-infra/zuul-jobs+branch:master+topic:zuulv3-tempest | 20:33 |
| andreaf | mordred, jeblair: if you want to review my zuul-jobs patches ^^^ | 20:33 |
| *** rhallisey has quit IRC | 20:34 | |
| openstackgerrit | Merged openstack-infra/project-config master: Migrate ansible-hardening to zuulv3 https://review.openstack.org/510622 | 20:37 |
| *** priteau_ has joined #openstack-infra | 20:37 | |
| *** priteau has quit IRC | 20:37 | |
| *** Goneri has joined #openstack-infra | 20:39 | |
| fungi | is http://logs.openstack.org/3e/3e51efa608dafa038fb909d668734fbf4bea5a1a/release/mistral-lib-tarball-signing/5778634/console.html#_2017-10-08_21_44_37_125818 an indication that we have stale remotes on signing01.ci? zuul-cloner wants zm03 but the error is from trying to hit zm07 (a v3 merger?) | 20:41 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add upload-npm role https://review.openstack.org/510686 | 20:42 |
| fungi | infra-root: i thought we cleaned something like this up already, but maybe that was on proposal.slave instead? and what was the fix, just temporarily stop the launcher on zlstatic01 and then blow away all the git repos on the node? | 20:42 |
| *** hashar has quit IRC | 20:43 | |
| * fungi goes digging in the status log | 20:44 | |
| *** felipemonteiro has joined #openstack-infra | 20:45 | |
| *** thorst has quit IRC | 20:46 | |
| *** felipemonteiro_ has quit IRC | 20:46 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add release-openstack-javascript job https://review.openstack.org/502185 | 20:46 |
| *** florianf has quit IRC | 20:47 | |
| jeblair | fungi: yes, we probably did that for a different static node | 20:50 |
| *** jcoufal has quit IRC | 20:50 | |
| jeblair | fungi: fix should be to remove all workspaces from the node | 20:50 |
| smcginnis | Gerrit issues? Really seems to have slowed down. | 20:50 |
| fungi | jeblair: cool, i guess i should gracefully stop the launcher when i do that. i'll get started | 20:50 |
| smcginnis | To the point of getting timeouts now. | 20:51 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/devstack-gate master: WIP/DNM: test disabling UCA if fixes ironic multinode https://review.openstack.org/510690 | 20:52 |
| *** srobert has joined #openstack-infra | 20:52 | |
| *** felipemonteiro_ has joined #openstack-infra | 20:52 | |
| jlvillal | FYI: I've seen two 502 errors from Gerrit in the last minute | 20:53 |
| fungi | #status log cleared all old workspaces on signing01.ci to deal with those which had cached git remotes to some no-longer-existing zuul v2 mergers | 20:53 |
| openstackstatus | fungi: finished logging | 20:53 |
| *** ijw has quit IRC | 20:54 | |
| johnsom | FYI, I am also seeing 502s from gerrit. Retries are working. | 20:54 |
| *** ijw has joined #openstack-infra | 20:55 | |
| *** felipemonteiro has quit IRC | 20:55 | |
| smcginnis | johnsom: That makes at least two of us. | 20:55 |
| *** jkilpatr has quit IRC | 20:56 | |
| *** srobert has quit IRC | 20:56 | |
| jlvillal | Three... | 20:56 |
| *** erlon has joined #openstack-infra | 20:57 | |
| melwitt | four | 20:58 |
| *** lukebrow_ has quit IRC | 21:00 | |
| *** lukebrowning has joined #openstack-infra | 21:00 | |
| *** priteau_ has quit IRC | 21:01 | |
| *** rockyg has joined #openstack-infra | 21:01 | |
| mnaser | i guess it couldn't be possible that it's gc-ing for 10+ minutes | 21:02 |
| *** lukebrowning has quit IRC | 21:02 | |
| *** lukebrowning has joined #openstack-infra | 21:02 | |
| *** jascott1 has quit IRC | 21:02 | |
| *** lukebrowning has quit IRC | 21:03 | |
| smcginnis | I could make a commentary on some of the patches I've seen related to GC, but I won't. ;) | 21:03 |
| thingee | mordred: about that project https://review.openstack.org/#/q/topic:contributor-guide+status:open :) | 21:03 |
| thingee | mordred: I think we're in better shape now | 21:04 |
| thingee | #thanks pabelanger for merging thanks bot. | 21:07 |
| openstackstatus | thingee: Added thanks to Thanks page | 21:07 |
| thingee | sweet https://wiki.openstack.org/wiki/Thanks | 21:07 |
| thingee | ttx: ^ | 21:08 |
| thingee | Formatting could be better | 21:08 |
| thingee | also the bot could give a link to the thanks page? | 21:08 |
| *** yamahata has joined #openstack-infra | 21:10 | |
| fungi | hard to tell if gerrit's current java gc volume is worse (it's almost certainly the cause of the current intermittent 502 errors judging from the graph), as the previous major gc event we've seen since the last restart happened around 03:00z sunday the 1st | 21:10 |
| *** e0ne has joined #openstack-infra | 21:11 | |
| fungi | it did eventually complete, but the graph from that timeframe suggests it lasted 6 hours (hard to tell whether that's a quantization/granularity artifact) | 21:12 |
| thingee | #thanks mordred for making project creator documentation more clearer with zuul v3 | 21:13 |
| openstackstatus | thingee: Added thanks to Thanks page | 21:13 |
| *** jascott1 has joined #openstack-infra | 21:15 | |
| smcginnis | fungi: Any way to configure it to gc more often so it's smaller, faster chunks? | 21:15 |
| *** apetrich has quit IRC | 21:16 | |
| *** apetrich has joined #openstack-infra | 21:16 | |
| *** dizquierdo has quit IRC | 21:16 | |
| *** lukebrowning has joined #openstack-infra | 21:16 | |
| fungi | smcginnis: great question... i don't know but will take a look at the tuning parameters it accepts | 21:17 |
| openstackgerrit | Mike Perez proposed openstack-infra/statusbot master: Thanks bot should provide the thanks page url https://review.openstack.org/510699 | 21:17 |
| fungi | smcginnis: odds are it's not a gerrit configuration option, but rather something we need to configure for the jvm | 21:18 |
| *** jascott1 has quit IRC | 21:18 | |
| smcginnis | fungi: Seems like I may have known this at one point, but I've tried to purge my java years from my brain. | 21:18 |
| *** ijw has quit IRC | 21:19 | |
| *** ijw has joined #openstack-infra | 21:19 | |
| *** jascott1 has joined #openstack-infra | 21:20 | |
| *** Apoorva has quit IRC | 21:21 | |
| *** lukebrowning has quit IRC | 21:21 | |
| *** armax has joined #openstack-infra | 21:21 | |
| *** Apoorva has joined #openstack-infra | 21:21 | |
| clarkb | i think it is a response to some request or requests | 21:21 |
| clarkb | as it goes away and server recovers | 21:21 |
| clarkb | unlike the old behavior | 21:22 |
| *** slaweq has quit IRC | 21:22 | |
| *** lukebrowning has joined #openstack-infra | 21:23 | |
| fungi | clarkb: oh! i wonder if election tooling queries are bogging it down | 21:25 |
| *** lukebrowning has quit IRC | 21:27 | |
| *** ldnunes has quit IRC | 21:28 | |
| *** lukebrowning has joined #openstack-infra | 21:29 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add git timeout https://review.openstack.org/509517 | 21:29 |
| openstackgerrit | Michael Johnson proposed openstack-infra/irc-meetings master: Moving the Octavia meeting time and channel https://review.openstack.org/510701 | 21:32 |
| *** lukebrowning has quit IRC | 21:33 | |
| *** jkilpatr has joined #openstack-infra | 21:34 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Create sub_nodes file before trying to modify it https://review.openstack.org/510540 | 21:34 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Make sure releasenotes always build on master https://review.openstack.org/510564 | 21:34 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Add a shared openstack-tox-pylint job https://review.openstack.org/510578 | 21:34 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: [rally] Add openstack/dib-utils to jobs requirements https://review.openstack.org/510555 | 21:34 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: [rally] Disable sahara for *-verify-* jobs https://review.openstack.org/510556 | 21:34 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove Zun legacy jobs https://review.openstack.org/509969 | 21:34 |
| *** Shrews has quit IRC | 21:36 | |
| *** e0ne has quit IRC | 21:38 | |
| *** lukebrowning has joined #openstack-infra | 21:39 | |
| *** Shrews has joined #openstack-infra | 21:39 | |
| *** felipemonteiro_ has quit IRC | 21:43 | |
| *** lukebrowning has quit IRC | 21:43 | |
| *** efried has joined #openstack-infra | 21:44 | |
| *** lukebrowning has joined #openstack-infra | 21:45 | |
| *** gouthamr has quit IRC | 21:45 | |
| *** rockyg has quit IRC | 21:46 | |
| *** lukebrowning has quit IRC | 21:50 | |
| *** lukebrowning has joined #openstack-infra | 21:51 | |
| *** slaweq has joined #openstack-infra | 21:53 | |
| *** wolverineav has quit IRC | 21:53 | |
| *** ijw has quit IRC | 21:53 | |
| *** threestrands has joined #openstack-infra | 21:55 | |
| *** lukebrowning has quit IRC | 21:55 | |
| *** aeng has joined #openstack-infra | 21:58 | |
| *** chlong_ has quit IRC | 21:59 | |
| *** chlong has quit IRC | 21:59 | |
| *** bobh has quit IRC | 21:59 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/devstack-gate master: WIP/DNM: test using UCA queen if fixes ironic multinode https://review.openstack.org/510690 | 22:00 |
| jeblair | it's zuul meeting time in #openstack-meeting-alt | 22:00 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/devstack-gate master: WIP/DNM: Set UBUNTU_CLOUD_ARCHIVE_RELEASE=ocata in devstack https://review.openstack.org/510690 | 22:00 |
| *** slaweq has quit IRC | 22:02 | |
| mriedem | jeblair: well, i think i see at least what's wrong with e-r http://paste.openstack.org/show/623151/ :) | 22:04 |
| *** ijw has joined #openstack-infra | 22:06 | |
| *** ganso has quit IRC | 22:06 | |
| *** lukebrowning has joined #openstack-infra | 22:10 | |
| *** jrist has quit IRC | 22:14 | |
| *** jaypipes has quit IRC | 22:14 | |
| *** lukebrowning has quit IRC | 22:15 | |
| *** hongbin has joined #openstack-infra | 22:15 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/devstack-gate master: WIP/DNM: Set UBUNTU_CLOUD_ARCHIVE_RELEASE=ocata in devstack https://review.openstack.org/510690 | 22:20 |
| *** thorst has joined #openstack-infra | 22:21 | |
| *** thorst has quit IRC | 22:21 | |
| *** thorst has joined #openstack-infra | 22:22 | |
| *** Sukhdev has joined #openstack-infra | 22:23 | |
| *** boris_42_ has quit IRC | 22:25 | |
| *** lbragstad has quit IRC | 22:25 | |
| *** edmondsw has quit IRC | 22:27 | |
| *** andreas_s has joined #openstack-infra | 22:28 | |
| *** claudiub has quit IRC | 22:28 | |
| *** baoli has quit IRC | 22:29 | |
| mriedem | heh, so i'm adding a retry on 502 check to e-r when building the graph | 22:31 |
| mriedem | so far it's working | 22:31 |
| *** andreas_s has quit IRC | 22:32 | |
| *** bobh has joined #openstack-infra | 22:34 | |
| *** jrist has joined #openstack-infra | 22:35 | |
| pabelanger | thingee: \o/ | 22:37 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements https://review.openstack.org/503645 | 22:40 |
| ianw | is it just me, or is gerrit being a bit tardy ... cpu is looking a bit spiked https://review.openstack.org/monitoring?part=graph&graph=cpu | 22:40 |
| mriedem | ianw: known issue | 22:41 |
| dansmith | mriedem: why no restart yet? | 22:43 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Retry change queries on a 502 response https://review.openstack.org/510706 | 22:46 |
| *** andreww has quit IRC | 22:46 | |
| mriedem | mtreinish: clarkb: ^ should fix the graph builder again | 22:46 |
| mriedem | dansmith: not sure, zuulv3 meeting? | 22:47 |
| dansmith | hrm | 22:47 |
| ianw | fungi / jeblair: https://review.openstack.org/monitoring ... looks like fairly standard "too much memory & cpu going nuts". i can babysit a restart if we agree | 22:50 |
| *** Sukhdev has quit IRC | 22:51 | |
| *** lukebrowning has joined #openstack-infra | 22:51 | |
| fungi | ianw: dansmith: clarkb posits that since the upgrade to 2.13 those java gc spikes are induced by some outside process/user and no longer the runaway death spiral we used to observe | 22:52 |
| fungi | if so, a restart may only help briefly | 22:52 |
| dansmith | fungi: but it was better last week, no? | 22:52 |
| fungi | dansmith: it happened last week too, and then finished on its own | 22:52 |
| fungi | but it happened on a sunday | 22:52 |
| fungi | according to the graphs, october 1 | 22:53 |
| dansmith | so is this the new status quo or are we hoping to get some more detail? | 22:53 |
| *** lukebrowning has quit IRC | 22:56 | |
| *** sdague has quit IRC | 22:56 | |
| fungi | dansmith: it's probable we could do a better job of tuning the jvm heap management. i think clarkb was conferring with some gerrit contributors on getting them to update their tuning guide for current releases | 22:57 |
| dansmith | okay | 22:57 |
| *** lukebrowning has joined #openstack-infra | 22:58 | |
| dansmith | I wouldn't consider it usable based on today's performance | 22:58 |
| fungi | if this is like the gc spike on sunday, it resolves itself after some hours | 22:58 |
| *** neon_icon has joined #openstack-infra | 22:58 | |
| fungi | sunday the 1st i mean | 22:58 |
| fungi | not sunday yesterday | 22:58 |
| jeblair | this looks to be a little more sustained than then, however, i don't know if there's an rrd rollup involved | 23:00 |
| *** aeng has quit IRC | 23:00 | |
| jeblair | hrm, imgur isn't working for me | 23:01 |
| fungi | yes, i expect the squareness of the "spike" from 8 days ago is a quantization/granularity artifact | 23:01 |
| jeblair | http://files.openstack.org/user/corvus/gc.png | 23:02 |
| *** bobh has quit IRC | 23:02 | |
| *** tosky has quit IRC | 23:02 | |
| *** lukebrowning has quit IRC | 23:02 | |
| clarkb | what does the day graph look like? | 23:03 |
| clarkb | is it an all day thing? iirc last timr was ~4 hours | 23:03 |
| jeblair | clarkb: generously i'd say it started 3 hours ago. or maybe 2.5. | 23:03 |
| jeblair | i'll get a copy | 23:03 |
| clarkb | then gc stopped and a few hours later memory use fell by ~1/3 | 23:04 |
| fungi | i did a graph from midnight to midnight for the sunday spike and it starts at 06:00 and ends at 12:00, with no variation, which i have a feeling means that's teh sample size on the rollup | 23:04 |
| clarkb | the old behavior was to never recover | 23:04 |
| jeblair | http://files.openstack.org/user/corvus/gc-jour.png | 23:05 |
| *** vhosakot has quit IRC | 23:05 | |
| fungi | feeling french today i guess | 23:05 |
| *** vhosakot has joined #openstack-infra | 23:05 | |
| jeblair | just going with the flow: https://review.openstack.org/monitoring?part=graph&graph=gc&period=jour | 23:05 |
| fungi | nice, i'd never noticed that before | 23:06 |
| jeblair | in unrelated news, i have learned the french word for month. | 23:06 |
| *** MasterOfBugs has joined #openstack-infra | 23:06 | |
| jeblair | dansmith, clarkb, fungi, ianw: it seems like maybe we should see if we can ride this out for another 30-60 minutes to observe whether it self-corrects? and if not (if it either does not correct, or becomes intolerable) restart? | 23:07 |
| dansmith | well, the only problem with that sampling method is that most activity is naturally dropping off right now anyway right? | 23:08 |
| dansmith | although I guess your point is to see if it never goes away per usual previous behavior? | 23:08 |
| fungi | yes | 23:08 |
| jeblair | dansmith: yeah, not much we can do about that except write more patches :) | 23:08 |
| dansmith | I'm fully willing to believe it's all load-based in this case, I just worry about what we can do if it is | 23:09 |
| jeblair | it's not a *perfect* experiment | 23:09 |
| fungi | gives us an additional datapoint to bring to the attention of gerrit upstream | 23:09 |
| dansmith | sure, no argument, I'm not going to be around much longer to benefit from the potential speed of a restart, | 23:09 |
| dansmith | just saying the data will likely be influenced by the getting-beer-ness of the day :) | 23:10 |
| dansmith | if it really used to just jam up and never unjam on its own, then that'll be good data if this does | 23:10 |
| fungi | also if there's no java tuning we can do to make gc activity itself less crippling on the service, then maybe we can configure it to perform scheduled gc at times when we have lower volumes of developer activity so as to be slightly more convenient | 23:10 |
| dansmith | do you know it's just gc behavior? | 23:11 |
| ianw | ok | 23:11 |
| *** mriedem1 has joined #openstack-infra | 23:11 | |
| fungi | doesn't help to mitigate event-driven gc activity thuogh, if clarkb's theory about some outside forces making heavy use of gerrit pan out | 23:11 |
| fungi | dansmith: well, we have a graph of increases in gc activity coincident with reports of increased proxy errors, so the two seem related at least (but of course correlation does not imply causation) | 23:12 |
| dansmith | yeah | 23:12 |
| dansmith | can't you tell java to only do X amount of synchronous gc'ing or something? | 23:13 |
| fungi | and it is known in the java world that furious gc activity will basically stop your vm dead until it finishes | 23:13 |
| *** aeng has joined #openstack-infra | 23:13 | |
| fungi | your jvm i mean | 23:13 |
| * dansmith feels like there are copious java slurs to be used here | 23:13 | |
| *** mriedem has quit IRC | 23:13 | |
| fungi | dansmith: i started poking around at tuning guides for java in general and it seems like there's a variety of dynamic gc modes you can set but i don't know what's relevant under the jvm we're using for this | 23:14 |
| dansmith | anye | 23:14 |
| dansmith | er, aye | 23:14 |
| dansmith | I'm not a java guy so I have nothing to offer but complaints :) | 23:15 |
| fungi | for example, http://blog.mgm-tp.com/2013/03/garbage-collection-tuning/ is old enough as to likely be relevant | 23:15 |
| dansmith | maybe we should call oracle and ask for help? they seem uuber friendly these days :P | 23:15 |
| clarkb | we can try a dufferent GC method, add memory, write fewer patches, drink more | 23:15 |
| fungi | dansmith: i heard that if you want help with java, oracle suggests talking to apache now ;) | 23:15 |
| dansmith | really? | 23:16 |
| fungi | dansmith: oracle is divesting java to the asf last i heard (or was it the lf?) | 23:16 |
| *** Goneri has quit IRC | 23:16 | |
| dansmith | orly, I don't really pay attention to oracle or java news.. I guess I'm missing out | 23:16 |
| fungi | happened at roughly the same time they laid off the bulk of the remaining sun/solaris devs, if memory serves | 23:17 |
| fungi | a month or so ago? | 23:17 |
| dansmith | https://developers.slashdot.org/story/17/08/19/1637231/oracle-now-wants-to-give-java-ee-to-an-open-source-foundation | 23:17 |
| fungi | ahh, yup, mighta been eclipse too | 23:18 |
| fungi | maybe they haven't decided who yet | 23:18 |
| dansmith | I think the first comment there is probably all the summary we need: | 23:18 |
| dansmith | "We can't milk this for licensing money anymore, so we no longer want to invest in it" | 23:18 |
| dansmith | Share | 23:18 |
| fungi | hah | 23:18 |
| fungi | i mean, it's not funny... but it's still funny | 23:19 |
| dansmith | yeah | 23:19 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add git timeout https://review.openstack.org/509517 | 23:22 |
| *** armaan has quit IRC | 23:23 | |
| *** armaan has joined #openstack-infra | 23:24 | |
| *** ijw has quit IRC | 23:26 | |
| ianw | fungi: yeah, after i got jstack working in a local directory, yeah the threads constantly hogging cpu are the gc threads -> http://paste.openstack.org/show/623158/ | 23:34 |
| ianw | so i think i figured out what you already knew :) | 23:34 |
| fungi | i don't pretend to know anything when it comes to java | 23:35 |
| ianw | i doesn't seem like javamelody gives you any way to see the system process id? | 23:35 |
| fungi | i don't even know what those words mean in this context | 23:36 |
| ianw | fungi: well https://review.openstack.org/monitoring?part=threadsDump gives you all the java threads | 23:36 |
| ianw | but the only way i could correlate that to "top" output was via jstack | 23:36 |
| ianw | (see /tmp/threads.txt) | 23:37 |
| ianw | which gives you a fairly unhelpful nid= in hex | 23:37 |
| ianw | although i get that it only corresponds to linux thread id's as an implementation detail | 23:37 |
| ianw | but, correlating that and top output, my end conclusion is, yeah, the gc threads are mostly responsible for the current spike in cpu | 23:39 |
| *** pt_15 has joined #openstack-infra | 23:43 | |
| *** aeng has quit IRC | 23:44 | |
| *** MasterOfBugs has quit IRC | 23:45 | |
| *** rlandy is now known as rlandy|bbl | 23:49 | |
| fungi | jeblair: pabelanger: mnaser: mrhillsman: here's the cutover announcement draft: https://etherpad.openstack.org/p/zuulv3-rollout-2-announcement | 23:49 |
| fungi | Shrews: ^ | 23:50 |
| fungi | should i mention anything about retention of the infra-check pipeline in there, or is this sufficiently detailed? | 23:50 |
| *** tpsilva has quit IRC | 23:51 | |
| dansmith | fungi: you want typo correction in there? | 23:51 |
| fungi | dansmith: by all means! | 23:51 |
| dansmith | fungi: smack me if I'm misreading, but here goes | 23:51 |
| fungi | i welcome all editors. my typos are the stuff of legend | 23:51 |
| jeblair | fungi: i think that's good, and infra-check mention isn't necessary. | 23:51 |
| fungi | dansmith: thanks, that's much better ;) | 23:52 |
| dansmith | heh | 23:52 |
| fungi | jeblair: we can probably mention in the followup announcement after cutover that we have a priority queue for fixes to legacy job configuration | 23:52 |
| * fungi makes a mental note | 23:53 | |
| *** stakeda has joined #openstack-infra | 23:55 | |
| *** xarses has joined #openstack-infra | 23:59 | |
| *** lukebrowning has joined #openstack-infra | 23:59 | |
| *** edmondsw has joined #openstack-infra | 23:59 | |
| fungi | announcement sent | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
