| *** rtjure has quit IRC | 00:00 | |
| *** rtjure has joined #openstack-infra | 00:03 | |
| *** salv-orlando has quit IRC | 00:03 | |
| *** gildub has joined #openstack-infra | 00:06 | |
| *** tosky has quit IRC | 00:08 | |
| *** ijw has joined #openstack-infra | 00:08 | |
| *** rloo has quit IRC | 00:09 | |
| *** claudiub|2 has quit IRC | 00:10 | |
| *** hongbin has quit IRC | 00:10 | |
| *** rfolco has quit IRC | 00:10 | |
| *** yamamoto has joined #openstack-infra | 00:14 | |
| *** eharney has quit IRC | 00:14 | |
| jeblair | dims: can you +3 https://review.openstack.org/516084 please? | 00:15 |
|---|---|---|
| *** yamamoto has quit IRC | 00:20 | |
| *** ijw has quit IRC | 00:20 | |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-zuul master: DNM: test https://review.openstack.org/522050 | 00:22 |
| *** mlavalle has quit IRC | 00:22 | |
| jungleboyj | dhinesh: Did the results look like this? http://dal05.objectstorage.softlayer.net/v1/AUTH_0bd27569-6310-44fc-8a8d-36a112cac4ec/IBM-STORAGE-CI/xiv-tempest-dsvm-full-iscsi/8742/console.txt | 00:22 |
| jungleboyj | dhinesh: I couldn't get to the host you sent earlier. Behind a high school's firewall, they might be blocking things. | 00:23 |
| dhinesh | jungleboyj: yes, it is running the same 237 tests i run | 00:24 |
| jungleboyj | dhinesh: Ok, then that looks right. The number of tests has increased since I last looked. | 00:24 |
| *** sree has joined #openstack-infra | 00:26 | |
| dhinesh | here http://dal05.objectstorage.softlayer.net/v1/AUTH_0bd27569-6310-44fc-8a8d-36a112cac4ec/IBM-STORAGE-CI/xiv-tempest-dsvm-full-iscsi/8742/ there is all kinds of logs here and my CI does not | 00:29 |
| *** slaweq has joined #openstack-infra | 00:30 | |
| *** sree has quit IRC | 00:30 | |
| *** sree has joined #openstack-infra | 00:33 | |
| *** ijw has joined #openstack-infra | 00:34 | |
| *** slaweq has quit IRC | 00:35 | |
| *** sree has quit IRC | 00:38 | |
| *** ijw has quit IRC | 00:40 | |
| *** ijw has joined #openstack-infra | 00:42 | |
| *** ijw has quit IRC | 00:49 | |
| *** jamesmcarthur has joined #openstack-infra | 00:49 | |
| *** Apoorva_ has quit IRC | 00:50 | |
| *** Apoorva has joined #openstack-infra | 00:51 | |
| *** jamesmcarthur has quit IRC | 00:53 | |
| *** Swami has quit IRC | 00:56 | |
| *** daidv has joined #openstack-infra | 01:15 | |
| *** yamamoto has joined #openstack-infra | 01:16 | |
| *** Daisy_ has joined #openstack-infra | 01:17 | |
| *** Daisy__ has joined #openstack-infra | 01:19 | |
| *** tnarg has quit IRC | 01:19 | |
| *** yamamoto has quit IRC | 01:21 | |
| *** Daisy_ has quit IRC | 01:23 | |
| *** caphrim007 has quit IRC | 01:25 | |
| *** rcernin has quit IRC | 01:29 | |
| *** rcernin has joined #openstack-infra | 01:30 | |
| *** larainema has joined #openstack-infra | 01:41 | |
| *** links has joined #openstack-infra | 01:44 | |
| *** Goneri has quit IRC | 01:46 | |
| *** Apoorva_ has joined #openstack-infra | 01:50 | |
| *** Apoorva_ has quit IRC | 01:50 | |
| *** kiennt26 has joined #openstack-infra | 01:52 | |
| *** Apoorva has quit IRC | 01:53 | |
| *** jascott1 has quit IRC | 01:56 | |
| *** jascott1 has joined #openstack-infra | 01:56 | |
| *** kukacz has quit IRC | 02:00 | |
| *** salv-orlando has joined #openstack-infra | 02:01 | |
| *** jascott1 has quit IRC | 02:01 | |
| *** kukacz_ has joined #openstack-infra | 02:01 | |
| *** ramishra has joined #openstack-infra | 02:01 | |
| *** sree has joined #openstack-infra | 02:01 | |
| *** cuongnv has joined #openstack-infra | 02:02 | |
| *** rosmaita has quit IRC | 02:04 | |
| *** salv-orlando has quit IRC | 02:05 | |
| *** sree has quit IRC | 02:07 | |
| *** liujiong has joined #openstack-infra | 02:09 | |
| *** zhurong has joined #openstack-infra | 02:16 | |
| *** bobh has joined #openstack-infra | 02:16 | |
| *** yamamoto has joined #openstack-infra | 02:17 | |
| *** yamamoto has quit IRC | 02:22 | |
| dims | jeblair : done! | 02:22 |
| *** andreas_s has joined #openstack-infra | 02:27 | |
| *** annp has joined #openstack-infra | 02:27 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: cistatus: update web page with most important jobs https://review.openstack.org/522068 | 02:29 |
| *** andreas_s has quit IRC | 02:32 | |
| *** gildub has quit IRC | 02:33 | |
| *** caphrim007 has joined #openstack-infra | 02:34 | |
| *** stakeda has joined #openstack-infra | 02:38 | |
| *** caphrim007 has quit IRC | 02:38 | |
| kiennt26 | hi all, i've faced an error "The following LIBS_FROM_GIT were not installed correct" in Heat & Zun gates. | 02:45 |
| kiennt26 | #link: http://logs.openstack.org/63/520263/3/check/heat-functional-orig-mysql-lbaasv2/ce3712d/logs/devstacklog.txt.gz#_2017-11-22_02_12_36_835 | 02:45 |
| EmilienM | I think there is a general gate failure | 02:45 |
| EmilienM | which looks pretty bad | 02:46 |
| EmilienM | http://logs.openstack.org/68/522068/1/check/tripleo-ci-centos-7-containers-multinode/feb97c4/job-output.txt.gz#_2017-11-22_02_40_02_671960 | 02:46 |
| EmilienM | 2017-11-22 02:40:02.671960 | primary | /bin/bash: /usr/zuul-env/bin/zuul-cloner: /usr/zuul-env/bin/python: bad interpreter: No such file or directory | 02:46 |
| EmilienM | jeblair: fyi ^ | 02:46 |
| EmilienM | unless I missed something | 02:47 |
| EmilienM | infra-root: ^ any idea? | 02:49 |
| ramishra | kiennt26: I think https://review.openstack.org/522077 would probably fix it | 02:50 |
| ramishra | ianw: ^^ | 02:50 |
| kiennt26 | ramishra: Yeah, thank you! | 02:51 |
| *** Nil_ has quit IRC | 02:56 | |
| *** liujiong has quit IRC | 02:56 | |
| *** liujiong has joined #openstack-infra | 02:57 | |
| *** Nil_ has joined #openstack-infra | 02:59 | |
| Jeffrey4l | what the difference between periodic pipeline and periodic-stable pipeline? | 03:01 |
| *** salv-orlando has joined #openstack-infra | 03:02 | |
| *** mat128 has quit IRC | 03:03 | |
| EmilienM | Jeffrey4l: the timer and the notifications on failures | 03:03 |
| EmilienM | https://github.com/openstack-infra/project-config/blob/2deb614a73c084e7267321a95d4c198eaf169adc/zuul.d/pipelines.yaml#L209-L224 | 03:04 |
| EmilienM | https://github.com/openstack-infra/project-config/blob/2deb614a73c084e7267321a95d4c198eaf169adc/zuul.d/pipelines.yaml#L156-L167 | 03:04 |
| clarkb | mordred: http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/fetch-zuul-cloner/templates/zuul-cloner-shim.py.j2#n1 it does use the venv python | 03:05 |
| clarkb | pabelanger: ^ fyi | 03:06 |
| *** salv-orlando has quit IRC | 03:06 | |
| clarkb | EmilienM: the issue is we removed the venv but not the cloner shim and the cloner shim uses the venv python | 03:06 |
| EmilienM | clarkb: is it what it's broken? | 03:06 |
| EmilienM | ok | 03:06 |
| clarkb | EmilienM: yes | 03:06 |
| EmilienM | clarkb: anything I can do? | 03:06 |
| clarkb | EmilienM: if you want to push a change that changes that line I linked to just be path to python (/usr/bin/python) I can review and approve thaf | 03:07 |
| clarkb | EmilienM: or we can revert the venv removal | 03:07 |
| clarkb | but I think just using system python should work | 03:08 |
| *** yamamoto has joined #openstack-infra | 03:09 | |
| EmilienM | clarkb: ok let me push a patch | 03:10 |
| *** ricolin has joined #openstack-infra | 03:10 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/zuul-jobs master: zuul-cloner-shim: load python from the system https://review.openstack.org/522093 | 03:12 |
| EmilienM | clarkb: ^ something like that? | 03:12 |
| *** dhinesh has quit IRC | 03:14 | |
| clarkb | yup | 03:14 |
| clarkb | I am hoping that will self test (not sure but will check logs and approve if it looks good) | 03:14 |
| clarkb | EmilienM: you can probably depends on it frkm where you had the failure to double check too | 03:15 |
| EmilienM | clarkb: I have so much failures | 03:15 |
| EmilienM | I'll just wait it lands | 03:15 |
| EmilienM | I'll recheck after dinner, ttyl | 03:15 |
| clarkb | bah it needs yaml so it needs the venv | 03:17 |
| *** iyamahat_ has quit IRC | 03:17 | |
| Jeffrey4l | EmilienM, roger, thanks a lot. | 03:18 |
| EmilienM | Jeffrey4l: np | 03:18 |
| clarkb | it was self testing and failed. I will work on a revert intsead | 03:18 |
| EmilienM | ok | 03:18 |
| EmilienM | feel free to abandon my patch | 03:18 |
| EmilienM | or take over | 03:18 |
| *** yamahata has quit IRC | 03:18 | |
| EmilienM | I go eat now | 03:18 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: Revert "Remove zuul-env from DIB images" https://review.openstack.org/522096 | 03:20 |
| clarkb | ok I've deleted the new centos and debian images, they appear to be the only two new enough to be having problems with this | 03:23 |
| clarkb | infra-root I'm going to self approve the above revert shortly | 03:24 |
| tbarron | so manila jobs are "all failing" and I suspect other projects as well | 03:25 |
| clarkb | tbarron: is it the above issue? | 03:25 |
| clarkb | I think its only affecting centos and debian currently | 03:25 |
| zhongjun | clarkb: http://logs.openstack.org/94/520994/3/check/manila-tempest-minimal-dsvm-lvm-centos-7/1bd9c62/job-output.txt.gz#_2017-11-22_02_18_27_619462 | 03:25 |
| tbarron | https://review.openstack.org/#/c/520994/ | 03:25 |
| tbarron | most of those jobs are ubuntu | 03:25 |
| tbarron | though some centos | 03:25 |
| tbarron | http://logs.openstack.org/94/520994/3/check/manila-tempest-minimal-dsvm-lvm-centos-7/1bd9c62/job-output.txt.gz#_2017-11-22_02_18_27_619462 | 03:26 |
| clarkb | tbarron: http://logs.openstack.org/94/520994/3/check/manila-tempest-dsvm-scenario/d65f26d/logs/devstacklog.txt.gz#_2017-11-22_02_38_34_087 that is the error on ubuntu | 03:26 |
| tbarron | we have legacy jobs in-tree defined in manila | 03:26 |
| tbarron | clarkb: so maybe we have two issues? | 03:27 |
| tbarron | we do run LIBS_FROM_GIT with python-manilaclient in our jobs | 03:27 |
| clarkb | tbarron: ya possible | 03:27 |
| clarkb | is python-manilaclient in the PROJECTS list? | 03:28 |
| tbarron | clarkb: should be, don't think we changed anything recently. | 03:28 |
| clarkb | http://logs.openstack.org/94/520994/3/check/manila-tempest-dsvm-scenario/d65f26d/logs/devstack-gate-setup-workspace-new.txt.gz#_2017-11-22_02_14_41_063 ya seems to be | 03:29 |
| *** erlon has quit IRC | 03:30 | |
| clarkb | the breaking change for zuul shim merged almost 8 hours ago. It only applies to new images. THe last xenial image was built 10 hours ago so I think it must be a separate issue | 03:30 |
| openstackgerrit | Ghanshyam Mann proposed openstack-infra/project-config master: Remove legacy novaclient jobs https://review.openstack.org/522101 | 03:33 |
| *** hongbin has joined #openstack-infra | 03:34 | |
| clarkb | ya just logged into a test xenial node and it still has the venv | 03:34 |
| clarkb | so fairly confident it is two issues | 03:34 |
| tbarron | clarkb: so reading the backlog a bit I see kiennt26 reporting LIBS_FROM_GIT not installed correctly issues for heat and zun, and that https://review.openstack.org/#/c/522077 will probably fix that one | 03:34 |
| tbarron | clarkb: and the venv issue you have a fix up for (for centos, etc.) | 03:35 |
| zhongjun | clarkb: The zuul is fine in latest patch https://review.openstack.org/#/c/493071/ | 03:35 |
| clarkb | tbarron: ya venv issue should be on its way to getting corrected | 03:35 |
| clarkb | I'll review the devstack fix | 03:35 |
| tbarron | clarkb: thanks, looks like both issues are being addressed | 03:35 |
| *** dhinesh has joined #openstack-infra | 03:35 | |
| *** caphrim007 has joined #openstack-infra | 03:36 | |
| openstackgerrit | Ghanshyam Mann proposed openstack-infra/openstack-zuul-jobs master: Remove legacy novaclient jobs https://review.openstack.org/522102 | 03:37 |
| tbarron | zhongjun: it's too bad you hit these at the beginning of the Wuhan bug smash, but good luck and it looks like the jobs will run better pretty soon now. | 03:37 |
| *** rlandy|rover|bbl is now known as rlandy|rover | 03:38 | |
| zhongjun | tbarron: Thanks for you quickly find the bug | 03:38 |
| *** ramishra has quit IRC | 03:39 | |
| openstackgerrit | Ghanshyam Mann proposed openstack-infra/project-config master: Remove legacy novaclient jobs https://review.openstack.org/522101 | 03:41 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool feature/zuulv3: Reorg non detailed instance listing columns https://review.openstack.org/522103 | 03:41 |
| *** rlandy|rover has quit IRC | 03:42 | |
| *** jkilpatr has quit IRC | 03:43 | |
| *** caphrim007_ has joined #openstack-infra | 03:45 | |
| *** Daisy__ has quit IRC | 03:46 | |
| openstackgerrit | Merged openstack-infra/project-config master: Revert "Remove zuul-env from DIB images" https://review.openstack.org/522096 | 03:46 |
| *** jascott1 has joined #openstack-infra | 03:47 | |
| *** caphrim007 has quit IRC | 03:49 | |
| *** sree has joined #openstack-infra | 03:53 | |
| *** slaweq has joined #openstack-infra | 03:59 | |
| *** salv-orlando has joined #openstack-infra | 04:02 | |
| *** slaweq has quit IRC | 04:04 | |
| clarkb | bah ok so when you delete the current dib image in nodepool it immedaitely starts trying to rebuild the images | 04:05 |
| clarkb | soif your fixes aren't in place you could end up right back where you started | 04:05 |
| clarkb | I'm waiting for puppet to apply the revert above on the builder(s) then will redelete images so there may be a short period of brokeness again depending on how those image uploads race me | 04:05 |
| clarkb | shrews jeblair pabelanger re ^ maybe we need a command for pause image build in addition to the config update? | 04:06 |
| clarkb | or some way to say "please leave this alone while we fix it" | 04:06 |
| *** salv-orlando has quit IRC | 04:07 | |
| *** jascott1 has quit IRC | 04:08 | |
| *** jascott1 has joined #openstack-infra | 04:09 | |
| *** ykarel|away has joined #openstack-infra | 04:12 | |
| *** jascott1 has quit IRC | 04:13 | |
| *** bobh has quit IRC | 04:15 | |
| *** Daisy_ has joined #openstack-infra | 04:15 | |
| clarkb | #status log had to revert the change to delete zuul-env on dib images as the zuul cloner shim does depend on that python installation and its pyyaml lib. Followed up by deleting the debian and centos images that were built without zuul envs | 04:18 |
| openstackstatus | clarkb: finished logging | 04:18 |
| *** bobh has joined #openstack-infra | 04:19 | |
| *** Daisy_ has quit IRC | 04:20 | |
| *** bobh has quit IRC | 04:20 | |
| *** bobh has joined #openstack-infra | 04:21 | |
| *** Daisy_ has joined #openstack-infra | 04:21 | |
| *** dbecker has quit IRC | 04:24 | |
| *** gongysh has joined #openstack-infra | 04:25 | |
| *** Daisy_ has quit IRC | 04:25 | |
| *** Daisy_ has joined #openstack-infra | 04:27 | |
| *** bobh has quit IRC | 04:30 | |
| *** ykarel|away is now known as ykarel | 04:31 | |
| *** Daisy_ has quit IRC | 04:32 | |
| *** janki has joined #openstack-infra | 04:32 | |
| *** ramishra has joined #openstack-infra | 04:35 | |
| *** dbecker has joined #openstack-infra | 04:37 | |
| *** jascott1 has joined #openstack-infra | 04:37 | |
| *** psachin has joined #openstack-infra | 04:38 | |
| *** sree_ has joined #openstack-infra | 04:39 | |
| *** sree_ is now known as Guest13106 | 04:39 | |
| *** sree has quit IRC | 04:41 | |
| *** sree has joined #openstack-infra | 04:42 | |
| *** Guest13106 has quit IRC | 04:43 | |
| clarkb | ok the second round of image deletes has been issued. I'm going to bed now but I think we should be in the clear at this point | 04:43 |
| *** Daisy_ has joined #openstack-infra | 04:46 | |
| *** zhurong has quit IRC | 04:48 | |
| *** jamesmcarthur has joined #openstack-infra | 04:49 | |
| *** Daisy_ has quit IRC | 04:51 | |
| *** jamesmcarthur has quit IRC | 04:54 | |
| *** kiennt26 has quit IRC | 04:58 | |
| *** dhajare has joined #openstack-infra | 04:59 | |
| *** chrisyang_0660 has joined #openstack-infra | 05:03 | |
| *** salv-orlando has joined #openstack-infra | 05:03 | |
| *** uberjay has quit IRC | 05:04 | |
| *** hongbin has quit IRC | 05:05 | |
| *** Daisy_ has joined #openstack-infra | 05:06 | |
| *** udesale has joined #openstack-infra | 05:06 | |
| *** salv-orlando has quit IRC | 05:07 | |
| *** slaweq has joined #openstack-infra | 05:09 | |
| *** claudiub|2 has joined #openstack-infra | 05:10 | |
| *** uberjay has joined #openstack-infra | 05:10 | |
| *** zhurong has joined #openstack-infra | 05:12 | |
| *** slaweq has quit IRC | 05:13 | |
| *** janki has quit IRC | 05:17 | |
| *** yamamoto_ has joined #openstack-infra | 05:18 | |
| AJaeger | mordred: http://logs.openstack.org/90/520890/3/check/build-openstack-releasenotes/e46e8d8/job-output.txt.gz is a failure where .venv does not exist | 05:21 |
| *** yamamoto has quit IRC | 05:21 | |
| mnaser | AJaeger: affecting openstack puppet modules as well http://logs.openstack.org/68/521868/1/gate/build-openstack-releasenotes/e3d68a9/job-output.txt.gz | 05:25 |
| mnaser | interestingly enough it does install sphinx successfully | 05:26 |
| *** bhavik1 has joined #openstack-infra | 05:26 | |
| mnaser | ooo | 05:27 |
| mnaser | i know whats happening | 05:27 |
| mnaser | /bin/bash: /home/zuul/.venv/bin/sphinx-build: No such file or directory | 05:27 |
| mnaser | New python executable in /home/zuul/src/git.openstack.org/openstack/puppet-ceilometer/.venv/bin/python | 05:28 |
| mnaser | strange there is chdir zuul_work_dir in both... | 05:29 |
| *** pgadiya has joined #openstack-infra | 05:30 | |
| *** pgadiya has quit IRC | 05:30 | |
| mnaser | zuul_work_dir: "{{ zuul.project.src_dir }}" for both roles... yet the first one is running correctly and the second one isnt | 05:33 |
| AJaeger | mnaser: check roles/build-releasenotes/defaults/main.yaml in *zuul-jobs* | 05:35 |
| AJaeger | zuul_work_virtualenv: "{{ ansible_user_dir }}/.venv" | 05:35 |
| AJaeger | That's the problem... | 05:35 |
| mnaser | wait we have 2 roles now? | 05:35 |
| *** janki has joined #openstack-infra | 05:35 | |
| mnaser | thats so vastly confusing but yes | 05:35 |
| AJaeger | mnaser: second one is getting removed... | 05:35 |
| mnaser | i believe that would be our issue | 05:35 |
| mnaser | okay cool, sorry i havent been around past few days to keep up | 05:35 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Base build-openstack-releasenotes on build-reno-releasenotes https://review.openstack.org/521144 | 05:36 |
| mnaser | yup you are right | 05:36 |
| AJaeger | mnaser: ^ removal of the extra build-releasenotes | 05:36 |
| mnaser | aha | 05:36 |
| AJaeger | mnaser: so, two options: Changing zuul_work_virtualenv or changing its usage? | 05:36 |
| AJaeger | roles/sphinx/defaults/main.yaml also has ansible_user_dir | 05:37 |
| AJaeger | ok, changing to $HOME which is ansible_user_dir. | 05:37 |
| mnaser | AJaeger: i'd probably be in favor of keeping zuul_work_virtualenv as is but updating ensure-reno | 05:37 |
| mnaser | that way we keep the virtualenv outside the source | 05:38 |
| AJaeger | yes | 05:38 |
| mnaser | just in case that might pollute stuff in any way | 05:38 |
| AJaeger | mnaser: https://review.openstack.org/#/c/522030/1/roles/ensure-reno/tasks/main.yaml | 05:38 |
| AJaeger | Let's +2A | 05:38 |
| mnaser | AJaeger: oh nice, approved | 05:39 |
| mnaser | that should fix things up | 05:39 |
| mnaser | probably long term it would be best to have zuul_work_virtualenv used across roles but thats a whole another can of worms :> | 05:40 |
| AJaeger | mnaser: and https://review.openstack.org/#/c/522028 as well | 05:40 |
| AJaeger | mordred: ^ | 05:40 |
| mnaser | AJaeger: just to understand that patch, we don't care about the constraints because we're just trying to get the python package installed to build docs (and if oslo.log version is capped at X then we cant install Y when Y>X to build the docs, right?) | 05:43 |
| AJaeger | mnaser: yes | 05:43 |
| AJaeger | we can't install oslo.log source since it's capped | 05:43 |
| mnaser | ok cool! +A as well, when that merges i can do a recheck for puppet patches for release notes | 05:43 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Follow changed .venv location https://review.openstack.org/522121 | 05:46 |
| AJaeger | mnaser: thanks. could you also +2A ^ | 05:46 |
| AJaeger | that's another fallout ;( | 05:46 |
| johnsom | I'm having a problem with this patch: https://review.openstack.org/#/c/510204 It started causing this in our gates: http://logs.openstack.org/09/519509/7/check/octavia-v1-dsvm-scenario/938d8d9/logs/devstacklog.txt.gz#_2017-11-22_02_27_32_877 because we install these libs in our devstack plugin here: https://github.com/openstack/octavia/blob/master/devstack/plugin.sh#L23 | 05:47 |
| mnaser | AJaeger: i see the rest of the file using $VENV | 05:47 |
| mnaser | could we not use that instead? | 05:47 |
| johnsom | Wondering if anyone has a thought about the right way to fix this. | 05:47 |
| AJaeger | mnaser: two different VENVs ;) We merge them in a later change... | 05:47 |
| mnaser | oh | 05:48 |
| mnaser | this venv is the docs venv | 05:48 |
| mnaser | ooh my lol | 05:48 |
| mnaser | +2 | 05:48 |
| mnaser | looking johnsom :) | 05:48 |
| AJaeger | thanks, will self-approve then | 05:48 |
| mnaser | johnsom: https://review.openstack.org/#/c/522077/ | 05:49 |
| AJaeger | hope we're green now - will be back later. | 05:49 |
| *** dimak has quit IRC | 05:50 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add missing required-projects to blazar-nova https://review.openstack.org/521782 | 05:50 |
| johnsom | mnaser Ha, ok | 05:50 |
| mnaser | johnsom: so that should be fixed whenever that gets merged in, which i can imagine will be soon | 05:50 |
| mnaser | that means my patch will be waiting D: aha | 05:51 |
| johnsom | Yep, I was looking through the merged to find the issue, but didn't look at open patches. | 05:51 |
| *** lihi has quit IRC | 05:51 | |
| johnsom | Yep | 05:51 |
| *** oanson_ has joined #openstack-infra | 05:52 | |
| *** oanson has quit IRC | 05:52 | |
| mnaser | johnsom: LIBS_FROM_GIT is used a fair amount so im sure it'll be merged soon, though i wonder if i can do a Depends-On: to get zuul to be green at least | 05:52 |
| *** oanson_ is now known as oanson | 05:52 | |
| johnsom | Yes, you should be able to | 05:52 |
| johnsom | It would be a good check too | 05:52 |
| mnaser | johnsom: ill do that and it'll give us confirmation that this fixes our issue as well | 05:52 |
| *** lihi has joined #openstack-infra | 05:53 | |
| *** iyamahat has joined #openstack-infra | 05:53 | |
| *** markvoelker has quit IRC | 05:53 | |
| openstackgerrit | Merged openstack-infra/project-config master: gerritbot: add puppet-pacemaker stable/0.6.x https://review.openstack.org/521924 | 05:53 |
| openstackgerrit | Merged openstack-infra/project-config master: gerritbot/mistral: add stable branches https://review.openstack.org/521986 | 05:53 |
| *** slaweq has joined #openstack-infra | 05:54 | |
| *** dimak has joined #openstack-infra | 05:54 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: remove constraints arg from pip installation of self in releasenotes jobs https://review.openstack.org/522028 | 05:58 |
| *** slaweq has quit IRC | 05:58 | |
| *** dhajare has quit IRC | 05:59 | |
| *** udesale has quit IRC | 06:00 | |
| *** udesale has joined #openstack-infra | 06:00 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: have ensure-reno create .venv in $HOME https://review.openstack.org/522030 | 06:01 |
| *** aeng has quit IRC | 06:03 | |
| *** salv-orlando has joined #openstack-infra | 06:04 | |
| *** Daisy_ has quit IRC | 06:05 | |
| *** salv-orlando has quit IRC | 06:05 | |
| *** salv-orlando has joined #openstack-infra | 06:05 | |
| *** Daisy_ has joined #openstack-infra | 06:05 | |
| *** Daisy_ has quit IRC | 06:06 | |
| openstackgerrit | Merged openstack-infra/project-config master: Follow changed .venv location https://review.openstack.org/522121 | 06:08 |
| *** kjackal has quit IRC | 06:09 | |
| *** yamahata has joined #openstack-infra | 06:10 | |
| *** gouthamr has quit IRC | 06:10 | |
| *** ramishra has quit IRC | 06:11 | |
| *** gongysh has quit IRC | 06:12 | |
| *** Daisy_ has joined #openstack-infra | 06:13 | |
| AJaeger | tonyb: could you review https://review.openstack.org/#/q/I851a06fbc07a84f70ee76e78062777ad2bade764 , please? We need these backports for docs job changes... | 06:16 |
| *** dingyichen has joined #openstack-infra | 06:17 | |
| *** ramishra has joined #openstack-infra | 06:18 | |
| tonyb | AJaeger: done. | 06:19 |
| AJaeger | thanks, tonyb | 06:20 |
| AJaeger | looks like releasenotes are working again... | 06:23 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Fix ansible comment https://review.openstack.org/522130 | 06:28 |
| AJaeger | mnaser: ' are evil, see http://logs.openstack.org/periodic/git.openstack.org/openstack/manila-ui/stable/ocata/propose-translation-update/49c44c7/ara/ | 06:28 |
| * AJaeger self-approves 522130 | 06:28 | |
| *** rtjure has quit IRC | 06:29 | |
| *** kjackal has joined #openstack-infra | 06:30 | |
| *** Daisy_ has quit IRC | 06:30 | |
| mnaser | AJaeger: ouch when it rains it pours :p | 06:31 |
| *** rtjure has joined #openstack-infra | 06:32 | |
| *** openstackgerrit has quit IRC | 06:33 | |
| *** dhajare has joined #openstack-infra | 06:34 | |
| *** ykarel_ has joined #openstack-infra | 06:36 | |
| *** openstackgerrit has joined #openstack-infra | 06:37 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Fix ansible comment https://review.openstack.org/522130 | 06:37 |
| *** Daisy_ has joined #openstack-infra | 06:37 | |
| *** ykarel has quit IRC | 06:38 | |
| *** ramishra has quit IRC | 06:38 | |
| *** gongysh has joined #openstack-infra | 06:41 | |
| *** liujiong has quit IRC | 06:41 | |
| *** liujiong has joined #openstack-infra | 06:42 | |
| *** bhavik1 has quit IRC | 06:45 | |
| openstackgerrit | zhangfei gao proposed openstack/diskimage-builder master: Add grub2 ci tests for debian https://review.openstack.org/522134 | 06:48 |
| *** markvoelker has joined #openstack-infra | 06:54 | |
| *** namnh has joined #openstack-infra | 06:54 | |
| *** xinliang has quit IRC | 06:58 | |
| *** ramishra has joined #openstack-infra | 06:59 | |
| pabelanger | clarkb: EmilienM: ah sorry, I ending up crashing for a few hours. looking | 07:00 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/project-config master: Add log-classify project https://review.openstack.org/522139 | 07:00 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/project-config master: Add log-classify to zuul https://review.openstack.org/522141 | 07:01 |
| *** gongysh has quit IRC | 07:02 | |
| *** iyamahat has quit IRC | 07:04 | |
| *** dsariel has joined #openstack-infra | 07:07 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Setup virtualenv for zuul-cloner https://review.openstack.org/522145 | 07:13 |
| pabelanger | clarkb: EmilienM: ^ should be our fix | 07:13 |
| *** xinliang has joined #openstack-infra | 07:14 | |
| pabelanger | I'll propose a few patch to make that a variable | 07:14 |
| pabelanger | actually | 07:15 |
| *** andreas_s has joined #openstack-infra | 07:16 | |
| zhongjun | Anyone could help for take a look at https://review.openstack.org/#/c/522077 thanks in advance | 07:16 |
| *** rcernin has quit IRC | 07:17 | |
| frickler | zhongjun: that is a devstack fix, I already pinged folks over in #openstack-qa | 07:17 |
| zhongjun | frickler: Oh, it is belong to #openstack-qa, I see now. thanks | 07:18 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Setup virtualenv for zuul-cloner https://review.openstack.org/522145 | 07:20 |
| pabelanger | clarkb: EmilienM: ^ now the fix | 07:20 |
| *** d0ugal has quit IRC | 07:21 | |
| *** coolsvap has joined #openstack-infra | 07:22 | |
| *** ykarel__ has joined #openstack-infra | 07:23 | |
| *** isaacb has joined #openstack-infra | 07:23 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Use username from node information if available https://review.openstack.org/453983 | 07:23 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Rename ssh_port to connection_port https://review.openstack.org/500799 | 07:23 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Use connection type supplied from nodepool https://review.openstack.org/501976 | 07:23 |
| *** ykarel_ has quit IRC | 07:25 | |
| *** yamamoto_ has quit IRC | 07:27 | |
| *** hashar has joined #openstack-infra | 07:31 | |
| *** Wei_Liu has quit IRC | 07:37 | |
| *** gongysh has joined #openstack-infra | 07:39 | |
| pabelanger | okay, tests seem to pass | 07:40 |
| pabelanger | so will be a no-op if we lan | 07:40 |
| pabelanger | AJaeger: 522145 might be of interest too | 07:40 |
| *** iyamahat has joined #openstack-infra | 07:41 | |
| *** d0ugal has joined #openstack-infra | 07:41 | |
| *** yamahata has quit IRC | 07:42 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/irc-meetings master: Update Monasca team meeting time and channel https://review.openstack.org/522156 | 07:43 |
| *** armax has quit IRC | 07:44 | |
| *** armax has joined #openstack-infra | 07:45 | |
| *** Wei_Liu has joined #openstack-infra | 07:45 | |
| *** armax has quit IRC | 07:45 | |
| *** armax has joined #openstack-infra | 07:46 | |
| *** ykarel__ is now known as ykarel|lunch | 07:46 | |
| *** armax has quit IRC | 07:46 | |
| *** dsariel has quit IRC | 07:46 | |
| *** armax has joined #openstack-infra | 07:46 | |
| *** armax has quit IRC | 07:47 | |
| *** armax has joined #openstack-infra | 07:47 | |
| *** threestrands has quit IRC | 07:47 | |
| *** armax has quit IRC | 07:47 | |
| *** armax has joined #openstack-infra | 07:48 | |
| *** armax has quit IRC | 07:48 | |
| *** pcaruana has joined #openstack-infra | 07:49 | |
| *** armax has joined #openstack-infra | 07:49 | |
| *** armax has quit IRC | 07:49 | |
| *** jamesmcarthur has joined #openstack-infra | 07:49 | |
| *** armax has joined #openstack-infra | 07:50 | |
| *** armax has quit IRC | 07:50 | |
| *** iyamahat has quit IRC | 07:51 | |
| *** tojuvone has joined #openstack-infra | 07:52 | |
| *** jamesmcarthur has quit IRC | 07:53 | |
| AJaeger | pabelanger: +2 | 07:53 |
| AJaeger | yeah, translation jobs work again after 522130 merged | 07:56 |
| *** shardy has joined #openstack-infra | 07:59 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/jobs route https://review.openstack.org/503270 | 08:08 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/builds route https://review.openstack.org/466561 | 08:08 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add Cache-Control to static files https://review.openstack.org/522163 | 08:08 |
| *** ccamacho has joined #openstack-infra | 08:10 | |
| *** florianf has joined #openstack-infra | 08:11 | |
| *** Hal has joined #openstack-infra | 08:13 | |
| *** apetrich has quit IRC | 08:13 | |
| *** tesseract has joined #openstack-infra | 08:14 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/project-config master: Add log-classify project https://review.openstack.org/522139 | 08:15 |
| *** slaweq has joined #openstack-infra | 08:16 | |
| *** slaweq has quit IRC | 08:17 | |
| *** slaweq has joined #openstack-infra | 08:17 | |
| *** alexchadin has joined #openstack-infra | 08:18 | |
| *** sree_ has joined #openstack-infra | 08:22 | |
| *** sree_ is now known as Guest39600 | 08:23 | |
| *** salv-orlando has quit IRC | 08:25 | |
| *** salv-orlando has joined #openstack-infra | 08:26 | |
| *** sree has quit IRC | 08:26 | |
| *** apetrich has joined #openstack-infra | 08:26 | |
| *** sree has joined #openstack-infra | 08:27 | |
| *** yamamoto has joined #openstack-infra | 08:27 | |
| *** salv-orlando has quit IRC | 08:30 | |
| *** Guest39600 has quit IRC | 08:30 | |
| openstackgerrit | Rui Chen proposed openstack-infra/nodepool feature/zuulv3: Apply floating ip for node according to configuration https://review.openstack.org/518875 | 08:34 |
| *** ykarel|lunch is now known as ykarel | 08:35 | |
| *** yamamoto has quit IRC | 08:35 | |
| *** amoralej|off is now known as amoralej | 08:36 | |
| *** salv-orlando has joined #openstack-infra | 08:42 | |
| *** jpena|off is now known as jpena | 08:46 | |
| *** wendar has quit IRC | 08:47 | |
| *** wendar has joined #openstack-infra | 08:52 | |
| *** slaweq has quit IRC | 08:55 | |
| *** udesale has quit IRC | 08:57 | |
| *** dbecker has quit IRC | 09:03 | |
| *** rossella_s has joined #openstack-infra | 09:03 | |
| *** dbecker has joined #openstack-infra | 09:03 | |
| openstackgerrit | Julien Danjou proposed openstack-infra/project-config master: Retire ceilometerclient https://review.openstack.org/522181 | 09:04 |
| *** jpich has joined #openstack-infra | 09:04 | |
| *** jascott1 has quit IRC | 09:05 | |
| *** jascott1 has joined #openstack-infra | 09:05 | |
| openstackgerrit | Julien Danjou proposed openstack-infra/openstack-zuul-jobs master: Retire ceilometerclient https://review.openstack.org/522182 | 09:07 |
| *** rossella_s has quit IRC | 09:08 | |
| *** zhurong has quit IRC | 09:09 | |
| *** jascott1 has quit IRC | 09:09 | |
| *** rossella_s has joined #openstack-infra | 09:10 | |
| *** yamamoto has joined #openstack-infra | 09:11 | |
| *** andreas_s has quit IRC | 09:13 | |
| *** sree has quit IRC | 09:13 | |
| openstackgerrit | Julien Danjou proposed openstack-infra/project-config master: Retire ceilometerclient https://review.openstack.org/522181 | 09:13 |
| openstackgerrit | Julien Danjou proposed openstack-infra/project-config master: Remove ceilometerclient from infra https://review.openstack.org/522185 | 09:13 |
| *** jtomasek has joined #openstack-infra | 09:14 | |
| *** sree has joined #openstack-infra | 09:14 | |
| *** sshnaidm|afk is now known as sshnaidm | 09:17 | |
| *** slaweq has joined #openstack-infra | 09:17 | |
| *** sree has quit IRC | 09:18 | |
| *** yamamoto has quit IRC | 09:19 | |
| *** dsariel has joined #openstack-infra | 09:19 | |
| *** slaweq has quit IRC | 09:23 | |
| *** yamamoto has joined #openstack-infra | 09:23 | |
| *** andreas_s has joined #openstack-infra | 09:24 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Update Monasca team meeting time and channel https://review.openstack.org/522156 | 09:26 |
| *** alexchadin has quit IRC | 09:26 | |
| *** alexchadin has joined #openstack-infra | 09:27 | |
| *** sree has joined #openstack-infra | 09:28 | |
| *** jascott1 has joined #openstack-infra | 09:28 | |
| *** sree has quit IRC | 09:29 | |
| *** andreas_s has quit IRC | 09:29 | |
| *** sree has joined #openstack-infra | 09:29 | |
| *** lucas-afk is now known as lucasagomes | 09:32 | |
| *** makowals has quit IRC | 09:32 | |
| *** makowals has joined #openstack-infra | 09:34 | |
| *** derekh has joined #openstack-infra | 09:36 | |
| *** andreas_s has joined #openstack-infra | 09:41 | |
| *** dhinesh has quit IRC | 09:43 | |
| *** isaacb has quit IRC | 09:44 | |
| *** andreas_s has quit IRC | 09:45 | |
| *** rossella_s has quit IRC | 09:47 | |
| *** rossella_s has joined #openstack-infra | 09:48 | |
| *** tosky has joined #openstack-infra | 09:51 | |
| *** Daisy_ has quit IRC | 09:52 | |
| *** Daisy_ has joined #openstack-infra | 09:53 | |
| *** andreas_s has joined #openstack-infra | 09:54 | |
| *** stakeda has quit IRC | 09:55 | |
| *** Daisy_ has quit IRC | 09:57 | |
| *** andreas_s has quit IRC | 09:57 | |
| *** andreas_s has joined #openstack-infra | 09:57 | |
| openstackgerrit | Rajat Sharma proposed openstack-infra/openstack-zuul-jobs master: Remove Blazar legacy jobs https://review.openstack.org/522197 | 10:00 |
| *** ralonsoh has joined #openstack-infra | 10:02 | |
| *** tosky has quit IRC | 10:06 | |
| openstackgerrit | Rajat Sharma proposed openstack-infra/project-config master: Remove legacy jobs from Blazar https://review.openstack.org/522200 | 10:09 |
| *** pilgrimstack has joined #openstack-infra | 10:12 | |
| *** ramishra has quit IRC | 10:13 | |
| *** cuongnv has quit IRC | 10:13 | |
| *** liujiong has quit IRC | 10:16 | |
| *** tosky has joined #openstack-infra | 10:17 | |
| *** alexchadin has quit IRC | 10:19 | |
| *** gongysh has quit IRC | 10:20 | |
| *** gmann is now known as gmann_afk | 10:24 | |
| *** annp has quit IRC | 10:24 | |
| *** electrofelix has joined #openstack-infra | 10:25 | |
| *** pilgrimstack has left #openstack-infra | 10:27 | |
| *** sree has quit IRC | 10:29 | |
| *** e0ne has joined #openstack-infra | 10:29 | |
| *** sree has joined #openstack-infra | 10:29 | |
| *** sree has quit IRC | 10:34 | |
| *** ldnunes has joined #openstack-infra | 10:34 | |
| openstackgerrit | Rajat Sharma proposed openstack-infra/project-config master: Remove legacy jobs from Blazar https://review.openstack.org/522200 | 10:35 |
| *** kjackal has quit IRC | 10:38 | |
| *** kjackal has joined #openstack-infra | 10:39 | |
| *** namnh has quit IRC | 10:42 | |
| *** igormarnat has quit IRC | 10:47 | |
| *** asilenkov has quit IRC | 10:48 | |
| *** rakhmerov has quit IRC | 10:49 | |
| *** dteselkin has quit IRC | 10:49 | |
| *** katyafervent_awa has quit IRC | 10:49 | |
| *** rakhmerov has joined #openstack-infra | 10:53 | |
| *** hashar is now known as hasharAway | 10:54 | |
| *** dteselkin has joined #openstack-infra | 10:55 | |
| *** asilenkov has joined #openstack-infra | 10:56 | |
| *** katyafervent_awa has joined #openstack-infra | 10:56 | |
| *** igormarnat has joined #openstack-infra | 10:57 | |
| *** wolverineav has joined #openstack-infra | 10:57 | |
| *** andreas_s has quit IRC | 10:57 | |
| *** andreas_s has joined #openstack-infra | 10:58 | |
| *** andreas_s has quit IRC | 11:03 | |
| *** makowals has quit IRC | 11:15 | |
| *** jpich has quit IRC | 11:15 | |
| *** andreas_s has joined #openstack-infra | 11:17 | |
| *** sdague has joined #openstack-infra | 11:17 | |
| *** jpich has joined #openstack-infra | 11:18 | |
| *** alexchadin has joined #openstack-infra | 11:18 | |
| *** makowals has joined #openstack-infra | 11:21 | |
| *** rossella_s has quit IRC | 11:27 | |
| *** rossella_s has joined #openstack-infra | 11:30 | |
| *** andreas_s has quit IRC | 11:31 | |
| *** asilenkov has quit IRC | 11:31 | |
| *** igormarnat has quit IRC | 11:31 | |
| *** dteselkin has quit IRC | 11:31 | |
| *** katyafervent_awa has quit IRC | 11:31 | |
| *** rakhmerov has quit IRC | 11:31 | |
| *** adisky_ has quit IRC | 11:32 | |
| *** rfolco has joined #openstack-infra | 11:32 | |
| *** rossella_s has quit IRC | 11:35 | |
| *** igormarnat has joined #openstack-infra | 11:36 | |
| *** andreas_s has joined #openstack-infra | 11:37 | |
| *** rossella_s has joined #openstack-infra | 11:38 | |
| *** rakhmerov has joined #openstack-infra | 11:38 | |
| *** dsariel has quit IRC | 11:40 | |
| *** dteselkin has joined #openstack-infra | 11:40 | |
| *** asilenkov has joined #openstack-infra | 11:41 | |
| *** katyafervent_awa has joined #openstack-infra | 11:44 | |
| *** jascott1 has quit IRC | 11:48 | |
| *** jascott1 has joined #openstack-infra | 11:49 | |
| *** jascott1 has quit IRC | 11:53 | |
| *** ianychoi has quit IRC | 12:00 | |
| *** ianychoi has joined #openstack-infra | 12:00 | |
| *** [HeOS] has quit IRC | 12:01 | |
| *** dsariel has joined #openstack-infra | 12:01 | |
| *** shardy has quit IRC | 12:04 | |
| *** shardy has joined #openstack-infra | 12:06 | |
| *** janki has quit IRC | 12:08 | |
| *** dtantsur|afk is now known as dtantsur | 12:10 | |
| *** Daisy_ has joined #openstack-infra | 12:11 | |
| *** ramishra has joined #openstack-infra | 12:14 | |
| *** sree has joined #openstack-infra | 12:15 | |
| *** shardy has quit IRC | 12:15 | |
| *** jkilpatr has joined #openstack-infra | 12:15 | |
| *** Daisy_ has quit IRC | 12:15 | |
| *** shardy has joined #openstack-infra | 12:16 | |
| lennyb | ianw, mordred, #link https://review.openstack.org/#/c/510204/ breaks our CI. More logs #link http://paste.openstack.org/show/627070/ | 12:18 |
| *** alexchadin has quit IRC | 12:18 | |
| *** lucasagomes is now known as lucas-hungry | 12:18 | |
| *** jpich has quit IRC | 12:19 | |
| *** alexchadin has joined #openstack-infra | 12:19 | |
| *** sree has quit IRC | 12:19 | |
| *** yamamoto has quit IRC | 12:20 | |
| *** kukacz_ is now known as kukacz | 12:20 | |
| AJaeger | lennyb: see https://review.openstack.org/#/c/522077 | 12:20 |
| *** alexchadin has quit IRC | 12:21 | |
| *** shardy has quit IRC | 12:23 | |
| lennyb | AJaeger, thanks. | 12:24 |
| *** yamamoto has joined #openstack-infra | 12:24 | |
| *** shardy has joined #openstack-infra | 12:28 | |
| *** sree has joined #openstack-infra | 12:29 | |
| *** dteselkin has quit IRC | 12:29 | |
| *** katyafervent_awa has quit IRC | 12:30 | |
| *** asilenkov has quit IRC | 12:30 | |
| *** igormarnat has quit IRC | 12:31 | |
| *** rakhmerov has quit IRC | 12:31 | |
| *** sree_ has joined #openstack-infra | 12:32 | |
| *** sree_ is now known as Guest23148 | 12:32 | |
| *** jpich has joined #openstack-infra | 12:33 | |
| *** dteselkin has joined #openstack-infra | 12:35 | |
| *** sree has quit IRC | 12:36 | |
| *** rakhmerov has joined #openstack-infra | 12:36 | |
| *** katyafervent_awa has joined #openstack-infra | 12:38 | |
| *** asilenkov has joined #openstack-infra | 12:40 | |
| *** igormarnat has joined #openstack-infra | 12:41 | |
| *** jaypipes has joined #openstack-infra | 12:42 | |
| *** LindaWang1 has joined #openstack-infra | 12:45 | |
| *** gmann_afk is now known as gmann | 12:51 | |
| *** jpena is now known as jpena|lunch | 12:53 | |
| *** spectr has joined #openstack-infra | 12:53 | |
| *** rossella_s has quit IRC | 12:54 | |
| *** alexchadin has joined #openstack-infra | 12:54 | |
| *** Guest23148 has quit IRC | 12:56 | |
| *** rhallisey has joined #openstack-infra | 12:58 | |
| *** weshay_pto is now known as weshay | 12:58 | |
| *** rossella_s has joined #openstack-infra | 12:58 | |
| *** andreas__ has joined #openstack-infra | 13:03 | |
| *** dprince has joined #openstack-infra | 13:03 | |
| *** andreas__ has quit IRC | 13:04 | |
| *** amoralej is now known as amoralej|brb | 13:04 | |
| *** spectr has quit IRC | 13:04 | |
| *** rossella_s has quit IRC | 13:04 | |
| *** andreas__ has joined #openstack-infra | 13:04 | |
| *** rossella_s has joined #openstack-infra | 13:06 | |
| *** andreas_s has quit IRC | 13:07 | |
| *** dsariel has quit IRC | 13:07 | |
| *** gmann is now known as gmann_afk | 13:11 | |
| *** lucas-hungry is now known as lucasagomes | 13:12 | |
| *** ramishra has quit IRC | 13:16 | |
| *** ramishra has joined #openstack-infra | 13:16 | |
| *** ykarel is now known as ykarel|afk | 13:18 | |
| *** dsariel has joined #openstack-infra | 13:20 | |
| *** yamamoto has quit IRC | 13:21 | |
| *** yamamoto has joined #openstack-infra | 13:22 | |
| mordred | morning AJaeger | 13:22 |
| *** sshnaidm is now known as sshnaidm|afk | 13:23 | |
| mordred | AJaeger: I'm getting going with writing the patches to update the tox_install.sh of the six projects that are different - anything else I should lok at first? | 13:24 |
| openstackgerrit | Daniel Alvarez proposed openstack-infra/devstack-gate master: Add a new hook ``collect_artifacts_hook`` https://review.openstack.org/522241 | 13:26 |
| *** [HeOS] has joined #openstack-infra | 13:26 | |
| *** janki has joined #openstack-infra | 13:26 | |
| *** rlandy has joined #openstack-infra | 13:28 | |
| *** raissa has joined #openstack-infra | 13:30 | |
| *** rossella_s has quit IRC | 13:31 | |
| mordred | lucasagomes: (do you know Daniel Alvarez's irc nick?) any reason to not just use post-run playbooks? we'd prefer to get folks migrated to the new devstack base job and retire devstack-gate (I don't want to block people from getting stuff done they need to get done, but there's a better way to do that task in the system now) | 13:33 |
| lucasagomes | mordred, dalvarez I think he's not in -infra | 13:33 |
| lucasagomes | mordred, hmm good point... I think we could try that as well we already migrated the playbooks to our tree | 13:33 |
| *** rossella_s has joined #openstack-infra | 13:34 | |
| lucasagomes | lemme pass the message along (also will tell him to join #openstack-infra | 13:34 |
| *** dalvarez has joined #openstack-infra | 13:35 | |
| *** arxcruz has quit IRC | 13:35 | |
| *** arxcruz_ is now known as arxcruz | 13:35 | |
| mordred | lucasagomes: cool - I'll also leave a note on the review | 13:35 |
| lucasagomes | mordred, thanks a lot! | 13:36 |
| lucasagomes | dalvarez, ^ | 13:36 |
| dalvarez | lucasagomes, mordred i joined late to the party but i think i got it, sry.. didn't know that | 13:36 |
| mordred | dalvarez: no worries! I didn't want to block you from doing good things, mostly just wanted to point out the option :) | 13:37 |
| dalvarez | mordred, appreciated! :) | 13:37 |
| *** ykarel|afk is now known as ykarel | 13:37 | |
| AJaeger | mordred: in meetings right now - please review the three changes we merged over the night if those are fine with you (see backscroll). Everything looks green now. | 13:38 |
| *** dsariel has quit IRC | 13:38 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Apply floating ip for node according to configuration https://review.openstack.org/518875 | 13:38 |
| *** dhajare has quit IRC | 13:39 | |
| *** trown|outtypewww is now known as trown | 13:39 | |
| mordred | AJaeger: woot | 13:40 |
| *** dave-mccowan has joined #openstack-infra | 13:40 | |
| *** mriedem has joined #openstack-infra | 13:41 | |
| *** dteselkin has quit IRC | 13:42 | |
| *** katyafervent_awa has quit IRC | 13:42 | |
| mordred | AJaeger: yah - those look good | 13:43 |
| dalvarez | mordred, im not familiar with this... but it looks like it was my first approach: https://review.openstack.org/#/c/520609/1 | 13:43 |
| *** igormarnat has quit IRC | 13:44 | |
| *** dave-mcc_ has joined #openstack-infra | 13:44 | |
| dalvarez | mordred, for some reason it didn't work and we also like to gzip the files, etc. do you have any example handy of existing similar thing? | 13:44 |
| *** links has quit IRC | 13:44 | |
| *** andreas__ has quit IRC | 13:44 | |
| *** andreas_s has joined #openstack-infra | 13:45 | |
| mordred | dalvarez: there's actually a nice role andreaf wrote to help with this use case ... | 13:45 |
| *** dave-mccowan has quit IRC | 13:45 | |
| dalvarez | mordred, that'd be great to look into :) thanks a lot | 13:45 |
| *** rossella_s has quit IRC | 13:46 | |
| mordred | andreaf: where did that role wind up? | 13:46 |
| *** katyafervent_awa has joined #openstack-infra | 13:47 | |
| *** dteselkin has joined #openstack-infra | 13:48 | |
| *** igormarnat has joined #openstack-infra | 13:48 | |
| *** rossella_s has joined #openstack-infra | 13:48 | |
| *** alexchadin has quit IRC | 13:49 | |
| *** andreas_s has quit IRC | 13:50 | |
| AJaeger | mordred: I was concerned whether $HOME is the right way to set or whether we need something else. There's some implicit information passing between those, we could use a global variable for all these jobs and pass it around ;) | 13:54 |
| *** jpena|lunch is now known as jpena | 13:54 | |
| AJaeger | mordred: in zuul-jobs | 13:54 |
| *** amoralej|brb is now known as amoralej | 13:55 | |
| AJaeger | mordred, dalvarez https://review.openstack.org/509234 | 13:55 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: unittests: make bindep role optional https://review.openstack.org/522255 | 13:56 |
| *** andreas_s has joined #openstack-infra | 13:56 | |
| *** rosmaita has joined #openstack-infra | 13:56 | |
| *** andreas_s has quit IRC | 13:58 | |
| *** andreas_s has joined #openstack-infra | 13:58 | |
| AJaeger | mordred: do you have a list of changes to merge next? | 14:00 |
| *** kgiusti has joined #openstack-infra | 14:01 | |
| mordred | AJaeger: yah - I agree - I think plumbing zuul_work_virtualenv through them all might be a better choice | 14:01 |
| *** andreas_s has quit IRC | 14:01 | |
| mordred | AJaeger: or alternately we could change it all to be {{ zuul_work_dir }}/.venv too ... | 14:01 |
| mordred | AJaeger: but I agree | 14:01 |
| *** andreas_s has joined #openstack-infra | 14:02 | |
| mordred | AJaeger: and no - not yet - I think I still have an issue from the etherpad to fix first | 14:03 |
| dalvarez | AJaeger, thanks a lot, looking now | 14:03 |
| *** sean-k-mooney has joined #openstack-infra | 14:04 | |
| *** andreas__ has joined #openstack-infra | 14:04 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: revoke-sudo: only revoke when zuul is sudoer https://review.openstack.org/522261 | 14:05 |
| *** andrea___ has joined #openstack-infra | 14:06 | |
| *** haleyb is now known as haleyb|away | 14:06 | |
| *** andreas_s has quit IRC | 14:07 | |
| *** Goneri has joined #openstack-infra | 14:08 | |
| *** andreas__ has quit IRC | 14:09 | |
| *** trown is now known as trown|brb | 14:13 | |
| AJaeger | infra-root, I'm approving the bindep-fallback cleanup changes now | 14:15 |
| AJaeger | ianw: do you want to remove the WIP from https://review.openstack.org/#/c/519534 now? | 14:15 |
| *** rossella_s has quit IRC | 14:16 | |
| *** andrea___ has quit IRC | 14:16 | |
| *** andreas_s has joined #openstack-infra | 14:17 | |
| *** dprince has quit IRC | 14:17 | |
| *** rossella_s has joined #openstack-infra | 14:19 | |
| *** trown|brb is now known as trown | 14:23 | |
| *** andreas_s has quit IRC | 14:26 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove some non-standard things from bindep-fallback https://review.openstack.org/519533 | 14:26 |
| *** spectr has joined #openstack-infra | 14:27 | |
| *** dprince has joined #openstack-infra | 14:30 | |
| *** esberglu has joined #openstack-infra | 14:34 | |
| *** dpawlik is now known as danpawlik | 14:37 | |
| *** danpawlik has quit IRC | 14:38 | |
| *** andreas_s has joined #openstack-infra | 14:38 | |
| mnaser | AJaeger: i dont think ceilometer currently has a PTL | 14:39 |
| *** danpawlik has joined #openstack-infra | 14:39 | |
| *** andreas__ has joined #openstack-infra | 14:39 | |
| mnaser | oh | 14:40 |
| mnaser | it's under telemetry | 14:40 |
| * mnaser was looking for ceilometer | 14:41 | |
| fungi | they renamed themselves several years ago | 14:41 |
| mnaser | gotcha | 14:42 |
| AJaeger | mnaser: so, telemetry - gordc | 14:43 |
| mnaser | yup :) | 14:43 |
| *** andreas_s has quit IRC | 14:43 | |
| *** rlandy is now known as rlandy|rover | 14:44 | |
| *** ramishra has quit IRC | 14:46 | |
| *** ramishra has joined #openstack-infra | 14:46 | |
| *** ykarel is now known as ykarel|away | 14:48 | |
| *** _ari_ has quit IRC | 14:49 | |
| *** pbourke has quit IRC | 14:49 | |
| *** pbourke has joined #openstack-infra | 14:50 | |
| *** jascott1 has joined #openstack-infra | 14:51 | |
| *** rcernin has joined #openstack-infra | 14:51 | |
| andreaf | mordred, dalvarez: are you still looking for my role? | 14:54 |
| raissa | hi! the manila team was trying to remove one job from the project definition while keeping the job definition in the .zuul.yaml: https://review.openstack.org/#/c/517647/ - the issue is that the job is still running regardless. are we missing something? | 14:55 |
| andreaf | ah no it looks like you found it | 14:55 |
| andreaf | mordred, dalvarez: sorry ignore me | 14:55 |
| *** jascott1 has quit IRC | 14:56 | |
| *** d0ugal has quit IRC | 14:56 | |
| *** smarcet has joined #openstack-infra | 14:57 | |
| AJaeger | config-core, please review these changes for sphinx/releasenotes publishing: https://review.openstack.org/522017 https://review.openstack.org/521996 https://review.openstack.org/#/c/521590 | 14:58 |
| *** armax has joined #openstack-infra | 14:58 | |
| smcginnis | There was a release job failure last night. Before I dig in to that, was this discussed already by anyone? | 14:59 |
| smcginnis | http://logs.openstack.org/e5/e5831f230bd29516dc202eb406270604f27e27f9/release/release-openstack-javascript/95af4ef/job-output.txt.gz#_2017-11-22_03_01_48_750499 | 14:59 |
| frickler | openstack health lists jobs running into timeout as success: http://status.openstack.org/openstack-health/#/g/build_name/openstack-tox-cover-neutron the last job is show as success, but did timeout: http://logs.openstack.org/70/521070/2/gate/openstack-tox-cover/ad54211/job-output.txt.gz#_2017-11-22_13_37_56_992376 | 14:59 |
| *** d0ugal has joined #openstack-infra | 14:59 | |
| *** d0ugal has quit IRC | 14:59 | |
| *** d0ugal has joined #openstack-infra | 14:59 | |
| dalvarez | andreaf, mordred i ended up doing this: https://review.openstack.org/#/c/520609/ | 14:59 |
| *** snapiri has quit IRC | 15:01 | |
| openstackgerrit | Michael Schmidt proposed openstack-infra/project-config master: initialises kubernikus project skeleton https://review.openstack.org/522272 | 15:02 |
| *** _ari_ has joined #openstack-infra | 15:02 | |
| *** spectr has quit IRC | 15:03 | |
| *** LindaWang1 has quit IRC | 15:03 | |
| *** ramishra has quit IRC | 15:05 | |
| *** bobh has joined #openstack-infra | 15:06 | |
| *** ramishra has joined #openstack-infra | 15:07 | |
| *** mat128 has joined #openstack-infra | 15:07 | |
| *** erlon has joined #openstack-infra | 15:08 | |
| *** ykarel|away has quit IRC | 15:09 | |
| AJaeger | mordred: did you see the comments on https://review.openstack.org/#/c/513927/ ? | 15:09 |
| *** rbrndt has joined #openstack-infra | 15:10 | |
| *** gouthamr has joined #openstack-infra | 15:12 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Make scenario007-container voting https://review.openstack.org/521614 | 15:16 |
| *** efoley has joined #openstack-infra | 15:16 | |
| *** Daisy_ has joined #openstack-infra | 15:17 | |
| *** rcernin has quit IRC | 15:18 | |
| openstackgerrit | Jan Kundrát proposed openstack-infra/zuul master: Prepare correct refspec on new Gerrit https://review.openstack.org/439057 | 15:21 |
| *** Daisy_ has quit IRC | 15:21 | |
| AJaeger | mordred: I tried fixing all your tox_install changes... | 15:23 |
| *** hongbin has joined #openstack-infra | 15:24 | |
| *** dtantsur is now known as dtantsur|brb | 15:25 | |
| *** dhinesh has joined #openstack-infra | 15:28 | |
| openstackgerrit | Michael Schmidt proposed openstack-infra/project-config master: initialises kubernikus project skeleton https://review.openstack.org/522272 | 15:28 |
| openstackgerrit | Michael Schmidt proposed openstack-infra/project-config master: adds kubernikus project https://review.openstack.org/522282 | 15:28 |
| *** iyamahat has joined #openstack-infra | 15:28 | |
| mordred | AJaeger: not yet - sorry, spotty airplane wifi today so far | 15:33 |
| mordred | AJaeger: oh - I did see the comments on that - lemme respond | 15:34 |
| openstackgerrit | Michael Schmidt proposed openstack-infra/project-config master: initialises kubernikus project skeleton https://review.openstack.org/522272 | 15:36 |
| openstackgerrit | Michael Schmidt proposed openstack-infra/project-config master: adds kubernikus project https://review.openstack.org/522282 | 15:36 |
| *** marst has joined #openstack-infra | 15:36 | |
| *** ramishra has quit IRC | 15:39 | |
| *** ykarel|away has joined #openstack-infra | 15:42 | |
| smcginnis | Looks like our upload-npm role is broken. | 15:42 |
| smcginnis | Missing the npm_info variable here: http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/upload-npm/templates/.npmrc.j2#n1 | 15:42 |
| smcginnis | Where should that be coming from? | 15:42 |
| mordred | smcginnis: should be a secret | 15:43 |
| * smcginnis could make a joke about that information being a secret | 15:43 | |
| mordred | smcginnis: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/secrets.yaml#n525 | 15:43 |
| *** bhavik1 has joined #openstack-infra | 15:43 | |
| smcginnis | mordred: So npm_info is part of npm_credentials? | 15:44 |
| mordred | smcginnis: well - it's more that we need a password too - so bundling the other bits winds up being a nice pattern ... :) | 15:44 |
| mordred | yah - something should be passing npm_credentials - it's possile we just have a variable name mismatch | 15:44 |
| mordred | smcginnis: yah - http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/jobs.yaml#n742 | 15:45 |
| smcginnis | mordred: Should it be s/npm_info/npm_credentials/g ? | 15:45 |
| mordred | smcginnis: the job uses npm_credentials.... so yah, let's just do that transform in the role | 15:45 |
| *** vhosakot has joined #openstack-infra | 15:45 | |
| smcginnis | mordred: Sorry, still learning. Are you saying rename it in the template? | 15:45 |
| mordred | smcginnis: yes - in the template, and anywhere else in roles/upload-npm that is using npm_info | 15:46 |
| smcginnis | mordred: Great - on it. Thanks for the help. | 15:46 |
| *** bhavik1 has quit IRC | 15:49 | |
| openstackgerrit | Sean McGinnis proposed openstack-infra/zuul-jobs master: Update upload-npm credentials secret https://review.openstack.org/522284 | 15:49 |
| *** psachin has quit IRC | 15:49 | |
| *** mat128_ has joined #openstack-infra | 15:51 | |
| *** iyamahat has quit IRC | 15:51 | |
| *** mat128 has quit IRC | 15:52 | |
| pabelanger | dmsimard: do you have elastic-recheck query for rdo dns failures? | 15:58 |
| pabelanger | dmsimard: curious if we've seen a decrease already | 15:58 |
| *** dtantsur|brb is now known as dtantsur | 15:58 | |
| dmsimard | pabelanger: I don't have an e-r query but I have an e query :P | 15:59 |
| dmsimard | sec | 15:59 |
| *** salv-orlando has quit IRC | 16:02 | |
| *** salv-orlando has joined #openstack-infra | 16:03 | |
| *** makowals has quit IRC | 16:06 | |
| *** salv-orlando has quit IRC | 16:07 | |
| *** makowals has joined #openstack-infra | 16:12 | |
| *** sshnaidm|afk is now known as sshnaidm | 16:13 | |
| *** janki has quit IRC | 16:13 | |
| dmsimard | pabelanger: logstash results are super weird, I don't know if it's because the logstash queue is super backed up | 16:16 |
| dmsimard | oh it actually empties over the night, heh http://grafana.openstack.org/dashboard/db/zuul-status | 16:17 |
| dmsimard | It was at 150k when I looked yesterday | 16:17 |
| dmsimard | pabelanger: I'm searching for either (message:"Could not resolve host" OR message:"Name or service not known") AND message:"github.com" | 16:18 |
| dmsimard | or (message:"Could not resolve host" OR message:"Name or service not known") AND message:"trunk.rdoproject.org" | 16:18 |
| pabelanger | dmsimard: yah, clarkb fixed our logstash worker issue, so we should be processing well now | 16:18 |
| dmsimard | and I'm not nearly getting the amount of hits I'm expecting | 16:18 |
| pabelanger | dmsimard: that's good right? | 16:18 |
| pabelanger | Cluster Health: red | 16:19 |
| pabelanger | could explain issue you might be seeing | 16:19 |
| *** rossella_s has quit IRC | 16:20 | |
| pabelanger | dmsimard: Hmm, also, that query doesn't make sense, because nothing should be going to trunk.rdoproject.org directly, we have reverse proxy cache setup | 16:20 |
| pabelanger | dmsimard: would be something like: http://mirror.dfw.rax.openstack.org:8080/rdo/centos7/ | 16:21 |
| pabelanger | I'm just seeing failures in openstack/kolla | 16:21 |
| *** slaweq has joined #openstack-infra | 16:21 | |
| pabelanger | they likley aren't using reverse proxy | 16:22 |
| *** eharney has joined #openstack-infra | 16:23 | |
| *** rossella_s has joined #openstack-infra | 16:24 | |
| dmsimard | pabelanger: but I keep hearing about DNS failures for trunk.rdo, no ? maybe it's just me | 16:24 |
| dmsimard | pabelanger: have we been getting DNS resolution failures for mirror hostnames as well ? | 16:25 |
| * dmsimard looks | 16:25 | |
| pabelanger | dmsimard: well, I haven't see a report honstly, but if you have a bug to look at. But yah, nothing should be hitting rdo directly, if so, we should change it to use reverse proxy cache | 16:26 |
| clarkb | there were definitely dns lookup failures for that name | 16:26 |
| pabelanger | dmsimard: but, it is possible our mirror is getting DNS issue | 16:26 |
| clarkb | in jobs | 16:26 |
| *** smarcet has quit IRC | 16:26 | |
| pabelanger | clarkb: happen to have link? I only see kolla right now | 16:26 |
| dmsimard | pabelanger: let's hold that up a bit and check why elasticsearch is red ? | 16:27 |
| clarkb | ya my search results from before say its kolla | 16:28 |
| clarkb | hitting trunk.rdoproject.org | 16:28 |
| pabelanger | clarkb: yah, so that to me is incorrect. We should have them update the jobs to use reverse proxy cache | 16:28 |
| pabelanger | dmsimard: cluster issue? I haven't looked yet | 16:28 |
| clarkb | dmsimard: pabelanger /dev/mapper/main-elasticsearch 1008G 979G 30G 98% /var/lib/elasticsearch | 16:29 |
| pabelanger | need to remember how to do that | 16:29 |
| clarkb | I think we've run out of disk | 16:29 |
| *** felipemonteiro has joined #openstack-infra | 16:29 | |
| pabelanger | okay then | 16:29 |
| clarkb | so we caught up on job backlog and in the process filled up our disks \o/ | 16:29 |
| clarkb | quick "fix" for that is to reduce number of days we retain | 16:29 |
| pabelanger | Yay for next issue | 16:29 |
| *** camunoz has joined #openstack-infra | 16:30 | |
| clarkb | patch in a sec | 16:30 |
| dmsimard | clarkb: how many days do we have right now ? | 16:31 |
| pabelanger | 10 IIRC | 16:31 |
| *** felipemonteiro_ has joined #openstack-infra | 16:31 | |
| clarkb | yup | 16:31 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Reduce elasticsearch retention to 7 days https://review.openstack.org/522292 | 16:31 |
| *** sshnaidm is now known as sshnaidm|off | 16:31 | |
| clarkb | ^^ reduces to 7 | 16:31 |
| clarkb | pabelanger: dmsimard the thing that tipped me off to check disk is that curl -X GET http://localhost:9200/_cluster/health?pretty=true on elasticsearch02.openstack.org showed we have all 6 data nodes present (so we didn't lose cluster nodes) | 16:32 |
| clarkb | if all cluster nodes are present but we are in bad state disk is likely cause | 16:32 |
| clarkb | (memory will cause cluster members to crash and then they won't show up in the health status) | 16:33 |
| *** raissa has quit IRC | 16:33 | |
| dmsimard | 7 days is so small :( | 16:33 |
| dmsimard | but I understand it's expensive | 16:34 |
| *** raissa has joined #openstack-infra | 16:34 | |
| dmsimard | clarkb: so we have 7 nodes at 1TB each ? logs.o.o is 12TB right ? | 16:34 |
| *** felipemonteiro has quit IRC | 16:34 | |
| clarkb | dmsimard: 6 nodes at 1TB each for total of 6TB disk but 5TB usable (because we can lose one cluster member and stay up) | 16:34 |
| clarkb | dmsimard: logs.o.o is 12TB of compressed logs. ES is 5TB useable of logs in a database so they aren't really directly relatable | 16:35 |
| clarkb | but shoving them into the db significantly increases their storage requirements hence then reduction in retention | 16:35 |
| dmsimard | clarkb: yeah, I understand but it gives an idea of the relationship/ratio between the two | 16:35 |
| dmsimard | I'm finding 6TB for 7 days a bit much | 16:36 |
| dmsimard | or, at the very least, we're really not doing a good job of making that 6TB profitable | 16:36 |
| clarkb | I dunno, it has been an incredibly valuable resource | 16:36 |
| dmsimard | We could probably do more with it is what I mean | 16:38 |
| *** jtomasek has quit IRC | 16:38 | |
| clarkb | definitely, thats why I am excited tristanC is interested in poking at the machine learning bits of this more | 16:38 |
| clarkb | but also the explosion in size is likely related to the regex collecting a lot more logs than it used to | 16:39 |
| clarkb | so we can likely tune the regexes further to exclude logs we don't need or want | 16:39 |
| rcarrillocruz | could someone please link me or point me for a job variant in zuul v3? I see they are mentioned in docs, but I need an example to make it click for me | 16:40 |
| clarkb | pabelanger: if you'd be willing to review (and hopefully approve) https://review.openstack.org/#/c/522292/ I can quickly run curator manually with a 7 day retention | 16:41 |
| clarkb | as I think it will only run once a day as is | 16:41 |
| *** ykarel|away has quit IRC | 16:42 | |
| clarkb | but I've got multiple back to back doctor appointments I have to shuttle family to shortly | 16:42 |
| dmsimard | Oh, yeah, I've seen tristanC's log stuff firsthand when he demo'ed it a LONG time ago already. It's pretty neat. | 16:43 |
| *** rhallisey has quit IRC | 16:44 | |
| *** slaweq has quit IRC | 16:45 | |
| *** yamahata has joined #openstack-infra | 16:48 | |
| pabelanger | clarkb: +3 | 16:48 |
| pabelanger | clarkb: did you see our fix for zuul-cloner issue that happend last night? https://review.openstack.org/522145/ | 16:49 |
| *** e0ne has quit IRC | 16:50 | |
| *** andreas__ has quit IRC | 16:51 | |
| *** andreas_s has joined #openstack-infra | 16:51 | |
| *** jascott1 has joined #openstack-infra | 16:52 | |
| clarkb | I haven't yet | 16:52 |
| pabelanger | rcarrillocruz: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/projects.yaml#n453 would be a variant job, because it modified voting | 16:53 |
| rcarrillocruz | i was thinking of using variants for run-blah-{node} | 16:53 |
| clarkb | /dev/mapper/main-elasticsearch 1008G 627G 382G 63% /var/lib/elasticsearch <- current disk after manually running curator at 7 days | 16:53 |
| rcarrillocruz | or is that pattern no longer a thing recommended on zuulv3 | 16:53 |
| rcarrillocruz | ? | 16:53 |
| clarkb | the initializing shards should manage to get homes now and cluster should go to yellow then green | 16:54 |
| pabelanger | rcarrillocruz: yah, you can do that still. because you want more then 1 run-blah-{node} to run on a patch | 16:54 |
| clarkb | rcarrillocruz: instead you just make a variant for run-blah that runs on a different nodeset | 16:54 |
| clarkb | basically you don't need -ubuntu -centos -fedora etc | 16:55 |
| clarkb | instead you just modify the nodesets in variants | 16:55 |
| rcarrillocruz | ah, that makes sense | 16:55 |
| clarkb | or you can do it that way as proper child jobs still I suppose it just isn't necessarty | 16:55 |
| rcarrillocruz | thanks! | 16:55 |
| pabelanger | but, we cannot have same job name for patchset right? | 16:55 |
| clarkb | pabelanger: that I don't know | 16:55 |
| pabelanger | like, run-foo, run-foo, run-foo, with all different nodesets | 16:55 |
| pabelanger | would be confusing :) | 16:55 |
| *** andreas_s has quit IRC | 16:56 | |
| pabelanger | I still use -ubuntu, -centos, -fedora in job name, mostly to know what the job is running on | 16:56 |
| *** tesseract has quit IRC | 16:57 | |
| *** jascott1 has quit IRC | 16:57 | |
| *** dsariel has joined #openstack-infra | 17:00 | |
| *** salv-orlando has joined #openstack-infra | 17:04 | |
| *** andreas_s has joined #openstack-infra | 17:07 | |
| *** salv-orlando has quit IRC | 17:08 | |
| * dmsimard prefers to use explicit child jobs with explicit names too | 17:11 | |
| *** andreas_s has quit IRC | 17:11 | |
| *** dtantsur is now known as dtantsur|afk | 17:12 | |
| rcarrillocruz | hmm, yeah, i think this can be even a mess for log pushing | 17:12 |
| rcarrillocruz | what's the naming pattern pabelanger and dmsimard use for this stuff, node_type as suffix ? | 17:12 |
| *** makowals has quit IRC | 17:13 | |
| dmsimard | rcarrillocruz: if it can help, I did a gist a while ago with three different examples that amount to the same thing https://gist.github.com/dmsimard/b5a85109cffde40c5e6699bdc4e5c2d1 | 17:13 |
| dmsimard | rcarrillocruz: zuul doesn't enforce any pattern because there's no template expansion like in JJB | 17:13 |
| dmsimard | rcarrillocruz: you can name them what you like :) | 17:14 |
| rcarrillocruz | yeah, but there's some naming convention from migration guide iirc | 17:14 |
| rcarrillocruz | ? | 17:14 |
| dmsimard | for ara I use like "ara-integration-debian-py27-2.2.3.0" or "ara-integration-ubuntu-py35-devel" | 17:14 |
| dmsimard | rcarrillocruz: I'm not sure if there are conventions | 17:14 |
| *** dsariel has quit IRC | 17:15 | |
| jeblair | if the purpose of the job is to test "foo running under centos" and there's another job that tests "foo running under fedora", make child jobs and name them foo-centos, foo-fedora | 17:15 |
| *** rossella_s has quit IRC | 17:15 | |
| jeblair | if the purpose of the job is to test "foo" and the stable version of foo runs under trusty but the master version of foo runs under xenial, just have one job, "foo" with a variant for stable which sets the node to trusty | 17:16 |
| pabelanger | yah, that is how I understood it | 17:17 |
| jeblair | rcarrillocruz: ^ maybe that helps explain when you should use one thing instead of the other | 17:17 |
| *** rossella_s has joined #openstack-infra | 17:17 | |
| *** lucasagomes is now known as lucas-afk | 17:17 | |
| dmsimard | jeblair: that's a good example | 17:18 |
| rcarrillocruz | yeah, it's the primer. the use case is 'test ios against iosv, ios-xe, etc' | 17:18 |
| rcarrillocruz | so same modules that need to be tested against different network nodes | 17:18 |
| rcarrillocruz | thanks | 17:18 |
| *** pcaruana has quit IRC | 17:18 | |
| *** salv-orlando has joined #openstack-infra | 17:31 | |
| *** salv-orlando has quit IRC | 17:31 | |
| *** salv-orlando has joined #openstack-infra | 17:31 | |
| *** Hal has quit IRC | 17:35 | |
| *** edmondsw has joined #openstack-infra | 17:41 | |
| openstackgerrit | Merged openstack-infra/system-config master: Reduce elasticsearch retention to 7 days https://review.openstack.org/522292 | 17:44 |
| *** e0ne has joined #openstack-infra | 17:44 | |
| andreaf | infra-core, can I have a 2nd d-g review on this? https://review.openstack.org/#/c/521237/ stacviz has been broken in legacy jobs for a while now... | 17:49 |
| *** smarcet has joined #openstack-infra | 17:51 | |
| *** yamamoto_ has joined #openstack-infra | 17:51 | |
| openstackgerrit | Ian Main proposed openstack-infra/tripleo-ci master: DNM, update logging for undercloud containers https://review.openstack.org/518972 | 17:53 |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: DNM, update logging for undercloud containers https://review.openstack.org/518972 | 17:53 |
| *** dmsimard is now known as dmsimard|afk | 17:53 | |
| *** makowals has joined #openstack-infra | 17:53 | |
| *** yamamoto has quit IRC | 17:55 | |
| *** rlandy|rover is now known as rlandy|rover|brb | 17:56 | |
| *** yamamoto has joined #openstack-infra | 17:57 | |
| *** trown is now known as trown|lunch | 17:58 | |
| *** yamamoto_ has quit IRC | 17:59 | |
| *** jascott1 has joined #openstack-infra | 18:00 | |
| *** ralonsoh has quit IRC | 18:02 | |
| *** rlandy|rover|brb is now known as rlandy|rover | 18:05 | |
| *** derekh has quit IRC | 18:09 | |
| *** efoley has quit IRC | 18:11 | |
| *** rossella_s has quit IRC | 18:11 | |
| *** coolsvap has quit IRC | 18:12 | |
| *** rossella_s has joined #openstack-infra | 18:12 | |
| *** ricolin has quit IRC | 18:12 | |
| *** tosky has quit IRC | 18:13 | |
| *** jpena is now known as jpena|off | 18:13 | |
| AJaeger | config-core, please review these changes for sphinx/releasenotes publishing: https://review.openstack.org/522017 https://review.openstack.org/521996 https://review.openstack.org/#/c/521590 | 18:16 |
| *** yamahata has quit IRC | 18:17 | |
| *** caphrim007_ has quit IRC | 18:18 | |
| *** dhinesh has quit IRC | 18:18 | |
| *** caphrim007 has joined #openstack-infra | 18:18 | |
| *** caphrim007 has quit IRC | 18:19 | |
| AJaeger | mordred: did you see http://logs.openstack.org/29/521729/1/check/build-openstack-sphinx-docs/daaf8d0/job-output.txt.gz - why is horizon not installed? I'll currently cleanup etherpad... | 18:19 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: zuul: Run OVB jobs only when it matters https://review.openstack.org/522321 | 18:19 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: zuul: Run OVB jobs only when it matters https://review.openstack.org/522321 | 18:22 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul-jobs master: Fix wrong dir for doc/requirements.txt https://review.openstack.org/522327 | 18:23 |
| AJaeger | mordred, config-core, trivial typo fix - please review ^ | 18:23 |
| mordred | AJaeger: whoops | 18:23 |
| mordred | AJaeger: is that the fix for the horizon not being installed link? | 18:24 |
| AJaeger | mordred: I created the first doc/requirements.txt - see https://review.openstack.org/522320 | 18:24 |
| pabelanger | AJaeger: +3 all around | 18:24 |
| AJaeger | mordred: nothing todo with horizon ;( No clue yet on that one | 18:24 |
| mordred | AJaeger: woohoo! | 18:24 |
| AJaeger | pabelanger: thanks! | 18:24 |
| mordred | k. I'll look at that one | 18:25 |
| AJaeger | mordred: just to see another interesting implementation, review 522320 ;( | 18:25 |
| AJaeger | mordred: thanks for checking the horizon stuff | 18:25 |
| mordred | AJaeger: sigh | 18:27 |
| mordred | AJaeger: it hardcodes horizon in the tox.in ifiles | 18:27 |
| pabelanger | mordred: jeblair: clarkb: I'd like to see if we can land our zuul-env virtualenv fixes and try again to delete zuul-env in DIB: https://review.openstack.org/522145/ this should now create the required virtuelenv with yaml dependencies for zuul-cloner shim to work | 18:28 |
| *** rbrndt has quit IRC | 18:29 | |
| pabelanger | I have time this afternoon to monitor image rebuilds again | 18:29 |
| AJaeger | mordred: ;( | 18:29 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: cistatus: update web page with most important jobs https://review.openstack.org/522068 | 18:30 |
| mordred | AJaeger: remote: https://review.openstack.org/522343 Align tox_install.sh with other projects | 18:31 |
| mordred | pabelanger: so it turned out that we DID use the virtualenv? | 18:32 |
| *** tlaxkit has joined #openstack-infra | 18:32 | |
| AJaeger | dhellmann: do you know why https://review.openstack.org/#/c/521211/ fails the sphinx build job? I'm puzzled... | 18:33 |
| mordred | pabelanger: +3 | 18:33 |
| pabelanger | mordred: yah, for pyyaml dependency when we installed zuul | 18:33 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Set success-url for sphinx-docs to html https://review.openstack.org/522017 | 18:34 |
| AJaeger | mordred: there are more repos that follow the same pattern as neutron-vpnaas-dashboard ;( | 18:34 |
| AJaeger | pabelanger: did you see tobiash's comment on 522145? Might be worth a cleanup followup... | 18:36 |
| pabelanger | AJaeger: good point, i can propose that | 18:37 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: zuul: Run OVB jobs only when it matters https://review.openstack.org/522321 | 18:37 |
| tobiash | AJaeger: wow, you're fast, just wrote that a minute ago | 18:37 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Only run whereto if htaccess file exists https://review.openstack.org/521996 | 18:39 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Update fetch sphinx output to use sphinx vars https://review.openstack.org/521590 | 18:39 |
| mordred | AJaeger: :( | 18:41 |
| AJaeger | mordred: updated etherpad, it's current with everything I know | 18:42 |
| mordred | AJaeger: so - we might have to think about this a little harder ... we're in good shape on releasenotes, right? | 18:42 |
| AJaeger | mordred: yes, with exception of aodh - that I fixed - I'm not aware of problems | 18:43 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Revert "Revert "Remove zuul-env from DIB images"" https://review.openstack.org/522347 | 18:44 |
| mordred | ok. that's good. | 18:44 |
| pabelanger | AJaeger: mordred: clarkb: ^ and patch to remove zuul-env from DIB again | 18:44 |
| AJaeger | pabelanger: let's try again... | 18:44 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Setup virtualenv for zuul-cloner https://review.openstack.org/522145 | 18:44 |
| pabelanger | AJaeger: indeed! | 18:45 |
| *** viktor_ has joined #openstack-infra | 18:48 | |
| *** dhinesh has joined #openstack-infra | 18:49 | |
| *** dhinesh_ has joined #openstack-infra | 18:51 | |
| *** shardy has quit IRC | 18:51 | |
| *** mihalis68 has joined #openstack-infra | 18:52 | |
| AJaeger | mordred: I'm fixing the dashboard projects like you did. cliff looks odd ;( | 18:53 |
| *** dhinesh has quit IRC | 18:53 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: zuul: Run OVB jobs only when it matters https://review.openstack.org/522321 | 18:54 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Fix wrong dir for doc/requirements.txt https://review.openstack.org/522327 | 18:56 |
| *** jesusaur has quit IRC | 18:58 | |
| *** makowals has quit IRC | 19:00 | |
| *** e0ne has quit IRC | 19:02 | |
| *** e0ne has joined #openstack-infra | 19:04 | |
| AJaeger | mordred: so, fixed everything with exception of cliff - not sure what's the best approach is for that one ;( | 19:04 |
| *** yamahata has joined #openstack-infra | 19:06 | |
| mordred | AJaeger: looking... | 19:07 |
| mordred | AJaeger: is it just the cliffdemo dep that's weird? | 19:09 |
| AJaeger | mordred: check the "os:" lines in tox.ini | 19:12 |
| AJaeger | the two src environments | 19:12 |
| mordred | AJaeger: oh | 19:12 |
| * AJaeger fixes aodh releasenotes, they build without error checking and now fail with it enabled | 19:13 | |
| *** e0ne has quit IRC | 19:15 | |
| *** e0ne has joined #openstack-infra | 19:15 | |
| *** edmondsw has quit IRC | 19:16 | |
| pabelanger | going to grab some lunch, if a config-core could approve https://review.openstack.org/522347/ before I get back that would be helpful :) | 19:17 |
| *** electrofelix has quit IRC | 19:19 | |
| *** jesusaur has joined #openstack-infra | 19:20 | |
| dhellmann | AJaeger : what's wrong with cliff? | 19:22 |
| AJaeger | dhellmann: the way it installs other projects from source causes some problems with tox_install.sh and docs builds. | 19:22 |
| mordred | dhellmann: nothing's *wrong* with it per-se - but ... what AJaeger ^^ said | 19:23 |
| * mordred is poking at options to see which is the least disruptive | 19:23 | |
| dhellmann | that's weird. I have no memory of tonyb making that change. | 19:23 |
| AJaeger | dhellmann: any idea what octavia is doing wrong? https://review.openstack.org/#/c/521211/ fails the sphinx build job? I'm puzzled... | 19:24 |
| *** salv-orlando has quit IRC | 19:24 | |
| dhellmann | AJaeger : looking | 19:24 |
| *** salv-orlando has joined #openstack-infra | 19:24 | |
| mordred | dhellmann: while I'm bothering you ... do you need cliffdemo installed for things that aren't docs? it looks like it's only referenced for docs things | 19:24 |
| dhellmann | AJaeger : why is the doc build importing pdb? | 19:25 |
| AJaeger | mordred: integration-tests/openstackclient-tip.sh could install the lib from source | 19:25 |
| AJaeger | dhellmann: a result of pip install . perhaps? Didn't check | 19:25 |
| dhellmann | mordred : it's used in the doc build as an example of generating the docs for a command, but we don't need it installed usually | 19:25 |
| mordred | dhellmann: cool | 19:25 |
| dhellmann | AJaeger : importing, not installing | 19:25 |
| *** dsariel has joined #openstack-infra | 19:25 | |
| mordred | AJaeger: I think if we make a doc/requirements.txt and put cliffdemo in it, then we can use the tox_install.sh there without modifications | 19:26 |
| *** niedbalski is now known as ikslabdein | 19:26 | |
| *** ikslabdein is now known as gyosaboy | 19:26 | |
| AJaeger | mordred, dhellmann - sorry, too tired to help further with these ;( | 19:26 |
| *** gyosaboy is now known as niedbalski | 19:27 | |
| clarkb | ok back from doctors /me checks on elasticsearch | 19:27 |
| *** e0ne has quit IRC | 19:27 | |
| dhellmann | AJaeger : np, I'll see what I can figure out | 19:27 |
| AJaeger | thanks, dhellmann ! | 19:27 |
| openstackgerrit | Merged openstack-infra/project-config master: Revert "Revert "Remove zuul-env from DIB images"" https://review.openstack.org/522347 | 19:27 |
| * AJaeger signs off for today... | 19:27 | |
| mordred | AJaeger: no worries! thank you for all of your help! | 19:27 |
| clarkb | cluster is still red but went from 5 initializing shards to 2, there are still 2 unitialized shards which is why it is red | 19:27 |
| clarkb | so I think it is healing itself | 19:28 |
| *** edmondsw has joined #openstack-infra | 19:28 | |
| *** salv-orlando has quit IRC | 19:28 | |
| clarkb | pabelanger: when you return from lunch might be good to trigger an image build to pikc up 522347 sooner than later | 19:28 |
| *** trown|lunch is now known as trown | 19:29 | |
| *** slaweq has joined #openstack-infra | 19:30 | |
| *** eharney has quit IRC | 19:31 | |
| dhellmann | oh, fun, octavia is doing things with diagrams | 19:31 |
| *** Daisy_ has joined #openstack-infra | 19:31 | |
| *** rbrndt has joined #openstack-infra | 19:32 | |
| *** edmondsw has quit IRC | 19:32 | |
| clarkb | [2017-11-22 19:33:47,922][WARN ][index.engine ] [elasticsearch03] [logstash-2017.11.22][4] failed to sync translog and [2017-11-22 19:33:47,931][WARN ][indices.cluster ] [elasticsearch03] [[logstash-2017.11.22][4]] marking and sending shard failed due to [failed recovery] are a thing on es03 | 19:34 |
| clarkb | I'm worried that we have corrupted shards and it won't ever recover this index | 19:34 |
| clarkb | (which I think means we delete it and move on) | 19:34 |
| fungi | is that the one which was being appended when we ran out of space? | 19:36 |
| *** Daisy_ has quit IRC | 19:36 | |
| clarkb | fungi: they are two shards in the active index yes | 19:39 |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Use stackviz tarball from cache directly https://review.openstack.org/521237 | 19:39 |
| *** Winwin has joined #openstack-infra | 19:40 | |
| Winwin | https://cryptosrevolution.wixsite.com/beta/ | 19:41 |
| mordred | dhellmann, AJaeger: I just pushed up two patches to cliff - one to do doc/requirements.txt which should work with current jobs and new jobs and one to test that the first one works with the new jobs | 19:42 |
| dhellmann | mordred : https://review.openstack.org/522371 may fix the octavia doc build. if it does, you can squash that into your patch | 19:42 |
| mordred | dhellmann: woot! | 19:43 |
| *** ChanServ sets mode: +b Winwin!*@* | 19:44 | |
| *** Winwin was kicked by ChanServ (User is banned from this channel) | 19:44 | |
| * mordred was just about to do that | 19:44 | |
| fungi | i meant to add a comment, but meh | 19:45 |
| *** bgmccollum_ is now known as bgmccollum | 19:45 | |
| fungi | my finger was too quick on the enter key | 19:45 |
| *** amoralej is now known as amoralej|off | 19:46 | |
| clarkb | fungi: I think I'll give it another half hour or so and if state doesn't change I'm going to cave and delete the index so we can move forward | 19:47 |
| clarkb | (because I think finding stability in logstash/es is more valuable now than a few hours of data that will be lost) | 19:47 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: OVB-HA: run when ci/environments/ovb-ha.yaml is touched https://review.openstack.org/522376 | 19:50 |
| fungi | clarkb: yeah, it's not a _huge_ loss honestly, given the current activity levels, but i guess there may have been a lot of backlog encapsulated there too | 19:51 |
| *** rwsu has quit IRC | 19:52 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Temporarily disable releasenotes for aodh https://review.openstack.org/522377 | 19:52 |
| clarkb | its only 90GB compared to 264GB for yesterday's index | 19:53 |
| mordred | ++ | 19:54 |
| AJaeger | ok, stepping out helped - config-core, please review 522377 | 19:55 |
| *** tosky has joined #openstack-infra | 19:55 | |
| *** esberglu has quit IRC | 20:00 | |
| erlon | @all, clarkb: hey, man Im trying to scratch my mind to remember the name o the apache module that give that nice colors and behaviour to the CIs, do you remember what it is? | 20:00 |
| erlon | http://logs.openstack.org/72/512572/7/check/manila-tempest-minimal-dsvm-cephfs-nfs-centos-7/6d00562/logs/screen-c-vol.txt.gz | 20:00 |
| *** esberglu has joined #openstack-infra | 20:00 | |
| *** esberglu has quit IRC | 20:00 | |
| *** esberglu has joined #openstack-infra | 20:01 | |
| clarkb | erlon: os-loganalyze, it is a wsgi app | 20:01 |
| erlon | clarkb: owww man thanks | 20:02 |
| *** edmondsw has joined #openstack-infra | 20:02 | |
| *** slaweq has quit IRC | 20:04 | |
| *** esberglu has quit IRC | 20:05 | |
| *** edmondsw has quit IRC | 20:06 | |
| clarkb | ok I went ahead and deleted the index, cluster is now yellow while it rebalances. | 20:09 |
| clarkb | it is moving shards off of 07 which has 270GB free to other nodes with have ~500GB free | 20:09 |
| clarkb | one shard on 07 to 06 and the other to 03 | 20:10 |
| *** efoley has joined #openstack-infra | 20:11 | |
| fungi | that sounds more sustainable | 20:12 |
| * fungi disappears to find food, bbiaw | 20:12 | |
| clarkb | logstash job queue size is shrinking again now too | 20:18 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Updated get member endpoint https://review.openstack.org/522381 | 20:18 |
| *** gouthamr has quit IRC | 20:20 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Stop using tox to build python tarballs https://review.openstack.org/513927 | 20:21 |
| *** gouthamr has joined #openstack-infra | 20:22 | |
| clarkb | it would be neat if you could tell elasticsaerch to delete the replica shards after some time (maybe this is possibile and I just don't know it) | 20:23 |
| clarkb | but then we could potentially say take 10 days of indexed logs and extend that to 14 days by deleting the replicas for days 8 - 14 | 20:23 |
| clarkb | and if we lose a node we say we are fine with deleting 8 - 14 | 20:24 |
| clarkb | (problem is it is also a query performance hit since multiple replicas increaes number of cpus that can sarch | 20:24 |
| *** salv-orlando has joined #openstack-infra | 20:25 | |
| AJaeger | config-core, could you please review https://review.openstack.org/522377 to allow fixing aodh releasenotes? also, here're some job removal reviews https://review.openstack.org/521779 https://review.openstack.org/521895 https://review.openstack.org/521655 | 20:25 |
| clarkb | AJaeger: 521779 depends on a change that has a zuul -1 (just fyi) | 20:28 |
| *** ldnunes has quit IRC | 20:29 | |
| smarcet | fungi: how are u doing , hope everythig is fine , got this error on openstackid-dev deployment seems that some job is failing under zuul side | 20:30 |
| smarcet | https://review.openstack.org/#/c/512685/ | 20:30 |
| *** salv-orlando has quit IRC | 20:31 | |
| smarcet | fungi:http://logs.openstack.org/85/512685/2/check/legacy-laravel-openstackid-unittests/e4bcd47/ara/ | 20:31 |
| smarcet | fungi : seems that sudo: php5enmod: command not found | 20:32 |
| clarkb | smarcet: I think that may be due to cleanup of the bindep fallback file | 20:32 |
| smarcet | clarkb: so i should update openstackid bindep file ? | 20:32 |
| clarkb | though I don't see any bindep happening in the pre tasks | 20:32 |
| *** slaweq has joined #openstack-infra | 20:32 | |
| clarkb | oh bindep happens during run | 20:33 |
| clarkb | smarcet: http://logs.openstack.org/85/512685/2/check/legacy-laravel-openstackid-unittests/e4bcd47/ara/result/99699f3d-9ada-47ca-859b-ac0328d26609/ that is where bindep is happening | 20:33 |
| clarkb | smarcet: looks like it is installing php7 not php5 | 20:33 |
| clarkb | smarcet: is this job running on xenial but should be running on trusty? | 20:34 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove multinode legacy gate https://review.openstack.org/521895 | 20:34 |
| smarcet | clarkb: well i never tried openstackid on xenial | 20:35 |
| smarcet | i think that should be trusty | 20:35 |
| clarkb | smarcet: ok we likely just need to update the job to run on trusty then. | 20:35 |
| clarkb | smarcet: do you want me to do that? | 20:35 |
| smarcet | clarkb: please :) and thx u! really appreciate it | 20:35 |
| clarkb | smarcet: do you know if the release jobs also need to run on trusty? | 20:36 |
| pabelanger | and back | 20:36 |
| pabelanger | clarkb: yah, looking into that now | 20:37 |
| smarcet | clarkb: well i guess so, at any rate i think that having all related jobs running on same version should be ideal :) | 20:37 |
| clarkb | smarcet: ya quick grep shows they use php5 too so I'll update them as well | 20:37 |
| smarcet | clarkb: thx u! | 20:37 |
| *** jkilpatr has quit IRC | 20:37 | |
| *** rwsu has joined #openstack-infra | 20:38 | |
| pabelanger | okay, centos-7 dib started | 20:38 |
| openstackgerrit | Clark Boylan proposed openstack-infra/openstack-zuul-jobs master: Run laravel openstackid jobs on trusty https://review.openstack.org/522384 | 20:39 |
| clarkb | config-core ^ review on that should fix smarcet's issue above | 20:40 |
| smarcet | clarkb thx u :) | 20:40 |
| dhellmann | mordred : I'm looking at the new failure in that octavia doc job | 20:44 |
| openstackgerrit | Michael Schmidt proposed openstack-infra/project-config master: initialises kubernikus project skeleton https://review.openstack.org/522272 | 20:45 |
| openstackgerrit | Michael Schmidt proposed openstack-infra/project-config master: adds kubernikus project https://review.openstack.org/522282 | 20:45 |
| *** kgiusti has left #openstack-infra | 20:48 | |
| dhellmann | mordred : it looks like the latest failure with octavia is it is relying on pbr's feature of auto-generating docs that link to python classes, etc. What's the new way to do that? | 20:50 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Run laravel openstackid jobs on trusty https://review.openstack.org/522384 | 20:51 |
| clarkb | smarcet: ^ you should be able to recheck nwo I think | 20:52 |
| clarkb | smarcet: config reloads happen almost immediately | 20:52 |
| smarcet | clarkb got it thx u :) | 20:52 |
| openstackgerrit | Michael Schmidt proposed openstack-infra/project-config master: initialises kubernikus project skeleton https://review.openstack.org/522272 | 20:57 |
| openstackgerrit | Michael Schmidt proposed openstack-infra/project-config master: adds kubernikus project https://review.openstack.org/522282 | 20:57 |
| *** puiterwijk is now known as puiterwijk|mnt | 20:57 | |
| *** viktor_ has quit IRC | 20:59 | |
| *** salv-orlando has joined #openstack-infra | 21:00 | |
| *** salv-orlando has quit IRC | 21:00 | |
| *** salv-orlando has joined #openstack-infra | 21:01 | |
| *** gouthamr has quit IRC | 21:03 | |
| *** trown is now known as trown|outtypewww | 21:04 | |
| *** hrubi has quit IRC | 21:08 | |
| *** hrubi has joined #openstack-infra | 21:09 | |
| *** efoley_ has joined #openstack-infra | 21:10 | |
| *** efoley has quit IRC | 21:10 | |
| *** andreas_s has joined #openstack-infra | 21:11 | |
| *** linkmark has joined #openstack-infra | 21:13 | |
| *** ijw has joined #openstack-infra | 21:14 | |
| *** jkilpatr has joined #openstack-infra | 21:15 | |
| *** gouthamr has joined #openstack-infra | 21:15 | |
| jeblair | huzzah! all of the patches to add file extensions to playbooks have merged! | 21:15 |
| *** andreas_s has quit IRC | 21:16 | |
| *** ijw has quit IRC | 21:18 | |
| SamYaple | jeblair: w00t! | 21:19 |
| *** Wei_Liu has quit IRC | 21:21 | |
| openstackgerrit | Merged openstack-infra/openstackid master: Fix Access Token reissue with Rotate Refresh Token policy active https://review.openstack.org/512685 | 21:22 |
| *** xingchao has joined #openstack-infra | 21:23 | |
| *** smarcet has quit IRC | 21:24 | |
| clarkb | ES cluster is green now \o/ | 21:25 |
| pabelanger | okay, centos-7 DIB uploading | 21:26 |
| pabelanger | http://logs.openstack.org/33/522033/2/gate/tripleo-ci-centos-7-undercloud-oooq/083ea25/job-output.txt.gz#_2017-11-22_20_37_50_984070 | 21:27 |
| pabelanger | EmilienM: ^ shouldn't be hitting trunk.rdo directly, but using the reverse proxy cache in regional mirrors | 21:28 |
| clarkb | also all logstash workers appear to still be running from the time they were last restarted (so no recent OOMs) | 21:35 |
| pabelanger | nice | 21:35 |
| *** Goneri has quit IRC | 21:36 | |
| clarkb | so I think the thing to check on monday is disk usage it should be on the low end assuming the US holiday results in job drop off | 21:36 |
| *** mat128_ has quit IRC | 21:36 | |
| clarkb | so if its high relative to available disk then we probably can't increase back to 10 days | 21:36 |
| clarkb | if it is low then we can probably give it a shot and watch it | 21:36 |
| EmilienM | pabelanger: do you ask me to send a patch? :) | 21:37 |
| EmilienM | pabelanger: it's here https://git.openstack.org/cgit/openstack/tripleo-quickstart/tree/roles/repo-setup/tasks/get-dlrn-hash.yml#n19 | 21:37 |
| pabelanger | EmilienM: just pointing out a job failure, a fix :) | 21:38 |
| *** dave-mcc_ has quit IRC | 21:39 | |
| EmilienM | weshay: ^ should we file a launchpad bug for this one? I don't have a quick fix handy | 21:39 |
| *** ijw has joined #openstack-infra | 21:40 | |
| *** slaweq has quit IRC | 21:41 | |
| pabelanger | removal of zuul-env looks to be good: http://zuulv3.openstack.org/static/stream.html?uuid=55ef444e761c4a3a88bf8eb5062f04fe&logfile=console.log has created the virtualenv as expected | 21:41 |
| clarkb | if we are creating a virtualenv anyways what is the benefit of deleting the venv from the image? | 21:42 |
| pabelanger | clarkb: next patch is to remove fetch-zuul-cloner from base jobs, and leaving only legacy jobs that will create it | 21:43 |
| pabelanger | https://review.openstack.org/513506/ once all new images have been uploaded | 21:45 |
| *** threestrands has joined #openstack-infra | 21:45 | |
| *** threestrands has quit IRC | 21:45 | |
| *** threestrands has joined #openstack-infra | 21:45 | |
| pabelanger | then, native zuulv3 jobs will no longer have any zuul-env | 21:45 |
| weshay | EmilienM, /me looks | 21:46 |
| weshay | pabelanger, EmilienM dmsimard|afk and apevec have been discussing how to resolve hitting rdo repos | 21:46 |
| weshay | EmilienM, there is already a bug opened | 21:47 |
| weshay | pabelanger, and a elastic-recheck query | 21:47 |
| clarkb | pabelanger: I see so its a way to ensure with a high degree of certainty that non legacy jobs can't use it | 21:47 |
| pabelanger | weshay: well, CI job in gate shouldn't need to hit trunk.rdo directly, we setup a reverse proxy cache. for example: http://mirror.dfw.rax.openstack.org:8080/rdo/centos7/ | 21:50 |
| pabelanger | clarkb: yah | 21:50 |
| *** Goneri has joined #openstack-infra | 21:51 | |
| pabelanger | okay, centos upload every where minus infracloud, slow network | 21:54 |
| jeblair | pabelanger, clarkb: also, new venv is much smaller | 21:55 |
| jeblair | only pyyaml | 21:55 |
| pabelanger | +1 | 21:55 |
| *** xingchao has quit IRC | 21:57 | |
| openstackgerrit | Joseph Davis proposed openstack-infra/project-config master: Mark requirements-check for monasca-ceilometer as non-voting https://review.openstack.org/522392 | 21:57 |
| *** felipemonteiro_ has quit IRC | 21:57 | |
| *** efoley_ has quit IRC | 21:58 | |
| *** efoley_ has joined #openstack-infra | 21:58 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Remove hack of ignoring 127.0.0.2 in logs collection https://review.openstack.org/506956 | 21:59 |
| *** jpich has quit IRC | 22:00 | |
| *** dbecker has quit IRC | 22:01 | |
| *** florianf has quit IRC | 22:02 | |
| *** wolverineav has quit IRC | 22:02 | |
| *** ijw has quit IRC | 22:02 | |
| *** ijw has joined #openstack-infra | 22:03 | |
| *** raissa has quit IRC | 22:04 | |
| *** raissa has joined #openstack-infra | 22:04 | |
| *** raissa has quit IRC | 22:05 | |
| *** tlaxkit has quit IRC | 22:05 | |
| *** raissa has joined #openstack-infra | 22:05 | |
| *** raissa has quit IRC | 22:05 | |
| *** xingchao has joined #openstack-infra | 22:06 | |
| *** raissa has joined #openstack-infra | 22:06 | |
| *** raissa has quit IRC | 22:06 | |
| openstackgerrit | Joseph Davis proposed openstack-infra/project-config master: Temporarily remove requirements-check for monasca-ceilometer https://review.openstack.org/522392 | 22:07 |
| *** raissa has joined #openstack-infra | 22:07 | |
| *** raissa has quit IRC | 22:07 | |
| *** Dinesh_Bhor has quit IRC | 22:11 | |
| *** jaypipes has quit IRC | 22:13 | |
| *** dprince has quit IRC | 22:13 | |
| clarkb | I guess it will also make our images slightly smaller too | 22:13 |
| clarkb | that reminds me, we may want to consider removing the trove image from our images if it is still there since it can be pulled from the tarballs proxy | 22:14 |
| clarkb | but that will likely be more complicated than it sounds due to how we list images to cache | 22:14 |
| *** Apoorva has joined #openstack-infra | 22:16 | |
| pabelanger | clarkb: are we still using trove image on stable branches? | 22:17 |
| pabelanger | I would guess not, and we could maybe propose patch to devstack to remove it there | 22:18 |
| *** efoley_ has quit IRC | 22:18 | |
| clarkb | pabelanger: that I'm not sure of | 22:18 |
| clarkb | amrith may know | 22:18 |
| *** efoley_ has joined #openstack-infra | 22:18 | |
| *** dingyichen has quit IRC | 22:20 | |
| *** rcernin has joined #openstack-infra | 22:20 | |
| *** tojuvone has quit IRC | 22:22 | |
| *** xingchao has quit IRC | 22:24 | |
| *** ijw has quit IRC | 22:26 | |
| *** tojuvone has joined #openstack-infra | 22:34 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-log_processor master: Collapse logically identically filenames for crm114 https://review.openstack.org/522399 | 22:41 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Consume collect-logs directly from zuul/post playbook https://review.openstack.org/522400 | 22:41 |
| clarkb | ^^ is totally untested but with tristanC proposing to make the crm114 stuff more robust I figure getting that running again in production is important | 22:41 |
| *** bobh has quit IRC | 22:42 | |
| clarkb | well I shouldn't say totally untested, I did run those specific bits through the python repl | 22:44 |
| *** hasharAway has quit IRC | 22:45 | |
| *** marst has quit IRC | 22:46 | |
| jeblair | clarkb: you're okay that nova.gz doesn't match? | 22:46 |
| clarkb | jeblair: oh thats a bug | 22:47 |
| clarkb | yay code review | 22:47 |
| jeblair | yay i helped | 22:47 |
| jeblair | left a 0 on that one too. really doesn't matter. :) | 22:48 |
| *** rosmaita has quit IRC | 22:49 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-log_processor master: Collapse logically identically filenames for crm114 https://review.openstack.org/522399 | 22:50 |
| clarkb | easy enough to fix | 22:50 |
| clarkb | likely easier to read that version too | 22:51 |
| *** felipemonteiro has joined #openstack-infra | 22:51 | |
| *** mihalis68 has quit IRC | 22:51 | |
| *** bobh has joined #openstack-infra | 22:51 | |
| *** felipemonteiro_ has joined #openstack-infra | 22:52 | |
| *** Apoorva has quit IRC | 22:52 | |
| *** smarcet has joined #openstack-infra | 22:52 | |
| *** Apoorva has joined #openstack-infra | 22:53 | |
| *** dsariel has quit IRC | 22:54 | |
| *** rwsu has quit IRC | 22:55 | |
| *** felipemonteiro has quit IRC | 22:56 | |
| dhinesh_ | is it possible for nodepool or jenkins to hold onto a virtual machine, after a job is complete on it? | 23:01 |
| pabelanger | dhinesh_: yes, you can setup an autohold in nodepool if needed | 23:02 |
| pabelanger | dhinesh_: https://docs.openstack.org/infra/nodepool/operation.html#job-create | 23:03 |
| *** sdague has quit IRC | 23:03 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-log_processor master: Collapse logically identical filenames for crm114 https://review.openstack.org/522399 | 23:05 |
| dhinesh_ | pabelanger: or can i just do 'nodepool hold nodeid' | 23:06 |
| pabelanger | dhinesh_: yes, but you'd need to know the node id to use that command, which you might | 23:08 |
| pabelanger | otherwise, you can use job-create and add a specific job | 23:09 |
| dhinesh_ | pabelanger: ok , got it thanks | 23:10 |
| *** claudiub|2 has quit IRC | 23:16 | |
| *** smarcet has quit IRC | 23:16 | |
| *** lbragstad has quit IRC | 23:23 | |
| *** jascott1 has quit IRC | 23:29 | |
| *** jascott1 has joined #openstack-infra | 23:29 | |
| *** vhosakot has quit IRC | 23:30 | |
| *** Apoorva has quit IRC | 23:31 | |
| *** gouthamr has quit IRC | 23:34 | |
| *** jascott1 has quit IRC | 23:34 | |
| *** rlandy|rover is now known as rlandy|rover|bbl | 23:36 | |
| *** rbrndt has quit IRC | 23:37 | |
| *** felipemonteiro_ has quit IRC | 23:38 | |
| *** edmondsw has joined #openstack-infra | 23:54 | |
| openstackgerrit | Jeremy Stanley proposed openstack-dev/pbr master: Support PEP 345 Project-URL metadata https://review.openstack.org/521689 | 23:57 |
| *** salv-orlando has quit IRC | 23:57 | |
| *** salv-orlando has joined #openstack-infra | 23:58 | |
| *** edmondsw has quit IRC | 23:59 | |
| *** dingyichen has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!