| *** slaweq has quit IRC | 00:00 | |
| fungi | ianw: i'm guessing it's something we need to update on the report comment template | 00:00 |
|---|---|---|
| fungi | but don't really know for sure. trying to get all these release project-template updates worked through currently | 00:00 |
| ianw | fungi: that's cool, i just didn't want to retrace already solved steps | 00:01 |
| fungi | yep, i think it's virgin ground | 00:01 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements https://review.openstack.org/503645 | 00:02 |
| ianw | tonyb: i'm assuming jeblair is ok with the perms and took the liberty of adding it (can always fiddle it later). want to try the abandon again? | 00:03 |
| ianw | just so we can get this EOL out of the way | 00:03 |
| tonyb | ianw: thanks ... trying kolla again | 00:04 |
| dhinesh_ | is there a nodepool/elements/puppet which can be used for CI setup? | 00:04 |
| *** slaweq has joined #openstack-infra | 00:05 | |
| pabelanger | dhinesh_: not any more, we've stopped using puppet on our images. | 00:05 |
| pabelanger | dhinesh_: you can look at git history for project-config to see how we did it | 00:05 |
| tonyb | ianw: that worked ... so I'll carry on with the rest of the projects | 00:05 |
| *** trown is now known as trown|outtypewww | 00:05 | |
| Jeffrey4l | what's the example url for the project public keys? "keys/<source>/<project>.pub" | 00:06 |
| *** rtjure has joined #openstack-infra | 00:07 | |
| Jeffrey4l | what's the example url for the project public keys? "keys/<source>/<project>.pub" | 00:07 |
| *** andreas_s has joined #openstack-infra | 00:09 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/openstack-zuul-jobs master: Use a more thorough release for API servers https://review.openstack.org/514856 | 00:10 |
| *** rtjure has quit IRC | 00:12 | |
| fungi | Jeffrey4l: you can use http://git.openstack.org/cgit/openstack-infra/zuul/tree/tools/encrypt_secret.py?h=feature/zuulv3 to handle it for you, but yes something like http://zuulv3.openstack.org/keys/gerrit/openstack/nova.pub | 00:13 |
| Jeffrey4l | got. thanks a lot. | 00:13 |
| fungi | you're welcome! | 00:14 |
| dhinesh_ | pabelanger: ok, i am building an image for a CI setup, what are the essential elements for jenkins slave? | 00:14 |
| openstackgerrit | Merged openstack-infra/system-config master: Add match for "-" in pipeline name https://review.openstack.org/514591 | 00:14 |
| *** bobh has joined #openstack-infra | 00:15 | |
| pabelanger | dhinesh_: we infact have an element called jenkins-slave: http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/elements/jenkins-slave you should be able to use that | 00:15 |
| jeblair | fungi, ianw: we're probably missing the pipeline config from zuulv2 which customized the 'build succeeded' messages. | 00:18 |
| jeblair | it's a config change, not a code change. | 00:18 |
| fungi | jeblair: that sounds about right | 00:18 |
| *** ijw has quit IRC | 00:18 | |
| fungi | it was like some sort of format string, if memory serves | 00:18 |
| ianw | jeblair: yeah, i'm not so sure ... in the debugger now | 00:19 |
| *** rcernin has joined #openstack-infra | 00:19 | |
| dhinesh_ | pabelanger: ok cool, and it can run it on top of any OS or is it specific to ubuntu? | 00:19 |
| *** rtjure has joined #openstack-infra | 00:20 | |
| jeblair | ianw: we have it for gate, but not check | 00:20 |
| jeblair | ianw: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/pipelines.yaml#n101 | 00:20 |
| jeblair | compare to http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n7 | 00:21 |
| *** andreas_s has quit IRC | 00:22 | |
| *** rcernin has quit IRC | 00:23 | |
| ianw | success-message: Build succeeded (check pipeline). ? | 00:23 |
| *** wolverineav has quit IRC | 00:23 | |
| jeblair | yep. also failure-message | 00:23 |
| *** wolverineav has joined #openstack-infra | 00:24 | |
| *** rcernin has joined #openstack-infra | 00:24 | |
| *** rtjure has quit IRC | 00:24 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config master: Use non-pypi for release instead of tarball-only https://review.openstack.org/514869 | 00:27 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Add success message for check pipeline https://review.openstack.org/514870 | 00:27 |
| ianw | jeblair: ^ like so? | 00:27 |
| *** bobh has quit IRC | 00:28 | |
| openstackgerrit | Merged openstack-infra/storyboard master: Consolidate old database migrations https://review.openstack.org/510370 | 00:28 |
| *** wolverineav has quit IRC | 00:29 | |
| *** thorst has joined #openstack-infra | 00:29 | |
| Zara | hm so I'm finally catching up with some long-overdue storyboard reviews | 00:30 |
| pabelanger | dhinesh_: it should work for ubuntu, fedora-26, centos-7 and opensuse | 00:30 |
| dhinesh_ | pabelanger: ok thanks | 00:31 |
| *** andreas_s has joined #openstack-infra | 00:32 | |
| *** rtjure has joined #openstack-infra | 00:32 | |
| Zara | johnsom sent us a patch, https://review.openstack.org/#/c/506812/2 to update some docs, but he currently has to link to the github mirror for releases because he wants to link a specific section of the page and apparently that's not possible with the git.o.o link | 00:32 |
| Zara | we'd prefer to use git.o.o and not github, so wondering if there are any solutions we've not considered. | 00:33 |
| Zara | well, that's a bit of a garbled question but the comments on that patch should convey things more clearly than I am rn | 00:34 |
| *** s-shiono has joined #openstack-infra | 00:36 | |
| fungi | you can link to a specific line in a file in cgit | 00:36 |
| *** rtjure has quit IRC | 00:37 | |
| *** slaweq has quit IRC | 00:37 | |
| fungi | anyway, it looks like the real solution there is to ask the release team to start publishing their readme on docs.o.o or releases.o.o | 00:40 |
| fungi | rather than abusing github's autorender of rst files | 00:41 |
| fungi | i'll leave a comment on the change | 00:41 |
| openstackgerrit | zhongshengping proposed openstack-infra/openstack-zuul-jobs master: Remove xenial beaker jobs from gate pipeline https://review.openstack.org/514872 | 00:42 |
| *** rtjure has joined #openstack-infra | 00:42 | |
| *** thorst has quit IRC | 00:46 | |
| openstackgerrit | zhongshengping proposed openstack-infra/openstack-zuul-jobs master: Remove extra voting https://review.openstack.org/514875 | 00:47 |
| *** rtjure has quit IRC | 00:47 | |
| *** slaweq has joined #openstack-infra | 00:49 | |
| *** andreas_s has quit IRC | 00:49 | |
| *** huanxie has joined #openstack-infra | 00:49 | |
| fungi | Zara: i've also asked the release team about getting the readme for that repo published in rendered form somewhere more durable | 00:50 |
| fungi | so maybe this will be solved soonish | 00:50 |
| Zara | fungi: thank you :) yeah, I wondered about linking to the line, but I thought that could be fragile since aiui it can only be done by line number and we'd want to identify it by heading | 00:50 |
| Zara | I've also noticed that the webclient draft builds behave strangely under 'Zuul' vs 'Jenkins check'. (there are examples here: https://review.openstack.org/#/c/511247/ ) | 00:51 |
| Zara | gate-storyboard-webclient-js-draft links to a storyboard-dev instance with the changes happily enough | 00:52 |
| Zara | legacy-storyboard-webclient-js-draft links to a dev instance that throws lots of errors and doesn't load any of the content | 00:52 |
| *** adisky_ has joined #openstack-infra | 00:52 | |
| *** rkukura has quit IRC | 00:52 | |
| Zara | I don't know what's going on since our error messages for that just say '-1' | 00:53 |
| *** rtjure has joined #openstack-infra | 00:54 | |
| *** andreas_s has joined #openstack-infra | 00:54 | |
| *** mriedem has quit IRC | 00:56 | |
| *** bnemec has quit IRC | 00:58 | |
| *** cuongnv has joined #openstack-infra | 00:58 | |
| *** Nil_ has quit IRC | 00:58 | |
| *** rtjure has quit IRC | 00:59 | |
| *** andreas_s has quit IRC | 00:59 | |
| *** Nil_ has joined #openstack-infra | 00:59 | |
| *** bnemec has joined #openstack-infra | 00:59 | |
| *** Apoorva_ has joined #openstack-infra | 01:00 | |
| *** Sukhdev has quit IRC | 01:00 | |
| *** jpena|off has quit IRC | 01:00 | |
| *** LindaWang has joined #openstack-infra | 01:00 | |
| *** ihrachys has quit IRC | 01:01 | |
| *** Apoorva has quit IRC | 01:02 | |
| *** amoralej|off has quit IRC | 01:04 | |
| *** ltomasbo has quit IRC | 01:04 | |
| *** pvaneck has quit IRC | 01:04 | |
| fungi | Zara: i've basically checked out for the night, but i would definitely suggest explaining/linking that on the zuulv3-issues etherpad | 01:04 |
| *** Apoorva_ has quit IRC | 01:04 | |
| fungi | in case nobody happens along in the next few minutes to look at that | 01:04 |
| *** rtjure has joined #openstack-infra | 01:05 | |
| *** ltomasbo has joined #openstack-infra | 01:06 | |
| *** jpena|off has joined #openstack-infra | 01:06 | |
| *** amoralej has joined #openstack-infra | 01:08 | |
| *** rtjure has quit IRC | 01:10 | |
| Zara | sure, thanks :) I wasn't sure if it was a zuul thing or a sb thing given the jobs might've run at different times | 01:10 |
| *** ijw has joined #openstack-infra | 01:11 | |
| *** jpena|off has quit IRC | 01:11 | |
| * Zara is tempted to make a story | 01:12 | |
| *** ltomasbo has quit IRC | 01:12 | |
| *** amoralej has quit IRC | 01:13 | |
| *** nhicher has quit IRC | 01:13 | |
| *** ansiwen[q] has quit IRC | 01:13 | |
| *** mriedem has joined #openstack-infra | 01:15 | |
| *** liujiong has joined #openstack-infra | 01:15 | |
| *** gyee has quit IRC | 01:17 | |
| tonyb | ianw: FWIW I've done the newton EOL's it looks like there were a couple of issues with repos having bad .gitreview files/ gerrit remotes I'll correct them after lunch and then move onto the old numeric releases | 01:18 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements https://review.openstack.org/503645 | 01:18 |
| *** ijw_ has joined #openstack-infra | 01:19 | |
| *** rtjure has joined #openstack-infra | 01:19 | |
| fungi | Zara: my understanding is that the draft job publishes a draft copy of the webclient to docs-draft, but those all connect back to a common api on the storyboard-dev server | 01:19 |
| *** gk_ has quit IRC | 01:19 | |
| fungi | so multiple parallel builds should never be a problem | 01:19 |
| *** Qiming has quit IRC | 01:20 | |
| *** jamielennox has quit IRC | 01:20 | |
| *** ltomasbo has joined #openstack-infra | 01:20 | |
| *** slaweq has quit IRC | 01:20 | |
| *** jpena|off has joined #openstack-infra | 01:20 | |
| *** amoralej has joined #openstack-infra | 01:20 | |
| *** ijw has quit IRC | 01:20 | |
| fungi | more likely the translated legacy job is just missing some necessary bit | 01:20 |
| *** ansiwen[q] has joined #openstack-infra | 01:21 | |
| Zara | yeah. I definitely haven't seen that issue before now-- just spent very little time investigating it so I didn't want to be too definite about it. I've added it to the etherpad. | 01:21 |
| Zara | 'night! :) | 01:21 |
| *** persia has quit IRC | 01:21 | |
| *** gus has quit IRC | 01:21 | |
| *** andreas_s has joined #openstack-infra | 01:21 | |
| *** gus has joined #openstack-infra | 01:22 | |
| *** persia has joined #openstack-infra | 01:22 | |
| *** gk_ has joined #openstack-infra | 01:22 | |
| *** jamielennox has joined #openstack-infra | 01:23 | |
| *** shu-mutou-AWAY has quit IRC | 01:23 | |
| *** rtjure has quit IRC | 01:23 | |
| *** nhicher has joined #openstack-infra | 01:25 | |
| *** sree has joined #openstack-infra | 01:27 | |
| *** Qiming has joined #openstack-infra | 01:28 | |
| *** mriedem has quit IRC | 01:28 | |
| *** rtjure has joined #openstack-infra | 01:28 | |
| *** salv-orlando has joined #openstack-infra | 01:28 | |
| *** slaweq has joined #openstack-infra | 01:29 | |
| gk_ | Hi everyone, our third-party cannot connect to gerrit since last saturday. Zuul raises this exception "SSHException: No existing session". I checked the public key set in gerrit and it is the same as in our server. Besides, I can connect to gerrit using the ssh command line but it failed with the paramiko client. Anyone has any ideas on how to fix it? Thanks in advance. | 01:29 |
| *** thorst has joined #openstack-infra | 01:29 | |
| *** andreas_s has quit IRC | 01:30 | |
| *** andreas_s has joined #openstack-infra | 01:31 | |
| *** sree has quit IRC | 01:32 | |
| *** rcernin has quit IRC | 01:32 | |
| *** rcernin has joined #openstack-infra | 01:32 | |
| *** salv-orlando has quit IRC | 01:33 | |
| *** Qiming has quit IRC | 01:33 | |
| *** rcernin has quit IRC | 01:33 | |
| *** rtjure has quit IRC | 01:33 | |
| *** thorst has quit IRC | 01:34 | |
| *** Qiming has joined #openstack-infra | 01:35 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements https://review.openstack.org/503645 | 01:37 |
| *** psachin has joined #openstack-infra | 01:37 | |
| *** Qiming has quit IRC | 01:39 | |
| *** rtjure has joined #openstack-infra | 01:40 | |
| *** Qiming has joined #openstack-infra | 01:40 | |
| clarkb | gk_: can you paste a full exception? | 01:41 |
| fungi | to somewhere like paste.openstack.org (not directly into this channel) | 01:41 |
| gk_ | clarkb: http://paste.openstack.org/show/624543/ | 01:42 |
| *** edmondsw has joined #openstack-infra | 01:43 | |
| clarkb | gk_: appare tly this can happen if it cannot find a useable key file so it is looking for an ssh agent session | 01:44 |
| clarkb | does your key have a passphrase? or is the key missing? | 01:44 |
| gk_ | clarkb: I don't have any passphrase and the key file is here | 01:45 |
| *** rtjure has quit IRC | 01:45 | |
| clarkb | maybe double check the zuul daemon is running as the expected user? | 01:46 |
| *** rlandy has quit IRC | 01:46 | |
| fungi | yeah, lack of read permission (or too loose permission as well under some circumstances) can cause it to get skipped | 01:47 |
| *** edmondsw has quit IRC | 01:47 | |
| gk_ | clarkb: The expected user is ok. About the permissions, I have read/write permission for this user. | 01:48 |
| *** annp has joined #openstack-infra | 01:48 | |
| clarkb | are the file contents a valid ssh private key? | 01:49 |
| clarkb | older versions if paramiko may also need rsa and not support ecdsa or other newer keh types | 01:49 |
| *** smatzek has joined #openstack-infra | 01:49 | |
| *** mat128 has quit IRC | 01:50 | |
| *** ijw_ has quit IRC | 01:50 | |
| *** wewe0901 has joined #openstack-infra | 01:51 | |
| gk_ | clarkb: -rw------- 1 zuul zuul 1679 Jun 14 20:37 /var/lib/zuul/ssh/id_rsa | 01:51 |
| *** mat128 has joined #openstack-infra | 01:51 | |
| *** kiennt26 has joined #openstack-infra | 01:51 | |
| gk_ | clarkb: it is a rsa key type | 01:52 |
| *** dhinesh_ has quit IRC | 01:52 | |
| *** esberglu has joined #openstack-infra | 01:52 | |
| *** mat128 has quit IRC | 01:53 | |
| clarkb | gk_: should it be in /var/lib/zuul/.ssh ? | 01:53 |
| *** kiennt26 has quit IRC | 01:53 | |
| *** rtjure has joined #openstack-infra | 01:53 | |
| *** kiennt26 has joined #openstack-infra | 01:54 | |
| gk_ | clarkb: in the zuul conf, it is set like this: sshkey=/var/lib/zuul/ssh/id_rsa | 01:54 |
| *** smatzek has quit IRC | 01:54 | |
| *** rcernin has joined #openstack-infra | 01:55 | |
| gk_ | clarkb: by using this command line, I can connect successfully to gerrit: ssh -p 29418 disco.ci@review.openstack.org -i /var/lib/zuul/ssh/id_rsa | 01:55 |
| *** esberglu has quit IRC | 01:57 | |
| *** hongbin has joined #openstack-infra | 01:57 | |
| *** andreas_s has quit IRC | 01:58 | |
| fungi | it's the connection count | 01:58 |
| *** rtjure has quit IRC | 01:58 | |
| clarkb | fungi: ? | 01:59 |
| fungi | gk_: the gerrit api believes it has 126 open connections from 113.196.154.248 | 01:59 |
| fungi | many of which are from the disco.ci account | 01:59 |
| fungi | we have a connlimit filter in place to reject new connections beyond 100 concurrent | 02:00 |
| clarkb | fungi: I wonder if they are all tcp connections made without getting ssh going due to this exception | 02:00 |
| fungi | entirely possible | 02:00 |
| fungi | i can close all open connections for that account and see what happens | 02:01 |
| gk_ | fungi: ok we can try it | 02:01 |
| fungi | it'll take a few minutes for all the close-connection calls to complete | 02:02 |
| *** slaweq has quit IRC | 02:02 | |
| fungi | looks like this same ip address is also being shared by other accounts | 02:02 |
| gk_ | fungi: ok please tell me when it is finished and I will start zuul to check if it happens again | 02:02 |
| openstackgerrit | zhongshengping proposed openstack-infra/openstack-zuul-jobs master: Remove xenial beaker jobs from gate pipeline for puppet modules https://review.openstack.org/514872 | 02:02 |
| *** andreas_s has joined #openstack-infra | 02:02 | |
| clarkb | gk_: if that doesnt address it then we may want to see you zuul config and the zuul process details (ps -elf probably works well enpugh) so that we can see if it is running as the correct user and has correct args to the process | 02:03 |
| fungi | gk_: go ahead and try again. there are no currently open connections for disco.ci though there are still 63 from that address for other accounts | 02:04 |
| clarkb | but reading the traceback I wonder if keyfile is None | 02:04 |
| *** slaweq has joined #openstack-infra | 02:04 | |
| fungi | i see 1 connection for disco.ci now | 02:05 |
| fungi | make that 2 | 02:05 |
| gk_ | fungi: it seems ok now. I don't see the exception | 02:06 |
| fungi | doesn't seem to have increased past 2 connections yet | 02:06 |
| *** gongysh has joined #openstack-infra | 02:06 | |
| fungi | odds are there's something flaky about the network from which you're originating which is preventing proper socket shutdown | 02:07 |
| *** andreas_s has quit IRC | 02:07 | |
| *** rtjure has joined #openstack-infra | 02:07 | |
| fungi | and that causes stale sockets to accumulate until it reaches the point where we begin refusing any subsequent connections | 02:08 |
| gk_ | fungi: ok, we have already asked the IT team to check the equipment and to change our network switch | 02:09 |
| gk_ | fungi: maybe the root cause is in our equipment. | 02:10 |
| *** bobh has joined #openstack-infra | 02:10 | |
| fungi | having worked in network engineering for very large environments for over a decade before i started doing this, i saw lots of examples of exactly what you're experiencing | 02:11 |
| fungi | mostly chalked up to oversubscribed nat devices | 02:11 |
| *** liujiong has quit IRC | 02:11 | |
| *** rtjure has quit IRC | 02:12 | |
| *** liujiong has joined #openstack-infra | 02:12 | |
| *** ltomasbo has quit IRC | 02:12 | |
| fungi | though there are also plenty of other potential causes from a network device side, like flow-based routing gone awry or other load distribution schemes sometimes sending flows down inappropriate paths | 02:12 |
| *** amoralej has quit IRC | 02:12 | |
| *** jpena|off has quit IRC | 02:12 | |
| *** yamamoto has joined #openstack-infra | 02:14 | |
| gk_ | fungi: ok thanks for your help and your advice. I will push our IT team to check it quickly | 02:14 |
| fungi | or state-tracking filters unceremoniously dropping flows they see as stale using unbelievably short timeout settings | 02:14 |
| gk_ | clarkb: thank you for help too | 02:14 |
| *** bobh has quit IRC | 02:15 | |
| fungi | also spontaneously rebooting firewalls can result in similar behavior | 02:15 |
| fungi | you're welcome, gk_ | 02:16 |
| * fungi disappears for the night | 02:17 | |
| fungi | oh, and i'm not around much tomorrow though i'll check back in when i wake up and remind everyone | 02:17 |
| *** rtjure has joined #openstack-infra | 02:18 | |
| tonyb | ianw: okay It seems that 2 repos failed but I have corrected them. I'll move onto the old numeric branches now | 02:18 |
| *** smatzek has joined #openstack-infra | 02:18 | |
| ianw | tonyb: ok ... how do they ave bad gitreviews? | 02:19 |
| ianw | no contribiutions? | 02:19 |
| *** jhesketh_ is now known as jhesketh | 02:22 | |
| *** Qiming has quit IRC | 02:22 | |
| *** rtjure has quit IRC | 02:23 | |
| tonyb | ianw: Yeah I guess so. It's pretty common with the deb-* repos but I also hit openstack/bareon | 02:24 |
| tonyb | oh bareon was my problem I had a stale gerrit remote from befoer: http://git.openstack.org/cgit/openstack/bareon/commit/.gitreview?h=newton-eol&id=e36577418f2deea7d140320c41085f7cc3d23864 | 02:27 |
| *** rtjure has joined #openstack-infra | 02:27 | |
| *** ltomasbo has joined #openstack-infra | 02:30 | |
| *** rtjure has quit IRC | 02:32 | |
| dmsimard | Huh, is that new ? E741 ambiguous variable name 'l' | 02:33 |
| dmsimard | Don't remember getting that error on pep8 | 02:33 |
| dmsimard | Damn you pep8, it's because of you I put silly variable names in the first place | 02:34 |
| *** andreas_s has joined #openstack-infra | 02:34 | |
| * dmsimard goes to watch some pep8 video by Raymond Hettinger | 02:34 | |
| tonyb | dmsimard: Yeah I'm preety sure it's new | 02:34 |
| *** slaweq has quit IRC | 02:37 | |
| *** rtjure has joined #openstack-infra | 02:37 | |
| *** tpsilva has quit IRC | 02:39 | |
| *** smatzek has quit IRC | 02:41 | |
| *** smatzek has joined #openstack-infra | 02:41 | |
| *** smatzek has quit IRC | 02:42 | |
| *** rtjure has quit IRC | 02:42 | |
| *** slaweq has joined #openstack-infra | 02:42 | |
| *** smatzek has joined #openstack-infra | 02:42 | |
| *** andreas_s has quit IRC | 02:43 | |
| *** ramishra has joined #openstack-infra | 02:44 | |
| *** thorst has joined #openstack-infra | 02:46 | |
| *** smatzek has quit IRC | 02:46 | |
| *** stakeda has joined #openstack-infra | 02:47 | |
| *** rtjure has joined #openstack-infra | 02:48 | |
| *** cody-somerville has joined #openstack-infra | 02:48 | |
| *** cody-somerville has quit IRC | 02:48 | |
| *** cody-somerville has joined #openstack-infra | 02:48 | |
| *** thorst has quit IRC | 02:50 | |
| *** baoli has joined #openstack-infra | 02:50 | |
| *** Qiming has joined #openstack-infra | 02:52 | |
| *** rtjure has quit IRC | 02:53 | |
| pabelanger | are we saying project-templates should be also in branches? Or should they stay in master branch? | 02:53 |
| *** baoli has quit IRC | 02:55 | |
| *** Qiming has quit IRC | 02:56 | |
| *** gildub has joined #openstack-infra | 02:57 | |
| *** rtjure has joined #openstack-infra | 02:57 | |
| *** daidv has joined #openstack-infra | 02:59 | |
| *** dave-mccowan has quit IRC | 02:59 | |
| *** namnh has joined #openstack-infra | 02:59 | |
| openstackgerrit | Kazunori Shinohara proposed openstack-infra/project-config master: Updated templates for heat-dashboard https://review.openstack.org/514895 | 03:02 |
| *** rtjure has quit IRC | 03:02 | |
| *** Qiming has joined #openstack-infra | 03:04 | |
| *** iyamahat has joined #openstack-infra | 03:06 | |
| *** yamahata has quit IRC | 03:08 | |
| *** rtjure has joined #openstack-infra | 03:10 | |
| kazsh | Hi team, could you someone review ^^^^^ thanks in advance. | 03:11 |
| *** andreas_s has joined #openstack-infra | 03:11 | |
| *** benj__ has joined #openstack-infra | 03:13 | |
| *** iyamahat has quit IRC | 03:14 | |
| *** rtjure has quit IRC | 03:15 | |
| *** Qiming has quit IRC | 03:15 | |
| *** slaweq has quit IRC | 03:15 | |
| *** esberglu has joined #openstack-infra | 03:20 | |
| *** slaweq has joined #openstack-infra | 03:21 | |
| *** rtjure has joined #openstack-infra | 03:22 | |
| openstackgerrit | Kazunori Shinohara proposed openstack-infra/project-config master: Updated templates for heat-dashboard https://review.openstack.org/514895 | 03:23 |
| *** dhinesh has joined #openstack-infra | 03:23 | |
| *** andreas_s has quit IRC | 03:24 | |
| *** esberglu has quit IRC | 03:25 | |
| *** rtjure has quit IRC | 03:26 | |
| *** dhinesh has quit IRC | 03:27 | |
| *** dhinesh has joined #openstack-infra | 03:28 | |
| *** andreas_s has joined #openstack-infra | 03:29 | |
| *** fanzhang has quit IRC | 03:29 | |
| *** salv-orlando has joined #openstack-infra | 03:29 | |
| *** edmondsw has joined #openstack-infra | 03:31 | |
| *** gk_ has quit IRC | 03:32 | |
| *** rtjure has joined #openstack-infra | 03:32 | |
| *** andreas_s has quit IRC | 03:33 | |
| *** salv-orlando has quit IRC | 03:33 | |
| *** fanzhang has joined #openstack-infra | 03:34 | |
| *** edmondsw has quit IRC | 03:35 | |
| tonyb | ianw: I'm done. with the numeric removals too. | 03:36 |
| *** rtjure has quit IRC | 03:37 | |
| tonyb | I'll generate a fresh report of all the stale things in about a month, and time permitting go to each prjects team with a list of (repo, branch) tuples and ask which ones can be dropped now and estimate the EOL date for the rest | 03:37 |
| ianw | tonyb: ok, both yourself & i are out of project bootstrappers again | 03:37 |
| tonyb | ianw: Thanks. | 03:38 |
| ianw | i think we made good progress :) more documentation and now we just need to flip a bit | 03:38 |
| tonyb | ianw: FWIW I dropped a summary onto the mailing list. | 03:38 |
| tonyb | ianw: Yeah. Should be much easier next time. | 03:38 |
| tonyb | infra-root: I did both EOL runs in a screen session with logging turned on so I can share the log if required | 03:39 |
| jhesketh | tonyb, ianw: sorry I didn't notice you guys were doing this today, I would have been happy to help | 03:39 |
| tonyb | jhesketh: np you did help by writing the eol_branch script :) | 03:40 |
| tonyb | jhesketh: the hardening stuff is really nice | 03:40 |
| jhesketh | tonyb: I've kept a copy of my screen sessions whenever I've ran the script (just locally) in case we ever discover any problems | 03:40 |
| jhesketh | I also usually run it with -x | 03:40 |
| jhesketh | yeah, it gets a little smarter each time :-) | 03:40 |
| ianw | "people can do the work, so machines have time to think" :) | 03:41 |
| tonyb | jhesketh: Yeah I shoudl have done that ... next time. | 03:41 |
| tonyb | jhesketh: I'll update my tools that nominate the repos to use -x also | 03:43 |
| *** Qiming has joined #openstack-infra | 03:44 | |
| jhesketh | -x is pretty ugly, but it's there if we ever need to debug some breakage | 03:44 |
| *** rtjure has joined #openstack-infra | 03:45 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move to a common lock-file directory https://review.openstack.org/514079 | 03:49 |
| *** rtjure has quit IRC | 03:50 | |
| *** andreas_s has joined #openstack-infra | 03:52 | |
| tonyb | jhesketh: Yeah it always is but as you say it helps a heap when debugging. | 03:54 |
| *** slaweq has quit IRC | 03:55 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move to a common lock-file directory https://review.openstack.org/514079 | 03:55 |
| *** rtjure has joined #openstack-infra | 03:56 | |
| *** Qiming has quit IRC | 04:00 | |
| *** hongbin has quit IRC | 04:00 | |
| *** andreas_s has quit IRC | 04:00 | |
| *** rtjure has quit IRC | 04:01 | |
| *** yamahata has joined #openstack-infra | 04:02 | |
| *** sree has joined #openstack-infra | 04:02 | |
| *** slaweq has joined #openstack-infra | 04:06 | |
| *** kiennt26 has quit IRC | 04:07 | |
| *** claudiub|2 has joined #openstack-infra | 04:08 | |
| *** rtjure has joined #openstack-infra | 04:08 | |
| *** Qiming has joined #openstack-infra | 04:08 | |
| *** rtjure has quit IRC | 04:12 | |
| *** ykarel|away has joined #openstack-infra | 04:16 | |
| *** esberglu has joined #openstack-infra | 04:16 | |
| *** rtjure has joined #openstack-infra | 04:18 | |
| openstackgerrit | Cao Xuan Hoang proposed openstack-infra/project-config master: Add docs job for VPNaaS and VPNaaS dashboard https://review.openstack.org/514906 | 04:18 |
| *** esberglu has quit IRC | 04:20 | |
| *** rcernin has quit IRC | 04:21 | |
| *** rtjure has quit IRC | 04:22 | |
| *** iyamahat has joined #openstack-infra | 04:23 | |
| *** rtjure has joined #openstack-infra | 04:27 | |
| *** coolsvap has joined #openstack-infra | 04:32 | |
| *** rtjure has quit IRC | 04:32 | |
| *** huanxie has quit IRC | 04:34 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements https://review.openstack.org/503645 | 04:35 |
| *** dhajare has joined #openstack-infra | 04:37 | |
| *** slaweq has quit IRC | 04:39 | |
| *** rtjure has joined #openstack-infra | 04:40 | |
| *** rcernin has joined #openstack-infra | 04:45 | |
| *** rtjure has quit IRC | 04:45 | |
| *** dhajare has quit IRC | 04:46 | |
| *** slaweq has joined #openstack-infra | 04:47 | |
| *** verdurin_ has joined #openstack-infra | 04:49 | |
| *** rtjure has joined #openstack-infra | 04:49 | |
| *** verdurin has quit IRC | 04:50 | |
| *** thorst has joined #openstack-infra | 04:51 | |
| *** rtjure has quit IRC | 04:54 | |
| *** udesale has joined #openstack-infra | 04:54 | |
| *** alex_xu has quit IRC | 04:56 | |
| *** thorst has quit IRC | 04:56 | |
| *** felipemonteiro_ has joined #openstack-infra | 04:57 | |
| *** mummer has joined #openstack-infra | 04:59 | |
| *** alex_xu has joined #openstack-infra | 04:59 | |
| *** rtjure has joined #openstack-infra | 04:59 | |
| *** mummer has quit IRC | 04:59 | |
| AJaeger | ianw: do you want to +2A the jenkins v2 removal? https://review.openstack.org/#/c/507180/ | 04:59 |
| *** felipemonteiro_ has quit IRC | 05:02 | |
| *** huanxie has joined #openstack-infra | 05:03 | |
| *** rtjure has quit IRC | 05:04 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add success message for check pipeline https://review.openstack.org/514870 | 05:04 |
| *** janki has joined #openstack-infra | 05:05 | |
| *** iyamahat has quit IRC | 05:06 | |
| *** ykarel|away has quit IRC | 05:06 | |
| *** jaosorior has quit IRC | 05:07 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Newton removal: zuul.d/jobs.yaml https://review.openstack.org/514909 | 05:07 |
| *** iyamahat has joined #openstack-infra | 05:09 | |
| *** rtjure has joined #openstack-infra | 05:10 | |
| openstackgerrit | Tomasz Trębski proposed openstack-infra/project-config master: Remove legacy job for monasca https://review.openstack.org/513722 | 05:11 |
| *** LindaWang has quit IRC | 05:12 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Newton removal: Remove periodic-newton https://review.openstack.org/514910 | 05:12 |
| *** LindaWang has joined #openstack-infra | 05:13 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Create job-output.txt together with JobDir https://review.openstack.org/514617 | 05:13 |
| *** rtjure has quit IRC | 05:15 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Newton removal: Remove periodic-newton https://review.openstack.org/514911 | 05:15 |
| *** dhajare has joined #openstack-infra | 05:15 | |
| openstackgerrit | Rico Lin proposed openstack-infra/project-config master: Remove legacy jobs from heat https://review.openstack.org/509194 | 05:16 |
| *** salv-orlando has joined #openstack-infra | 05:18 | |
| *** slaweq has quit IRC | 05:19 | |
| *** rtjure has joined #openstack-infra | 05:20 | |
| *** ykarel|away has joined #openstack-infra | 05:23 | |
| *** jaosorior has joined #openstack-infra | 05:23 | |
| *** rtjure has quit IRC | 05:24 | |
| *** slaweq has joined #openstack-infra | 05:26 | |
| AJaeger | config-core, please review my newton cleanup changes https://review.openstack.org/#/q/status:open+topic:newton-eol | 05:26 |
| *** rtjure has joined #openstack-infra | 05:30 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Newton removal: Remove some stable/newton https://review.openstack.org/514915 | 05:31 |
| *** gongysh has quit IRC | 05:31 | |
| chandankumar | AJaeger: please have a look https://review.openstack.org/#/c/514269/ i want to make this job failing | 05:32 |
| *** armaan has joined #openstack-infra | 05:32 | |
| *** cloudnull has quit IRC | 05:32 | |
| *** eglute has quit IRC | 05:33 | |
| *** cshastri has joined #openstack-infra | 05:33 | |
| *** eglute has joined #openstack-infra | 05:34 | |
| AJaeger | chandankumar: sorry, can't help this morning ;( | 05:34 |
| *** cloudnull has joined #openstack-infra | 05:34 | |
| *** rtjure has quit IRC | 05:35 | |
| *** tushar has joined #openstack-infra | 05:35 | |
| *** xinliang has joined #openstack-infra | 05:36 | |
| *** rtjure has joined #openstack-infra | 05:40 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Newton EOl: Remove trove legacy jobs https://review.openstack.org/514916 | 05:41 |
| ianw | AJaeger: I dunno, is it ceremonial? | 05:43 |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config master: tap-as-a-service: Use -horizon variant of templates https://review.openstack.org/514917 | 05:43 |
| AJaeger | ianw: we tagged yesterday, so I added my +2A now on top of yours ;) | 05:44 |
| AJaeger | I rebased so often that I didn't want to be the 2nd reviewer ;) | 05:44 |
| AJaeger | thanks, ianw | 05:44 |
| ianw | the end of a era :) | 05:44 |
| AJaeger | ianw: yeah - and thansk for the end of another one - the big branch cleanup... | 05:45 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Newton-EOL: Remove trove legacy jobs https://review.openstack.org/514918 | 05:47 |
| *** cloudnull has quit IRC | 05:48 | |
| *** cloudnull has joined #openstack-infra | 05:50 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove zuul v2 and jjb content https://review.openstack.org/507180 | 05:55 |
| openstackgerrit | Merged openstack-infra/project-config master: Run pep8 everywhere https://review.openstack.org/513953 | 05:55 |
| *** slaweq has quit IRC | 05:59 | |
| *** gongysh has joined #openstack-infra | 06:00 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Document executor/merger stats https://review.openstack.org/514343 | 06:02 |
| *** armaan_ has joined #openstack-infra | 06:03 | |
| *** esberglu has joined #openstack-infra | 06:03 | |
| *** umbarkar has joined #openstack-infra | 06:04 | |
| *** tushar has quit IRC | 06:04 | |
| *** armaan has quit IRC | 06:04 | |
| *** armaan has joined #openstack-infra | 06:05 | |
| *** armaan_ has quit IRC | 06:06 | |
| *** kiennt26 has joined #openstack-infra | 06:07 | |
| *** gongysh has quit IRC | 06:07 | |
| *** slaweq has joined #openstack-infra | 06:07 | |
| *** esberglu has quit IRC | 06:07 | |
| *** alex_xu has quit IRC | 06:09 | |
| *** andreas_s has joined #openstack-infra | 06:11 | |
| *** Qiming has quit IRC | 06:11 | |
| *** yolanda_ has joined #openstack-infra | 06:12 | |
| *** vsaienk0 has joined #openstack-infra | 06:12 | |
| *** bhavik1 has joined #openstack-infra | 06:13 | |
| *** gouthamr has quit IRC | 06:14 | |
| *** alex_xu has joined #openstack-infra | 06:15 | |
| *** jchhatbar has joined #openstack-infra | 06:27 | |
| *** iyamahat has quit IRC | 06:29 | |
| *** janki has quit IRC | 06:29 | |
| *** tosky has joined #openstack-infra | 06:36 | |
| *** pcaruana has joined #openstack-infra | 06:39 | |
| *** csomerville has joined #openstack-infra | 06:40 | |
| *** yolanda_ has quit IRC | 06:40 | |
| *** slaweq has quit IRC | 06:40 | |
| *** ykarel|away is now known as ykarel | 06:40 | |
| *** umbarkar has quit IRC | 06:40 | |
| openstackgerrit | Masayuki Igawa proposed openstack-infra/subunit2sql master: Switch to use stestr completely https://review.openstack.org/514933 | 06:41 |
| openstackgerrit | Dirk Mueller proposed openstack-dev/hacking master: Transition to flake8 2.6.x https://review.openstack.org/514934 | 06:42 |
| *** cody-somerville has quit IRC | 06:43 | |
| *** dhinesh has quit IRC | 06:43 | |
| *** salv-orlando has quit IRC | 06:44 | |
| *** salv-orlando has joined #openstack-infra | 06:45 | |
| AJaeger | #status log zuul v2/jenkins config has been removed from project-config | 06:45 |
| openstackstatus | AJaeger: finished logging | 06:45 |
| *** jchhatbar is now known as janki | 06:47 | |
| *** salv-orlando has quit IRC | 06:49 | |
| *** slaweq has joined #openstack-infra | 06:49 | |
| *** salv-orlando has joined #openstack-infra | 06:49 | |
| *** florianf has joined #openstack-infra | 06:52 | |
| *** thorst has joined #openstack-infra | 06:53 | |
| *** ralonsoh has joined #openstack-infra | 06:53 | |
| *** tesseract has joined #openstack-infra | 06:54 | |
| openstackgerrit | Masayuki Igawa proposed openstack-infra/subunit2sql master: Switch to use stestr completely https://review.openstack.org/514933 | 06:55 |
| *** thorst has quit IRC | 06:57 | |
| *** gcb has joined #openstack-infra | 07:00 | |
| *** pgadiya has joined #openstack-infra | 07:02 | |
| *** gildub has quit IRC | 07:03 | |
| *** rossella_s has joined #openstack-infra | 07:06 | |
| *** jtomasek has joined #openstack-infra | 07:09 | |
| *** armax has quit IRC | 07:09 | |
| *** spectr has joined #openstack-infra | 07:13 | |
| *** esberglu has joined #openstack-infra | 07:16 | |
| tonyb | Is there an API (or similar to find all the jobs and templates associated with a project (like used to be in layout.yaml) In not I guess I need to process project-config/zuul.d and $project/.zuul.yaml $project/.zuul.d anywheer else? | 07:17 |
| openstackgerrit | Jens Harbott (frickler) proposed openstack-infra/infra-manual master: Clarify which jobs belong in the project block https://review.openstack.org/513197 | 07:17 |
| tonyb | IIUC openstack-zuul-jobs and zull-jobs are the definitions but the list of the jobs run will come from ^^ | 07:18 |
| *** esberglu has quit IRC | 07:21 | |
| *** slaweq has quit IRC | 07:21 | |
| *** Qiming has joined #openstack-infra | 07:21 | |
| *** slaweq has joined #openstack-infra | 07:23 | |
| *** Qiming has quit IRC | 07:26 | |
| *** Hal has joined #openstack-infra | 07:28 | |
| *** Hal is now known as Guest97785 | 07:28 | |
| *** gongysh has joined #openstack-infra | 07:30 | |
| AJaeger | tonyb: list of jobs is $project/.zuul.yaml, $project/zuul.yaml, $project/zuul.d/* $project/.zuul.d AFAIK plus the jobs defined in openstack-zuul-jobs, project-config and zuul-jobs. Each of these repos can define new jobs and use them | 07:31 |
| AJaeger | tonyb: list of jobs that run in a repo comes from only two places: The repo itself and project-config. | 07:32 |
| vsaienk0 | AJaeger: please add to your review queue https://review.openstack.org/#/c/514596/ - don't launch ironic-python-agent legacy jobs. We are pretty close in landing https://review.openstack.org/#/c/513380 | 07:33 |
| AJaeger | So, rereading your question: Add the two other in-tree places and you're fine ;) | 07:33 |
| AJaeger | vsaienk0: LGTM. Please push a change for openstack-zuul-jobs to remove them as well. | 07:36 |
| AJaeger | bbl | 07:37 |
| *** andreas_s has quit IRC | 07:38 | |
| *** andreas_s has joined #openstack-infra | 07:39 | |
| mtreinish | infra-root: is there a reason zuul hsan't merged this: https://review.openstack.org/#/c/491827/ we've tried a few rechecks and it comes back with a +2 from the gate | 07:41 |
| mtreinish | but never merges | 07:41 |
| vsaienk0 | AJaeger: thanks will prepare a patch | 07:42 |
| *** yolanda_ has joined #openstack-infra | 07:42 | |
| *** yolanda_ is now known as yolanda | 07:43 | |
| *** Qiming has joined #openstack-infra | 07:43 | |
| *** andreas_s has quit IRC | 07:44 | |
| *** dhinesh has joined #openstack-infra | 07:44 | |
| *** amoralej has joined #openstack-infra | 07:45 | |
| *** jpena has joined #openstack-infra | 07:46 | |
| *** slaweq_ has joined #openstack-infra | 07:48 | |
| *** dhinesh has quit IRC | 07:48 | |
| *** ethfci has quit IRC | 07:50 | |
| AJaeger | mtreinish: I suggest to rebase and +2A again, the parent is outdated (orange mark). | 07:51 |
| *** salv-orlando has quit IRC | 07:51 | |
| *** salv-orlando has joined #openstack-infra | 07:52 | |
| mtreinish | AJaeger: well it's not a super important change, and if it's exposing a potential bug in zuul do you think there is value in leaving it as is until that's figured out? | 07:52 |
| mtreinish | because it should be a clean rebase so there shouldn't be a merge conflict so I'd expect everything to just work | 07:53 |
| *** ykarel is now known as ykarel|lunch | 07:53 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/subunit2sql master: Update python3 versions in tox.ini envlist https://review.openstack.org/491827 | 07:53 |
| *** andreas_s has joined #openstack-infra | 07:53 | |
| *** esberglu has joined #openstack-infra | 07:53 | |
| *** esberglu has quit IRC | 07:54 | |
| *** Goneri has joined #openstack-infra | 07:54 | |
| *** esberglu has joined #openstack-infra | 07:54 | |
| *** esberglu has quit IRC | 07:54 | |
| *** gmann is now known as gmann_afk | 07:54 | |
| *** slaweq has quit IRC | 07:56 | |
| *** Qiming has quit IRC | 07:56 | |
| *** salv-orlando has quit IRC | 07:56 | |
| *** gcb has quit IRC | 07:57 | |
| *** armaan has quit IRC | 07:59 | |
| *** ethfci has joined #openstack-infra | 08:00 | |
| *** armaan has joined #openstack-infra | 08:00 | |
| *** ccamacho has joined #openstack-infra | 08:00 | |
| *** martinkopec has joined #openstack-infra | 08:01 | |
| *** slaweq has joined #openstack-infra | 08:02 | |
| *** salv-orlando has joined #openstack-infra | 08:05 | |
| *** slaweq has quit IRC | 08:06 | |
| *** amoralej has left #openstack-infra | 08:07 | |
| *** amoralej has joined #openstack-infra | 08:08 | |
| amoralej | hi, we are having some issues in CI jobs which seems to be related to how repos in PROJECTS are cloned in CI nodes | 08:08 |
| amoralej | it seems git remote does not longer exist in those repos | 08:09 |
| amoralej | does it make sense? | 08:09 |
| *** martinkopec has quit IRC | 08:09 | |
| *** Qiming has joined #openstack-infra | 08:09 | |
| *** martinkopec has joined #openstack-infra | 08:10 | |
| *** ralonsoh has quit IRC | 08:11 | |
| *** ralonsoh has joined #openstack-infra | 08:13 | |
| *** Qiming has quit IRC | 08:14 | |
| *** thorst has joined #openstack-infra | 08:14 | |
| *** dbecker has joined #openstack-infra | 08:18 | |
| *** danpawlik has quit IRC | 08:18 | |
| *** ralonsoh has quit IRC | 08:18 | |
| *** danpawlik has joined #openstack-infra | 08:18 | |
| *** danpawlik has quit IRC | 08:18 | |
| *** danpawlik has joined #openstack-infra | 08:19 | |
| *** thorst has quit IRC | 08:19 | |
| *** bhavik1 has quit IRC | 08:19 | |
| *** bcafarel has quit IRC | 08:20 | |
| AJaeger | mtreinish: this would be a gerrit issue AFAIK, let's see whether it merges now | 08:20 |
| *** namnh_ has joined #openstack-infra | 08:21 | |
| AJaeger | amoralej: yes, that's correct. How is that causing you a problem? | 08:21 |
| AJaeger | the checked out tree should have all the branches you need | 08:21 |
| *** apevec has joined #openstack-infra | 08:21 | |
| amoralej | we use output of "git remote -v" in some jobs to get the name of the upstream repo | 08:22 |
| amoralej | which we need to build packages on the fly | 08:22 |
| amoralej | so probably we need to find other way | 08:22 |
| openstackgerrit | Tomasz Trębski proposed openstack-infra/project-config master: Enable documentation building for monascaclient https://review.openstack.org/514954 | 08:23 |
| *** namnh has quit IRC | 08:23 | |
| openstackgerrit | Tomasz Trębski proposed openstack-infra/project-config master: Enable documentation building for monascaclient https://review.openstack.org/514954 | 08:24 |
| apevec | amoralej, what did I miss? | 08:24 |
| apevec | ok, I see on eavesdrop, no replies yet | 08:25 |
| amoralej | apevec, so the problem is that with zuulv3, when repos are cloned in ci servers, the git remote doesn't exist | 08:25 |
| openstackgerrit | Yaguo Zhou proposed openstack-infra/subunit2sql master: Replace DbMigrationError with DBMigrationError https://review.openstack.org/500858 | 08:25 |
| *** ralonsoh has joined #openstack-infra | 08:25 | |
| amoralej | and we use it in DLRN | 08:25 |
| amoralej | to figure out the name of the upstream repo | 08:25 |
| amoralej | AJaeger just confirmed that | 08:25 |
| apevec | jpena, ^ where is that, any other ideas? | 08:25 |
| amoralej | AJaeger, is there any configuration that can be used to change that behavior ? | 08:26 |
| amoralej | we may use directory name | 08:26 |
| apevec | amoralej, but the other thing is why those puppet-* co-gated with tripleoci change, I'd like to understand that | 08:26 |
| apevec | http://zuulv3.openstack.org/ gate queue tripleo looks very bad | 08:26 |
| *** s-shiono has quit IRC | 08:26 | |
| amoralej | apevec, zuul "magically" relates revies in gate pipeline | 08:26 |
| jpena | apevec: it's in https://github.com/softwarefactory-project/DLRN/blob/master/scripts/build_rpm.sh#L101-L105 | 08:27 |
| amoralej | s/reviews/reviews/ | 08:27 |
| apevec | it's not magic | 08:27 |
| amoralej | i never understood that relations | 08:27 |
| apevec | IIUC it is co-gated when sharing gate jobs | 08:27 |
| apevec | so let's first understand how all those projects ended up in tripleo gate queue | 08:27 |
| jpena | apevec: git remote -v is used to find the project name for puppet modules | 08:28 |
| amoralej | apevec, anyway we need to fix the issue with git remote also | 08:28 |
| apevec | jpena, aah, special case for puppet | 08:28 |
| AJaeger | apevec: if you have shared queue (check queue: paremeter in project-config), you share a queue. Also, if you have common jobs you're gating on. | 08:28 |
| AJaeger | jpena, apevec, apevec, I cannot help much right now, best to wait for rest of the team to discuss later. | 08:29 |
| apevec | AJaeger, thanks, we need to fix dlrn first | 08:29 |
| apevec | jpena, let's use dirname instead of remote? | 08:30 |
| AJaeger | amoralej: best explain later your exact use case, we might have other means to support that... | 08:30 |
| amoralej | ok, thanks AJaeger | 08:30 |
| *** eumel8 has joined #openstack-infra | 08:31 | |
| frickler | AJaeger: requirements gate failure because it seems to be referring to the now removed zuul/layout.yaml : http://logs.openstack.org/02/511202/9/gate/legacy-requirements-tox-validate-projects/38f987d/job-output.txt.gz#_2017-10-25_06_59_00_640788 | 08:31 |
| *** tosky has quit IRC | 08:31 | |
| eumel8 | morning | 08:31 |
| frickler | config-core: ^^ I guess I'll propose a revert until that gets fixed | 08:31 |
| openstackgerrit | Jens Harbott (frickler) proposed openstack-infra/project-config master: Revert "Remove zuul v2 and jjb content" https://review.openstack.org/514957 | 08:33 |
| *** lucas-afk is now known as lucasagomes | 08:34 | |
| *** electrofelix has joined #openstack-infra | 08:34 | |
| *** slaweq has joined #openstack-infra | 08:36 | |
| *** kjackal_ has joined #openstack-infra | 08:37 | |
| AJaeger | frickler: can't we fix the job? | 08:37 |
| ianw | frickler: urgh, is it hard to fix? | 08:37 |
| frickler | not sure, it is used in requirements/openstack_requirements/project_config.py, will have to look at how to get this data otherwise | 08:38 |
| *** Qiming has joined #openstack-infra | 08:39 | |
| AJaeger | frickler: that one is broken anyhow, since we do not touch layout.yaml anymore. | 08:40 |
| AJaeger | frickler: So, either fix in requirements - or make non-voting. | 08:40 |
| AJaeger | prometheanfire, dirk, any suggestions? ^ | 08:40 |
| AJaeger | frickler: the file needs to parse zuul.d/projects.yaml instead | 08:42 |
| *** dingyichen has quit IRC | 08:43 | |
| *** Qiming has quit IRC | 08:44 | |
| frickler | o.k., so make the job non-voting for now and then make it work with zuul v3 config seems like a sane approach | 08:44 |
| *** dhinesh has joined #openstack-infra | 08:45 | |
| *** derekh has joined #openstack-infra | 08:46 | |
| openstackgerrit | Jens Harbott (frickler) proposed openstack-infra/project-config master: Make legacy-requirements-tox-validate-projects non-voting until fixed https://review.openstack.org/514958 | 08:49 |
| *** dhinesh has quit IRC | 08:49 | |
| AJaeger | prometheanfire, dirk, tonyb ^ | 08:50 |
| frickler | woa, that revert has so many lint errors it would have needed a force-merge anyway it seems :-/ | 08:51 |
| *** namnh has joined #openstack-infra | 08:51 | |
| *** Qiming has joined #openstack-infra | 08:52 | |
| AJaeger | frickler: the pep8 change merged after it, we would need to merge that as well | 08:52 |
| AJaeger | we didn't lint those files before | 08:52 |
| *** namnh_ has quit IRC | 08:53 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Check that zuul.d/projects.yaml is sorted https://review.openstack.org/514061 | 08:53 |
| AJaeger | trivial linter test for review, please ^ | 08:53 |
| *** ykarel|lunch is now known as ykarel | 08:56 | |
| *** Qiming has quit IRC | 08:56 | |
| *** Qiming has joined #openstack-infra | 08:56 | |
| *** tosky has joined #openstack-infra | 08:57 | |
| *** yamahata has quit IRC | 08:58 | |
| *** gildub has joined #openstack-infra | 08:58 | |
| *** rcernin has quit IRC | 08:59 | |
| *** Qiming has quit IRC | 09:01 | |
| *** yamamoto has quit IRC | 09:05 | |
| *** slaweq has quit IRC | 09:09 | |
| openstackgerrit | Merged openstack-infra/project-config master: Make legacy-requirements-tox-validate-projects non-voting until fixed https://review.openstack.org/514958 | 09:10 |
| *** yamamoto has joined #openstack-infra | 09:12 | |
| jd_ | howdy! | 09:13 |
| *** Qiming has joined #openstack-infra | 09:15 | |
| *** jascott1 has quit IRC | 09:15 | |
| *** jascott1 has joined #openstack-infra | 09:15 | |
| jd_ | question about zuul v3: in https://review.openstack.org/#/c/512700/ I remove the jobs ceilometer-tox-py27-mongodb and others, but Zuul complains about not finding the playbook, which is weird since I don't want those tests to run and I wonder where they are still referenced | 09:15 |
| *** armaan has quit IRC | 09:16 | |
| *** armaan has joined #openstack-infra | 09:16 | |
| *** armaan has quit IRC | 09:16 | |
| *** vsaienk0 has quit IRC | 09:16 | |
| *** Qiming has quit IRC | 09:19 | |
| *** jpena is now known as jpena|off | 09:20 | |
| *** sdague has joined #openstack-infra | 09:20 | |
| *** jascott1 has quit IRC | 09:20 | |
| *** vsaienk0 has joined #openstack-infra | 09:21 | |
| *** jpena|off is now known as jpena | 09:21 | |
| *** slaweq has joined #openstack-infra | 09:21 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/release-tools master: Add some notes on eol_branch.sh https://review.openstack.org/514846 | 09:22 |
| ianw | jd_: that's interesting and i bet due to the way the changes are stacked | 09:35 |
| eumel8 | hello yolanda, infra-core! We're still looking forward for a merge to move on Zanata upgrade: https://review.openstack.org/#/c/506795/ | 09:35 |
| *** rcarrill1 is now known as rcarrillocruz | 09:48 | |
| *** yamamoto has quit IRC | 09:49 | |
| *** rpittau_ has quit IRC | 09:49 | |
| jd_ | ianw: hmmm anything I can do to "fix" it? | 09:50 |
| *** e0ne has joined #openstack-infra | 09:53 | |
| *** slaweq has quit IRC | 09:55 | |
| *** rossella_s has quit IRC | 09:57 | |
| *** slaweq has joined #openstack-infra | 10:02 | |
| openstackgerrit | Jens Harbott (frickler) proposed openstack-infra/project-config master: Revert "Make legacy-requirements-tox-validate-projects non-voting until fixed" https://review.openstack.org/515007 | 10:03 |
| tosky | AJaeger: question about migrating jobs in-tree: sahara-dashboards has 3 jobs to be migrated; 2 of them are failing in project-config (so they would require a fix) but I have a working version in-tree | 10:03 |
| tosky | AJaeger: can I migrate only those 2 jobs, leaving the 3rd one still as legacy in project-config for now (I'm aiming at a clear conversion), or should just fix the 2 jobs in project-config and then do the migration in one shot? | 10:04 |
| tosky | the question is more: which of the two would you approve and which would you -2? :D | 10:04 |
| AJaeger | tosky: I would say, let's be pragmatic: REmove the jobs first in project-config, then merge the change for sahara-dashboard | 10:05 |
| AJaeger | tosky: we prefer the other order so that you have full coverage on testing but in this case it'S not worth it IMHO | 10:05 |
| AJaeger | So, go ahead migrate those two jobs for now, and let'S do the 3rd later... | 10:05 |
| tosky | ack, thanks | 10:06 |
| *** cuongnv has quit IRC | 10:06 | |
| tosky | I think I already said that, but I'm trying in general to not import converted jobs but to directly rewrite them natively | 10:06 |
| tosky | but it's taking a while :) | 10:07 |
| *** kiennt26 has quit IRC | 10:07 | |
| AJaeger | tosky: recommendation is to import and then convert - but your call ;) | 10:09 |
| *** Qiming has joined #openstack-infra | 10:09 | |
| tosky | I know, and I will be do if I can't do in time, but I hope to finish before other people even start :) | 10:10 |
| tosky | for example, the tempest jobs are stuck on the reviews for devstack-tempest | 10:10 |
| tosky | a job which publish some artefacts is blocked by the proposed change to the way publishing works | 10:10 |
| AJaeger | tosky: best ask the rest of team later for help ... | 10:11 |
| tosky | yep, already did - and commented on bug on andreaf's patches | 10:11 |
| tosky | I just had a pass on the native v3 sahara-scenario job, and now I have a question about multinode jobs (I see the logs for all services on both nodes) | 10:12 |
| tosky | which should not probably happen | 10:12 |
| tosky | so I'm waiting to catch people | 10:12 |
| *** andreas_s has quit IRC | 10:13 | |
| *** liujiong has quit IRC | 10:13 | |
| AJaeger | understood | 10:13 |
| *** udesale has quit IRC | 10:13 | |
| openstackgerrit | Luigi Toscano proposed openstack-infra/project-config master: sahara-dashboard: remove the migrated unittests https://review.openstack.org/515015 | 10:21 |
| *** annp has quit IRC | 10:24 | |
| *** LindaWang has quit IRC | 10:27 | |
| openstackgerrit | Luigi Toscano proposed openstack-infra/openstack-zuul-jobs master: sahara-dashboard: remove the migrated unittests https://review.openstack.org/515019 | 10:30 |
| *** salv-orlando has quit IRC | 10:32 | |
| *** salv-orlando has joined #openstack-infra | 10:33 | |
| *** andreas_s has joined #openstack-infra | 10:34 | |
| *** ldnunes has joined #openstack-infra | 10:35 | |
| *** slaweq has quit IRC | 10:36 | |
| *** pcaruana has quit IRC | 10:37 | |
| *** sree has quit IRC | 10:38 | |
| ianw | jd_: sorry, i'd probably bring it up with jeblair. since both patches touch the jobs: maybe there's something funny going on | 10:38 |
| *** sree has joined #openstack-infra | 10:38 | |
| *** namnh has quit IRC | 10:38 | |
| *** slaweq has joined #openstack-infra | 10:39 | |
| *** gmann_afk is now known as gmann | 10:40 | |
| *** pbourke has quit IRC | 10:40 | |
| *** pbourke has joined #openstack-infra | 10:42 | |
| *** sree has quit IRC | 10:43 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Do not rely on openstack-puppet-modules in Queens and beyond https://review.openstack.org/512783 | 10:43 |
| *** andreas_s has quit IRC | 10:45 | |
| *** priteau has joined #openstack-infra | 10:45 | |
| *** boden has joined #openstack-infra | 10:48 | |
| *** tpsilva has joined #openstack-infra | 10:49 | |
| *** yamamoto has joined #openstack-infra | 10:49 | |
| *** andreas_s has joined #openstack-infra | 10:51 | |
| *** nicolasbock has joined #openstack-infra | 10:55 | |
| *** andreas_s has quit IRC | 10:55 | |
| *** andreas_s has joined #openstack-infra | 10:55 | |
| frickler | cool, merging the patch to remove the failing test seems to have changed https://review.openstack.org/511202 back from failing to passing in the gate queue without resubmitting. | 10:55 |
| *** ramishra has quit IRC | 11:00 | |
| *** yamamoto has quit IRC | 11:01 | |
| *** dtantsur|afk is now known as dtantsur | 11:01 | |
| *** armaan has joined #openstack-infra | 11:02 | |
| *** sree has joined #openstack-infra | 11:02 | |
| *** pgadiya has quit IRC | 11:02 | |
| *** ramishra has joined #openstack-infra | 11:02 | |
| openstackgerrit | Merged openstack-infra/subunit2sql master: Update python3 versions in tox.ini envlist https://review.openstack.org/491827 | 11:03 |
| *** verdurin_ is now known as verdurin | 11:05 | |
| *** huanxie has quit IRC | 11:05 | |
| odyssey4me | Are there any project-config cores available to review https://review.openstack.org/514592 please. The legacy jobs aren't working, so we'd like to get them cleared out. | 11:07 |
| *** sree has quit IRC | 11:07 | |
| odyssey4me | Well, some are working, some are not - but we have replacements in play already - either merged or proposed... so it's time to clean up. | 11:07 |
| *** stakeda has quit IRC | 11:07 | |
| *** huanxie has joined #openstack-infra | 11:08 | |
| *** pgadiya has joined #openstack-infra | 11:10 | |
| *** slaweq has quit IRC | 11:11 | |
| *** jkilpatr has quit IRC | 11:11 | |
| *** gildub has quit IRC | 11:12 | |
| *** LindaWang has joined #openstack-infra | 11:12 | |
| *** yamamoto has joined #openstack-infra | 11:14 | |
| *** armaan has quit IRC | 11:17 | |
| *** armaan has joined #openstack-infra | 11:17 | |
| *** dbecker has quit IRC | 11:19 | |
| *** yolanda has quit IRC | 11:19 | |
| *** lucasagomes is now known as lucas-hungry | 11:20 | |
| *** bobh has joined #openstack-infra | 11:20 | |
| *** thorst has joined #openstack-infra | 11:21 | |
| *** bobh has quit IRC | 11:23 | |
| *** slaweq has joined #openstack-infra | 11:23 | |
| *** thorst has quit IRC | 11:26 | |
| AJaeger | odyssey4me: +3 | 11:26 |
| odyssey4me | thanks AJaeger | 11:28 |
| *** smatzek has joined #openstack-infra | 11:28 | |
| odyssey4me | heh, and ianw :) | 11:28 |
| odyssey4me | MOAR WORKFLOW! | 11:28 |
| AJaeger | odyssey4me: you'll get the revert merged automatically now ;) | 11:29 |
| odyssey4me | AJaeger I'm not sure what you mean? Which revert? | 11:30 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove remaining legacy openstack-ansible jobs https://review.openstack.org/514592 | 11:30 |
| *** dave-mccowan has joined #openstack-infra | 11:31 | |
| *** rossella_s has joined #openstack-infra | 11:33 | |
| *** bobh has joined #openstack-infra | 11:38 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/openstack-zuul-jobs master: Remove remaining legacy openstack-ansible tests https://review.openstack.org/512635 | 11:40 |
| *** Guest97785 has quit IRC | 11:41 | |
| *** mkolesni has joined #openstack-infra | 11:42 | |
| *** wolverineav has joined #openstack-infra | 11:43 | |
| *** pcaruana has joined #openstack-infra | 11:43 | |
| mkolesni | howdy, does anyone know why 'recheck' doesnt kill the currently running zuul job? | 11:43 |
| mkolesni | i have one thats definitely going to fail and issued 'recheck' on the change, but it didnt seem to do anything | 11:44 |
| mkolesni | i remember in the old zuul it used to work like this | 11:44 |
| *** mat128 has joined #openstack-infra | 11:45 | |
| *** bobh has quit IRC | 11:45 | |
| AJaeger | odyssey4me: that was a lame joke | 11:46 |
| AJaeger | mkolesni: it never did, you misrembered | 11:46 |
| AJaeger | mkolesni: if you want to stop testing, push a new change up | 11:46 |
| mkolesni | AJaeger, i dont need to push a new change | 11:47 |
| mkolesni | AJaeger, i need to redcheck the existing one since it failed on something sporadic and now its waiting for the job to time out | 11:47 |
| mkolesni | *recheck | 11:47 |
| *** huanxie has quit IRC | 11:48 | |
| *** sileht has quit IRC | 11:50 | |
| *** huanxie has joined #openstack-infra | 11:50 | |
| AJaeger | mkolesni: And recheck only works once it's finished, so you have to kill it otherwise like with a new change. | 11:50 |
| *** sileht has joined #openstack-infra | 11:50 | |
| *** bobh has joined #openstack-infra | 11:52 | |
| mkolesni | AJaeger, ok so can we make it a feature then? | 11:54 |
| AJaeger | mkolesni: I'm the wrong person to ask for that... | 11:55 |
| *** jkilpatr has joined #openstack-infra | 11:55 | |
| apevec | mkolesni, you can upload dummy change in the commit msg? | 11:56 |
| mkolesni | apevec, i have a tree of chains of other changes depending on this one | 11:57 |
| prometheanfire | AJaeger: bug I can point to? | 11:59 |
| fungi | infra-root: config-core: anyone else who cares... i'm afk the majority of today but will try to catch up a bit remotely around 1500z and am hoping to be home by 2100z to help out with whatever needs fixing at that point | 12:00 |
| *** pgadiya has quit IRC | 12:00 | |
| *** jpena is now known as jpena|lunch | 12:01 | |
| *** yamamoto has quit IRC | 12:02 | |
| *** apevec has quit IRC | 12:02 | |
| *** sree has joined #openstack-infra | 12:02 | |
| *** dprince has joined #openstack-infra | 12:03 | |
| *** pcaruana has quit IRC | 12:03 | |
| *** pcaruana has joined #openstack-infra | 12:04 | |
| *** masuberu has joined #openstack-infra | 12:06 | |
| *** mat128_ has joined #openstack-infra | 12:06 | |
| *** sree has quit IRC | 12:06 | |
| odyssey4me | jeblair mordred when you're in, I'd like to follow up on what's going on in https://review.openstack.org/513706 where, for some reason, the newton branch is being used instead of the review patch | 12:08 |
| *** thorst has joined #openstack-infra | 12:08 | |
| prometheanfire | frickler: was there a bug created for the requirements non-voting change? | 12:09 |
| *** masber has quit IRC | 12:09 | |
| *** pgadiya has joined #openstack-infra | 12:09 | |
| *** mat128 has quit IRC | 12:10 | |
| *** smatzek has quit IRC | 12:13 | |
| *** smatzek has joined #openstack-infra | 12:14 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: test containers update https://review.openstack.org/511175 | 12:16 |
| AJaeger | prometheanfire: I doubt it. frickler noticed that it fails since it reads a Zuul v2 file that we just removed, you need changing it for v3. frickler has a fix for requiremetns repo up, see the depends-on in https://review.openstack.org/515007 | 12:18 |
| *** smatzek has quit IRC | 12:18 | |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/openstack-zuul-jobs master: Cleanup ironic-python-agent legacy jobs https://review.openstack.org/515039 | 12:19 |
| AJaeger | prometheanfire: so, please review 515007 and https://review.openstack.org/#/c/515005/ - and let's fix 515007 so that the test works again | 12:19 |
| AJaeger | fungi, safe travels! | 12:19 |
| *** pgadiya has quit IRC | 12:21 | |
| *** rosmaita has joined #openstack-infra | 12:22 | |
| *** hemna_ has joined #openstack-infra | 12:23 | |
| *** lucas-hungry is now known as lucasagomes | 12:24 | |
| *** thorst_ has joined #openstack-infra | 12:24 | |
| *** sbezverk has joined #openstack-infra | 12:26 | |
| *** thorst has quit IRC | 12:26 | |
| *** rlandy has joined #openstack-infra | 12:27 | |
| openstackgerrit | Dirk Mueller proposed openstack-infra/zuul-jobs master: Remove pep8/pyflakes https://review.openstack.org/515041 | 12:27 |
| *** yolanda has joined #openstack-infra | 12:27 | |
| *** kiennt26 has joined #openstack-infra | 12:27 | |
| *** slaweq has quit IRC | 12:28 | |
| prometheanfire | AJaeger: ok, looking at those two | 12:28 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Do late decoding of log stream buffer https://review.openstack.org/515043 | 12:28 |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/openstack-zuul-jobs master: Cleanup ironic-python-agent legacy jobs https://review.openstack.org/515039 | 12:30 |
| *** huanxie has quit IRC | 12:30 | |
| *** esberglu has joined #openstack-infra | 12:31 | |
| AJaeger | thanks, prometheanfire | 12:32 |
| *** edmondsw has joined #openstack-infra | 12:32 | |
| *** slaweq has joined #openstack-infra | 12:33 | |
| prometheanfire | the error in 515005 looks odd to me, but looking | 12:35 |
| *** pgadiya has joined #openstack-infra | 12:35 | |
| *** edmondsw has quit IRC | 12:37 | |
| *** edmondsw has joined #openstack-infra | 12:37 | |
| *** trown|outtypewww is now known as trown | 12:38 | |
| AJaeger | prometheanfire: looks like frickler renamed a function but missed a place | 12:39 |
| frickler | AJaeger: prometheanfire: yeah, just pushed a new PS | 12:39 |
| *** bobh has quit IRC | 12:39 | |
| prometheanfire | frickler: thanks | 12:41 |
| *** armaan has quit IRC | 12:41 | |
| *** armaan has joined #openstack-infra | 12:41 | |
| *** pgadiya has quit IRC | 12:41 | |
| *** vsaienk0 has quit IRC | 12:41 | |
| *** danpawlik has quit IRC | 12:42 | |
| andreaf | tosky: sorry, travelling this week so slow progress there | 12:42 |
| *** markvoelker has joined #openstack-infra | 12:43 | |
| tosky | andreaf: np! Apart from the post-failure, my biggest question is why I can't make my regex selector work (no jobs found) | 12:43 |
| *** danpawlik has joined #openstack-infra | 12:43 | |
| tosky | but let's see when you are back | 12:43 |
| openstackgerrit | Blair Bethwaite proposed openstack-infra/irc-meetings master: Convert Scientific from Working Group to Special Interest Group https://review.openstack.org/515047 | 12:44 |
| *** vsaienk0 has joined #openstack-infra | 12:44 | |
| *** dbecker has joined #openstack-infra | 12:47 | |
| *** udesale has joined #openstack-infra | 12:48 | |
| *** esberglu has quit IRC | 12:49 | |
| *** mriedem has joined #openstack-infra | 12:50 | |
| *** Hal has joined #openstack-infra | 12:51 | |
| *** Hal is now known as Guest12599 | 12:51 | |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add a generic stage-artifacts role https://review.openstack.org/509233 | 12:52 |
| *** LindaWang has quit IRC | 12:52 | |
| *** lindycoder has joined #openstack-infra | 12:53 | |
| *** LindaWang has joined #openstack-infra | 12:53 | |
| *** pgadiya has joined #openstack-infra | 12:53 | |
| *** vsaienk0 has quit IRC | 12:54 | |
| andreaf | mordred, jeblair: starting to rework my staging role to match mordred proposal in http://lists.openstack.org/pipermail/openstack-infra/2017-October/005606.html -> https://review.openstack.org/509233 | 12:55 |
| andreaf | let me know what you think - I'd like to get the patch series done and start merging the patches on tempest side then | 12:55 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Fix manual publishing https://review.openstack.org/515050 | 12:55 |
| AJaeger | config-core, please review ^ - seems the only remaining one with the problem | 12:56 |
| *** pgadiya has quit IRC | 12:56 | |
| AJaeger | yolanda: ^ | 12:58 |
| dmsimard | AJaeger: ah, the good old trailing slash (or lack thereof) with rsync. | 13:01 |
| dmsimard | Still manages to confuses me after all these years | 13:01 |
| dmsimard | Just like I always manage to screw up "ln -s" between source and target | 13:01 |
| dmsimard | my brain is not wired properly to use these commands right :) | 13:02 |
| prometheanfire | you aren't the only one | 13:02 |
| prometheanfire | EVERY time I use ln I get it backwards | 13:02 |
| *** yamamoto has joined #openstack-infra | 13:02 | |
| AJaeger | dmsimard: yeah ;( Thanks for review! | 13:03 |
| *** baoli has joined #openstack-infra | 13:03 | |
| dmsimard | prometheanfire: happy to know I'm not an anomaly | 13:03 |
| dmsimard | prometheanfire: I've had someone tell me, "but, it's easy, it's source first then target" .. but then, depending from the perspective you look at it, these words are interchangeable :/ | 13:03 |
| openstackgerrit | Merged openstack-infra/project-config master: sahara-dashboard: remove the migrated unittests https://review.openstack.org/515015 | 13:06 |
| *** bexelbie has joined #openstack-infra | 13:06 | |
| *** slaweq has quit IRC | 13:06 | |
| AJaeger | dmsimard, yolanda , frickler, please review my newton cleanup changes as well https://review.openstack.org/#/q/status:open+topic:newton-eol - and https://review.openstack.org/515050 | 13:06 |
| *** camunoz has joined #openstack-infra | 13:06 | |
| dmsimard | AJaeger: I'm behind on reviews, will try to catch up asap | 13:07 |
| *** jcoufal has joined #openstack-infra | 13:07 | |
| *** jpena|lunch is now known as jpena | 13:08 | |
| AJaeger | dmsimard: thanks | 13:09 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add a generic stage-artifacts role https://review.openstack.org/509233 | 13:09 |
| *** mat128_ has quit IRC | 13:09 | |
| *** vsaienk0 has joined #openstack-infra | 13:09 | |
| *** esberglu has joined #openstack-infra | 13:09 | |
| yolanda | AJaeger, taking a look, as time permits. I have been on meetings this and last week | 13:10 |
| tosky | do changes to project-config takes few minutes to be applied (it was ~20 in v2 iirc?) or are they now applied immediately because they are directly re-read? | 13:10 |
| openstackgerrit | Merged openstack-infra/project-config master: tap-as-a-service: Use -horizon variant of templates https://review.openstack.org/514917 | 13:10 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy ironic-python-agent jobs https://review.openstack.org/514596 | 13:10 |
| AJaeger | yolanda: thanks! Thankfully the team has grown as well, so if you have no time, go back to your meetings ;) | 13:11 |
| yolanda | yes, is good to see new people joining and reviewing :) | 13:11 |
| *** yamamoto has quit IRC | 13:12 | |
| *** slaweq has joined #openstack-infra | 13:12 | |
| yolanda | AJaeger, about indentation on https://review.openstack.org/#/c/514526/2/roles/revoke-sudo/tasks/main.yaml , on tags | 13:14 |
| yolanda | that looks basd | 13:14 |
| yolanda | bad | 13:14 |
| yolanda | is that done with some purpose? | 13:15 |
| *** camunoz has quit IRC | 13:15 | |
| EmilienM | pabelanger: quick question - is there a parameter in zuul that could make tripleo patches in *gate* not dependent each others, unless they are in Depends-On (in that case, yes make them dependent) | 13:15 |
| *** ykarel has quit IRC | 13:15 | |
| *** ykarel has joined #openstack-infra | 13:15 | |
| AJaeger | yolanda: I'm an ansible novice - how should it be done instead? | 13:16 |
| yolanda | 2 spaces after tags | 13:16 |
| AJaeger | yolanda: feel free to -1 and educate me | 13:16 |
| yolanda | https://review.openstack.org/#/c/514526/2/roles/version-from-git/tasks/main.yaml is fine, 2 spaces there | 13:16 |
| AJaeger | yolanda: got it, will udpate | 13:17 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul-jobs master: Silence ansible-lint https://review.openstack.org/514526 | 13:17 |
| *** bobh has joined #openstack-infra | 13:17 | |
| AJaeger | yolanda: better? | 13:17 |
| yolanda | ok wanted to be sure if that was some ansible trick of some sort :) | 13:18 |
| EmilienM | AJaeger: when you have time this week, could you please help me on https://review.openstack.org/#/q/status:open+topic:tripleo/migrate-to-zuulv3 ? this is not urgent at all, it can even wait next week. | 13:18 |
| AJaeger | EmilienM: hope others can help as well... | 13:19 |
| *** armaan has quit IRC | 13:20 | |
| EmilienM | right | 13:20 |
| *** armaan has joined #openstack-infra | 13:20 | |
| openstackgerrit | Merged openstack-infra/infra-manual master: Clarify which jobs belong in the project block https://review.openstack.org/513197 | 13:23 |
| *** eharney has joined #openstack-infra | 13:24 | |
| *** eumel8 has quit IRC | 13:24 | |
| *** huanxie has joined #openstack-infra | 13:25 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy job for monasca https://review.openstack.org/513722 | 13:27 |
| *** camunoz has joined #openstack-infra | 13:28 | |
| AJaeger | EmilienM: very brief review: You should import the playbooks as well - see the documentation on https://docs.openstack.org/infra/manual/zuulv3.html#howto-update-legacy-jobs | 13:31 |
| EmilienM | AJaeger: ok, I'll take a second look | 13:31 |
| *** armaan has quit IRC | 13:32 | |
| AJaeger | EmilienM: and rename all the jobs | 13:32 |
| EmilienM | ok | 13:33 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove nodepool legacy jobs from project https://review.openstack.org/514799 | 13:36 |
| *** dbecker has quit IRC | 13:37 | |
| *** kiennt26 has quit IRC | 13:38 | |
| *** dhajare has quit IRC | 13:40 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Fix space character for bash command https://review.openstack.org/513435 | 13:40 |
| *** dhajare has joined #openstack-infra | 13:42 | |
| EmilienM | dmsimard: quick question - is there a parameter in zuul that could make tripleo patches in *gate* not dependent each others, unless they are in Depends-On (in that case, yes make them dependent) | 13:42 |
| dmsimard | EmilienM: you mean for co-gating in the queue ? | 13:43 |
| EmilienM | dmsimard: yeah | 13:43 |
| EmilienM | we want our gate to stop resetting when a patch fails on top | 13:43 |
| EmilienM | at least find a solution | 13:43 |
| dmsimard | EmilienM: I have no idea, that'd be a question for infra-root | 13:43 |
| openstackgerrit | Merged openstack-infra/project-config master: Change osc-plugin-jobs to openstackclient-plugin-jobs https://review.openstack.org/514849 | 13:44 |
| *** slaweq has quit IRC | 13:44 | |
| dmsimard | I'm not very familiar with how queues and co-gating works in the first place | 13:44 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove Oslo.log legacy job https://review.openstack.org/511385 | 13:44 |
| AJaeger | EmilienM: see https://review.openstack.org/513222 | 13:44 |
| AJaeger | and https://review.openstack.org/512140 - that's part of creating the queue | 13:45 |
| *** yamahata has joined #openstack-infra | 13:45 | |
| EmilienM | AJaeger: ok so what I asked is to removed this queue, correct? | 13:45 |
| EmilienM | remove* | 13:45 |
| AJaeger | EmilienM: https://review.openstack.org/512140 | 13:45 |
| AJaeger | EmilienM: yes, revert those changes ;) | 13:45 |
| EmilienM | you would accept it? | 13:45 |
| dmsimard | EmilienM: you do what you want, we review that it doesn't break anything :P | 13:45 |
| AJaeger | EmilienM: Let's ask pabelanger , please | 13:46 |
| EmilienM | ok | 13:46 |
| EmilienM | I'll prepare the patch though | 13:46 |
| AJaeger | EmilienM: best discuss further with him | 13:46 |
| EmilienM | yep | 13:46 |
| *** sshnaidm is now known as sshnaidm|mtg | 13:46 | |
| openstackgerrit | Dean Troyer proposed openstack/os-client-config master: Consume the osc-functional-devstack-tips job https://review.openstack.org/514847 | 13:47 |
| openstackgerrit | Merged openstack-infra/project-config master: Run linters job instead of py27 https://review.openstack.org/514610 | 13:48 |
| openstackgerrit | Merged openstack-infra/project-config master: Fix manual publishing https://review.openstack.org/515050 | 13:48 |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Convert Scientific from Working Group to Special Interest Group https://review.openstack.org/515047 | 13:48 |
| *** dhajare has quit IRC | 13:48 | |
| *** slaweq has joined #openstack-infra | 13:48 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: Remove usage of 'tripleo' gate queue https://review.openstack.org/515070 | 13:49 |
| *** sbezverk has quit IRC | 13:50 | |
| *** andreas_s has quit IRC | 13:50 | |
| *** gus has quit IRC | 13:51 | |
| *** andreas_s has joined #openstack-infra | 13:51 | |
| EmilienM | pabelanger: when you're up, please take a look at https://review.openstack.org/#/c/515070 | 13:51 |
| *** gus has joined #openstack-infra | 13:51 | |
| *** smatzek has joined #openstack-infra | 13:52 | |
| *** gridinv has quit IRC | 13:52 | |
| *** iyamahat has joined #openstack-infra | 13:53 | |
| openstackgerrit | Tin Lam proposed openstack-infra/project-config master: Remove legacy jobs https://review.openstack.org/515072 | 13:53 |
| *** armax has joined #openstack-infra | 13:53 | |
| *** smatzek has quit IRC | 13:53 | |
| *** smatzek has joined #openstack-infra | 13:54 | |
| *** smatzek has quit IRC | 13:54 | |
| *** bexelbie has quit IRC | 13:54 | |
| *** smatzek has joined #openstack-infra | 13:55 | |
| haleyb | clarkb: follow-up question on graphite stats. is there a difference between something in stats.zuul vs stats_counts.zuul ? or is one preferred ? | 13:55 |
| *** huanxie has quit IRC | 13:55 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: remove legacy panko jobs https://review.openstack.org/512430 | 13:55 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove legacy jobs in Tooz https://review.openstack.org/513296 | 13:55 |
| openstackgerrit | Merged openstack-infra/project-config master: remove some debugging for release git clone https://review.openstack.org/514729 | 13:55 |
| openstackgerrit | Merged openstack-infra/project-config master: add neutron to vmware-nsx releasenotes jobs https://review.openstack.org/514630 | 13:55 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy oslo.db job https://review.openstack.org/511414 | 13:55 |
| vsaienk0 | AJaeger: yolanda please add to your review queue https://review.openstack.org/#/c/515039/ cleanup legacy ironic-python-agent jobs | 13:55 |
| *** sree has joined #openstack-infra | 13:55 | |
| *** yamahata has quit IRC | 13:55 | |
| *** jkilpatr has quit IRC | 13:56 | |
| *** huanxie has joined #openstack-infra | 13:56 | |
| AJaeger | infra-root, please remove http://files.openstack.org/docs/publish-docs/ - the directory and content is wrong, fixed now with just merged 515050 | 13:57 |
| *** gmann is now known as gmann_afk | 13:58 | |
| *** sree has quit IRC | 14:00 | |
| openstackgerrit | Dean Troyer proposed openstack-infra/project-config master: Remove legacy-osc-dsvm-functional* jobs https://review.openstack.org/515075 | 14:01 |
| *** eharney has quit IRC | 14:04 | |
| *** andreas_s has quit IRC | 14:05 | |
| *** hongbin has joined #openstack-infra | 14:08 | |
| *** yamahata has joined #openstack-infra | 14:08 | |
| *** slaweq_ has quit IRC | 14:09 | |
| *** andreas_s has joined #openstack-infra | 14:11 | |
| *** iyamahat has quit IRC | 14:11 | |
| openstackgerrit | Tin Lam proposed openstack-infra/project-config master: Change documentation job for OpenStack-Helm https://review.openstack.org/515080 | 14:11 |
| *** andreas_s has quit IRC | 14:17 | |
| openstackgerrit | Dean Troyer proposed openstack-infra/openstack-zuul-jobs master: Remove osc-dsvm-functional* jobs https://review.openstack.org/515088 | 14:19 |
| *** rossella_s has quit IRC | 14:19 | |
| *** dbecker has joined #openstack-infra | 14:20 | |
| *** rbrndt has joined #openstack-infra | 14:20 | |
| *** salv-orlando has quit IRC | 14:21 | |
| *** slaweq has quit IRC | 14:21 | |
| *** salv-orlando has joined #openstack-infra | 14:22 | |
| *** mat128 has joined #openstack-infra | 14:23 | |
| *** pblaho has quit IRC | 14:24 | |
| jeblair | odyssey4me: a fix for that stable branch problem has merged, but we need to restart the scheduler. i'm planning to do that now. | 14:24 |
| *** yamahata has quit IRC | 14:25 | |
| *** salv-orlando has quit IRC | 14:26 | |
| odyssey4me | jeblair ah ok, thanks ! which review was it? I'm curious | 14:26 |
| *** udesale has quit IRC | 14:26 | |
| flaper87 | is there some documentation on how I can re-use the jobs defined in a different repo? Say repo A has a .zuul.yml file in it with some jobs defined and some ansible roles. Is there a way for project B to use those jobs/roles? | 14:27 |
| jeblair | odyssey4me: Id1266a5d553f88d9354b259d523821a707d4ac98 | 14:27 |
| AJaeger | flaper87: the jobs: Just do it! | 14:28 |
| *** huanxie has quit IRC | 14:28 | |
| jeblair | flaper87: nothing special, just add them to your project dfn. :) | 14:28 |
| *** slaweq has joined #openstack-infra | 14:28 | |
| odyssey4me | jeblair ah yeah - saw that one, awesome - let me know when zuul's been bounced | 14:28 |
| tosky | flaper87: yes: something like: https://git.openstack.org/cgit/openstack/sahara-tests/tree/.zuul.yaml?id=b68cc677422ee7da6fda2a7c722bffca66b1a7fe#n58 | 14:28 |
| odyssey4me | (please) | 14:28 |
| *** yolanda has quit IRC | 14:28 | |
| AJaeger | flaper87: http://git.openstack.org/cgit/openstack/security-doc/tree/.zuul.yaml reuses what'S in http://git.openstack.org/cgit/openstack/openstack-manuals/tree/.zuul.yaml | 14:28 |
| *** pblaho has joined #openstack-infra | 14:29 | |
| *** udesale has joined #openstack-infra | 14:30 | |
| *** coolsvap has quit IRC | 14:31 | |
| *** fanzhang__ has joined #openstack-infra | 14:31 | |
| *** fanzhang has quit IRC | 14:32 | |
| *** huanxie has joined #openstack-infra | 14:33 | |
| *** dhill_ has quit IRC | 14:33 | |
| *** andreas_s has joined #openstack-infra | 14:34 | |
| *** apevec has joined #openstack-infra | 14:38 | |
| *** apevec has quit IRC | 14:38 | |
| *** apevec has joined #openstack-infra | 14:38 | |
| *** dhill_ has joined #openstack-infra | 14:38 | |
| openstackgerrit | Trevor McCasland proposed openstack-infra/subunit2sql master: Add subunit2sql CLI option to use non_subunit_name https://review.openstack.org/507192 | 14:40 |
| *** ykarel_ has joined #openstack-infra | 14:40 | |
| *** jaypipes has joined #openstack-infra | 14:40 | |
| *** ykarel has quit IRC | 14:41 | |
| *** gouthamr has joined #openstack-infra | 14:41 | |
| *** sree has joined #openstack-infra | 14:42 | |
| jeblair | andreaf: cool! i left some comments on 509233 | 14:43 |
| *** andreas_s has quit IRC | 14:44 | |
| *** david-lyle has quit IRC | 14:44 | |
| johnsom | Still looking for a +A on https://review.openstack.org/513781 to cut out the duplicate legacy gates for octavia. Thanks. | 14:44 |
| *** slaweq_ has joined #openstack-infra | 14:45 | |
| AJaeger | johnsom: +3 | 14:45 |
| *** andreas_s has joined #openstack-infra | 14:45 | |
| johnsom | Thank you! | 14:45 |
| *** sree has quit IRC | 14:47 | |
| *** dbecker has quit IRC | 14:50 | |
| *** andreas_s has quit IRC | 14:50 | |
| AJaeger | dmsimard: could you remove your -1 from https://review.openstack.org/#/c/513785 - the change is fine as is. | 14:51 |
| *** andreas_s has joined #openstack-infra | 14:52 | |
| *** david-lyle has joined #openstack-infra | 14:53 | |
| dmsimard | AJaeger: sure, I suppose the change is correct even though you can't use a depends-on against project-config | 14:54 |
| *** felipemonteiro_ has joined #openstack-infra | 14:54 | |
| dmsimard | I guess it has for effect to ensure the patch does not merge until the project-config patch is merged but it also prevents gate jobs from running because zuul considers it an error | 14:54 |
| AJaeger | dmsimard: you can use it - to block the merge. Our documentation recommends to do it since we have no better way ;( | 14:54 |
| dmsimard | AJaeger: btw I meant to ask, does 'needed-by' really work ? | 14:54 |
| AJaeger | in this case it would not merge anyhow since the removed jobs are used. | 14:54 |
| AJaeger | dmsimard: just as pointer - you can click on it and follow the link. So, purely documentation | 14:55 |
| *** yamahata has joined #openstack-infra | 14:55 | |
| flaper87 | AJaeger: jeblair tosky thanks | 14:55 |
| *** eharney has joined #openstack-infra | 14:55 | |
| dmsimard | AJaeger: ok, so it doesn't effectively do a "reverse depends-on" | 14:56 |
| *** sbezverk has joined #openstack-infra | 14:56 | |
| *** ramishra has quit IRC | 14:56 | |
| AJaeger | dmsimard: indeed, it does not enforce any order | 14:57 |
| *** felipemonteiro__ has joined #openstack-infra | 14:57 | |
| *** ykarel_ has quit IRC | 14:57 | |
| jeblair | i'm restarting the scheduler now | 14:57 |
| *** lewo has quit IRC | 14:59 | |
| *** ykarel_ has joined #openstack-infra | 14:59 | |
| *** lewo has joined #openstack-infra | 14:59 | |
| dmsimard | EmilienM: btw ^ needed-by | 14:59 |
| EmilienM | dmsimard: ok, good to know. Thanks! | 15:00 |
| pabelanger | EmilienM: AJaeger: You can move tripleo jobs into non-shared queue, but if those projects share the same code, it will be dangerous and broken code could land. | 15:00 |
| *** felipemonteiro_ has quit IRC | 15:01 | |
| jeblair | they probably shared a queue under v2, right? | 15:01 |
| pabelanger | 1yes | 15:01 |
| *** ykarel_ is now known as ykarel | 15:01 | |
| EmilienM | yes we did | 15:01 |
| *** slaweq has quit IRC | 15:01 | |
| EmilienM | pabelanger: can you please give me an example of risk? | 15:01 |
| EmilienM | I'm a bit confused on the why, sorry | 15:01 |
| *** slaweq has joined #openstack-infra | 15:01 | |
| *** andreas_s has quit IRC | 15:01 | |
| jeblair | scheduler restarted, changes are being re-enqueued now | 15:02 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add a generic stage-artifacts role https://review.openstack.org/509233 | 15:02 |
| dmsimard | EmilienM: if tripleo-quickstart clones something from tripleo-ci and there's no explicit depends-on, it'll be tested without in-flight tripleo-ci code and things could break after the fact | 15:02 |
| dmsimard | (for example) | 15:03 |
| EmilienM | ok now I understand | 15:03 |
| clarkb | EmilienM: dmsimard no there isnt as gating implies dependency due to needing a serliazed set of git commits | 15:03 |
| EmilienM | trown: ^ fyi, it's usefult o knnow that | 15:03 |
| pabelanger | right | 15:03 |
| *** andreas_s has joined #openstack-infra | 15:03 | |
| pabelanger | its first code that lands wins, and could break everything behind it | 15:03 |
| *** cshastri has quit IRC | 15:04 | |
| *** huanxie has quit IRC | 15:04 | |
| trown | only if we dont do explicit depends-on? | 15:04 |
| *** gongysh has quit IRC | 15:05 | |
| pabelanger | TBH, your better leaving triplo queue and making all votes non-voting | 15:05 |
| andreaf | jeblair: thanks for your review - I replied to your comments | 15:05 |
| andreaf | jeblair: I'm at osseu so I just put up the patch, I did not really manage to test it yet - but https://review.openstack.org/#/c/509272 should verify it | 15:06 |
| jeblair | note that if you *do* use a depends-on but don't share a queue, the changes have to merge serially, so it could slow things down significantly. | 15:06 |
| *** efried has left #openstack-infra | 15:06 | |
| *** martinkopec has quit IRC | 15:06 | |
| *** eumel8 has joined #openstack-infra | 15:07 | |
| jeblair | (you can't test+merge changes in parallel that don't share a queue because the order isn't guaranteed) | 15:07 |
| trown | well merging serially is actually what we are looking for... because right now the queue is so large that we cant get parallel to work at all | 15:07 |
| jeblair | trown: serially is exactly what it's already doing in that case | 15:07 |
| trown | im confused | 15:08 |
| *** psachin has quit IRC | 15:08 | |
| jeblair | the algorithm is O(1) best case and O(n) worst. | 15:08 |
| *** martinkopec has joined #openstack-infra | 15:09 | |
| EmilienM | pabelanger: non voting? | 15:13 |
| *** andreas_s has quit IRC | 15:13 | |
| *** andreas_s has joined #openstack-infra | 15:15 | |
| robcresswell | Is there a suggested base job for Neutron plugins? Just running tox-driven tests and required-projects is currently requirements, neutronclient and neutron. Wondered if there was a base job that includes these, as they are fairly standard for Neutron plugins :) | 15:15 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Add a Python release job for API services https://review.openstack.org/514855 | 15:15 |
| odyssey4me | jeblair I see zuul's been restarted - are we happy to go ahead and check whether our log patch is doing the right things now? | 15:15 |
| jeblair | odyssey4me: yep! | 15:15 |
| pabelanger | EmilienM: non-voting jobs are not run in gate, so they cannot be kicked out and gate reset if they fail | 15:16 |
| jeblair | robcresswell: i think rather than a custom job, there's a project-template that adds those. | 15:16 |
| *** dhinesh has joined #openstack-infra | 15:16 | |
| *** martinkopec has quit IRC | 15:16 | |
| AJaeger | robcresswell, jeblair : The project-template only add neutron (requiremetns is there by default). If all repos need neutronclient as well, we could add it | 15:17 |
| * robcresswell needs to learn how the templates work | 15:17 | |
| robcresswell | I'll go read some more before I waste time - but thanks for the pointers jeblair AJaeger | 15:18 |
| *** jascott1 has joined #openstack-infra | 15:18 | |
| EmilienM | pabelanger: I don't think we want non votig jobs at this stage | 15:18 |
| AJaeger | robcresswell: template is defined in openstack-infra/openstack-zuul-jobs/zuul.d/project-templates.yaml | 15:18 |
| *** dhinesh has quit IRC | 15:21 | |
| jeblair | https://docs.openstack.org/infra/openstack-zuul-jobs/project-templates.html#project_template-openstack-python-jobs-neutron is what i was thinking of | 15:21 |
| *** jascott1 has quit IRC | 15:22 | |
| AJaeger | exactly | 15:23 |
| pabelanger | I keep forgetting about those docs :) They look awesome | 15:23 |
| *** Apoorva has joined #openstack-infra | 15:25 | |
| openstackgerrit | Merged openstack-infra/project-config master: Removes migrated legacy-octavia-* https://review.openstack.org/513781 | 15:25 |
| pabelanger | AJaeger: looking at 514526, I don't see where you updated .zuul.yaml and added openstack-zuul-jobs-linters. | 15:26 |
| *** huanxie has joined #openstack-infra | 15:26 | |
| AJaeger | pabelanger: I didn't update .zuul.yaml yet... | 15:29 |
| AJaeger | pabelanger: indeed, that would have been the solution to get dependency working | 15:29 |
| *** slaweq has quit IRC | 15:29 | |
| *** sree has joined #openstack-infra | 15:29 | |
| AJaeger | pabelanger: do you want to merge 514526 as is? And then fix the other problems? | 15:29 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add a generic stage-artifacts role https://review.openstack.org/509233 | 15:30 |
| andreaf | jeblair: this version is tested, it should work fine ^^^ | 15:30 |
| * AJaeger signs of for the day | 15:31 | |
| *** hemna_ has quit IRC | 15:31 | |
| *** andreas_s has quit IRC | 15:31 | |
| *** thiagolib has quit IRC | 15:32 | |
| *** LindaWang has quit IRC | 15:32 | |
| openstackgerrit | Merged openstack-infra/project-config master: Newton removal: Remove some stable/newton https://review.openstack.org/514915 | 15:34 |
| *** sree has quit IRC | 15:34 | |
| *** thorst_ has quit IRC | 15:34 | |
| *** thorst has joined #openstack-infra | 15:35 | |
| *** andreas_s has joined #openstack-infra | 15:37 | |
| pabelanger | infra-root: EmilienM I've enqueue 513701 into the gate pipeline to help tripleo with gate reset issue. It seems ansible 2.4 might be part of the issue | 15:38 |
| *** slaweq has joined #openstack-infra | 15:39 | |
| *** thorst has quit IRC | 15:39 | |
| EmilienM | pabelanger: thank you | 15:39 |
| *** LindaWang has joined #openstack-infra | 15:42 | |
| *** LindaWang has quit IRC | 15:42 | |
| jeblair | pabelanger: ++ | 15:43 |
| *** slaweq has quit IRC | 15:44 | |
| *** vhosakot has joined #openstack-infra | 15:45 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Use non-pypi for release instead of tarball-only https://review.openstack.org/514869 | 15:45 |
| *** andreas_s has quit IRC | 15:46 | |
| *** smatzek_ has joined #openstack-infra | 15:46 | |
| *** rkukura has joined #openstack-infra | 15:46 | |
| jeblair | fungi, dhellmann, smcginnis: i updated fungi's patch to address comments: https://review.openstack.org/514855 | 15:47 |
| jeblair | fungi: you have a sec to give that a once over? | 15:48 |
| jeblair | pabelanger: ^ you too? | 15:48 |
| fungi | jeblair: lgtm, thanks! | 15:48 |
| smcginnis | jeblair: Thanks! | 15:49 |
| *** apevec_ has joined #openstack-infra | 15:49 | |
| *** pcaruana has quit IRC | 15:49 | |
| *** smatzek has quit IRC | 15:49 | |
| dhellmann | jeblair : lgtm | 15:49 |
| openstackgerrit | Merged openstack-infra/project-config master: Newton removal: Remove periodic-newton https://review.openstack.org/514910 | 15:50 |
| *** apevec has quit IRC | 15:50 | |
| *** apevec_ is now known as apevec | 15:50 | |
| *** yamahata has quit IRC | 15:51 | |
| pabelanger | jeblair: fungi: lgtm +3 | 15:51 |
| fungi | awesome | 15:52 |
| fungi | okay, disappearing back into a black hole again | 15:52 |
| fungi | i should hopefully be back online around 21:00z | 15:52 |
| frickler | is it possible to have ansible look for roles in a directory named other than "roles"? I'm facing a clash with chef using that directory for the same purpose in openstack-chef-repo | 15:53 |
| openstackgerrit | Merged openstack-infra/project-config master: Newton removal: zuul.d/jobs.yaml https://review.openstack.org/514909 | 15:53 |
| pabelanger | frickler: ANSIBLE_ROLES_PATH variable | 15:55 |
| jeblair | frickler, pabelanger: is this in the context of zuul? | 15:55 |
| pabelanger | jeblair: I am assuming no | 15:55 |
| jeblair | frickler: if you are writing a role that isn't going to be used by anything other than the playbooks in that repo, you can put the roles adjacent to the playbooks. | 15:56 |
| jeblair | pabelanger: i'm assuming yes, since it was asked here and not #ansible :) | 15:56 |
| *** huanxie has quit IRC | 15:57 | |
| frickler | jeblair: pabelanger: yes, I'm starting to build a native v3 job for openstack-chef-repo | 15:58 |
| pabelanger | Yah, so in that case ANSIBLE_ROLES_PATH will not work :) | 15:59 |
| *** salv-orlando has joined #openstack-infra | 16:00 | |
| frickler | jeblair: so "adjacent to the playbooks" means within the playbooks directory? I'm already having a toplevel dir "roles" in that repo filled with chef roles and I don't want to put ansible roles into it to avoid confusion | 16:00 |
| jeblair | frickler: right. if you have 'playbooks/foo.yaml' use a role 'bar', it can be in 'playbooks/roles/bar/...' | 16:00 |
| frickler | jeblair: o.k., that should do, thx | 16:01 |
| *** coolsvap has joined #openstack-infra | 16:01 | |
| jeblair | frickler: basically, if the role is meant to be shared outside the repo, then zuul assumes it's constructed as an ansible role repo. there's a bit more info on the rationale here: https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html#attr-job.roles | 16:02 |
| openstackgerrit | Merged openstack-infra/project-config master: Add a Python release job for API services https://review.openstack.org/514855 | 16:02 |
| *** jascott1 has joined #openstack-infra | 16:02 | |
| jeblair | frickler: but if you're not using it outside of that repo, none of that applies | 16:02 |
| *** hashar has joined #openstack-infra | 16:03 | |
| *** ijw has joined #openstack-infra | 16:05 | |
| *** apevec has quit IRC | 16:05 | |
| *** huanxie has joined #openstack-infra | 16:05 | |
| frickler | jeblair: I'll define the job within the same repo, but that job is then to be used also from all the cookbook repos | 16:05 |
| jeblair | frickler: that should still work fine -- that playbook will still have it's private roles accessible | 16:06 |
| *** jascott1 has quit IRC | 16:07 | |
| *** andreas_s has joined #openstack-infra | 16:07 | |
| *** jascott1 has joined #openstack-infra | 16:08 | |
| *** lindycoder has quit IRC | 16:08 | |
| *** bhavik1 has joined #openstack-infra | 16:08 | |
| *** janki has quit IRC | 16:09 | |
| *** andreas_s has quit IRC | 16:12 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: Remove legacy-puppet-beaker-rspec from gate https://review.openstack.org/515121 | 16:12 |
| *** andreas_s has joined #openstack-infra | 16:13 | |
| *** panda|rover|off is now known as panda|ruck | 16:15 | |
| openstackgerrit | Merged openstack-infra/project-config master: Use non-pypi for release instead of tarball-only https://review.openstack.org/514869 | 16:15 |
| *** Rockyg has joined #openstack-infra | 16:15 | |
| *** dhinesh has joined #openstack-infra | 16:17 | |
| *** jschlueter has joined #openstack-infra | 16:17 | |
| *** andreas_s has quit IRC | 16:18 | |
| *** kuchi has quit IRC | 16:19 | |
| *** andreas_s has joined #openstack-infra | 16:19 | |
| *** erlon has quit IRC | 16:20 | |
| *** dhinesh has quit IRC | 16:21 | |
| *** e0ne has quit IRC | 16:21 | |
| *** vsaienk0 has quit IRC | 16:21 | |
| openstackgerrit | Dan Prince proposed openstack-infra/openstack-zuul-jobs master: Make tripleo undercloud-containers voting https://review.openstack.org/513163 | 16:23 |
| *** ykarel has quit IRC | 16:23 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Create job-output.txt together with JobDir https://review.openstack.org/514617 | 16:24 |
| *** kuchi has joined #openstack-infra | 16:24 | |
| *** tesseract has quit IRC | 16:25 | |
| *** iyamahat has joined #openstack-infra | 16:26 | |
| *** thorst has joined #openstack-infra | 16:27 | |
| jschlueter | pabelanger: so to rephrase what I was saying ... it might be interesting to see if min queue size can be dropped below min-threshold if you get X queue resets in a row at min size ... where X is probably 10 times the queue size, but then it can grow as normal afterwards | 16:27 |
| *** trown is now known as trown|lunch | 16:27 | |
| *** andreas_s has quit IRC | 16:28 | |
| pabelanger | jschlueter: might be worth discussing it with jeblair | 16:29 |
| *** armaan has joined #openstack-infra | 16:29 | |
| jeblair | jschlueter, pabelanger: is dependent queue window size the topic? | 16:29 |
| jschlueter | one downside is that this could allow a patch which causes a flapping test to get through easier | 16:29 |
| *** andreas_s has joined #openstack-infra | 16:30 | |
| jschlueter | jeblair: yea | 16:30 |
| jeblair | clarkb: ping (clarkb wrote most of this code) | 16:30 |
| jschlueter | just brainstorming on my side because of tripleo's current issue ... | 16:30 |
| jeblair | jschlueter: well, the changes are tested the same regardless of the window size | 16:30 |
| jeblair | jschlueter: a smaller window size basically just lets the system "waste" less resources on changes that are statistically unlikely to be used. | 16:31 |
| jschlueter | yep but queue resets can throw off being able to get a linch-pin patch to land | 16:31 |
| *** erlon has joined #openstack-infra | 16:32 | |
| *** daidv has quit IRC | 16:32 | |
| pabelanger | I think wasting of resources was one of the topics at PTG, we found that when tripleo gets into a high gate reset state, they do start consuming more resources | 16:32 |
| jschlueter | just an idea ... queue min size => default size can grow when successful passes, and in bad cases (where many queue resets in a row) allow for it to shrink below min size a bit more to limit how much is tested together | 16:32 |
| jeblair | jschlueter: every change is individually tested, so once that linch-pin patch gets to the top, it has an opportunity to demonstrate that it works, regardless of what has come ahead of it or what's behind it. | 16:32 |
| *** Goneri has quit IRC | 16:33 | |
| jeblair | jschlueter: the window size grows when patches land, and shrinks when the gate resets. the minimum is hand-tuned by the admins (us) so that we don't lose all parallelism, even when things get bad. otherwise we would degrade to serial testing very quickly | 16:34 |
| *** jpena is now known as jpena|brb | 16:34 | |
| jschlueter | yep... but then also the flapping test can throw a monkey wrench in the works if the one out of 5 runs it fails is the one where that patch was top of the list | 16:34 |
| jschlueter | jeblair: yep | 16:34 |
| jeblair | jschlueter: right, but the window size won't have an effect on that | 16:34 |
| jschlueter | ok tell me more... | 16:35 |
| * jschlueter not seeing enough of the picture then | 16:35 | |
| jeblair | jschlueter: the window size is how many changes behind the change currently at the head of the queue should also be tested. basically, it's the degree of parellism. | 16:35 |
| *** udesale has quit IRC | 16:35 | |
| jeblair | jschlueter: but ultimately, the process works as if each patch was tested individually. | 16:36 |
| jschlueter | jeblair: ok how is the head of the queue picked? | 16:36 |
| *** huanxie has quit IRC | 16:36 | |
| jeblair | jschlueter: every patch becomes the head of the queue in series. the order is determined by approval order of changes. | 16:36 |
| jeblair | jschlueter: this is not a batch system. this is a serial system that operates in parallel for speed. | 16:36 |
| jschlueter | if I have patch queue size of 3 and pending patch set of 40 ... | 16:36 |
| *** gyee has joined #openstack-infra | 16:37 | |
| jschlueter | first in first tested? | 16:37 |
| jeblair | here are the docs, btw... it has some examples that may be helpful: https://docs.openstack.org/infra/zuul/feature/zuulv3/user/gating.html#project-gating | 16:37 |
| jeblair | jschlueter: yep | 16:37 |
| jschlueter | jeblair: ack thanks /me goes to read more | 16:37 |
| *** lucasagomes is now known as lucas-afk | 16:38 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Silence ansible-lint https://review.openstack.org/514526 | 16:39 |
| *** andreas_s has quit IRC | 16:39 | |
| clarkb | jeblair: hi, currebtly getting a tooth checked out. ehats up? | 16:39 |
| jschlueter | and when a patch fails while in the check queue ... it get's dropped out and added to end of list or excluded? until external influnce? | 16:39 |
| jeblair | clarkb: not urgent; just wanted to bring this conversation to your attention if you're interested. it can wait. :) | 16:39 |
| *** sshnaidm|mtg is now known as sshnaidm | 16:40 | |
| jschlueter | jeblair: I see dropped from queue | 16:40 |
| *** andreas_s has joined #openstack-infra | 16:40 | |
| jeblair | jschlueter: the 'queue' aspect of check is less important. all changes are treated independently there. the order only matters in that zuul will assign nodes in the order changes arrive there. | 16:41 |
| jeblair | jschlueter: the gate pipeline is where most of this really comes into play. | 16:41 |
| *** iyamahat has quit IRC | 16:43 | |
| *** iyamahat has joined #openstack-infra | 16:43 | |
| *** Goneri has joined #openstack-infra | 16:44 | |
| *** hashar is now known as hasharAway | 16:45 | |
| *** Swami has joined #openstack-infra | 16:48 | |
| *** andreas_s has quit IRC | 16:49 | |
| boden | hi.. I found for the vmware-nsx project’s release notes build we are getting an old version of reno and thus failing http://logs.openstack.org/10/513510/11/check/build-openstack-releasenotes/9487aeb/job-output.txt.gz#_2017-10-25_15_39_29_327289 I see neutrons is much newer in http://logs.openstack.org/83/511983/11/check/build-openstack-releasenotes/4b0910a/job-output.txt.gz#_2017-10-24_19_01_58_446340 | 16:51 |
| boden | could this be because we dont use the release-notes-jobs template for vmware-nsx? | 16:51 |
| *** andreas_s has joined #openstack-infra | 16:51 | |
| *** claudiub|2 has quit IRC | 16:52 | |
| *** andreas_s has quit IRC | 16:55 | |
| jschlueter | jeblair, pabelanger: thanks for the info ... will have to read what's there and think some more | 16:56 |
| *** derekh has quit IRC | 16:56 | |
| *** ccamacho has left #openstack-infra | 16:57 | |
| *** ccamacho has quit IRC | 16:57 | |
| *** ralonsoh has quit IRC | 16:57 | |
| *** panda|ruck is now known as panda|ruck|bbl | 16:58 | |
| *** baoli has quit IRC | 16:58 | |
| jeblair | jschlueter: you're welcome, thansk for reading! | 16:58 |
| jeblair | jschlueter: let me know if there's anything i can clarify | 16:59 |
| openstackgerrit | boden proposed openstack-infra/project-config master: add neutron release notes job template to vmware-nsx https://review.openstack.org/515136 | 16:59 |
| * jschlueter nods to jeblair | 16:59 | |
| *** baoli has joined #openstack-infra | 16:59 | |
| jeblair | boden: you think it's relying on neutron to bring in the updated version of reno? | 17:01 |
| *** dtantsur is now known as dtantsur|afk | 17:01 | |
| boden | jeblair: I suspect thats part of the template, but frankly I’m not sure.. I was using the patch I just pushed to get some input | 17:02 |
| jeblair | boden: hrm, neutron has the same reno requirement. i would expect upper-constraints to determine what gets installed. | 17:03 |
| boden | jeblair: in that case I’m a bit clueless | 17:04 |
| *** ijw has quit IRC | 17:04 | |
| *** dhinesh has joined #openstack-infra | 17:04 | |
| boden | limping along just trying to get things back to running with our gate jobs :) | 17:04 |
| jeblair | boden: oh i don't think that's the version of reno | 17:04 |
| boden | jeblair is it the version for release notes? | 17:05 |
| boden | to generate I mean | 17:05 |
| jeblair | boden: i think that's meant to be the version of vmware-nsx that reno is backtracking to... | 17:06 |
| *** lindycoder has joined #openstack-infra | 17:06 | |
| boden | jeblair: ack that very well could be, but either way the failure in vmware-nsx never happened with zuul v2 and based on the error it “seems” to indicate something changed with builder | 17:06 |
| jeblair | dhellmann: ^ can you shed some light on http://logs.openstack.org/10/513510/11/check/build-openstack-releasenotes/9487aeb/job-output.txt.gz#_2017-10-25_15_39_29_327289 ? | 17:07 |
| boden | nothing changed with our release notes and they built with zuul v2 jobs | 17:07 |
| *** ijw has joined #openstack-infra | 17:07 | |
| dhellmann | jeblair , boden : looking | 17:07 |
| *** camunoz has quit IRC | 17:08 | |
| *** rkukura has quit IRC | 17:08 | |
| AJaeger | boden: this should not make any kind of change - and you use a template that publishes to docs.o.o so cannot be used by your repo | 17:08 |
| dhellmann | jeblair, boden : it looks like a mis-formatted release notes file. Does the build work locally ? | 17:08 |
| *** ijw has quit IRC | 17:12 | |
| *** huanxie has joined #openstack-infra | 17:13 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add log streaming logging and exception handling https://review.openstack.org/513811 | 17:16 |
| boden | dhellmann: I guess not, tho I was under the impression it used to build.. what are other non-stadium projects doing W/R/T to release notes in the gate? nothing? | 17:16 |
| boden | I’m wondering if we even need this job | 17:16 |
| dhellmann | boden : I'm not sure this is in any way related to the stadium-ness of the project. I'm running the build locally to see if I can identify the bad file, but I have to step out for a few minutes. I should be back in ~30 | 17:17 |
| *** hemna_ has joined #openstack-infra | 17:17 | |
| tosky | jeblair: hi! Previously we used to define DEVSTACK_SUBNODE_CONFIG=" " for multinode jobs in order to run the full devstack only on the primary node; what should I do now for a multinode devstack job? | 17:17 |
| *** rbrndt has quit IRC | 17:17 | |
| boden | dhellmann: np: I see the file it’s unhappy about… I’m trying to understand if we even need a releasenotes job.. I guess I’ll have to dig into what that job does | 17:18 |
| clarkb | jschlueter: jeblair pabelanger the reason we kept a min queue size was we are basically always at the min queue size | 17:20 |
| clarkb | I'd expect that if we allowed it to drop to 1 we would spend a significant amount of time there (which maybe that is a good thing) | 17:21 |
| *** smatzek_ is now known as smatzek | 17:21 | |
| pabelanger | Yah, 1 might be too low, but I could maybe see 10 in the case of tripleo | 17:23 |
| clarkb | also keep in mind the jobs in that pipeline are expected to pass | 17:23 |
| *** dmsimard is now known as dmsimard|afk | 17:23 | |
| clarkb | so I think optimizing for the expectation that they won't is the wrong tactic | 17:24 |
| pabelanger | but, would only result in less wasted resources i believe | 17:24 |
| pabelanger | clarkb: completely agree | 17:24 |
| pabelanger | first steps should be to make jobs less flappy, what ever is the cause of that is | 17:24 |
| pabelanger | window size changes, won't fix that | 17:24 |
| clarkb | did fungi's tarball signing changes make it in? | 17:25 |
| * clarkb goes looking | 17:25 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Add project-template for functional tips jobs https://review.openstack.org/515143 | 17:28 |
| *** vhosakot_ has joined #openstack-infra | 17:28 | |
| openstackgerrit | Monty Taylor proposed openstack/os-client-config master: Add shade-functional-tips jobs https://review.openstack.org/514067 | 17:29 |
| clarkb | looks like https://review.openstack.org/#/c/514856/2/zuul.d/project-templates.yaml is the last piece needed to fix tarball signing. I've rechecked it as it depends on a project-config change that merged | 17:29 |
| *** vhosako__ has joined #openstack-infra | 17:29 | |
| *** bhavik1 has quit IRC | 17:29 | |
| *** camunoz has joined #openstack-infra | 17:30 | |
| openstackgerrit | boden proposed openstack-infra/project-config master: remove release notes jobs from vmware-nsx https://review.openstack.org/515136 | 17:30 |
| *** bobh has quit IRC | 17:30 | |
| *** vhosakot has quit IRC | 17:30 | |
| clarkb | can I get a second config-core reviewer on 514856? it is queued into check which I think means it passes the syntax checking and all that config loading | 17:31 |
| pabelanger | looking | 17:32 |
| *** lindycoder_ has joined #openstack-infra | 17:32 | |
| clarkb | with that in we can knock an item off the issues etherpad | 17:32 |
| boden | BTW AJaeger, we are still having some issues with the vmware-nsx stable branch builds… I added it to the issues needing triage (https://etherpad.openstack.org/p/zuulv3-issues) as I’m unsure how to proceed ATM | 17:32 |
| *** lindycoder has quit IRC | 17:32 | |
| pabelanger | clarkb: +3 | 17:33 |
| *** vhosakot_ has quit IRC | 17:33 | |
| clarkb | pabelanger: tyty | 17:33 |
| clarkb | jeblair: dhellmann: I'm going to remove the release py35 job irrelevant files issue from the issues with jobs list as we understand that problem and have a general plan for addressing it (so I think we can just address that generally rather than targeted fixing) | 17:33 |
| mordred | jeblair, clarkb, pabelanger, tonyb: I'm not 100% here ... but https://review.openstack.org/#/c/514058/ is worth looking at - it's a followup to the constraints patch from jeblair that landed and allows removing tox_install.sh from projects (although there is still another issue for neutron/horizon projects) | 17:34 |
| *** trown|lunch is now known as trown | 17:34 | |
| mordred | https://review.openstack.org/#/c/508061/ is a shade patch removing tox_install.sh for proving that it dtrt | 17:34 |
| mordred | it's also worth noting that it adds collecting the generated/edited requirements and constraints files into the logs: http://logs.openstack.org/61/508061/6/check/shade-tox-py35-tips/7eab935/shade/ | 17:36 |
| mordred | (although we might want to change the name of the dir into which it goes) | 17:36 |
| jeblair | mordred: we're not running tox-siblings everywhere yet, right? | 17:37 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Avoid tox_install.sh for constraints support https://review.openstack.org/508061 | 17:38 |
| mordred | jeblair: I think we merged the patch to re-enable it | 17:38 |
| jeblair | oh, i thought we were waiting for the release to finish | 17:38 |
| *** Goneri has quit IRC | 17:38 | |
| jeblair | mordred: question on 514058 | 17:38 |
| clarkb | jeblair: you wanted to discuss https://review.openstack.org/#/c/512408/ more, https://review.openstack.org/#/c/512653/ has been pushed as an alternative (this is the tox + sudo thing) | 17:38 |
| mordred | jeblair: yah - https://review.openstack.org/#/c/513199 was merged - there'sa comment fromyou on it saying it was safe to merge now | 17:39 |
| mordred | jeblair: looking | 17:39 |
| *** rkukura has joined #openstack-infra | 17:39 | |
| openstackgerrit | Tin Lam proposed openstack-infra/project-config master: Remove legacy jobs https://review.openstack.org/515072 | 17:40 |
| *** rwsu has quit IRC | 17:40 | |
| *** lindycoder__ has joined #openstack-infra | 17:41 | |
| mordred | jeblair: responded - but tl;dr for other folks, that's in support of https://review.openstack.org/#/c/508694/ - update to PTI about how we build docs | 17:41 |
| jeblair | mordred: oh wow, i misparsed a diff between ps4 and ps5. i should not have merged https://review.openstack.org/513199 | 17:42 |
| mordred | jeblair: I'll work on adding more general support for that when I get back from vacation if nobody has beaten me to it, but I figured might as well have this approach for siblings be aware of it | 17:42 |
| jeblair | er, approved. | 17:42 |
| mordred | jeblair: nod. should be easy enough to push up a fix to the defaults file if we want to re-disable - want me to make that patch real quick? | 17:43 |
| jeblair | i had not intended to re-enable that until after the release. for some reason, i thought ps5 removed the re-enablement | 17:43 |
| *** Goneri has joined #openstack-infra | 17:43 | |
| *** lindycoder_ has quit IRC | 17:44 | |
| *** huanxie has quit IRC | 17:44 | |
| jeblair | mordred: yeah, that's probably the best solution so you can continue to work on this without making any surprises for the release team | 17:45 |
| jeblair | though i do worry slightly that we might have grown an accidental dependence on it | 17:45 |
| jeblair | (if it works) | 17:46 |
| mordred | yah. I'm betting we'd have heard about it otherwise | 17:46 |
| jeblair | mordred: anyway, i think we should either hold off on your change or merge a disable-by-default patch first | 17:47 |
| mordred | jeblair: the new patch is testable via depends-on - perhaps we just find a bunch of representative projects and push up depends-on patches to it and e careful aboutlanding it | 17:47 |
| jeblair | mordred: sorry, after delaying the release by as much as we have, i'm *really* sensitive about doing anything else that might break things. we're finally getting a handle on all the release jobs, we don't need to break any more. | 17:48 |
| *** jpena|brb is now known as jpena | 17:48 | |
| mordred | jeblair: and it's not urgent - it's in suport of being able to remove toxinstall.sh | 17:48 |
| mordred | jeblair: how about I mark it WIP for now | 17:49 |
| jeblair | mordred: wfm, thx | 17:49 |
| *** Rockyg has quit IRC | 17:49 | |
| jeblair | mordred: hopefully we'll be in the clear soon. | 17:49 |
| mordred | tonyb: if you get bored though - you may want to play with https://review.openstack.org/#/c/514058 via depends-on patches in the mean time | 17:49 |
| *** tosky has quit IRC | 17:50 | |
| openstackgerrit | melanie witt proposed openstack-infra/project-config master: Don't ignore nova/tests for the live-migration jobs https://review.openstack.org/515152 | 17:53 |
| jeblair | mordred: left one more comment on 514058 | 17:53 |
| mordred | jeblair: oh. nice catch, thanks | 17:54 |
| jschlueter | clarkb: ack and then if a queue is constantly getting reset then that is an indicator of a flappy job somewhere | 17:54 |
| AJaeger | config-core, could I get a second +2A on https://review.openstack.org/514916 to remove some newton related jobs, please? | 17:54 |
| openstackgerrit | melanie witt proposed openstack-infra/project-config master: Don't ignore nova/tests for the live-migration jobs https://review.openstack.org/515152 | 17:55 |
| *** salv-orlando has quit IRC | 17:55 | |
| *** salv-orlando has joined #openstack-infra | 17:56 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Have tox-siblings edit requirements files instead https://review.openstack.org/514058 | 17:56 |
| *** electrofelix has quit IRC | 17:57 | |
| mriedem | this fixes a regression in the move to zuulv3 https://review.openstack.org/#/c/515152/ - pretty important for testing changes to the live-migration job itself which is run via a hook in the nova tree | 17:58 |
| *** smatzek_ has joined #openstack-infra | 17:59 | |
| pabelanger | infra-root: do you mind looking at https://etherpad.openstack.org/p/zuulv2-removal for the next steps on zuulv2 removal of servers. We have 4 sections, delete, rebuild, build, rename. I'd like to start on delete this afternoon, if everybody is good | 18:00 |
| *** salv-orlando has quit IRC | 18:00 | |
| pabelanger | we do have a question about nodepool.o.o / zookeeper future | 18:00 |
| *** jascott1 has quit IRC | 18:01 | |
| *** smatzek has quit IRC | 18:02 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove ACLs for retired branches https://review.openstack.org/515153 | 18:03 |
| *** e0ne has joined #openstack-infra | 18:03 | |
| *** jpena is now known as jpena|off | 18:06 | |
| *** Apoorva_ has joined #openstack-infra | 18:07 | |
| haleyb | clarkb: ping, have another question on graphite stats | 18:07 |
| *** amoralej is now known as amoralej|off | 18:09 | |
| *** Apoorva has quit IRC | 18:10 | |
| clarkb | haleyb: ask away | 18:11 |
| haleyb | clarkb: is there a difference between something in stats.zuul vs stats_counts.zuul ? or is one preferred ? | 18:11 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: test playbook with ansible debug https://review.openstack.org/515157 | 18:13 |
| *** baoli has quit IRC | 18:13 | |
| *** strigazi_ has joined #openstack-infra | 18:13 | |
| *** baoli has joined #openstack-infra | 18:13 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: test playbook with ansible debug https://review.openstack.org/515157 | 18:14 |
| clarkb | haleyb: I think they have different data type | 18:15 |
| *** strigazi has quit IRC | 18:16 | |
| clarkb | stats_counts are counters and stats are gauges and timers? | 18:16 |
| clarkb | jeblair: ^ is that right? | 18:16 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Newton EOL: Remove some branch conditions https://review.openstack.org/515160 | 18:16 |
| *** strigazi_ is now known as strigazi | 18:17 | |
| *** rbrndt has joined #openstack-infra | 18:17 | |
| haleyb | clarkb: if it's any help, our "old" grafana page used stats_counts, but i think yesterday i saw both as an answer to my question | 18:17 |
| *** baoli has quit IRC | 18:18 | |
| *** slaweq has joined #openstack-infra | 18:19 | |
| haleyb | clarkb: by follow-up question is that what happens when a counter isn't there? for example, existing page pulls a foo.FAILURE stat, but i don't see a .FAILURE for the new graphite entries | 18:19 |
| clarkb | haleyb: I think we've always had both. You should use counts if you want raw incremented counters like how many failures are there aiui | 18:19 |
| clarkb | haleyb: I think that implies there are no failures for that job (yet) | 18:19 |
| openstackgerrit | Merged openstack-infra/project-config master: Newton EOl: Remove trove legacy jobs https://review.openstack.org/514916 | 18:20 |
| openstackgerrit | Merged openstack-infra/project-config master: Don't ignore nova/tests for the live-migration jobs https://review.openstack.org/515152 | 18:20 |
| haleyb | clarkb: thanks regarding stats_counts, will use that, and i guess i'm surprised of no failures, but could be | 18:20 |
| *** Sukhdev has joined #openstack-infra | 18:21 | |
| *** tosky has joined #openstack-infra | 18:22 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Use a more thorough release for API servers https://review.openstack.org/514856 | 18:22 |
| *** slaweq has quit IRC | 18:24 | |
| openstackgerrit | Anup Navare proposed openstack-infra/devstack-gate master: [Test] DNM Checking if tinyIPA builds with py3 https://review.openstack.org/509641 | 18:25 |
| *** Goneri has quit IRC | 18:25 | |
| clarkb | haleyb: you could cross reference against logstash maybe | 18:26 |
| haleyb | clarkb: yeah, i might do that. Perhaps these were always for gate jobs, which sometimes don't have as many failures | 18:27 |
| inc0 | hey guys, so I'm fairly close to get publisher to work, jsut to confirm periodic pipelines will allow me to use secrets right? | 18:30 |
| clarkb | inc0: they should since the operate on merged code | 18:30 |
| inc0 | is there any way for us to kick off periodic job before scheduled time just to see if it works? | 18:31 |
| *** slaweq has joined #openstack-infra | 18:31 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Remove duplicate post-review key in pipelines https://review.openstack.org/515165 | 18:32 |
| pabelanger | clarkb: something I noticed in look at periodic pipeline^ | 18:33 |
| *** baoli has joined #openstack-infra | 18:33 | |
| clarkb | inc0: I'm not sure if we can easily directly trigger periodic jobs | 18:34 |
| clarkb | jeblair: ^ | 18:34 |
| pabelanger | clarkb: I had to enqueue them using zuul client in the past | 18:35 |
| inc0 | yeah I'm just trying to figure out good workflow for developing gates with secrets | 18:35 |
| inc0 | did you decide on this "allow secrets with one +2"? | 18:36 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul feature/zuulv3: Support cgit urls on Gerrit connections https://review.openstack.org/515168 | 18:36 |
| *** slaweq has quit IRC | 18:36 | |
| *** Goneri has joined #openstack-infra | 18:36 | |
| openstackgerrit | Krzysztof Klimonda proposed openstack-infra/zuul feature/zuulv3: [WIP] Allow autohold on specific changes https://review.openstack.org/515169 | 18:36 |
| pabelanger | inc0: not yet, but something we have discussed | 18:37 |
| *** Sukhdev has quit IRC | 18:38 | |
| inc0 | another alternative would be to sed -i 's/<< contents of sectet >>/*redacted*/g | 18:38 |
| inc0 | for all that jobs publishes at the end | 18:39 |
| clarkb | inc0: to be clear no gates (at least today) can use secrets | 18:39 |
| inc0 | I know | 18:39 |
| *** csomerville has quit IRC | 18:39 | |
| clarkb | one thing you can do is run it locally using ansible | 18:39 |
| *** csomerville has joined #openstack-infra | 18:39 | |
| clarkb | this won't check all the zuul specific things but should make you fairly confident in the playbooks/roles | 18:39 |
| jeblair | catching up | 18:40 |
| *** yolanda has joined #openstack-infra | 18:40 | |
| *** eharney has quit IRC | 18:40 | |
| inc0 | right, that's not ideal, but possible | 18:40 |
| clarkb | I think SamYaple developed them in post so they ran after merging | 18:41 |
| clarkb | you could then move the job into periodic once happy with it | 18:41 |
| clarkb | (but depending on merge volume that could end up running a lot until you get there) | 18:41 |
| jeblair | haleyb: agree with clarkb. also, it's possible that zuulv3 stats are broken in some way, so do please investigate the lack of failures and let me know what you find. a specific failed job that should have incremented a counter but didn't would be especially useful, for example. | 18:41 |
| inc0 | well that still goes to merging untested code, but true it will give me feedback faster | 18:41 |
| clarkb | inc0: right by definition you can't get away from that | 18:42 |
| clarkb | because secrets cannot be used pre merge | 18:42 |
| inc0 | yeah I know, I'm just thinking of some "fake" secret mechanism | 18:42 |
| inc0 | I guess I can just add var: fake_secret and test on that | 18:43 |
| pabelanger | you can mock your secrets today, but just adding them into vars section. Just don't use your real secret | 18:43 |
| haleyb | jeblair: thanks, will do | 18:43 |
| jeblair | clarkb: i believe we should be able to enqueue a single project-branch into a periodic pipeline with 'zuul enqueue-ref'. i have not tested that though. | 18:43 |
| inc0 | right, but will allow me to test | 18:43 |
| jeblair | clarkb, inc0: you can use secrets in 'gate'. just not check. | 18:44 |
| *** rlandy is now known as rlandy|brb | 18:44 | |
| SamYaple | clarkb: im doing them in post because i see no other way without actually adding a job that affects other projects gates | 18:44 |
| jeblair | the key thing is pre-review is dangerous, not pre-merge per se | 18:44 |
| openstackgerrit | Brian Haley proposed openstack-infra/project-config master: Fix neutron-failure-rate grafana dashboard https://review.openstack.org/509291 | 18:44 |
| inc0 | SamYaple: what about periodic? | 18:44 |
| *** catintheroof has joined #openstack-infra | 18:44 | |
| inc0 | I'll just mock it with vars and maybe even push to fake dockerhub account | 18:45 |
| clarkb | jeblair: I thought we weren't configured that way though? | 18:48 |
| clarkb | or did I misunderstand the talk of adding the run this pipeline after +2 thing | 18:48 |
| jeblair | clarkb: gate allows secrets (ie, has post-review: True) | 18:48 |
| *** martinkopec has joined #openstack-infra | 18:48 | |
| jeblair | clarkb: the run after +2 is more like check | 18:48 |
| pabelanger | jeblair: what reason would there be for a secret in gate? Since a job in check couldn't use the secret, it would never get a +2 from zuul to be enqueued into gate to run with secret. Unless job only ran in gate and not check | 18:49 |
| *** wolverineav has quit IRC | 18:49 | |
| inc0 | so I think fake secret mock is acutally better - you're one mistake away from security breach | 18:49 |
| clarkb | jeblair: gotcha | 18:49 |
| *** catintheroof has quit IRC | 18:49 | |
| inc0 | but consider this sed at the end of the gate too - just additional layer of protection | 18:49 |
| *** wolverineav has joined #openstack-infra | 18:49 | |
| jeblair | clarkb: it would be so you can run a job that uses secrets on a proposed change and examine the results before gating. the +2 means you've verified it isn't a trojan change that exposes your secret. | 18:49 |
| SamYaple | inc0: i want to have new images more often than that for the purpose of gating | 18:50 |
| *** wolverineav has quit IRC | 18:50 | |
| inc0 | allright, I'm fine with 24h | 18:50 |
| haleyb | clarkb, jeblair: btw my infra change was ^^^ a few minutes ago, if you could take a look. don't think i broke anything | 18:50 |
| jeblair | pabelanger: it would be for jobs that only run in gate and not check. the pipeline i describe to clarkb above could improve that situation though. | 18:50 |
| *** wolverineav has joined #openstack-infra | 18:50 | |
| *** catintheroof has joined #openstack-infra | 18:50 | |
| jeblair | pabelanger: imagine for example if shade had a gate job that used mordred's cloud credentials to actually exercise public clouds. | 18:51 |
| jeblair | pabelanger: shade has lots of tests, so you'd be just as comfortable approving a change as you would be today, but you'd have the extra assurance that the gate would stop it from landing if it broke use with a certain public cloud. | 18:51 |
| pabelanger | jeblair: right, so it seems like something new we'd consider allowing. jobs only in gate. So far, I've always seen jobs both in check / gate or check only | 18:52 |
| SamYaple | inc0: right, kolla should be fine with 24h based on what you do. loci has different end goals | 18:52 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul feature/zuulv3: Support cgit urls on Gerrit connections https://review.openstack.org/515168 | 18:52 |
| SamYaple | inc0: additionally, we will be building images on the release/tag pipelines so we have tagged images when, say, cinder tags a new version | 18:52 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Newton EOL: Remove some branch conditions https://review.openstack.org/515160 | 18:52 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Check that zuul.d/projects.yaml is sorted https://review.openstack.org/514061 | 18:55 |
| *** salv-orlando has joined #openstack-infra | 18:55 | |
| inc0 | well here goes nothing, gate to push batch of images to fake account is out | 18:55 |
| *** eharney has joined #openstack-infra | 18:55 | |
| jeblair | AJaeger: wow! zuul emitted a regex syntax error! neat! i've never seen that: https://review.openstack.org/515160 | 18:58 |
| *** rbrndt has quit IRC | 19:00 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Newton EOL: Remove branches https://review.openstack.org/515177 | 19:03 |
| *** rbrndt has joined #openstack-infra | 19:03 | |
| pabelanger | jeblair: clarkb: I'm going to start deleting zl01-zl06 from rackspace and removal of DNS | 19:04 |
| AJaeger | jeblair: I was happy to see that ;) | 19:04 |
| jeblair | pabelanger: w00t! | 19:04 |
| clarkb | pabelanger: sounds good | 19:05 |
| *** slaweq has joined #openstack-infra | 19:05 | |
| *** Apoorva_ has quit IRC | 19:06 | |
| *** Apoorva has joined #openstack-infra | 19:06 | |
| *** armax has quit IRC | 19:08 | |
| *** xarses has joined #openstack-infra | 19:08 | |
| *** trown is now known as trown|brb | 19:09 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove duplicate post-review key in pipelines https://review.openstack.org/515165 | 19:12 |
| *** mummer has joined #openstack-infra | 19:16 | |
| EmilienM | pabelanger: could we promote 513701 to gate for a last time please, just to give a last change to merge before we disable voting on the job | 19:19 |
| *** trown|brb is now known as trown | 19:20 | |
| *** nikhil has joined #openstack-infra | 19:21 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Remove zuul-launcher support https://review.openstack.org/515181 | 19:21 |
| pabelanger | jeblair: clarkb:^ zuulv2-removal | 19:22 |
| pabelanger | EmilienM: why you know why it failed? I haven't looked at logs | 19:22 |
| EmilienM | pabelanger: it's an overcloud timeout, we might have a fix | 19:23 |
| EmilienM | pabelanger: https://review.openstack.org/#/c/515123/ | 19:23 |
| EmilienM | pabelanger: but it's hard to debug when this problem happens | 19:23 |
| pabelanger | enqueued again | 19:25 |
| EmilienM | pabelanger: thx. If that fails again, then we'll disable voting | 19:25 |
| pabelanger | yah, sounds like there is a larger issue then the fix for 513701 | 19:26 |
| *** huanxie has joined #openstack-infra | 19:27 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Check that zuul.d/projects.yaml is sorted https://review.openstack.org/514061 | 19:30 |
| pabelanger | #status log zl01.o.o to zl06.o.o, zlstatic01.o.o have been deleted and DNS entries removed from rackspace. | 19:30 |
| openstackstatus | pabelanger: finished logging | 19:30 |
| *** marst has joined #openstack-infra | 19:32 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Newton EOL: Remove periodic newton jobs https://review.openstack.org/515182 | 19:32 |
| AJaeger | one more periodic newton change - config-cores, please +2A ^ | 19:32 |
| *** salv-orlando has quit IRC | 19:33 | |
| *** salv-orlando has joined #openstack-infra | 19:35 | |
| *** iyamahat has quit IRC | 19:35 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/openstack-zuul-jobs master: Disable voting on 2 multinode tripleo jobs https://review.openstack.org/515183 | 19:35 |
| EmilienM | pabelanger: would that be enough to disable voting on these jobs? I didn't remove any job from gate, just disable voting. I'm sure I missed something | 19:36 |
| pabelanger | EmilienM: yes, you'll need to remove them from gate pipeline also, as they will still run | 19:37 |
| pabelanger | (and waste resources) | 19:37 |
| *** vaidy has quit IRC | 19:37 | |
| *** isviridov_away has quit IRC | 19:37 | |
| *** rwsu has joined #openstack-infra | 19:38 | |
| EmilienM | pabelanger: yeah ok let me find where again | 19:38 |
| EmilienM | in projcet-config ok | 19:38 |
| pabelanger | EmilienM: I've been trying not to use voting in job stanza, and just making changes in pipeline or template stanza. To limit files that get touched | 19:39 |
| *** rlandy|brb is now known as rlandy | 19:40 | |
| *** armax has joined #openstack-infra | 19:42 | |
| EmilienM | pabelanger: ok so I'll just patch projcet-config. Gotcha | 19:42 |
| pabelanger | ++ | 19:44 |
| *** Goneri has quit IRC | 19:46 | |
| pabelanger | AJaeger: clarkb: jeblair: what are our plans for infra-check? Think the pressure is off and we can move our projects that back into check now? | 19:47 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Create root ssh folder if not existing https://review.openstack.org/515185 | 19:48 |
| *** coolsvap has quit IRC | 19:51 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: test playbook with ansible debug https://review.openstack.org/515157 | 19:51 |
| *** Goneri has joined #openstack-infra | 19:53 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: Disable voting on 2 multinode tripleo jobs https://review.openstack.org/515188 | 19:54 |
| EmilienM | pabelanger: ^ is it good? | 19:55 |
| pabelanger | looking | 19:55 |
| pabelanger | +2 | 19:56 |
| EmilienM | we'll see how it pass | 19:56 |
| EmilienM | thanks | 19:56 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: test https://review.openstack.org/515189 | 19:58 |
| *** huanxie has quit IRC | 19:58 | |
| *** Goneri has quit IRC | 20:01 | |
| EmilienM | I swear to update my linkedin skills once we migrate tripleo to zuul v3 proper layout | 20:01 |
| *** eharney has quit IRC | 20:01 | |
| *** iyamahat has joined #openstack-infra | 20:02 | |
| *** rbrndt has quit IRC | 20:04 | |
| *** edmondsw_ has joined #openstack-infra | 20:06 | |
| *** rbrndt has joined #openstack-infra | 20:06 | |
| EmilienM | pabelanger: after some thoughts with tripleo folks, can we go ahead and land https://review.openstack.org/#/c/515188 please ? (just need approval) | 20:08 |
| jeblair | EmilienM: i'm looking at that one now | 20:09 |
| *** edmondsw has quit IRC | 20:09 | |
| EmilienM | thanks, very appreciated | 20:09 |
| EmilienM | jeblair: thx for the quick review | 20:11 |
| jeblair | np | 20:12 |
| jeblair | pabelanger: i lean toward keeping infra-check for a bit longer. project-config still sees a lot of churn. | 20:12 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Move test_model.test_job_inheritance_configloader https://review.openstack.org/509495 | 20:14 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove test_job_inheritance https://review.openstack.org/509509 | 20:14 |
| pabelanger | jeblair: wfm | 20:17 |
| openstackgerrit | Merged openstack-infra/project-config master: Disable voting on 2 multinode tripleo jobs https://review.openstack.org/515188 | 20:19 |
| *** smatzek_ has quit IRC | 20:21 | |
| *** rbrndt_ has joined #openstack-infra | 20:21 | |
| *** rbrndt has quit IRC | 20:22 | |
| *** smatzek has joined #openstack-infra | 20:22 | |
| *** ldnunes has quit IRC | 20:22 | |
| mriedem | question about https://review.openstack.org/#/c/514685/ | 20:24 |
| mriedem | it's been queued in the gate for 5.5 hours | 20:25 |
| mriedem | passed check a few times, | 20:25 |
| mriedem | does it need to be kicked out of the gate? | 20:25 |
| pabelanger | mriedem: looks like the integrate queue has reset a few times today | 20:25 |
| *** smatzek has quit IRC | 20:26 | |
| mriedem | pabelanger: so another recheck? | 20:27 |
| *** thorst has quit IRC | 20:27 | |
| mriedem | or should i +W again? | 20:27 |
| *** e0ne_ has joined #openstack-infra | 20:27 | |
| pabelanger | mriedem: no, it is in gate pipeline, just behind 16 other patches | 20:28 |
| pabelanger | recheck or +w won't help, in fact take it longer to merge | 20:28 |
| mriedem | ok now i see it moving again | 20:28 |
| pabelanger | http://logs.openstack.org/83/500783/5/gate/openstack-tox-py35/69c885b/ failed for some reason, which is why the gate reset | 20:28 |
| pabelanger | so, should see why that is | 20:28 |
| *** e0ne has quit IRC | 20:29 | |
| pabelanger | looks like requirements team has approve a few changes too, guess a busy day today | 20:30 |
| jeblair | yeah, the cinder test traceback here is interesting: http://logs.openstack.org/83/500783/5/gate/openstack-tox-py35/69c885b/job-output.txt.gz#_2017-10-25_19_31_02_403294 | 20:30 |
| dtroyer | The v3 devstack job in the devstack repo doesn't collect config files like v2 did, anyone know if that was intentional? That seems like the right place to re-add it? | 20:30 |
| mriedem | interesting as in super hard to read | 20:30 |
| smcginnis | jeblair: I've seen that before and I believe it was usually some kind of transient failure. | 20:31 |
| jeblair | dtroyer: it was not intentional! yes we should re-add it. i was going to start on some of that and then we switched and i became a firefighter.... let me see if i started anything... | 20:31 |
| dtroyer | jeblair: it looks like it should be easy enough for me to tackle if you haven't already | 20:32 |
| jeblair | dtroyer: looks like the little i have done landed in https://review.openstack.org/506744 (and did get copied over to devstack repo) | 20:34 |
| jeblair | dtroyer: also may be worth checking in with andreaf who was pursuing some general-purpose log collection routines | 20:34 |
| jeblair | dtroyer: https://review.openstack.org/509233 is related | 20:35 |
| *** huanxie has joined #openstack-infra | 20:35 | |
| jeblair | dtroyer: however -- those fancy new things should be upward compatible with anything we add now, so we don't need to wait for them. | 20:36 |
| EmilienM | pabelanger: is https://review.openstack.org/515204 enough to fix the zuul failure on https://review.openstack.org/#/c/515135 ? | 20:39 |
| *** slaweq has quit IRC | 20:39 | |
| *** kjackal_ has quit IRC | 20:39 | |
| *** slaweq has joined #openstack-infra | 20:40 | |
| *** e0ne has joined #openstack-infra | 20:42 | |
| pabelanger | EmilienM: no, I would read that as http://git.openstack.org/cgit/openstack/puppet-openstack-integration/tree/.zuul.yaml#n320 doesn't exist on stable/pike | 20:42 |
| pabelanger | zuul-executor couldn't find the playbook to run | 20:43 |
| pabelanger | I'm about to sit down for food, but can help when I return | 20:43 |
| EmilienM | pabelanger: shouldn'it failover on master? | 20:43 |
| *** tristanC has quit IRC | 20:43 | |
| EmilienM | mnaser: ^ you around? | 20:44 |
| pabelanger | EmilienM: no, depends on how your .zuul.yaml is setup. Does stable/pike have jobs defined too? | 20:44 |
| *** slaweq has quit IRC | 20:44 | |
| EmilienM | pabelanger: we have syntax on pike | 20:44 |
| jeblair | EmilienM: there was some confusion about this when we wrote the migration doc. but i think what we need to do is to backport the jobs as well as playbooks to stable branches. | 20:44 |
| EmilienM | https://review.openstack.org/#/c/509202/ | 20:45 |
| jeblair | EmilienM: it's a little extra work right now, but the next time we fork a stable branch from master, the right thing should just happen | 20:45 |
| *** tristanC has joined #openstack-infra | 20:45 | |
| *** e0ne_ has quit IRC | 20:45 | |
| *** eumel8 has quit IRC | 20:46 | |
| *** spiffxp_ has joined #openstack-infra | 20:46 | |
| *** spiffxp has quit IRC | 20:46 | |
| pabelanger | yah, I quick look shows the correct playbooks on pike. But I have to run now. I'll help after I eat | 20:47 |
| EmilienM | ok thank you | 20:47 |
| *** thorst has joined #openstack-infra | 20:48 | |
| *** thorst has quit IRC | 20:48 | |
| dtroyer | jeblair: thanks. I'm mostly after local.conf, was going to basically mimic the fetch-devstack-logs role | 20:48 |
| jeblair | EmilienM: where is puppet-openstack-syntax-4 defined? | 20:48 |
| jeblair | dtroyer: cool | 20:48 |
| jeblair | dtroyer: (or maybe just add to it?) | 20:49 |
| *** thorst has joined #openstack-infra | 20:49 | |
| dtroyer | sure? it isn't logs so I wasn't certain that would be kosher :) | 20:49 |
| dtroyer | but it will go to the same place | 20:49 |
| jeblair | dtroyer: oh, fetch-devstack-log-dir copies everything in ~/logs back to the executor. so you really just need something that copies local.conf into that location. | 20:50 |
| dtroyer | right | 20:51 |
| *** masuberu has quit IRC | 20:51 | |
| jeblair | yeah, that might best be added to a new role | 20:51 |
| jeblair | but then you can throw any other log collection stuff in there | 20:51 |
| tonyb | AJaeger: Thanks for confirming that re zuul job locations | 20:52 |
| *** thorst has quit IRC | 20:53 | |
| EmilienM | jeblair: in puppet-openstack-integration | 20:53 |
| jeblair | EmilienM: ah thanks. found it. | 20:53 |
| *** andreas_s has joined #openstack-infra | 20:54 | |
| *** edmondsw_ is now known as edmondsw | 20:55 | |
| *** yolanda has quit IRC | 20:56 | |
| *** sree has joined #openstack-infra | 20:58 | |
| tonyb | mordred: Thanks I have that open to look at but it keeps getting pre-empted :/ | 20:58 |
| *** rosmaita has quit IRC | 21:00 | |
| tonyb | mordred: bored? LOL but yeah I will play with it. IIRC openstackclient was special with it constraints support so that'd be a good one. There was another that had a bespoke solution ... Can't recall which though :/ | 21:02 |
| *** lindycoder__ has quit IRC | 21:02 | |
| EmilienM | jeblair: I need to leave irc for a few hours but I'll catch-up - let me know what I should do or send a patch if you can. Thanks for your help anyway | 21:02 |
| jeblair | EmilienM: will do | 21:02 |
| EmilienM | thx again | 21:02 |
| jeblair | np | 21:02 |
| *** mummer has quit IRC | 21:02 | |
| *** sree has quit IRC | 21:03 | |
| *** andreas_s has quit IRC | 21:03 | |
| *** hemna_ has quit IRC | 21:03 | |
| *** mummer has joined #openstack-infra | 21:03 | |
| *** andreas_s has joined #openstack-infra | 21:05 | |
| *** huanxie has quit IRC | 21:06 | |
| *** hasharAway has quit IRC | 21:07 | |
| *** dmsimard|afk is now known as dmsimard | 21:08 | |
| dtroyer | jeblair: simple config file role: https://review.openstack.org/515209 | 21:09 |
| *** andreas_s has quit IRC | 21:10 | |
| *** mummer has quit IRC | 21:10 | |
| *** andreas_s has joined #openstack-infra | 21:11 | |
| *** mat128 has quit IRC | 21:11 | |
| *** baoli has quit IRC | 21:13 | |
| *** mummer has joined #openstack-infra | 21:13 | |
| *** rcernin has joined #openstack-infra | 21:16 | |
| *** andreas_s has quit IRC | 21:16 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Move to dictionary list of projects zuul._projects (take 2) https://review.openstack.org/514117 | 21:16 |
| *** andreas_s has joined #openstack-infra | 21:16 | |
| jeblair | EmilienM, pabelanger: i think (though i'm not certain), that is a manifestation of one of the problems i described in http://lists.openstack.org/pipermail/openstack-infra/2017-October/005634.html | 21:17 |
| jeblair | EmilienM, pabelanger: i pushed up a patch to verify and reparented 515135 on it | 21:17 |
| jeblair | dtroyer: ++ | 21:18 |
| tosky | jeblair: hi again: about my question on multinode devstack jobs without using devstack-gate... | 21:20 |
| tosky | how do I make sure that a compute node is only a compute node without using DEVSTACK_SUBNODE_CONFIG=" " ? | 21:21 |
| *** dprince has quit IRC | 21:23 | |
| *** e0ne has quit IRC | 21:25 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements https://review.openstack.org/503645 | 21:27 |
| *** sbezverk has quit IRC | 21:27 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Newton EOL: Remove branches https://review.openstack.org/515177 | 21:29 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Newton-EOL: Remove trove legacy jobs https://review.openstack.org/514918 | 21:29 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove monasca legacy job https://review.openstack.org/514251 | 21:29 |
| *** esberglu has quit IRC | 21:29 | |
| *** priteau has quit IRC | 21:30 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Cleanup ironic-python-agent legacy jobs https://review.openstack.org/515039 | 21:31 |
| *** rosmaita has joined #openstack-infra | 21:33 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove legacy nodepool job defs and playbooks https://review.openstack.org/514321 | 21:35 |
| *** panda|ruck|bbl is now known as panda|ruck | 21:35 | |
| EmilienM | jeblair: back | 21:35 |
| EmilienM | jeblair: ok thx, let's see | 21:35 |
| EmilienM | jeblair: do you want me to Depends-On your patch in my failing patch in puppet-tripleo? | 21:36 |
| EmilienM | oh you did | 21:36 |
| EmilienM | it's failing | 21:37 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Removes migrated legacy-octavia-* https://review.openstack.org/513785 | 21:38 |
| *** boden has quit IRC | 21:38 | |
| jeblair | tosky: i don't think we've done any multinode stuff with the new devstack jobs yet. most or all of the pieces should be in place though -- the relevant roles should be in zuul-jobs or devstack repos. i'd love to, though i can't help with that right now. clarkb, pabelanger, dtroyer, and andreaf may be able to assist. hopefully i can catch up later this week or next. | 21:42 |
| *** jcoufal has quit IRC | 21:43 | |
| jeblair | EmilienM: hrm, okay, i'm going to need to do some more intensive debugging then. i'll WIP my poi change and remove the depends-on after i set all the executors to keep jobs. | 21:43 |
| tosky | jeblair: ack, thanks! I will recheck the open reviews and wait for some other hints | 21:43 |
| *** esberglu has joined #openstack-infra | 21:43 | |
| clarkb | jeblair: tosky ya we haven't gotten there but I imagine we'd supply a localrc for each of the nodes | 21:43 |
| clarkb | and in the subnode(s) localrc you'd specify what services you want | 21:44 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: sahara-dashboard: remove the migrated unittests https://review.openstack.org/515019 | 21:44 |
| jeblair | infra-root: i'm going to set 'keep' on all the executors | 21:44 |
| EmilienM | ok | 21:44 |
| jeblair | EmilienM: done. the next time it tries to run that job, i'll be able to inspect the git repo state it prepared | 21:45 |
| EmilienM | ok | 21:45 |
| *** iyamahat has quit IRC | 21:45 | |
| tosky | clarkb: the basic idea is easy, but how to organize this configuration? Add a devstack_subnode: key indexed by the host/group with subkeys like devstack_localrc, devstack_services, etc? | 21:45 |
| *** sree has joined #openstack-infra | 21:45 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Update README to reflect current status https://review.openstack.org/514195 | 21:46 |
| *** andreas_s has quit IRC | 21:47 | |
| jeblair | infra-root: disabling keep | 21:47 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Run gerrit periodic on openstack/2.13 only https://review.openstack.org/514127 | 21:47 |
| *** esberglu has quit IRC | 21:48 | |
| openstackgerrit | Merged openstack-infra/project-config master: Newton EOL: Remove some branch conditions https://review.openstack.org/515160 | 21:49 |
| *** sree has quit IRC | 21:50 | |
| *** edmondsw has quit IRC | 21:51 | |
| *** wolverineav has quit IRC | 21:52 | |
| *** wolverineav has joined #openstack-infra | 21:52 | |
| *** andreas_s has joined #openstack-infra | 21:53 | |
| *** rwsu has quit IRC | 21:54 | |
| *** rwsu has joined #openstack-infra | 21:54 | |
| *** edmondsw has joined #openstack-infra | 21:56 | |
| *** sdague has quit IRC | 21:56 | |
| *** rwsu has quit IRC | 21:56 | |
| *** wolverineav has quit IRC | 21:57 | |
| *** andreas_s has quit IRC | 21:57 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul feature/zuulv3: Default change and patchset to NULL in SQLReporter https://review.openstack.org/514423 | 21:58 |
| *** marst has quit IRC | 21:58 | |
| *** owalsh has quit IRC | 22:00 | |
| *** owalsh has joined #openstack-infra | 22:00 | |
| *** kzaitsev_pi has quit IRC | 22:02 | |
| *** kzaitsev_pi has joined #openstack-infra | 22:04 | |
| *** edmondsw has quit IRC | 22:04 | |
| *** smatzek has joined #openstack-infra | 22:10 | |
| *** kzaitsev_pi has quit IRC | 22:11 | |
| EmilienM | pabelanger: just a heads-up, https://review.openstack.org/513701 is merged which will help. We identified 2 other things that will stabilize our gate | 22:12 |
| EmilienM | jeblair: ^ as well | 22:12 |
| EmilienM | thank you folks, we have some good progress | 22:12 |
| EmilienM | we also think that we have identified why we have so much timeouts, we did a patch and it's in gate now | 22:12 |
| EmilienM | we'll see if that help over the next hours/days | 22:13 |
| EmilienM | (I found useful to share that with you, to see there is some little progress) | 22:13 |
| *** kzaitsev_pi has joined #openstack-infra | 22:13 | |
| *** smatzek has quit IRC | 22:15 | |
| jeblair | EmilienM: yay! | 22:15 |
| jeblair | EmilienM, pabelanger: i think i have identified the problem with the syntax-4 change | 22:15 |
| jeblair | er job | 22:15 |
| EmilienM | yay² | 22:15 |
| jeblair | EmilienM, pabelanger: briefly, there's a bug related to variants and inheritance on stable branches. the variant of that job on the master branch is correctly picking up file matchers, but the variant on the stable branch is not. | 22:16 |
| *** andreas_s has joined #openstack-infra | 22:17 | |
| jeblair | EmilienM, pabelanger: the work i'm doing in that area should make this easy to fix when it's complete, but i don't think i can fix it in zuul before that work is complete. i think having those fixes in production friday is a reasonable time estimate. | 22:18 |
| *** catintheroof has quit IRC | 22:18 | |
| EmilienM | jeblair: it's ok for me | 22:18 |
| EmilienM | jeblair: it's not critical for now to have my puppet-tripleo patches merged if we can fix it on Friday | 22:18 |
| jeblair | EmilienM, pabelanger: so the options i can think of are: a) wait until friday. b) move tripleo configuration back into project-config (until friday). c) copy a bunch of extra irrelevant-files matchers around to those jobs. | 22:19 |
| EmilienM | jeblair: a° | 22:19 |
| EmilienM | a) | 22:19 |
| jeblair | EmilienM: okay. i think that would be best if we have time (everything else would be a temporary measure which just means more work) | 22:19 |
| EmilienM | yep | 22:19 |
| jeblair | i'll get back to to that now then. thanks :) | 22:19 |
| *** iyamahat has joined #openstack-infra | 22:20 | |
| EmilienM | cool, thanks to *you* | 22:20 |
| *** xinliang has quit IRC | 22:20 | |
| jeblair | also, thanks for pointing it out. this is related, but it's also a new sub-case i didn't catch earlier. | 22:20 |
| jeblair | (i think i know how to add a test for it) | 22:21 |
| EmilienM | jeblair: I guess tripleo is here to make zuul even better :P even all our weirdness :) | 22:21 |
| jeblair | EmilienM: my hope is that the opposite will be true soon as well :) | 22:22 |
| *** felipemonteiro__ has quit IRC | 22:24 | |
| *** andreas_s has quit IRC | 22:26 | |
| *** andreas_s has joined #openstack-infra | 22:28 | |
| *** iyamahat has quit IRC | 22:28 | |
| *** edmondsw has joined #openstack-infra | 22:28 | |
| *** edmondsw has quit IRC | 22:34 | |
| *** panda|ruck is now known as panda|ruck|off | 22:35 | |
| *** thorst has joined #openstack-infra | 22:35 | |
| tonyb | frickler: Are you around? | 22:40 |
| openstackgerrit | Merged openstack-infra/shade master: Add project-template for functional tips jobs https://review.openstack.org/515143 | 22:40 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements https://review.openstack.org/503645 | 22:41 |
| *** thorst has quit IRC | 22:42 | |
| *** felipemonteiro_ has joined #openstack-infra | 22:43 | |
| *** thorst has joined #openstack-infra | 22:43 | |
| *** andreas_s has quit IRC | 22:44 | |
| openstackgerrit | Merged openstack-infra/project-config master: Newton EOL: Remove periodic newton jobs https://review.openstack.org/515182 | 22:44 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy jobs https://review.openstack.org/515072 | 22:44 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Copy Gerrit delay to GitHub driver https://review.openstack.org/515224 | 22:45 |
| SpamapS | jeblair: ^ copies over quite nicely I thik | 22:46 |
| SpamapS | think even | 22:46 |
| *** claudiub|2 has joined #openstack-infra | 22:46 | |
| *** thorst has quit IRC | 22:46 | |
| *** wolverineav has joined #openstack-infra | 22:48 | |
| *** martinkopec has quit IRC | 22:49 | |
| *** rbrndt_ has quit IRC | 22:50 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP: Add implied branch matchers on 'master' https://review.openstack.org/514459 | 22:51 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP: experiment with late-binding inheritance https://review.openstack.org/511352 | 22:51 |
| *** Sukhdev has joined #openstack-infra | 22:52 | |
| *** felipemonteiro_ has quit IRC | 22:53 | |
| jlvillal | I've pushed two patches to openstack/ironic in the last 10 minutes, but 'openstackgerrit' did not announce either of them. Any ideas? | 22:55 |
| jlvillal | This patch done twice: https://review.openstack.org/#/c/515225/ | 22:56 |
| *** tpsilva has quit IRC | 22:57 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul feature/zuulv3: Default change and patchset to NULL in SQLReporter https://review.openstack.org/514423 | 22:58 |
| ianw | jlvillal: has it announced anything else? | 23:01 |
| jlvillal | ianw: It did 30 minutes ago on a patch merging | 23:02 |
| *** tosky has quit IRC | 23:02 | |
| jlvillal | ianw: And a few hours ago some proposed patches to openstack/ironic | 23:02 |
| *** Sukhdev has quit IRC | 23:03 | |
| jlvillal | ianw: It just announced a merge | 23:03 |
| *** xarses has quit IRC | 23:04 | |
| *** hongbin has quit IRC | 23:05 | |
| *** masuberu has joined #openstack-infra | 23:05 | |
| ianw | it thinks it sent it | 23:05 |
| ianw | root@review:/var/log/gerritbot# cat gerritbot.log | grep 515225 | grep Sending | 23:06 |
| ianw | 2017-10-25 22:45:59,552 INFO gerritbot: Sending "John L. Villalovos proposed openstack/python-ironicclient master: Simplify logic for API version '1' and improve unit tests https://review.openstack.org/515225" to #openstack-ironic | 23:06 |
| ianw | 2017-10-25 22:55:05,400 INFO gerritbot: Sending "John L. Villalovos proposed openstack/python-ironicclient master: Simplify logic for API version '1' and improve unit tests https://review.openstack.org/515225" to #openstack-ironic | 23:06 |
| openstackgerrit | Merged openstack-infra/release-tools master: Add some notes on eol_branch.sh https://review.openstack.org/514846 | 23:06 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements https://review.openstack.org/503645 | 23:07 |
| *** andreas_s has joined #openstack-infra | 23:10 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements https://review.openstack.org/503645 | 23:11 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP: experiment with late-binding inheritance https://review.openstack.org/511352 | 23:13 |
| *** vaidy has joined #openstack-infra | 23:14 | |
| *** isviridov_away has joined #openstack-infra | 23:14 | |
| jlvillal | ianw: Hmm. I never saw it on IRC | 23:15 |
| clarkb | it doesn't seem to be in the log either | 23:15 |
| clarkb | its possible that gerritbot was netsplit or effectively split? | 23:15 |
| jlvillal | ianw: And I don't see it here: http://eavesdrop.openstack.org/irclogs/%23openstack-ironic/%23openstack-ironic.2017-10-25.log.html | 23:15 |
| jlvillal | I'll just assume it will start working again at some point :) | 23:15 |
| jlvillal | I'm going to push an update in a minute or so. | 23:16 |
| jlvillal | clarkb: netsplit sounds reasonable | 23:16 |
| *** andreas_s has quit IRC | 23:21 | |
| *** gouthamr has quit IRC | 23:22 | |
| *** gouthamr has joined #openstack-infra | 23:22 | |
| *** andreas_s has joined #openstack-infra | 23:23 | |
| *** gridinv has joined #openstack-infra | 23:23 | |
| *** gmann_afk is now known as gmann | 23:25 | |
| dmsimard | clarkb: hey, I'm curious.. how long does a typical log prune on logs.o.o take ? the massive find that deletes files and dirs. | 23:25 |
| clarkb | dmsimard: I think a few days | 23:25 |
| dmsimard | clarkb: oh, okay, I thought something was horribly wrong on our end -- I'm running a slightly modified version for the RDO logserver | 23:26 |
| clarkb | that is why it only runs weekly | 23:26 |
| dmsimard | The fact that it'd run daily would not hurt, right ? there's a lockfile | 23:26 |
| *** yamahata has joined #openstack-infra | 23:27 | |
| clarkb | ya we could let the lock figure it out | 23:29 |
| dmsimard | I figured the file deletion was not necessary (and skip straight to directories) but then quickly understood why it was necessary :/ | 23:29 |
| jlvillal | ianw: clarkb: My latest push was announced. Thanks. | 23:31 |
| *** andreas_s has quit IRC | 23:31 | |
| *** yamahata has quit IRC | 23:32 | |
| *** andreas_s has joined #openstack-infra | 23:33 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove ACLs for retired branches https://review.openstack.org/515153 | 23:35 |
| openstackgerrit | Merged openstack-infra/project-config master: Change documentation job for OpenStack-Helm https://review.openstack.org/515080 | 23:36 |
| pabelanger | EmilienM: jeblair: great, glad we figured out the issue | 23:40 |
| *** andreas_s has quit IRC | 23:43 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/puppet-nodepool master: Make the secure.conf template configurable https://review.openstack.org/515228 | 23:43 |
| openstackgerrit | Merged openstack-infra/project-config master: Check that zuul.d/projects.yaml is sorted https://review.openstack.org/514061 | 23:44 |
| openstackgerrit | Merged openstack-infra/project-config master: remove release notes jobs from vmware-nsx https://review.openstack.org/515136 | 23:44 |
| openstackgerrit | Merged openstack-infra/project-config master: avoid creating extra temporary directory in release job https://review.openstack.org/514787 | 23:44 |
| openstackgerrit | Merged openstack-infra/project-config master: remove unused release-tools scripts https://review.openstack.org/514816 | 23:44 |
| *** mriedem is now known as mriedem_away | 23:44 | |
| *** mriedem_away has quit IRC | 23:44 | |
| openstackgerrit | Eric Kao proposed openstack-infra/irc-meetings master: Change Congress meeting time https://review.openstack.org/513467 | 23:45 |
| *** andreas_s has joined #openstack-infra | 23:45 | |
| *** thorst has joined #openstack-infra | 23:45 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP: experiment with late-binding inheritance https://review.openstack.org/511352 | 23:46 |
| *** thorst has quit IRC | 23:46 | |
| *** andreas_s has quit IRC | 23:54 | |
| *** rlandy has quit IRC | 23:54 | |
| *** baoli has joined #openstack-infra | 23:57 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP: Add implied branch matchers on 'master' https://review.openstack.org/514459 | 23:58 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP: experiment with late-binding inheritance https://review.openstack.org/511352 | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!