| *** dave-mccowan has quit IRC | 00:00 | |
| clarkb | 08 starting now | 00:00 |
|---|---|---|
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 00:00 |
| *** dave-mccowan has joined #openstack-infra | 00:01 | |
| clarkb | 2017-10-17 00:01:57,297 DEBUG zuul.AnsibleJob: [build: de1319e83b0649ed955fb2ba81c9fe8d] Ansible complete, result RESULT_NORMAL code 0 that is 08 looking done | 00:02 |
| clarkb | I'm going to restart 01 03 09 too to pick up the latest code | 00:04 |
| *** dhinesh has joined #openstack-infra | 00:04 | |
| clarkb | 09 is restarted | 00:08 |
| *** baoli has quit IRC | 00:08 | |
| *** dhinesh_ has joined #openstack-infra | 00:08 | |
| *** dhinesh has quit IRC | 00:09 | |
| dmsimard | mordred: I remember discussing at the PTG some sort of super quick installer (or was it an image?) you wanted to work on for testing things like shade or openstackclient. Was there ever anything done ? | 00:10 |
| clarkb | jeblair: pabelanger fwiw I accidentally attempted to start 'zuul' and not 'zuul-executor' on 09. It seems to hae correctly failed to start (and then I explicitly stopped it too) | 00:10 |
| clarkb | reading the init script I would've expected it to maybe work a bit more than it did but am happy it didn't work | 00:10 |
| clarkb | 10 is up 2017-10-17 00:10:52,641 DEBUG zuul.AnsibleJob: [build: 52d4552b03f544eb83830a9755893233] Ansible complete, result RESULT_NORMAL code 0 | 00:11 |
| clarkb | the inode situation should now be addressed. Just doing the last cleanup of restarting 01 and 03 so they run the same code as the others | 00:12 |
| clarkb | looks like init script would've failed becase DAEMON_ARGS is not defined? | 00:13 |
| *** edmondsw has joined #openstack-infra | 00:13 | |
| clarkb | so there would've been missing required params to start the daemon | 00:13 |
| clarkb | ok now I feel better about that derp | 00:13 |
| mordred | dmsimard: when you say "quick installer" - you mean like for zuul itself? perhaps you're referring to "some tool to let one easily run the ansible for a zuul job locally" | 00:14 |
| dmsimard | mordred: no, for openstack | 00:14 |
| mordred | dmsimard: oh! yah. that | 00:14 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Graphana: Add Zuul v3 executor graphs https://review.openstack.org/512451 | 00:14 |
| mordred | dmsimard: no - work has not yet been done - I intend to do tha tsoon, otherwise dtroyer and I lose testing environment for some legacy things | 00:15 |
| jeblair | clarkb, dmsimard: 512451 starts to polish up the zuul dashboard | 00:15 |
| mordred | dmsimard: tl;dr "apt-get install keystone nova cinder glance" on an ubuntu-xenail node theoretically gets a 'working' but probably highly unoptimized cloud | 00:15 |
| *** rossella_s has quit IRC | 00:16 | |
| mordred | jeblair: ooh | 00:16 |
| dmsimard | mordred: yeah, I was just curious what kind of approach you were going for -- I guess speed of deployment becomes important so I thought maybe container images? Dunno. | 00:16 |
| mordred | dmsimard: no - simplicity and supportedness is the main concern | 00:16 |
| dmsimard | jeblair: s/ph/f/ :p | 00:16 |
| clarkb | 2017-10-17 00:16:25,304 DEBUG zuul.AnsibleJob: [build: a00b90bbafd54287be96adba4b584c7f] Ansible exit code: 0 03 is done | 00:16 |
| mordred | dmsimard: becuase what we want is newton or earlier clouds | 00:16 |
| dmsimard | mordred: fair, the triangle of simplicity, supportedness and speed, can't have all 3 | 00:16 |
| mordred | yup | 00:16 |
| mordred | :)_ | 00:16 |
| mordred | also - we don't need these to be good - in fact, what we need is for them to seem like old antiquated clouds | 00:17 |
| *** salv-orlando has joined #openstack-infra | 00:17 | |
| mordred | so that we can ensure shade and osc still work with them | 00:17 |
| jeblair | i would point out that "old" is orthogonal to "good" | 00:17 |
| *** markvoelker_ has quit IRC | 00:17 | |
| clarkb | just one more restart then I can call it a day | 00:17 |
| mordred | jeblair: yup | 00:17 |
| mordred | jeblair: in this case, old is more important than good - although if we could hav both I'd be find with that | 00:18 |
| *** edmondsw has quit IRC | 00:18 | |
| jeblair | mordred: that zuul-sphinx change 512449 passed tests if you want to review it | 00:19 |
| jeblair | example output is available | 00:19 |
| *** rossella_s has joined #openstack-infra | 00:19 | |
| mordred | jeblair: looks great! | 00:21 |
| *** salv-orlando has quit IRC | 00:22 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Document base job change procedure https://review.openstack.org/512138 | 00:22 |
| clarkb | fwiw stopping 01 doesn't seem fast; however I will be happy to see it stop reliably when it is ready | 00:22 |
| clarkb | oh wait 01 didn't get the stop fast patch did it? | 00:22 |
| clarkb | it just got the dev null one that explain it | 00:22 |
| jeblair | clarkb: correct. i cherry-picked the second one since i wanted to verify the emergency fix without more variables | 00:23 |
| jeblair | so this *should* be the last time we have slow stops | 00:23 |
| *** yamamoto has joined #openstack-infra | 00:24 | |
| *** markvoelker has joined #openstack-infra | 00:25 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Create more inodes on ephemeral partition fs https://review.openstack.org/512453 | 00:27 |
| clarkb | ok ^ si a follow up so we don't have to do this again in the future | 00:27 |
| *** yamamoto has quit IRC | 00:29 | |
| *** markvoelker has quit IRC | 00:29 | |
| *** esberglu has joined #openstack-infra | 00:30 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/storyboard master: Bump minimum os-testr requirement to 1.0.0 https://review.openstack.org/504525 | 00:30 |
| anupn | I see my legacy-nodejs4-npm-lint job failed but i do not see any logs under it | 00:31 |
| anupn | http://logs.openstack.org/57/473957/18/check/legacy-nodejs4-npm-run-lint/243f3c3/ | 00:31 |
| dtroyer | mordred: fwiw, I've started on that, have keystone done. it looks a lot like a single-purpose devstack: https://github.com/dtroyer/x-stack | 00:31 |
| anupn | do i have to migrate it? I think we are not running zuul jobs for ironic-ui | 00:31 |
| SamYaple | /win/win 28 | 00:32 |
| clarkb | anupn: can you recheck it? we've had to restart the zuul executors to fix an inode issue and there is a chance that process resulted in the lack of logs for that job | 00:32 |
| anupn | clarkb, i last rechecked on 13th oct | 00:33 |
| anupn | were there any changes after that | 00:33 |
| clarkb | anupn: yes all day today fixing a problem that could result in no logs | 00:33 |
| *** wolverineav has quit IRC | 00:33 | |
| anupn | clarkb, and on that recheck my jenkins are passing | 00:33 |
| mordred | dtroyer: woot! | 00:33 |
| clarkb | anupn: it would be helpful to work off of current data | 00:33 |
| anupn | clarkb, hmm alright. I will recheck it again | 00:34 |
| anupn | clarkb, thanks | 00:34 |
| *** markvoelker has joined #openstack-infra | 00:34 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 00:34 |
| anupn | clarkb, yes because i am sure it is due to some migration as jenkins gate jobs are passing | 00:35 |
| * anupn hopes this recheck helps :) | 00:35 | |
| *** cshastri has joined #openstack-infra | 00:35 | |
| mordred | anupn, clarkb: also - I'm working on new-style npm/node jobs - but am a little behind | 00:35 |
| anupn | mordred, what do you mean new style, sorry did not get you | 00:36 |
| *** rossella_s has quit IRC | 00:36 | |
| mordred | anupn: native zuul v3 - not autoconverted from jenkins-job-uilder macros | 00:36 |
| anupn | mordred, oh i see. | 00:37 |
| anupn | mordred, nice | 00:37 |
| anupn | mordred, thumbs up :) | 00:37 |
| SamYaple | in the next few weeks i am going to be attempting a gerrit+zuulv3 infra deploy internally. so thatlly be fun. im sure ill have lots of questions | 00:37 |
| *** psachin has joined #openstack-infra | 00:38 | |
| *** rossella_s has joined #openstack-infra | 00:39 | |
| *** markvoelker has quit IRC | 00:39 | |
| clarkb | anupn: http://zuulv3.openstack.org/static/stream.html?uuid=867e7700e45645d3a2253f93ea81b338&logfile=console.log you can follow your rechecked job there | 00:39 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 00:40 |
| anupn | clarkb, cool | 00:40 |
| clarkb | I have to run an errand in about 10 minutes so starting 01 back up again may be delayed while I do that (but will be sure to get back and restart on 01 when I return) | 00:41 |
| *** markvoelker has joined #openstack-infra | 00:43 | |
| *** Swami has quit IRC | 00:43 | |
| *** Apoorva_ has joined #openstack-infra | 00:44 | |
| *** esberglu has quit IRC | 00:45 | |
| anupn | clarkb, woot it passed | 00:46 |
| clarkb | anupn: yup, not sure if it did the right thing it ran a tox -e py27 thing which seems odd for a lint job? | 00:47 |
| clarkb | it did install eslint though | 00:47 |
| clarkb | anyways I haev to run now | 00:47 |
| *** markvoelker has quit IRC | 00:48 | |
| anupn | clarkb, yes it installed eslint. And there is another gate job for py27 under zuul i think it did that. | 00:48 |
| anupn | clarkb, yes will talk later | 00:48 |
| *** Apoorva has quit IRC | 00:48 | |
| *** huanxie has joined #openstack-infra | 00:48 | |
| *** Apoorva_ has quit IRC | 00:48 | |
| pabelanger | clarkb: cool, that means we now have https://review.openstack.org/512158/ on executors too> | 00:48 |
| pabelanger | ssh persistent connections | 00:49 |
| clarkb | pabelanger: everywhere but 01 | 00:49 |
| clarkb | or no we have it there too | 00:49 |
| pabelanger | cool | 00:49 |
| clarkb | 01 needs restart for fast stop (working in that) | 00:50 |
| pabelanger | kk | 00:50 |
| *** rossella_s has quit IRC | 00:52 | |
| *** markvoelker has joined #openstack-infra | 00:52 | |
| *** mriedem1 has joined #openstack-infra | 00:54 | |
| *** mriedem has quit IRC | 00:55 | |
| *** mriedem1 is now known as mriedem | 00:55 | |
| *** rossella_s has joined #openstack-infra | 00:55 | |
| *** dingyichen has joined #openstack-infra | 00:56 | |
| *** markvoelker has quit IRC | 00:57 | |
| *** cuongnv has joined #openstack-infra | 00:59 | |
| *** edmondsw has joined #openstack-infra | 01:00 | |
| *** markvoelker has joined #openstack-infra | 01:01 | |
| *** LindaWang has joined #openstack-infra | 01:02 | |
| *** LindaWang has quit IRC | 01:02 | |
| *** sree has joined #openstack-infra | 01:04 | |
| *** LindaWang has joined #openstack-infra | 01:04 | |
| *** markvoelker has quit IRC | 01:06 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: Use tempfile for get_url task https://review.openstack.org/512458 | 01:07 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: Use tempfile for get_url task set-service-type-data-fact https://review.openstack.org/512458 | 01:08 |
| pabelanger | AJaeger: mordred ^fix for api-ref post jobs | 01:08 |
| *** sree has quit IRC | 01:08 | |
| *** markvoelker has joined #openstack-infra | 01:11 | |
| clarkb | pabelanger: is src not https://service-types.openstack.org/service-types.json ? | 01:13 |
| pabelanger | clarkb: no see: http://docs.ansible.com/ansible/latest/get_url_module.html | 01:13 |
| pabelanger | clarkb: in fact along with the tmp file, in /tmp | 01:14 |
| pabelanger | it will also create /tmp/service-types.json | 01:14 |
| pabelanger | so result.src and result.dest | 01:14 |
| pabelanger | both appear to be same contents in local testing | 01:15 |
| *** markvoelker has quit IRC | 01:15 | |
| *** aeng has quit IRC | 01:17 | |
| ianw | http://zuulv3.openstack.org/static/stream.html?uuid=84897ed6b9bf4d8fbc0f94792effef00&logfile=console.log appears to be a bit stuck | 01:17 |
| pabelanger | ianw: which jobs is that? | 01:18 |
| *** salv-orlando has joined #openstack-infra | 01:18 | |
| ianw | pabelanger: that's dib-dsvm-functests-python2-ubuntu-trusty in 512164 | 01:18 |
| pabelanger | ianw: as I guess, ze01 | 01:19 |
| pabelanger | and I think clarkb is stopping ze01 | 01:19 |
| ianw | ok, i won't go to any debugging effort atm | 01:20 |
| pabelanger | ianw: I added a issue to https://etherpad.openstack.org/p/zuulv3-issues about stuck jobs on ze02 in issues with zuul | 01:20 |
| pabelanger | might want to add this one too | 01:20 |
| pabelanger | likely related | 01:20 |
| clarkb | ya stopping 01 to pick up the last of the fixes | 01:20 |
| *** markvoelker has joined #openstack-infra | 01:20 | |
| clarkb | it is almost done | 01:20 |
| *** kiennt26 has joined #openstack-infra | 01:21 | |
| *** iyamahat has quit IRC | 01:21 | |
| ianw | dib-dsvm-functests-python2-ubuntu-trusty | Unknown | Unknown ago | None | | 01:21 |
| *** salv-orlando has quit IRC | 01:22 | |
| *** yamahata has quit IRC | 01:23 | |
| ianw | 2017-10-17 00:12:39,421 INFO zuul.ExecutorClient: Execute job dib-dsvm-functests-python2-ubuntu-trusty (uuid: 84897ed6b9bf4d8fbc0f94792effef00) on nodes <NodeSet legacy-ubuntu-trusty OrderedDict([('primary', <Node 0000243791 primary:ubuntu-trusty>)])OrderedDict()> for change <Change 0x7f15612b0c18 512164,3> with dependent changes [] | 01:24 |
| ianw | how do you determine which launcher picked it up easily? | 01:24 |
| *** yamamoto has joined #openstack-infra | 01:24 | |
| *** ijw has joined #openstack-infra | 01:25 | |
| clarkb | ianw: I grep the uuid in the executor-debug logs on all the hosts :/ | 01:25 |
| *** ijw has quit IRC | 01:25 | |
| clarkb | 22 processes owned by zuul on 01 remaining | 01:26 |
| clarkb | this restart will pick up the fix to make future restarts more quick | 01:26 |
| ianw | clarkb: ok, that was what i thought | 01:26 |
| *** ijw has joined #openstack-infra | 01:27 | |
| clarkb | pabelanger: how are src and dest different? | 01:30 |
| clarkb | is dest just going to be /tmp in this case? | 01:31 |
| pabelanger | clarkb: I'm not sure, result.dest will be /tmp/service-types.json after we set dest: /tmp | 01:32 |
| pabelanger | mordred: ^ | 01:32 |
| pabelanger | jlk: maybe you know^ | 01:33 |
| pabelanger | 512458 | 01:33 |
| clarkb | from a readability perspecitive if dest is the file path I think it would be a much better name to use in this case as src makes little sense to me even after reading the docs | 01:34 |
| *** aeng has joined #openstack-infra | 01:34 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Move subunit generate to oooq playbooks https://review.openstack.org/490032 | 01:36 |
| clarkb | oh is src the local tmpfile location it downloads to then sources in a file move/copy? | 01:37 |
| *** jkilpatr_ has quit IRC | 01:37 | |
| ianw | pabelanger / clarkb: http://paste.openstack.org/show/623773/ ... so basically it got lost? | 01:38 |
| *** aeng has quit IRC | 01:39 | |
| ianw | oh, it's on ze01 | 01:39 |
| clarkb | ianw: the file is executor-debug.log on ze01-ze10 | 01:39 |
| ianw | yeah, sorry | 01:40 |
| clarkb | and ze01 is in process of losing its process but its slow | 01:40 |
| clarkb | I need to eat dinner and check in on 01 after | 01:40 |
| *** daidv_ has joined #openstack-infra | 01:41 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 01:42 |
| *** links has joined #openstack-infra | 01:44 | |
| openstackgerrit | Merged openstack-infra/system-config master: Set SETUP_PIP=false for tools/prep-apply.sh https://review.openstack.org/512137 | 01:45 |
| *** mriedem has quit IRC | 01:45 | |
| *** links is now known as Jaison|away | 01:45 | |
| clarkb | oh hey it finally finished up for the most part so I killed the probably deadlocked -executor processes | 01:46 |
| clarkb | it is starting up again | 01:46 |
| clarkb | 2017-10-17 01:46:46,285 DEBUG zuul.AnsibleJob: [build: 445611fc81544eff83febf16a377bf09] Ansible complete, result RESULT_NORMAL code 0 | 01:46 |
| clarkb | ok I think it is working | 01:46 |
| clarkb | and with that really dinner | 01:47 |
| *** chlong has joined #openstack-infra | 01:49 | |
| *** hongbin has joined #openstack-infra | 01:49 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 01:50 |
| *** deikiyoch has joined #openstack-infra | 01:51 | |
| *** deikiyoch has left #openstack-infra | 01:51 | |
| *** daidv_ has quit IRC | 01:51 | |
| *** daidv_ has joined #openstack-infra | 01:51 | |
| *** daidv_ has quit IRC | 01:51 | |
| openstackgerrit | Merged openstack-infra/system-config master: Match HOST_ID using tab character too https://review.openstack.org/512376 | 01:51 |
| *** daidv_ has joined #openstack-infra | 01:52 | |
| *** aeng has joined #openstack-infra | 01:52 | |
| *** markvoelker has quit IRC | 01:53 | |
| *** GregHouse has quit IRC | 01:55 | |
| *** GregHous- has joined #openstack-infra | 01:55 | |
| *** bobh has quit IRC | 01:58 | |
| *** markvoelker has joined #openstack-infra | 01:59 | |
| *** s-shiono has joined #openstack-infra | 02:00 | |
| *** markvoelker has quit IRC | 02:03 | |
| *** hongbin_ has joined #openstack-infra | 02:04 | |
| *** masber has quit IRC | 02:07 | |
| *** markvoelker has joined #openstack-infra | 02:08 | |
| *** markvoelker has quit IRC | 02:12 | |
| *** markvoelker has joined #openstack-infra | 02:17 | |
| *** shu-mutou has joined #openstack-infra | 02:17 | |
| *** salv-orlando has joined #openstack-infra | 02:18 | |
| *** markvoelker has quit IRC | 02:21 | |
| *** salv-orlando has quit IRC | 02:23 | |
| *** markvoelker has joined #openstack-infra | 02:26 | |
| *** ihrachys_ has quit IRC | 02:29 | |
| *** markvoelker has quit IRC | 02:30 | |
| *** ijw has quit IRC | 02:31 | |
| *** toshii has joined #openstack-infra | 02:32 | |
| *** dhinesh_ has quit IRC | 02:32 | |
| toshii | where did logstash.openstack.org go? | 02:32 |
| *** ramishra has joined #openstack-infra | 02:33 | |
| *** sree has joined #openstack-infra | 02:34 | |
| *** dteselkin has quit IRC | 02:34 | |
| *** katyafervent_awa has quit IRC | 02:34 | |
| *** markvoelker has joined #openstack-infra | 02:35 | |
| *** gildub has quit IRC | 02:35 | |
| *** jhesketh has quit IRC | 02:35 | |
| jlk | wait, what might I know? | 02:36 |
| *** tonyb has quit IRC | 02:36 | |
| *** cinerama has quit IRC | 02:36 | |
| dmsimard | toshii: it didn't go anywhere ? | 02:37 |
| dmsimard | oh, the data in logstash.openstack.org | 02:37 |
| dmsimard | :D | 02:37 |
| *** bradm has quit IRC | 02:37 | |
| *** tonyb has joined #openstack-infra | 02:37 | |
| toshii | dmsimard: year it's still there but empty ;( | 02:37 |
| *** jhesketh has joined #openstack-infra | 02:38 | |
| dmsimard | infra-root: ^ I suspect we're sending gearman jobs for logstash workers (because the post jobs aren't failing?) but nothing's getting picked up | 02:38 |
| *** katyafervent_awa has joined #openstack-infra | 02:38 | |
| clarkb | dmsimard: toshii yes es was put to lunch. I havent had a chance to restart the logstash workers nwo that es is happy again | 02:39 |
| clarkb | but I think that id what needs to haplen | 02:39 |
| *** dteselkin has joined #openstack-infra | 02:39 | |
| *** markvoelker has quit IRC | 02:40 | |
| openstackgerrit | Nam Nguyen Hoai proposed openstack-infra/project-config master: Remove legacy jobs from Congress https://review.openstack.org/510437 | 02:40 |
| *** kota_ has left #openstack-infra | 02:41 | |
| *** cinerama has joined #openstack-infra | 02:41 | |
| *** mat128 has quit IRC | 02:42 | |
| *** rlandy has quit IRC | 02:43 | |
| *** markvoelker has joined #openstack-infra | 02:44 | |
| ianw | clarkb: want me to just run through and restart them? | 02:45 |
| *** lbragstad has joined #openstack-infra | 02:45 | |
| *** bradm has joined #openstack-infra | 02:45 | |
| clarkb | ianw: if you want its jenkins-log-worker-A through D and logstash | 02:45 |
| *** greghaynes has quit IRC | 02:45 | |
| clarkb | logstash will likely need a kill -9 | 02:46 |
| clarkb | that appears to be the behavior associated with this es derp | 02:46 |
| ianw | ok | 02:46 |
| mtreinish | clarkb, ianw if you're restarting wokers could you kick the subunit-workers too | 02:47 |
| ianw | mtreinish: ok, will after logstash | 02:47 |
| mtreinish | it looks like the recent rename changes (https://review.openstack.org/#/q/status:merged+project:openstack-infra/puppet-subunit2sql+branch:master+topic:restore-name-sanity )didn't take effect automagically | 02:48 |
| mtreinish | ianw: cool, thanks | 02:48 |
| *** masber has joined #openstack-infra | 02:49 | |
| *** markvoelker has quit IRC | 02:49 | |
| openstackgerrit | Merged openstack-infra/project-config master: Document base job change procedure https://review.openstack.org/512138 | 02:49 |
| *** hongbin_ has quit IRC | 02:51 | |
| *** stakeda has joined #openstack-infra | 02:52 | |
| *** markvoelker has joined #openstack-infra | 02:53 | |
| *** gildub has joined #openstack-infra | 02:53 | |
| ianw | clarkb: they all seem to have disappeared without the use of -9 | 03:00 |
| pabelanger | yah, cacti.o.o email now 932K (down from 2.9M) | 03:02 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Move subunit generate to oooq playbooks https://review.openstack.org/490032 | 03:02 |
| pabelanger | I'll continue looking at it in the morning | 03:02 |
| clarkb | ianw oh good | 03:03 |
| *** udesale has joined #openstack-infra | 03:08 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add tip on restarting logstash client and workers https://review.openstack.org/512467 | 03:10 |
| ianw | clarkb: ^ that being what i did ... | 03:11 |
| ianw | mtreinish: hmm, do i want to ask what subunit-worker01.o.o is running? it won't accept my ed25519 key i don't think | 03:14 |
| ianw | so worker02 is restarted, but clarkb or someone might have to do the other one | 03:14 |
| clarkb | I think 01 may actually not exist anymore because it was precise | 03:15 |
| ianw | it rejects my key, but maybe it's not running? | 03:16 |
| ianw | not running as a subunit-worker i mean | 03:17 |
| *** salv-orlando has joined #openstack-infra | 03:19 | |
| *** Jaison|away is now known as links | 03:23 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate master: Ansiblise ephemeral/swap setup https://review.openstack.org/499467 | 03:24 |
| *** salv-orlando has quit IRC | 03:24 | |
| *** dave-mccowan has quit IRC | 03:24 | |
| *** coolsvap has joined #openstack-infra | 03:25 | |
| ianw | clarkb: ^ might be good to roll that out soonish; i just ported the inode creation into it | 03:25 |
| *** rosmaita has quit IRC | 03:26 | |
| *** markvoelker has quit IRC | 03:27 | |
| ianw | there's an error on openstack-dev about /opt permissions, and i just got http://logs.openstack.org/50/512450/6/check/dib-functests-python3-xenial-extras/86f78b8/job-output.txt.gz#_2017-10-17_01_55_35_795275 | 03:27 |
| ianw | oh, i guess i was sudoing it in the old job, so that's my fault | 03:28 |
| ianw | i wonder if we should 777 /opt | 03:28 |
| *** kota_ has joined #openstack-infra | 03:30 | |
| clarkb | ianw I think that thread is about v2 jobs running in third party ci not caching the etcd file? | 03:32 |
| clarkb | its using opt stack new which is v2 pathing and that file is in thr cache for us | 03:32 |
| clarkb | (I think) | 03:32 |
| ianw | oh, well it says "text file busy" ... etcd must be running i guess. yeah, should have read it closer :) | 03:33 |
| openstackgerrit | Sam Yaple proposed openstack-infra/irc-meetings master: Add LOCI meeting https://review.openstack.org/512471 | 03:34 |
| *** hongbin has quit IRC | 03:37 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/infra-manual master: Correct a tiny typo https://review.openstack.org/512472 | 03:40 |
| *** nicolasbock has quit IRC | 03:41 | |
| *** mtreinish has quit IRC | 03:42 | |
| *** mtreinish has joined #openstack-infra | 03:42 | |
| openstackgerrit | Merged openstack-infra/system-config master: Create more inodes on ephemeral partition fs https://review.openstack.org/512453 | 03:44 |
| *** kiennt26 has quit IRC | 03:45 | |
| *** cuongnv has quit IRC | 03:45 | |
| *** yamahata has joined #openstack-infra | 03:45 | |
| *** cuongnv has joined #openstack-infra | 03:46 | |
| *** kiennt26 has joined #openstack-infra | 03:46 | |
| *** masber has quit IRC | 03:46 | |
| *** udesale has quit IRC | 03:47 | |
| *** udesale has joined #openstack-infra | 03:50 | |
| *** udesale has quit IRC | 03:52 | |
| *** udesale has joined #openstack-infra | 03:53 | |
| *** bhavik1 has joined #openstack-infra | 03:54 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Import legacy playbooks https://review.openstack.org/512164 | 03:57 |
| *** lbragstad has quit IRC | 03:58 | |
| *** gouthamr has quit IRC | 03:59 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove diskimage-builder functional test jobs https://review.openstack.org/512169 | 04:03 |
| *** sbezverk has quit IRC | 04:06 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 04:10 |
| *** claudiub has joined #openstack-infra | 04:11 | |
| *** bhavik1 has quit IRC | 04:13 | |
| *** ykarel|away has joined #openstack-infra | 04:13 | |
| *** ykarel|away is now known as ykarel | 04:13 | |
| *** huanxie has quit IRC | 04:15 | |
| *** markvoelker has joined #openstack-infra | 04:17 | |
| *** bobh has joined #openstack-infra | 04:18 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 04:20 |
| *** salv-orlando has joined #openstack-infra | 04:20 | |
| *** dbecker has quit IRC | 04:22 | |
| *** bobh has quit IRC | 04:22 | |
| *** markvoelker has quit IRC | 04:22 | |
| *** salv-orlando has quit IRC | 04:25 | |
| *** stakeda has quit IRC | 04:26 | |
| *** markvoelker has joined #openstack-infra | 04:27 | |
| *** stakeda has joined #openstack-infra | 04:32 | |
| *** eumel8 has joined #openstack-infra | 04:32 | |
| *** markvoelker has quit IRC | 04:33 | |
| *** markvoelker has joined #openstack-infra | 04:34 | |
| *** dbecker has joined #openstack-infra | 04:35 | |
| *** kiennt26 has quit IRC | 04:41 | |
| openstackgerrit | Nam Nguyen Hoai proposed openstack-infra/project-config master: Remove legacy jobs from Networking-cisco https://review.openstack.org/512197 | 04:43 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 04:49 |
| SamYaple | so im trying to collect some ansible logs, but i cannot because they are outside the work directory. | 04:49 |
| SamYaple | http://logs.openstack.org/79/512479/1/check/loci-glance/6d91bfb/job-output.txt.gz#_2017-10-17_04_43_57_017113 | 04:49 |
| SamYaple | https://review.openstack.org/#/c/512479/1/playbooks/post.yaml | 04:49 |
| *** ifat_afek has joined #openstack-infra | 04:52 | |
| *** ifat_afek has left #openstack-infra | 04:52 | |
| openstackgerrit | Merged openstack-infra/infra-manual master: Correct a tiny typo https://review.openstack.org/512472 | 04:56 |
| *** greghaynes has joined #openstack-infra | 04:58 | |
| *** huanxie has joined #openstack-infra | 04:59 | |
| SamYaple | oh im dumb. or tired. either way disregard, thanks | 04:59 |
| *** claudiub has quit IRC | 05:05 | |
| *** CrayZee has joined #openstack-infra | 05:07 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: web: make console-stream tenant scoped https://review.openstack.org/505452 | 05:08 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: web: add /{source}/{project}.pub route https://review.openstack.org/502530 | 05:08 |
| *** esberglu has joined #openstack-infra | 05:09 | |
| *** esberglu has quit IRC | 05:09 | |
| AJaeger | ianw: care to +2A https://review.openstack.org/512458 , please? | 05:14 |
| chandankumar | ianw: please have a look on this https://review.openstack.org/#/c/510529/ | 05:14 |
| chandankumar | ianw: thanks :-) | 05:16 |
| *** edmondsw has quit IRC | 05:17 | |
| AJaeger | chandankumar: it's system-required now, you need to rebase + update | 05:17 |
| AJaeger | ianw: ^ | 05:17 |
| chandankumar | AJaeger: yes i also need to add python-jobs | 05:18 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: web: add /tenants route https://review.openstack.org/503268 | 05:18 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/status route https://review.openstack.org/503269 | 05:18 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/jobs route https://review.openstack.org/503270 | 05:18 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: web: add /{tenant}/builds route https://review.openstack.org/466561 | 05:18 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: web: make console-stream tenant scoped https://review.openstack.org/505452 | 05:18 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: web: add /{source}/{project}.pub route https://review.openstack.org/502530 | 05:18 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Adapt relative streaming urls to tenant scope https://review.openstack.org/512483 | 05:18 |
| *** bobh has joined #openstack-infra | 05:19 | |
| openstackgerrit | Chandan Kumar proposed openstack-infra/project-config master: Added initial jobs for magnum-tempest-plugin https://review.openstack.org/510529 | 05:20 |
| *** salv-orlando has joined #openstack-infra | 05:21 | |
| chandankumar | ianw: AJaeger ^^ | 05:21 |
| *** bobh has quit IRC | 05:23 | |
| AJaeger | chandankumar: thanks, approved | 05:24 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Use tempfile for get_url task set-service-type-data-fact https://review.openstack.org/512458 | 05:24 |
| *** salv-orlando has quit IRC | 05:25 | |
| *** armaan has quit IRC | 05:27 | |
| *** ccamacho has left #openstack-infra | 05:28 | |
| *** armaan has joined #openstack-infra | 05:28 | |
| openstackgerrit | Merged openstack-infra/project-config master: Added initial jobs for magnum-tempest-plugin https://review.openstack.org/510529 | 05:29 |
| AJaeger | ianw, https://review.openstack.org/#/c/512426/ and https://review.openstack.org/511781 https://review.openstack.org/510835 are ready as well if you have a few minutes | 05:31 |
| *** salv-orlando has joined #openstack-infra | 05:34 | |
| *** armaan has quit IRC | 05:34 | |
| *** armaan has joined #openstack-infra | 05:34 | |
| *** slaweq has joined #openstack-infra | 05:35 | |
| *** dhinesh has joined #openstack-infra | 05:42 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Adapt relative streaming urls to tenant scope https://review.openstack.org/512483 | 05:46 |
| openstackgerrit | Merged openstack-infra/system-config master: Stop mirroring openSUSE 42.2 https://review.openstack.org/502316 | 05:46 |
| *** slaweq has quit IRC | 05:47 | |
| dirk | pabelanger: I guess you have to do the 42.2 mirroring cleanup once more as the change for removing mirroring was stuck in the gate -- https://review.openstack.org/#/c/502316/ | 05:48 |
| *** liusheng has quit IRC | 05:49 | |
| *** dfflanders has joined #openstack-infra | 05:50 | |
| *** slaweq has joined #openstack-infra | 05:52 | |
| *** slaweq has quit IRC | 05:54 | |
| *** markvoelker has quit IRC | 05:54 | |
| eumel8 | AJaeger: morning! Is it ready to go live or you want to check more on https://review.openstack.org/#/c/510273/ | 05:56 |
| *** armaan has quit IRC | 06:02 | |
| *** armaan has joined #openstack-infra | 06:03 | |
| *** kiennt26 has joined #openstack-infra | 06:04 | |
| AJaeger | eumel8: not ready yet - waiting for new translation imports... | 06:04 |
| *** `Kevin has quit IRC | 06:04 | |
| AJaeger | eumel8: every day fixing another bit, we're getting there slowly... | 06:04 |
| *** `Kevin has joined #openstack-infra | 06:05 | |
| eumel8 | AJaeger: ah okay, thx, the current checks were already green the last days | 06:05 |
| AJaeger | eumel8: I wait for the sync from Zanata with v3 | 06:06 |
| chandankumar | AJaeger: i want to add tripleo ci job against python-tempestconf how can i do that? | 06:06 |
| *** e0ne has joined #openstack-infra | 06:07 | |
| eumel8 | AJaeger: is it possible to trigger a periodic job manually? | 06:07 |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder master: cloud-init element systemctl support https://review.openstack.org/512489 | 06:08 |
| AJaeger | eumel8: not that I know | 06:10 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Run gerrit periodic jobs on proper branch https://review.openstack.org/512490 | 06:10 |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder master: cloud-init element systemctl support https://review.openstack.org/512489 | 06:11 |
| eumel8 | ok | 06:11 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: translations: Linking to /usr/local/bin needs sudo https://review.openstack.org/512491 | 06:14 |
| AJaeger | ianw, yolanda, care to quickly +2a, please? ^ | 06:14 |
| *** Guest68844 has quit IRC | 06:14 | |
| AJaeger | Let's see whether we get this in for today's run that just started... | 06:14 |
| AJaeger | eumel8: that's today's problem ^ | 06:14 |
| *** bcafarel has quit IRC | 06:14 | |
| *** florianf has joined #openstack-infra | 06:15 | |
| eumel8 | AJaeger: it would be easier if ansible runs under root ;) | 06:15 |
| *** aeng has quit IRC | 06:16 | |
| yolanda | hi good morning | 06:16 |
| yolanda | will do | 06:16 |
| AJaeger | thanks, yolanda - and good morning to you as well! | 06:16 |
| *** bcafarel has joined #openstack-infra | 06:17 | |
| eumel8 | hi yolanda | 06:17 |
| *** gcb has joined #openstack-infra | 06:18 | |
| *** Hal has joined #openstack-infra | 06:18 | |
| *** Hal is now known as Guest17047 | 06:18 | |
| mpeterson | hello! In networking-odl we have quite a few dependencies to include to required-projects in each job. The idea was to do it through project-templates like http://paste.openstack.org/show/623780/ , is that the best way to do it? That would be used on the templates section of the project definition inside the project repo and not project-config, right? | 06:19 |
| *** bobh has joined #openstack-infra | 06:20 | |
| *** liusheng has joined #openstack-infra | 06:21 | |
| *** masber has joined #openstack-infra | 06:21 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Add support for result data in child jobs https://review.openstack.org/504808 | 06:21 |
| *** dhinesh has quit IRC | 06:22 | |
| AJaeger | mpeterson: don't create a project-template for a single repo | 06:22 |
| AJaeger | mpeterson: just do it like other repos have done it, let me show you ... | 06:23 |
| AJaeger | mpeterson: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/projects.yaml#n15357 | 06:24 |
| AJaeger | Just add the additional required repos there | 06:24 |
| *** bobh has quit IRC | 06:24 | |
| AJaeger | and use the -neutron templates | 06:24 |
| *** armax has quit IRC | 06:25 | |
| *** armax has joined #openstack-infra | 06:25 | |
| *** armax has quit IRC | 06:25 | |
| mpeterson | AJaeger: okey, no problem. Is there not a way to avoid repetition of the required-projects in each pipeline definition? | 06:26 |
| AJaeger | mpeterson: not today, but the team is looking for a solution mid-term. | 06:28 |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder master: cloud-init element systemctl support https://review.openstack.org/512489 | 06:28 |
| openstackgerrit | Merged openstack-infra/project-config master: translations: Linking to /usr/local/bin needs sudo https://review.openstack.org/512491 | 06:30 |
| *** martinkopec has joined #openstack-infra | 06:34 | |
| *** jtomasek has joined #openstack-infra | 06:34 | |
| *** jtomasek has quit IRC | 06:34 | |
| *** jtomasek has joined #openstack-infra | 06:34 | |
| *** slaweq has joined #openstack-infra | 06:37 | |
| openstackgerrit | Merged openstack-infra/project-config master: zuul changes for vmware-nsx https://review.openstack.org/512426 | 06:38 |
| *** slaweq_ has joined #openstack-infra | 06:38 | |
| *** psachin has quit IRC | 06:38 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy jobs in Mogan https://review.openstack.org/511781 | 06:39 |
| openstackgerrit | Merged openstack-infra/project-config master: Follow up change for Heat Dashboard creation https://review.openstack.org/510835 | 06:39 |
| *** slaweq has quit IRC | 06:41 | |
| *** andreas_s has joined #openstack-infra | 06:42 | |
| *** armaan has quit IRC | 06:44 | |
| *** pcaruana has joined #openstack-infra | 06:44 | |
| *** andreas_s has quit IRC | 06:45 | |
| *** e0ne has quit IRC | 06:45 | |
| *** claudiub has joined #openstack-infra | 06:46 | |
| *** andreas_s has joined #openstack-infra | 06:47 | |
| *** andreas_s has quit IRC | 06:50 | |
| *** janki has joined #openstack-infra | 06:50 | |
| *** markvoelker has joined #openstack-infra | 06:50 | |
| *** psachin has joined #openstack-infra | 06:51 | |
| janki | hey. one of my patch is geeting git related error on zuul. http://logs.openstack.org/65/511765/1/check/gate-tripleo-heat-templates-pep8-ubuntu-xenial/59e783b/console.html. Is this a known issue? | 06:51 |
| janki | AJaeger, ^ | 06:52 |
| *** andreas__ has joined #openstack-infra | 06:52 | |
| *** ykarel_ has joined #openstack-infra | 06:57 | |
| *** ykarel has quit IRC | 06:59 | |
| openstackgerrit | Michel Peterson proposed openstack-infra/project-config master: networking-odl: Fix dependencies and reorder jobs https://review.openstack.org/512511 | 07:00 |
| *** namnh has joined #openstack-infra | 07:01 | |
| *** edmondsw has joined #openstack-infra | 07:01 | |
| *** andreas_s has joined #openstack-infra | 07:02 | |
| *** andreas__ has quit IRC | 07:05 | |
| *** edmondsw has quit IRC | 07:06 | |
| *** gildub has quit IRC | 07:06 | |
| *** shardy has joined #openstack-infra | 07:08 | |
| *** tmorin has joined #openstack-infra | 07:09 | |
| *** slaweq has joined #openstack-infra | 07:11 | |
| *** kiennt26 has quit IRC | 07:12 | |
| *** kiennt26 has joined #openstack-infra | 07:13 | |
| *** slaweq has quit IRC | 07:15 | |
| openstackgerrit | Michel Peterson proposed openstack-infra/project-config master: networking-odl: Fix dependencies and reorder jobs https://review.openstack.org/512511 | 07:16 |
| *** tesseract has joined #openstack-infra | 07:16 | |
| *** dasanind has quit IRC | 07:17 | |
| *** dasanind has joined #openstack-infra | 07:17 | |
| openstackgerrit | Michel Peterson proposed openstack-infra/project-config master: networking-odl: Fix dependencies and reorder jobs https://review.openstack.org/512511 | 07:18 |
| *** aviau has quit IRC | 07:19 | |
| AJaeger | janki: "HTTP 500 curl 22 The requested URL returned error: 500 Internal Server Error" | 07:19 |
| *** aviau has joined #openstack-infra | 07:19 | |
| AJaeger | infra-root, this is cloning http://zm03.openstack.org/p/openstack/tripleo-heat-templates | 07:20 |
| AJaeger | could anybody check zum03, please? | 07:20 |
| *** bobh has joined #openstack-infra | 07:20 | |
| *** armaan has joined #openstack-infra | 07:21 | |
| *** bobh has quit IRC | 07:25 | |
| *** gildub has joined #openstack-infra | 07:26 | |
| *** ccamacho has joined #openstack-infra | 07:30 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: periodic-neutron-dsvm-functional runs only on master https://review.openstack.org/512524 | 07:35 |
| *** armaan has quit IRC | 07:37 | |
| *** jpich has joined #openstack-infra | 07:37 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: periodic-neutron-dsvm-functional runs only on master https://review.openstack.org/512524 | 07:40 |
| openstackgerrit | Tetsuro Nakamura proposed openstack-infra/project-config master: Add new project networking-spp https://review.openstack.org/512223 | 07:42 |
| *** CrayZee has left #openstack-infra | 07:42 | |
| *** jpena|off is now known as jpena | 07:43 | |
| *** egonzalez has joined #openstack-infra | 07:45 | |
| *** gildub has quit IRC | 07:46 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Use proper script for translations https://review.openstack.org/512529 | 07:48 |
| *** amoralej|off is now known as amoralej | 07:49 | |
| *** ralonsoh has joined #openstack-infra | 07:49 | |
| *** ralonsoh_ has joined #openstack-infra | 07:50 | |
| *** dhajare has joined #openstack-infra | 07:51 | |
| *** hashar has joined #openstack-infra | 07:52 | |
| *** ralonsoh has quit IRC | 07:54 | |
| *** ralonsoh_ is now known as ralonsoh | 07:55 | |
| *** dhajare has quit IRC | 08:00 | |
| openstackgerrit | Nam Nguyen Hoai proposed openstack-infra/project-config master: Remove legacy job from Trio2o https://review.openstack.org/512532 | 08:02 |
| *** salv-orlando has quit IRC | 08:05 | |
| *** salv-orlando has joined #openstack-infra | 08:05 | |
| openstackgerrit | Nam Nguyen Hoai proposed openstack-infra/openstack-zuul-jobs master: Remove Trio2o legacy job https://review.openstack.org/512533 | 08:05 |
| *** iyamahat has joined #openstack-infra | 08:10 | |
| *** iyamahat_ has joined #openstack-infra | 08:11 | |
| *** gildub has joined #openstack-infra | 08:12 | |
| *** iyamahat has quit IRC | 08:14 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Remove dd from LVM element https://review.openstack.org/498153 | 08:15 |
| *** iyamahat_ has quit IRC | 08:20 | |
| *** iyamahat_ has joined #openstack-infra | 08:21 | |
| *** bobh has joined #openstack-infra | 08:21 | |
| openstackgerrit | Tetsuro Nakamura proposed openstack-infra/project-config master: Add new project networking-spp https://review.openstack.org/512223 | 08:23 |
| openstackgerrit | Tetsuro Nakamura proposed openstack-infra/project-config master: Follow up change for networking-spp creation https://review.openstack.org/512536 | 08:23 |
| *** iyamahat_ has quit IRC | 08:25 | |
| *** yamahata has quit IRC | 08:26 | |
| *** bobh has quit IRC | 08:27 | |
| *** ykarel_ is now known as ykarel | 08:32 | |
| *** lucas-afk is now known as lucasagomes | 08:33 | |
| *** derekh has joined #openstack-infra | 08:34 | |
| *** s-shiono has quit IRC | 08:35 | |
| *** toshii has quit IRC | 08:35 | |
| *** priteau has joined #openstack-infra | 08:43 | |
| *** dtantsur|afk is now known as dtantsur | 08:43 | |
| *** yamamoto has quit IRC | 08:47 | |
| *** coolsvap has quit IRC | 08:49 | |
| *** edmondsw has joined #openstack-infra | 08:50 | |
| *** electrofelix has joined #openstack-infra | 08:51 | |
| *** dingyichen has quit IRC | 08:52 | |
| *** gildub has quit IRC | 08:53 | |
| *** edmondsw has quit IRC | 08:54 | |
| openstackgerrit | Julien Danjou proposed openstack-infra/openstack-zuul-jobs master: Ceilometer: remove legacy-ceilometer-dsvm-tempest-plugin-es job https://review.openstack.org/512546 | 08:55 |
| *** yamamoto has joined #openstack-infra | 08:55 | |
| *** yamamoto has quit IRC | 08:55 | |
| openstackgerrit | Julien Danjou proposed openstack-infra/project-config master: Ceilometer: disable Tempest tests for master and Queens https://review.openstack.org/512342 | 08:56 |
| *** markmcd has quit IRC | 09:02 | |
| *** markmcd has joined #openstack-infra | 09:02 | |
| *** rhallisey has joined #openstack-infra | 09:03 | |
| *** markvoelker has quit IRC | 09:03 | |
| *** markvoelker has joined #openstack-infra | 09:04 | |
| *** esberglu has joined #openstack-infra | 09:04 | |
| *** esberglu has quit IRC | 09:09 | |
| *** florianf has quit IRC | 09:10 | |
| *** ccamacho has quit IRC | 09:11 | |
| *** e0ne has joined #openstack-infra | 09:13 | |
| *** dhinesh has joined #openstack-infra | 09:14 | |
| *** yamamoto has joined #openstack-infra | 09:15 | |
| *** florianf has joined #openstack-infra | 09:15 | |
| *** sree has quit IRC | 09:16 | |
| *** sdague has joined #openstack-infra | 09:17 | |
| *** gildub has joined #openstack-infra | 09:18 | |
| *** dhinesh has quit IRC | 09:18 | |
| *** udesale__ has joined #openstack-infra | 09:19 | |
| *** sree has joined #openstack-infra | 09:19 | |
| *** udesale has quit IRC | 09:19 | |
| *** sambetts|afk is now known as sambetts | 09:20 | |
| *** andreas_s has quit IRC | 09:21 | |
| *** udesale__ has quit IRC | 09:22 | |
| *** udesale has joined #openstack-infra | 09:22 | |
| *** liusheng has quit IRC | 09:22 | |
| *** liusheng has joined #openstack-infra | 09:23 | |
| *** sree has quit IRC | 09:23 | |
| *** jascott1 has quit IRC | 09:24 | |
| *** bobh has joined #openstack-infra | 09:24 | |
| *** jascott1 has joined #openstack-infra | 09:24 | |
| *** liusheng has quit IRC | 09:28 | |
| *** liusheng has joined #openstack-infra | 09:28 | |
| *** jascott1 has quit IRC | 09:29 | |
| *** panda|rover|off is now known as panda|rover | 09:29 | |
| *** bobh has quit IRC | 09:29 | |
| *** andreas_s has joined #openstack-infra | 09:30 | |
| *** slaweq has joined #openstack-infra | 09:30 | |
| *** gfidente has joined #openstack-infra | 09:30 | |
| gfidente | hi guys | 09:30 |
| gfidente | not sure if anybody can help me delete https://etherpad.openstack.org/p/tripleo-ceph-luminous-migration ? | 09:30 |
| gfidente | it looks like it is necessary to have a key for the API request which is in a file on the server so I couldn't do it myself | 09:31 |
| *** shardy has quit IRC | 09:32 | |
| *** eumel8 has quit IRC | 09:33 | |
| *** jchhatbar has joined #openstack-infra | 09:34 | |
| *** slaweq has quit IRC | 09:35 | |
| *** janki has quit IRC | 09:37 | |
| *** cshastri has quit IRC | 09:39 | |
| *** gildub has quit IRC | 09:41 | |
| *** yolanda has quit IRC | 09:44 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config master: Networking-ovn: Use publish-openstack-sphinx-docs-neutron https://review.openstack.org/512565 | 09:45 |
| *** shardy has joined #openstack-infra | 09:45 | |
| openstackgerrit | Duong Ha-Quang proposed openstack-infra/openstack-zuul-jobs master: Remove legacy job in Oaktree https://review.openstack.org/512566 | 09:45 |
| openstackgerrit | Duong Ha-Quang proposed openstack-infra/project-config master: Remove legacy job in Oaktree https://review.openstack.org/512567 | 09:46 |
| *** ykarel is now known as ykarel|lunch | 09:47 | |
| *** yolanda has joined #openstack-infra | 09:47 | |
| *** salv-orlando has quit IRC | 09:50 | |
| openstackgerrit | Flavio Percoco proposed openstack-infra/project-config master: Add ansible-role-k8s-cookiecutter https://review.openstack.org/512323 | 09:50 |
| openstackgerrit | Flavio Percoco proposed openstack-infra/project-config master: ansible-role-k8s-cookiecutter to zuul.d/projects https://review.openstack.org/512330 | 09:50 |
| *** ociuhandu has quit IRC | 09:50 | |
| *** salv-orlando has joined #openstack-infra | 09:50 | |
| *** tosky has joined #openstack-infra | 09:53 | |
| mpeterson | can anyone please +2 and +W these patches? https://review.openstack.org/#/q/topic:change/migrate_networking_odl_to_zuulv3+(status:open+OR+status:merged) | 09:54 |
| mpeterson | CI is a blocking issue for networking-odl at the moment | 09:55 |
| *** salv-orlando has quit IRC | 09:55 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config master: Networking-ovn: Use publish-openstack-sphinx-docs-neutron https://review.openstack.org/512565 | 09:55 |
| *** rhallisey has quit IRC | 09:58 | |
| *** esberglu has joined #openstack-infra | 09:59 | |
| *** esberglu has quit IRC | 09:59 | |
| *** shardy has quit IRC | 10:01 | |
| *** sree has joined #openstack-infra | 10:02 | |
| *** andreas_s has quit IRC | 10:02 | |
| *** shardy has joined #openstack-infra | 10:04 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add new repo for Senlin tempest plugin https://review.openstack.org/504870 | 10:05 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy jobs from Ceilometer https://review.openstack.org/510459 | 10:06 |
| *** sree has quit IRC | 10:06 | |
| *** gmann is now known as gmann_afk | 10:09 | |
| openstackgerrit | Merged openstack-infra/infra-manual master: Clarify job removal section https://review.openstack.org/512304 | 10:09 |
| openstackgerrit | Merged openstack-infra/infra-manual master: mention playbooks for backporting https://review.openstack.org/512381 | 10:12 |
| *** cuongnv has quit IRC | 10:12 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove legacy jobs in Mixmatch https://review.openstack.org/511763 | 10:13 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Add files section in the legact tripleo ci scenario007 jobs https://review.openstack.org/512007 | 10:13 |
| sambetts | AJaeger: if your around can I pick your brains on getting the networking-cisco CI back up and running, its broken due to changes in how requirements are installed | 10:13 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove workflow +1 on glance_store from swift-core https://review.openstack.org/511517 | 10:13 |
| openstackgerrit | Merged openstack-infra/project-config master: Publish cinder loci images to DockerHub https://review.openstack.org/512398 | 10:13 |
| openstackgerrit | Merged openstack-infra/project-config master: networking-midonet: Use publish-openstack-sphinx-docs-neutron https://review.openstack.org/512250 | 10:14 |
| openstackgerrit | Merged openstack-infra/project-config master: networking-midonet: Use release-notes-jobs-neutron https://review.openstack.org/512260 | 10:15 |
| *** yamamoto has quit IRC | 10:18 | |
| *** priteau has quit IRC | 10:19 | |
| openstackgerrit | Merged openstack-infra/project-config master: networking-midonet: Use publish-to-pypi-neutron https://review.openstack.org/512261 | 10:19 |
| *** priteau has joined #openstack-infra | 10:21 | |
| *** Jeffrey4l has quit IRC | 10:24 | |
| *** ccamacho has joined #openstack-infra | 10:24 | |
| *** Jeffrey4l has joined #openstack-infra | 10:24 | |
| *** yamamoto has joined #openstack-infra | 10:25 | |
| *** pbourke has quit IRC | 10:25 | |
| *** bobh has joined #openstack-infra | 10:25 | |
| openstackgerrit | Dmitry Tantsur proposed openstack-infra/tripleo-ci master: Add ironic-inspector to the undercloud-containers job https://review.openstack.org/476130 | 10:26 |
| *** pbourke has joined #openstack-infra | 10:26 | |
| *** ociuhandu has joined #openstack-infra | 10:27 | |
| *** yamamoto has quit IRC | 10:30 | |
| *** bobh has quit IRC | 10:30 | |
| *** liusheng has quit IRC | 10:32 | |
| *** salv-orlando has joined #openstack-infra | 10:32 | |
| *** liusheng has joined #openstack-infra | 10:32 | |
| *** slaweq has joined #openstack-infra | 10:32 | |
| *** Jeffrey4l has quit IRC | 10:33 | |
| *** openstackgerrit has quit IRC | 10:33 | |
| *** hashar has quit IRC | 10:33 | |
| *** Jeffrey4l has joined #openstack-infra | 10:33 | |
| *** andreas_s has joined #openstack-infra | 10:35 | |
| *** namnh has quit IRC | 10:36 | |
| *** slaweq has quit IRC | 10:37 | |
| *** edmondsw has joined #openstack-infra | 10:38 | |
| *** ldnunes has joined #openstack-infra | 10:38 | |
| mpeterson | yolanda: hey, how are you? I didn't understand your comment on why we need a neutron liasion +1 to merge the patches | 10:38 |
| yolanda | hi mpeterson , so for all neutron related changes, a +1 from the listed people as neutron liaisons, is needed | 10:40 |
| yolanda | https://wiki.openstack.org/wiki/CrossProjectLiaisons, https://governance.openstack.org/tc/reference/projects/neutron.html | 10:41 |
| mpeterson | yolanda: but this is not really neutron related and we are trying to fix the jobs that broke on the zuul deployment | 10:41 |
| *** dprince has joined #openstack-infra | 10:42 | |
| *** edmondsw has quit IRC | 10:42 | |
| *** dprince has quit IRC | 10:42 | |
| *** tesseract has quit IRC | 10:43 | |
| *** tesseract has joined #openstack-infra | 10:43 | |
| yolanda | if you can add those people under review, it's normally done fast. If that's not reviewing in a while, and jobs are broken, we can approve. But usually Yamamoto for example, reviews very fast | 10:44 |
| mpeterson | yolanda: what I mean, is we are not modifying anything in neutron, this is part of networking-odl only and have been +1 by core maintainers of networking-odl | 10:46 |
| mpeterson | btw, I'm not complaining, just trying to understand because I'm sitting and talking with mkolesni about it and he is as perplexed as me | 10:48 |
| yolanda | yep, i know, but even for testing we are always asking the +1 from neutron infra liaison, we always follow that rule | 10:48 |
| yolanda | AJaeger, can you consider that change as well? shall we wait for neutron liaison or we approve to fix broken jobs? ^ | 10:48 |
| mpeterson | thanks, in the meanwhile I added all the liasons as reviewers | 10:50 |
| *** priteau has quit IRC | 10:52 | |
| *** esberglu has joined #openstack-infra | 10:52 | |
| *** sree has joined #openstack-infra | 10:55 | |
| odyssey4me | AJaeger I'm a little confused - I see the reference you made to https://docs.openstack.org/infra/manual/zuulv3.html#what-not-to-convert but that says that the templates must live in project-config... it doesn't say that our consumption of those templates must be defined in project-config? | 10:56 |
| *** esberglu has quit IRC | 10:57 | |
| *** openstackgerrit has joined #openstack-infra | 10:57 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Handle tox environments via tox config https://review.openstack.org/512448 | 10:57 |
| mordred | morning yolanda AJaeger ! | 10:57 |
| electrofelix | does nodepool have support for static machines? ones that are reused a number of times before being discarded? | 10:57 |
| mordred | electrofelix: not today - but the patches to add it have been written and are on the schedule for review now that v3 is rolled out | 10:58 |
| *** ykarel|lunch is now known as ykarel | 10:58 | |
| mordred | electrofelix: although I believe we've discovered one additional feature we want to add while writing zuul jobs to make it truly usable - that is restricting where a set of nodes can be requested from | 10:59 |
| mordred | electrofelix: that is - now that job config is distributed across repos and jobs define which nodes they want to use - there is nothing to prevent a job in a random repo from requesting to run a job on a 'special' node | 10:59 |
| electrofelix | mordred: thanks, good to know, we're looking at bringing up nodepool and being able to handle some static slaves would make our lives much easier | 11:00 |
| mordred | electrofelix: depending on environment though that may not be as important - if your users are all 'trusted' then you can just tell them to not use any of the special nodes | 11:00 |
| electrofelix | mordred: ah, that probably doesn't impact us as much | 11:00 |
| mordred | electrofelix: yah | 11:00 |
| openstackgerrit | Raissa Sarmento proposed openstack-infra/project-config master: Change manila tempest plugin jobs https://review.openstack.org/512583 | 11:00 |
| AJaeger | mpeterson: neutron PTL asked that the liaisons +1 all neutron changes, see https://wiki.openstack.org/wiki/CrossProjectLiaisons#Infra as well | 11:01 |
| AJaeger | mpeterson: in this specific case, I'm fine doing an exception - but discuss the general policy with the neutron PTL | 11:02 |
| *** nicolasbock has joined #openstack-infra | 11:02 | |
| AJaeger | odyssey4me: leave those templates in project-config, please. That's what we're requesting. IT will not make a difference for you... | 11:03 |
| odyssey4me | AJaeger but I was - I was just consuming them in-repo | 11:03 |
| odyssey4me | but ok - perhaps the docs should clarify then | 11:03 |
| mpeterson | AJaeger: thanks that would be lovely | 11:04 |
| *** jkilpatr has joined #openstack-infra | 11:06 | |
| * mordred waves at odyssey4me from closer to his timezone | 11:08 | |
| odyssey4me | ohai mordred - I'm busy doing a horrible hack to get our jobs working seeing as the sudo issue is still up in the air | 11:09 |
| odyssey4me | horrible hacks are all the rage in the world of continuous delivery, of course | 11:09 |
| AJaeger | odyssey4me: if you have ideas on how to improve the docs, changes are welcome ;) | 11:09 |
| cmurphy | horrible hacks ftw | 11:10 |
| *** sps has joined #openstack-infra | 11:11 | |
| sps | hi alll.................... | 11:11 |
| *** sps is now known as Guest94812 | 11:11 | |
| Guest94812 | not able to open any gerrit topic | 11:12 |
| AJaeger | Guest94812: what do you mean exactly? | 11:12 |
| Guest94812 | AJaeger: the gerrit topic that automatically gets created when we add Partial-Implements: blueprint | 11:13 |
| *** lucasagomes is now known as lucas-hungry | 11:13 | |
| Guest94812 | AJaeger: for example https://review.openstack.org/#q,topic:bp/ks-rolling-upgrade-role,n,z | 11:14 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: networking-odl: Add required-projects for functional tests https://review.openstack.org/512254 | 11:15 |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config master: Ensure branch overrides are correctly set for networking-cisco https://review.openstack.org/512588 | 11:15 |
| AJaeger | Guest94812: try https://review.openstack.org/#/q/topic:bp/ks-rolling-upgrade-role | 11:16 |
| AJaeger | Guest94812: the gerrit update changed that URL... | 11:16 |
| openstackgerrit | Merged openstack-infra/project-config master: networking-odl: Fix dependencies and reorder jobs https://review.openstack.org/512511 | 11:16 |
| AJaeger | mordred: do you know why the "upstream-translation" jobs just report "error" without any log files? | 11:17 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP]vremove dlrnapi report at the end of the run https://review.openstack.org/512590 | 11:17 |
| *** shardy is now known as shardy_lunch | 11:18 | |
| Guest94812 | AJaeger: IIUC old one should be redirected to new one | 11:18 |
| Guest94812 | that should be fix | 11:19 |
| *** huanxie has quit IRC | 11:20 | |
| AJaeger | Guest94812: please update any old docs you find, we don't plan to add redirects for these, that can be really tricky ;( | 11:21 |
| Guest94812 | AJaeger: ok | 11:23 |
| Guest94812 | thanks | 11:23 |
| *** nicolasbock has quit IRC | 11:23 | |
| *** yamamoto has joined #openstack-infra | 11:26 | |
| *** bobh has joined #openstack-infra | 11:26 | |
| *** kjackal_ has joined #openstack-infra | 11:27 | |
| *** sarad has joined #openstack-infra | 11:27 | |
| *** jkilpatr has quit IRC | 11:27 | |
| *** jkilpatr has joined #openstack-infra | 11:27 | |
| *** dtantsur is now known as dtantsur|brb | 11:28 | |
| sarad | Hi. I want to know if i can used nodepool service with vSphere as cloud provider? | 11:28 |
| *** bobh has quit IRC | 11:31 | |
| *** yamamoto has quit IRC | 11:32 | |
| *** rhallisey has joined #openstack-infra | 11:34 | |
| *** nicolasbock has joined #openstack-infra | 11:35 | |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config master: Add neutron to required projects for net-cisco custom jobs https://review.openstack.org/512593 | 11:36 |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config master: Ensure branch overrides are correctly set for networking-cisco https://review.openstack.org/512594 | 11:36 |
| sarad | @here anyone can help me with my query above? | 11:37 |
| *** udesale has quit IRC | 11:38 | |
| *** smatzek has joined #openstack-infra | 11:39 | |
| *** rosmaita has joined #openstack-infra | 11:39 | |
| cmurphy | sarad: afaik nodepool only supports openstack as a backend but you might try https://github.com/puppetlabs/vmpooler which i think is designed to run on vsphere | 11:40 |
| sarad | thanks cmurphy, let me check vmpooler :) | 11:41 |
| *** yamamoto has joined #openstack-infra | 11:41 | |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config master: Ensure branch overrides are correctly set for networking-cisco https://review.openstack.org/512588 | 11:42 |
| *** yamamoto_ has joined #openstack-infra | 11:42 | |
| flaper87 | mmh, what am I missing? Why isn't Zuul happy? https://review.openstack.org/#/c/512330/ Maybe the patch this one depends on needs to be merged first | 11:44 |
| *** udesale has joined #openstack-infra | 11:44 | |
| *** yamamoto has quit IRC | 11:46 | |
| openstackgerrit | Tetsuro Nakamura proposed openstack-infra/infra-manual master: Update PyPI registration instruction https://review.openstack.org/512596 | 11:51 |
| *** tpsilva has joined #openstack-infra | 11:53 | |
| *** salv-orlando has quit IRC | 11:54 | |
| frickler | flaper87: yes, iiuc that needs https://review.openstack.org/512323 to be merged first | 11:55 |
| *** mat128 has joined #openstack-infra | 11:55 | |
| AJaeger | flaper87: see the important note in https://docs.openstack.org/infra/manual/creators.html#add-system-required-template | 11:55 |
| flaper87 | frickler: thanks | 11:55 |
| *** yamamoto_ has quit IRC | 11:56 | |
| AJaeger | flaper87: any suggestions on how to improve that? | 11:56 |
| *** amoralej is now known as amoralej|lunch | 11:57 | |
| flaper87 | AJaeger: weeeeeeeeeeeeell, for one, I should read the important notes till the end, instead of just reading half of it | 11:57 |
| chandankumar | AJaeger: is there a way to change node of openstack-python-jobs from ubuntu to fedora or centos for a particular project? | 11:59 |
| AJaeger | chandankumar: those jobs should be OS agnostic ;) | 12:00 |
| rosmaita | AJaeger: i think at the end of the "Add Project to Zuul" section, it would be good to say "commit everything up to here and submit a patch before proceeding" ... that Important Note comes too late IMO | 12:00 |
| *** chlong has quit IRC | 12:01 | |
| AJaeger | chandankumar: best ask the rest of the team later... | 12:01 |
| chandankumar | AJaeger: sure :-) | 12:01 |
| AJaeger | rosmaita: do you want to sent a patch? | 12:01 |
| *** udesale has quit IRC | 12:01 | |
| rosmaita | i can do that | 12:01 |
| *** shardy_lunch is now known as shardy | 12:02 | |
| *** jascott1 has joined #openstack-infra | 12:02 | |
| AJaeger | rosmaita: looking at the order, the gerritbot one could be in the initial change, so might need some reordering... | 12:02 |
| AJaeger | rosmaita: please go for it! | 12:02 |
| *** liusheng has quit IRC | 12:02 | |
| rosmaita | AJaeger i'll put up an initial patch and then you can comment and tell me how to improve it (because i really don't know a lot about the CI infrastructure) | 12:03 |
| *** liusheng has joined #openstack-infra | 12:03 | |
| AJaeger | rosmaita: thanks! | 12:04 |
| *** jpena is now known as jpena|lunch | 12:04 | |
| *** sileht has quit IRC | 12:05 | |
| *** sileht has joined #openstack-infra | 12:05 | |
| *** jascott1 has quit IRC | 12:06 | |
| *** sarad has quit IRC | 12:07 | |
| *** edmondsw has joined #openstack-infra | 12:09 | |
| mpeterson | just to confirm, in-repo we can have .zuul.yaml or .zuul.d/ with yaml inside, right? | 12:13 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: Remove dlrnapi report at the end of the run https://review.openstack.org/512590 | 12:17 |
| *** dprince has joined #openstack-infra | 12:17 | |
| *** yamamoto has joined #openstack-infra | 12:19 | |
| *** lucas-hungry is now known as lucasagomes | 12:19 | |
| *** dizquierdo has quit IRC | 12:20 | |
| odyssey4me | mpeterson yes | 12:20 |
| mpeterson | odyssey4me: thx | 12:21 |
| *** dave-mccowan has joined #openstack-infra | 12:22 | |
| *** wolverineav has joined #openstack-infra | 12:22 | |
| *** rlandy has joined #openstack-infra | 12:22 | |
| AJaeger | mpeterson: please read https://docs.openstack.org/infra/manual/zuulv3.html | 12:22 |
| *** boden has joined #openstack-infra | 12:23 | |
| boden | hi, I’m trying to get zuul v3 working with stable/ocata on the vmware-nsx project in https://review.openstack.org/#/c/512390/ However I’m seeing the wrong version of dependencies getting installed which is causing the job to fail http://logs.openstack.org/90/512390/3/check/openstack-tox-pep8/169ad0e/job-output.txt.gz#_2017-10-17_10_45_17_765865 it’s not clear where it’s getting neutron-lib-1.10.0… any ideas? | 12:25 |
| AJaeger | boden, your requirements file has "neutron-lib>=1.1.0 ", so 1.10.0 is not wrong ;) | 12:27 |
| AJaeger | boden: but indeed https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt?h=stable/ocata has 1.1.0, not 1.10.0 and you are using constraints | 12:27 |
| boden | AJaeger: sorry for being unclear, but yes your latter point is what I’m referring to | 12:28 |
| mpeterson | AJaeger: I've read it and I use it as a reference. There are some cases where it is not clear enough or where things are not documented. And that's when I turn to this resource to ask. | 12:28 |
| AJaeger | mpeterson: Sorry, I was under the impression that your question was answered ;) Could you update the document so that it's really clear, please? | 12:29 |
| AJaeger | boden, please add to the etherpad, somebody needs to look at it. | 12:30 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Harden exception handling around instance deletes https://review.openstack.org/512413 | 12:31 |
| AJaeger | boden, we run "tox --notest -e venv" - could you check locally whether that takes the constraints in - or ignores it? | 12:32 |
| boden | AJaeger: added to https://etherpad.openstack.org/p/zuulv3-issues … let me check on your other question | 12:33 |
| *** trown|outtypewww is now known as trown | 12:33 | |
| mpeterson | AJaeger: the confusion for me raised because yes, there is a mention about .zuul.d early on the document but then later on all references talk about .zuul.yaml instead of being more general. And there are some implicit conventions (ie: project-templates should not be in the in-repo templates section, project-templates should only be used when you need to share, requirements should be repeated | 12:34 |
| mpeterson | for every job in project-config, etc) | 12:34 |
| *** ykarel is now known as ykarel|away | 12:34 | |
| mpeterson | AJaeger: said that. Sure, I'll make a note of it and try to commit a patch to clarify it | 12:34 |
| *** gongysh has joined #openstack-infra | 12:35 | |
| *** gongysh has quit IRC | 12:35 | |
| AJaeger | mpeterson: we expect the majority of the projects to use .zuul.yaml that's why it's used in most cases. | 12:35 |
| AJaeger | You can have project-templates in-repo... | 12:35 |
| mpeterson | yes, having them yes. But it was mentioned earlier today that you prefer them being used in project.templates in project-config instead of in the in-repo counterpart | 12:38 |
| boden | AJaeger if I run "tox --notest -e venv" on stable/ocata of vmware-nsx, it does install neutron-lib==1.1.0 which is what I’d expect from the contraints file (not lib 1.10.0 as I’m seeing in the v3 job) | 12:39 |
| *** salv-orlando has joined #openstack-infra | 12:40 | |
| *** jcoufal has joined #openstack-infra | 12:41 | |
| AJaeger | mpeterson: see the "what not to convert section" - you used a couple of jobs that are standard. And just definining a template in a standard place for one time usage is useless. | 12:41 |
| AJaeger | mpeterson: Feel free to create templates in-repo as long as you respect the global namespace | 12:42 |
| AJaeger | boden: please put on the etherpad, others need to dig into this | 12:42 |
| AJaeger | mpeterson: does that clarify it? | 12:43 |
| jd_ | AJaeger: question; in https://review.openstack.org/512342 you say that the jobs are now defined in .zuul.yaml, but what does happen for stable/* branches, where are the jobs defined? | 12:43 |
| *** LindaWang has quit IRC | 12:44 | |
| sambetts | jd_: they can be defined in the stable branch itself in a .zuul.yaml or in the master branch with a branch filter | 12:44 |
| AJaeger | jd_: https://docs.openstack.org/infra/manual/zuulv3.html has a section on stable branches | 12:45 |
| jd_ | thanks guys, you rock, i'm gonna read that | 12:45 |
| AJaeger | basically: The jobs can be in master, the project lines need to be backported | 12:45 |
| *** kgiusti has joined #openstack-infra | 12:46 | |
| AJaeger | jd_: so, backport the last "section" of your .zuul.yaml file, the one starting with "- project" and customize for each branch | 12:46 |
| AJaeger | jd_: if you want a sanity review, just ask ;) | 12:46 |
| *** gongysh has joined #openstack-infra | 12:48 | |
| jd_ | thanks AJaeger :) | 12:48 |
| *** gongysh has quit IRC | 12:48 | |
| *** mriedem has joined #openstack-infra | 12:49 | |
| *** jpena|lunch is now known as jpena | 12:51 | |
| *** esberglu has joined #openstack-infra | 12:52 | |
| *** yamamoto has quit IRC | 12:54 | |
| *** bobh has joined #openstack-infra | 12:55 | |
| boden | AJaeger: it’s on https://etherpad.openstack.org/p/zuulv3-issues maybe you missed my msg about that earlier | 12:58 |
| *** bh526r has joined #openstack-infra | 12:58 | |
| openstackgerrit | Brian Rosmaita proposed openstack-infra/infra-manual master: Clarify patchsets for new project configuration https://review.openstack.org/512607 | 13:00 |
| rosmaita | AJaeger: ^^ | 13:01 |
| AJaeger | rosmaita: thanks, will review later. | 13:01 |
| AJaeger | boden: thanks | 13:01 |
| *** pabelanger has quit IRC | 13:01 | |
| *** pabelanger has joined #openstack-infra | 13:01 | |
| *** LindaWang has joined #openstack-infra | 13:02 | |
| robcresswell | If we're running standard jobs in v3 (things like "openstack-tox-pep8", but we customise them (see https://review.openstack.org/#/c/512588/2/zuul.d/projects.yaml) should those jobs live in our local .zuul.yaml, or upstream? | 13:02 |
| robcresswell | I've been reading the migration doc and it was a little unclear to me what happens in these semi-custom scenarios | 13:03 |
| AJaeger | robcresswell: leave them in project-config - and please help make the docs clearer. | 13:05 |
| AJaeger | robcresswell: networking-cisco is not an official project, so you could move them out if you really want. For official projects they should be in. And my recommendation for others is to keep them in (=project-config) as well | 13:06 |
| *** amoralej|lunch is now known as amoralej | 13:07 | |
| *** andreas_s has quit IRC | 13:07 | |
| *** andreas_s has joined #openstack-infra | 13:07 | |
| robcresswell | AJaeger: Personally it would be easier to move them out, as we track against eol branches for some tests (so its faster for us to update locally than bug infra for reviews) | 13:07 |
| pabelanger | janki: AJaeger: jobs no longer clone from zm03.o.o, that looks to be a jenkins job. I'd recommend a recheck | 13:07 |
| robcresswell | AJaeger: But, happy to stick to the status quo if its easier to oversee. | 13:08 |
| AJaeger | might be easier overall to keep them in for now... | 13:10 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: WIP: Migrate legacy jobs https://review.openstack.org/512611 | 13:11 |
| AJaeger | pabelanger: yeah, that was Zuul v2 - should have checked janki's report better ;( | 13:11 |
| *** thorst has joined #openstack-infra | 13:14 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: WIP: Migrate legacy jobs https://review.openstack.org/512611 | 13:15 |
| dhellmann | I'm trying to debug a bindep change to the releases repo. Is there some way for me to see what bindep actually installs in a job? http://logs.openstack.org/09/512609/1/check/ | 13:18 |
| dhellmann | the real thing I'm trying to fix is some missing packages on the "trusted" nodes that caused a new tag-releases failure. Are those jobs still running on a single signing node, or are we building nodes dynamically? http://logs.openstack.org/0e/0ef82df332aad8e827d41d879cd2c207ddd76f93/release-post/tag-releases/a48d209/job-output.txt.gz#_2017-10-17_12_55_04_697209 | 13:19 |
| *** baoli has joined #openstack-infra | 13:20 | |
| mriedem | looks like the live migration job is failing due to pypi mirrors not having python3-rados or python3-rbd http://logs.openstack.org/06/512406/1/check/legacy-tempest-dsvm-multinode-live-migration/95adf30/job-output.txt.gz | 13:20 |
| mriedem | i'm not sure why this is even trying to install python3 pips since it's not a py3 job | 13:20 |
| mriedem | ^ is against an ocata change btw | 13:20 |
| *** dtantsur|brb is now known as dtantsur | 13:21 | |
| *** andreas_s has quit IRC | 13:21 | |
| AJaeger | dhellmann: http://logs.openstack.org/09/512609/1/check/openstack-tox-pep8/8fedc38/job-output.txt.gz#_2017-10-17_13_15_12_536459 | 13:21 |
| *** lbragstad has joined #openstack-infra | 13:22 | |
| AJaeger | dhellmann: AFAIU we have no trusted separate nodes, we use a new node dynamically every time | 13:23 |
| dhellmann | AJaeger : oh! I saw an earlier line that made me thing it was running bindep and just saying "ok" | 13:23 |
| * dhellmann should not debug before tea | 13:23 | |
| AJaeger | pabelanger, do you know why the "upstream-translation" jobs just report "error" without any log files? | 13:23 |
| dhellmann | yeah, I knew the change to dynamic nodes was coming, I wasn't sure we were there | 13:23 |
| * AJaeger sends some green tea west... | 13:24 | |
| pabelanger | AJaeger: no, I can look into it | 13:24 |
| pabelanger | I need a coffee first | 13:24 |
| * AJaeger sends some coffee as well - and grabs some for himself ;) | 13:24 | |
| *** sree has quit IRC | 13:25 | |
| *** sree has joined #openstack-infra | 13:26 | |
| *** andreas_s has joined #openstack-infra | 13:26 | |
| frickler | mriedem: thats looks to me like not pypi but ubuntu pkg installation failing. iirc python3-rados et al only was added for pike uca | 13:27 |
| mriedem | frickler: hmm, wasn't a problem before, i'm not sure what's trying to pull those packages in | 13:29 |
| mriedem | will check the job definition in openstack-zuul-jobs first | 13:29 |
| *** sree has quit IRC | 13:30 | |
| mriedem | https://github.com/openstack/devstack-plugin-ceph/commit/54b360cdd3dc73465d362f2d0b15672d82478e5e | 13:30 |
| mriedem | duh duh duh | 13:30 |
| mriedem | melwitt: ^ | 13:30 |
| mriedem | devstack-plugin-ceph is branchless | 13:30 |
| mriedem | so any changes to master are going to affect stable | 13:30 |
| *** smatzek has quit IRC | 13:31 | |
| fungi | only 300 lines of scrollback while i was asleep? things must be going well! | 13:31 |
| openstackgerrit | Julien Danjou proposed openstack-infra/infra-manual master: Fix typo in zuulv3 https://review.openstack.org/512614 | 13:31 |
| mordred | morning fungi ! | 13:34 |
| mordred | AJaeger: I do not :( | 13:34 |
| fungi | AJaeger: we might consider a (temporary?) high-frequency periodic pipeline where we can run a representative handful of translation management jobs so we can iterate on them more quickly | 13:34 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Remove .yaml from upstream-translation-update post-run https://review.openstack.org/512617 | 13:35 |
| pabelanger | AJaeger: mordred: ^lets try for an upstream-translate-update fix | 13:35 |
| pabelanger | on a related note, i totally think we should support file extensions in pre-run, run and post-run | 13:35 |
| pabelanger | :) | 13:35 |
| *** andreas_s has quit IRC | 13:36 | |
| mordred | pabelanger: yah - OR - we should synax error on them | 13:36 |
| pabelanger | yah, I believe there is a storyboard for syntax error someplace | 13:36 |
| pabelanger | not the first time we have it this | 13:36 |
| fungi | AJaeger: still catching up, so apologies if someone else has already answered, but in zuul v3 jobs don't clone from the zuul mergers, they get their repositories pushed onto the nodes by the executors. the job janki linked ran under zuul v2 back on october 13 | 13:38 |
| pabelanger | http://logs.openstack.org/80/507780/7/gate/build-openstack-releasenotes/85c21ce/ is a new error for me | 13:39 |
| pabelanger | SSH error during task run | 13:39 |
| pabelanger | 2017-10-17 11:43:26.134958 | ConnectionResetError: [Errno 104] Connection reset by peer | 13:39 |
| pabelanger | from zuul_stream too | 13:39 |
| *** boden_ has joined #openstack-infra | 13:40 | |
| pabelanger | going to add the to zuulv3-issue | 13:40 |
| *** andreas_s has joined #openstack-infra | 13:41 | |
| fungi | gfidente: i'm missing some context on the luminous testing. i've skimmed the etherpad you linked, but no idea what key, api request, file and server you're talking about | 13:41 |
| *** andreas_s has quit IRC | 13:41 | |
| *** andreas_s has joined #openstack-infra | 13:41 | |
| *** sree has joined #openstack-infra | 13:41 | |
| gfidente | fungi it looks like to delete an etherpad it is necessary to submit a request to the etherpad-lite api | 13:42 |
| AJaeger | fungi: yes, might propose that later today... | 13:42 |
| gfidente | https://docs.openstack.org/infra/system-config/etherpad.html#deleting-a-pad | 13:42 |
| *** chlong has joined #openstack-infra | 13:42 | |
| AJaeger | fungi: yes, pabelanger pointed out that I needed more coffee this morning ;) | 13:42 |
| *** boden has quit IRC | 13:42 | |
| *** boden_ is now known as boden | 13:42 | |
| AJaeger | fungi: thanks! | 13:43 |
| pabelanger | question! should ironic be in the integrated change queue in gate? | 13:43 |
| pabelanger | because, it currently is not | 13:43 |
| *** gouthamr has joined #openstack-infra | 13:43 | |
| *** felipemonteiro_ has joined #openstack-infra | 13:43 | |
| AJaeger | pabelanger: my hero! That might explain the upstream translation thing! | 13:43 |
| mordred | pabelanger: that's a great question - if it's not I'm guessing it didn't share jobs | 13:43 |
| mordred | pabelanger: oh - speaking of errors ... | 13:43 |
| AJaeger | care to review https://review.openstack.org/512529 as well? That's another obvious translation bug ;( | 13:44 |
| *** felipemonteiro__ has joined #openstack-infra | 13:44 | |
| mordred | pabelanger: https://review.openstack.org/#/c/512448/ is failing - and isn't running its run playbook - I can't see why | 13:44 |
| *** gcb has quit IRC | 13:45 | |
| mordred | pabelanger: but I might just have eyeball fatigue - do you see anyhting? | 13:45 |
| pabelanger | mordred: yah, I'm unsure if it should be sharing jobs. | 13:45 |
| pabelanger | looking | 13:45 |
| *** sree has quit IRC | 13:46 | |
| *** gcb has joined #openstack-infra | 13:47 | |
| fungi | jd_: on the point of backporting the projects section of .zuul.yaml, jeblair sent a summary to the -dev ml yesterday as well: http://lists.openstack.org/pipermail/openstack-dev/2017-October/123688.html | 13:47 |
| jd_ | thanks fungi | 13:48 |
| *** felipemonteiro_ has quit IRC | 13:48 | |
| pabelanger | mordred: comment on 512448 typo | 13:49 |
| arxcruz | hi, how can I create a group in gerrit? requesting in o-infra mailing list, or can I make a review ? | 13:49 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove .yaml from upstream-translation-update post-run https://review.openstack.org/512617 | 13:50 |
| pabelanger | arxcruz: project-config we have ACLs for gerrit, you'd propose a change there | 13:51 |
| mordred | pabelanger: thank you! | 13:51 |
| fungi | gfidente: oh! you're talking about the etherpad api specifically, and asking for one of us to delete that pad. yes, the api key is an admin-only secret, but i can delete that pad if that's what you want... why did you want it deleted? | 13:51 |
| *** psachin has quit IRC | 13:52 | |
| mordred | dmsimard: ^^ we should, at some point, figureout why that isn't logging | 13:52 |
| arxcruz | pabelanger: thanks :) | 13:52 |
| mordred | dmsimard: it's not showing up in ara or in log stream | 13:52 |
| mriedem | oh wow, nice http://zuulv3.openstack.org/static/stream.html?uuid=89722725151544118fd72866b3530f6f&logfile=console.log | 13:53 |
| pabelanger | mordred: yah, I noticed it in gerrit, but did look on ze05 to confirm the exception. | 13:53 |
| mriedem | no more telnet :) | 13:53 |
| fungi | mriedem: even better, it's using finger protocol now nehind the scenes! | 13:53 |
| fungi | s/nehind/behind/ | 13:53 |
| *** trown is now known as trown|brb | 13:53 | |
| *** esberglu has quit IRC | 13:54 | |
| pabelanger | mriedem: finger 89722725151544118fd72866b3530f6f@ze06.openstack.org | 13:54 |
| pabelanger | much cooler :D | 13:54 |
| fungi | mriedem: you could finger that job at the executor responsible for it, but we have plans to add a finger multiplexer/dispatcher to get you to the right executor automatically | 13:54 |
| *** esberglu has joined #openstack-infra | 13:54 | |
| fungi | here in infra-land, console/command-line users are first-class citizens | 13:55 |
| * pabelanger still hoping will add finger:// URL to zuulv3.o.o, come on JS expert! Save me now | 13:55 | |
| *** yamamoto has joined #openstack-infra | 13:55 | |
| mriedem | should i bare my soul and tell you all i have no experience with finger protocol? | 13:55 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Handle tox environments via tox config https://review.openstack.org/512448 | 13:56 |
| tosky | finger <something>@<host> | 13:56 |
| fungi | mriedem: aww, c'mon. how did you ever get by without the finger utility? | 13:56 |
| *** Jeffrey4l has quit IRC | 13:56 | |
| mordred | mriedem: that's ok - it's very similar to just netcatting a port ... except it expects a single line to be sent to tell it what to return | 13:56 |
| fungi | finger fungi@cthulhu.yuggoth.org | 13:56 |
| jd_ | i'm sure finger is offensive in certain culture | 13:57 |
| pabelanger | tosky: finger <something>@<host>/var/log/syslog | 13:57 |
| jd_ | :) | 13:57 |
| mordred | jd_: everything is offensive in certain culture | 13:57 |
| mriedem | jd_: i was about to say it | 13:57 |
| tosky | pabelanger: uh, also in the original version? | 13:57 |
| jd_ | mordred++ | 13:57 |
| tosky | pabelanger: or is it the openstack extension? | 13:57 |
| fungi | jd_: i'm sure i'm offensive in most cultures... i try not to let it bother me | 13:57 |
| mordred | including but certainly not limited to me | 13:57 |
| mordred | fungi: jinx | 13:57 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Make executor hostname configurable https://review.openstack.org/512629 | 13:57 |
| *** armax has joined #openstack-infra | 13:58 | |
| *** Jeffrey4l has joined #openstack-infra | 13:59 | |
| *** smatzek has joined #openstack-infra | 13:59 | |
| *** esberglu has quit IRC | 13:59 | |
| *** mlavalle has joined #openstack-infra | 13:59 | |
| mlavalle | can someone here help me with an IRC channel question? | 13:59 |
| pabelanger | tosky: I think I have it backwards | 14:00 |
| *** smatzek has quit IRC | 14:00 | |
| fungi | mlavalle: likely, but only if you ask it | 14:00 |
| pabelanger | mlavalle: sure, best to just ask the question | 14:00 |
| *** dhinesh has joined #openstack-infra | 14:00 | |
| *** smatzek has joined #openstack-infra | 14:01 | |
| *** sbezverk has joined #openstack-infra | 14:01 | |
| mlavalle | fungi, pabelanger: last week I started, by mistake, a meeting in #openstack-neutron: http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-neutron.2017-10-12.log.html#t2017-10-12T15:00:25 | 14:01 |
| mlavalle | I ended it right away, as you will see a few lines below | 14:01 |
| mlavalle | but neutron_l3 still shows at the top of the channel | 14:02 |
| *** trown|brb is now known as trown | 14:02 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: add bindep step to release jobs https://review.openstack.org/512633 | 14:03 |
| *** jchhatbar has quit IRC | 14:03 | |
| fungi | mlavalle: yes, it looks like the openstack bot failed to restore the previous channel topic once the meeting ended. that is unusual | 14:03 |
| dhellmann | fungi, mordred: I'm not sure I have the path right in ^^, could you give it a look, please? | 14:04 |
| *** e0ne_ has joined #openstack-infra | 14:04 | |
| *** e0ne has quit IRC | 14:04 | |
| pabelanger | fungi: mlavalle: no, it did but to a status notice | 14:04 |
| pabelanger | *** openstack changes topic to "Job log uploads are failing due to lack of inodes. Jobs also fail due to mismatches in gnutls packages. Workarounds for both in progress with proper fixes to follow." | 14:05 |
| fungi | mlavalle: oh, i see, we have the openstack bot and the openstackstatus bot fighting over channel topocs | 14:05 |
| *** dhinesh has quit IRC | 14:05 | |
| *** yamamoto has quit IRC | 14:05 | |
| pabelanger | yah | 14:05 |
| *** rbrndt has joined #openstack-infra | 14:05 | |
| fungi | we actually have openstackstatus not joining official meeting channels to avoid this particular scenario | 14:05 |
| mlavalle | pabelanger, fungi: so how do we re-store the Neutron channel to its correct topic? | 14:06 |
| *** slaweq has joined #openstack-infra | 14:06 | |
| fungi | but basically what happened was openstack set your channel topic to the meeting name, then openstackstatus set your channel topic from that to the status alert message we issued, then later openstackstatus set the topic back to the one it saw when the alert was first issued, which happened to be the one set for the mistaken meeting | 14:06 |
| fungi | mlavalle: i can do that now, or anyone listed with topic editing access in `/msg chanserv access #openstack-neutron list` can do it | 14:07 |
| *** thingee has quit IRC | 14:07 | |
| mlavalle | fungi: would you do it please? | 14:08 |
| *** esberglu has joined #openstack-infra | 14:09 | |
| *** thingee has joined #openstack-infra | 14:09 | |
| fungi | mlavalle: yep, do you happen to have the original topic in scrollback? i apparently don't lurk in that channel so i don't have it handy | 14:10 |
| fungi | i can grep it out of channel logs on eavesdrop if necessary, but that will take a few minutes to get the pattern right to find the last time it was (re)set | 14:11 |
| *** slaweq has quit IRC | 14:11 | |
| pabelanger | dhellmann: -1 on 512633 with comment | 14:11 |
| dhellmann | pabelanger : thanks! | 14:11 |
| *** Nil_ has joined #openstack-infra | 14:12 | |
| mlavalle | fungi: no, I don't have it :-( | 14:12 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: add bindep step to release jobs https://review.openstack.org/512633 | 14:12 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/openstack-zuul-jobs master: Remove legacy OSA role tests https://review.openstack.org/512635 | 14:13 |
| mlavalle | fungi: "Discussion of OpenStack Networking || for support join #openstack" | 14:13 |
| mordred | dhellmann: ++ looks good | 14:13 |
| fungi | mlavalle: done | 14:13 |
| mlavalle | fungi: thanks :-) | 14:13 |
| mordred | pabelanger: that makes me wonder if we shoulnd't update the defaults on the bindep role - at least bindep_dir | 14:14 |
| dhellmann | pabelanger , mordred : thanks for your help there | 14:14 |
| fungi | mlavalle: we can add some channel regulars to the access list for that channel too if you want, and then they can issue things like `/msg chanserv topic #openstack-neutron Discussion of OpenStack Networking || for support join #openstack` too | 14:14 |
| mordred | dhellmann: sure thing! here's hoping that's the last issue with your jobs ... | 14:14 |
| * dhellmann crosses his fingers | 14:14 | |
| pabelanger | mordred: yah, I cannot remember why we've been doing that | 14:14 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 14:15 |
| pabelanger | mordred: oh, because we default bindep_dir: . | 14:15 |
| mordred | pabelanger: yah - I'm thinking that's from WAY long ago when we were still learning how to write things | 14:16 |
| pabelanger | mordred: or we just setup bindep_path in site-varaibles.yaml to what we want | 14:17 |
| pabelanger | err | 14:17 |
| pabelanger | bindep_dir | 14:17 |
| stephenfin | dhellmann: Just an FYI, I've a few pbr patches up for review that could be worth looking at. I'd like to cut pbr 4.0 before EOW | 14:17 |
| dhellmann | stephenfin : the release jobs are currently broken, so we're prioritizing fixing them to unblock a bunch of releases. I'll put pbr on my list for after that's done. | 14:18 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Update bindep role defaults to be more sane https://review.openstack.org/512639 | 14:18 |
| mordred | pabelanger: ^^ how about that | 14:18 |
| fungi | pabelanger: mordred: so we're not able to reuse the normal bindep role with the way it's currently written? | 14:18 |
| mlavalle | fungi: yeah, that would be great. please add armax, haleyb, ihrachys, davidsha | 14:18 |
| openstackgerrit | Nguyen Van Trung proposed openstack-infra/infra-manual master: Change http to https spec link https://review.openstack.org/512640 | 14:18 |
| stephenfin | dhellmann: Oh, I didn't know. Yup yup, no prob :) | 14:19 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 14:19 |
| dhellmann | stephenfin : no worries; we'll get it all done :-) | 14:19 |
| mordred | pabelanger, fungi: not currently - and yah - that reminds me why we set the default value to '.' - it was becaue that role in particular seemed like one that could be a place where we used a non-zuul specific role | 14:20 |
| *** yamahata has joined #openstack-infra | 14:20 | |
| openstackgerrit | Merged openstack-infra/project-config master: add bindep step to release jobs https://review.openstack.org/512633 | 14:20 |
| fungi | mlavalle: ihrachys was already there, but i've granted similar access for the other 3 now | 14:20 |
| mlavalle | thanks | 14:20 |
| *** gongysh has joined #openstack-infra | 14:20 | |
| *** gongysh has quit IRC | 14:20 | |
| *** dave-mccowan has quit IRC | 14:21 | |
| *** gongysh has joined #openstack-infra | 14:21 | |
| jeblair | fungi, mordred: should we restart the scehduler? | 14:22 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Update jobs for features/zuulv3 branch https://review.openstack.org/512642 | 14:22 |
| *** reed has quit IRC | 14:23 | |
| *** reed has joined #openstack-infra | 14:25 | |
| *** dizquierdo has joined #openstack-infra | 14:25 | |
| openstackgerrit | Michel Peterson proposed openstack-infra/openstack-zuul-jobs master: networking-odl: Removing legacy jobs definitions https://review.openstack.org/512644 | 14:27 |
| openstackgerrit | Michel Peterson proposed openstack-infra/project-config master: networking-odl: Removing legacy jobs references https://review.openstack.org/512645 | 14:27 |
| fungi | jeblair: did we land more fixes? | 14:27 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 14:28 |
| jeblair | fungi: we landed several over the weekend and yesterday which are not in effect | 14:28 |
| fungi | ahh, right, we didn't restart it during sunday's maintenance | 14:28 |
| fungi | can't hurt, i suppose | 14:28 |
| jeblair | ignore parents on variants and use weakref for change cache come to mind. also, it needs a restart for statsd. | 14:28 |
| jeblair | i mean, it'll hurt a little | 14:29 |
| jeblair | i'll go ahead and do that now | 14:29 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Update jobs for features/zuulv3 branch https://review.openstack.org/512642 | 14:29 |
| AJaeger | team, we're one step further with upstream translation, they now produce logs (thanks, pabelanger !): http://paste.openstack.org/show/623845/ - there's an ansible syntax error. could anybody help, please? | 14:29 |
| fungi | well, hurt as in delay stuff currently in pipelines and lose a handful of things in post i guess | 14:30 |
| jeblair | restarting now | 14:30 |
| pabelanger | AJaeger: looking | 14:30 |
| jeblair | mordred: did you have a chance to think about something like an openstack-tox-functional job which omits revoke-sudo? | 14:30 |
| *** gongysh has quit IRC | 14:31 | |
| mordred | jeblair: I have thoght about it -but I have not yet had a chance to type those thoughts into an editor | 14:31 |
| mordred | jeblair: why don't I do that real quick | 14:31 |
| pabelanger | AJaeger: I see an isse | 14:31 |
| pabelanger | working on patch | 14:31 |
| *** slaweq_ has quit IRC | 14:32 | |
| AJaeger | great, thanks | 14:32 |
| jeblair | scheduler is running again | 14:32 |
| jeblair | oh hey it's down to 3m from cold start :) | 14:33 |
| Shrews | oh, would have nice to have gotten https://review.openstack.org/512360 in before that. oh well | 14:33 |
| fungi | wow, that's quite the speedup | 14:33 |
| *** tmorin has left #openstack-infra | 14:33 | |
| jeblair | Shrews: yep :( | 14:33 |
| Shrews | i'm sure we'll restart it again in the future :) | 14:34 |
| *** chlong has quit IRC | 14:34 | |
| *** hongbin has joined #openstack-infra | 14:34 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Fix type with java_cert in prep-zanata https://review.openstack.org/512647 | 14:35 |
| pabelanger | AJaeger: ^ | 14:35 |
| jeblair | gate restored; restoring check now | 14:35 |
| jeblair | pabelanger: +3 https://review.openstack.org/512451 ? | 14:35 |
| openstackgerrit | Merged openstack-infra/zuul-sphinx master: Add support for project-templates https://review.openstack.org/512449 | 14:36 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Fix type with java_cert in prep-zanata https://review.openstack.org/512647 | 14:36 |
| AJaeger | jeblair: do you need a new release of zuul-sphinx to use the changes that just merged? | 14:36 |
| pabelanger | jeblair: yes please | 14:36 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Update jobs for features/zuulv3 branch https://review.openstack.org/512642 | 14:36 |
| *** camunoz has joined #openstack-infra | 14:37 | |
| *** dave-mccowan has joined #openstack-infra | 14:37 | |
| AJaeger | pabelanger: thanks! | 14:37 |
| pabelanger | AJaeger: np! | 14:37 |
| AJaeger | pabelanger: could you look at lines 162 and 163 of the etherpad - we have another certs problem | 14:38 |
| pabelanger | sure | 14:39 |
| pabelanger | AJaeger: Oh, hmm | 14:39 |
| tinwood | Hi. I'm still having problems configuring my project for zuul v3. review is: https://review.openstack.org/#/c/508304/ and I've written a gist here about the problem: https://gist.github.com/ajkavanagh/a09fe4e94d1aec7ccf655e71b9bc7329 -- I'm sure it's really simple!? Anybody have any cycles to have a peek please? | 14:40 |
| pabelanger | that might be an issue to run on local ( zuul-executors ). We likely are missing a SSL info in /etvc | 14:40 |
| jeblair | AJaeger: yes | 14:40 |
| pabelanger | AJaeger: mordred: does set-service-type-data-fact need to run on localhost, or can that run on a node? | 14:40 |
| *** andreww has joined #openstack-infra | 14:40 | |
| mordred | pabelanger: it can run anywhere | 14:40 |
| pabelanger | kk | 14:40 |
| pabelanger | lets move it to !localhost | 14:41 |
| pabelanger | will propose fix | 14:41 |
| mordred | cool | 14:41 |
| mordred | jeblair: we maywant to call this something other than "openstack-tox-functional" ... | 14:41 |
| *** iyamahat has joined #openstack-infra | 14:41 | |
| *** links has quit IRC | 14:41 | |
| pabelanger | Oh | 14:41 |
| mordred | jeblair: since there are also a bunch of tox -efunctional jobs that don't need sudo | 14:41 |
| pabelanger | Hmm | 14:41 |
| jeblair | mordred: point | 14:41 |
| jeblair | mordred: there are a lot that do, right? | 14:42 |
| mordred | yah | 14:42 |
| mordred | well - wait a sec ... | 14:42 |
| mordred | I answered too quickly | 14:42 |
| jeblair | mordred: i'm under the impression there are -- just wanted to double check i wasn't misremembering | 14:42 |
| fungi | pabelanger: oh! like we're not exposing /etc/ssl inside the bwrap chroot? | 14:42 |
| mordred | there are a lot of tox -efunctional jobs | 14:42 |
| mordred | there are also at least *some* tox jobs (openstack-ansible has some specifically) that want to run without sudo revoked | 14:42 |
| AJaeger | tinwood: deploy-guide jobs should stay in project-config/openstack-zuul-jobs. If you want to convert them, see what we did for releasenotes or api-guide. | 14:43 |
| mordred | jeblair: how about "openstack-tox-with-sudo" | 14:43 |
| *** jpich has quit IRC | 14:43 | |
| AJaeger | tinwood: please discuss also with dhellmann on where those should life - perhaps doc/source is not a bad idea.. | 14:43 |
| *** dizquierdo has quit IRC | 14:44 | |
| pabelanger | fungi: right, inside bubblewrap is super minimal, package installs with apt-get don't even work | 14:44 |
| *** dizquierdo has joined #openstack-infra | 14:44 | |
| fungi | pabelanger: yeah, and we'd only want access to non-private contents of /etc/ssl there | 14:44 |
| *** sree has joined #openstack-infra | 14:44 | |
| *** andreww has quit IRC | 14:44 | |
| fungi | (not that the executors should have any private ssl keys installed on them, now that i think about it) | 14:45 |
| mordred | fungi, pabelanger: I think we can run this thing not on localhost - but we might also want to expose ssl things - the trigger-readthedocs job hits an external url .. that should probably be able to do ssl properly | 14:45 |
| pabelanger | mordred: minor issue with moving to not localhost, we need to switch to host vars to access fact | 14:45 |
| *** iyamahat has quit IRC | 14:45 | |
| pabelanger | trying to figure out best way forward on that | 14:45 |
| fungi | mordred: yeah, i think if we actually want hitting rest apis from the executor to be a thing, then they'll need the ability to validate https ssl/tls certs | 14:45 |
| mordred | maybe let's just figure out fixing ssl then ... | 14:45 |
| tinwood | AJaeger, I really don't know. I was just documenting my findings so far. Something seems 'broken'. Is dhellmann the person to ask on this? Thanks. | 14:46 |
| mordred | I cna't think of any reason we want to prevent things from being able to hit ssl urls if they can hit normal ones | 14:46 |
| *** xarses has joined #openstack-infra | 14:46 | |
| jeblair | mordred: wfm. in jjb there were 816 out of 863 jobs with 'func' in the name and did not use 'revoke-sudo' | 14:46 |
| jeblair | mordred: er to make that more clear | 14:46 |
| mordred | jeblair: ah. fascinating | 14:46 |
| pabelanger | mordred: we can update puppet-zuul and add read-only mount to zuul.conf on executors | 14:46 |
| jeblair | mordred: there were 863 jobs with func in the name. 816 of themdid not use revoke-sudo | 14:46 |
| pabelanger | that should be enough | 14:46 |
| AJaeger | tinwood: discuss with dhellmann on where the deploy guide should life. And then let's discuss how to build it. But you don't need to move the job in-tree, we should share it with other deployment guides. | 14:47 |
| AJaeger | tinwood: let's move to #openstack-doc quickly, ok? | 14:47 |
| *** yamamoto has joined #openstack-infra | 14:47 | |
| *** yamamoto has quit IRC | 14:47 | |
| *** chlong has joined #openstack-infra | 14:48 | |
| mpeterson | something happened with Zuul? I've seen a bunch of jobs that I was waiting to finish just restarting again | 14:48 |
| mordred | jeblair: how hard is it for you to tell what tox env those used? | 14:48 |
| mpeterson | ie: https://review.openstack.org/#/c/500461/ | 14:48 |
| *** smarcet has joined #openstack-infra | 14:49 | |
| pabelanger | mpeterson: we restarted zuul scheduler, which means jobs were stopped / then started again | 14:49 |
| tinwood | AJaeger, ok | 14:50 |
| mpeterson | pabelanger: ah okey :) thx | 14:50 |
| fungi | mpeterson: specifically, to get it running on some recent patches we'd landed over the weekend | 14:50 |
| mpeterson | that's alright, I wasn't sure if it was something expected or perhaps a bug | 14:51 |
| jeblair | mordred: still looking | 14:52 |
| mordred | jeblair: I'm thinking "openstack-tox-with-sudo" - "openstack-tox-functional" and "openstack-tox-functional-with-sudo" ... so the diffrence can be clear | 14:53 |
| *** smarcet has quit IRC | 14:53 | |
| *** jtomasek has quit IRC | 14:54 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Update jobs for features/zuulv3 branch https://review.openstack.org/512642 | 14:54 |
| *** dizquierdo has quit IRC | 14:54 | |
| jeblair | mordred: among non-ansible-related jobs (the minority), maybe 10 use 'functional', 10 use 'func' and the rest (25 or so) use very specific envs | 14:54 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: New client command for printing autohold requests https://review.openstack.org/512360 | 14:54 |
| jeblair | mordred: the ansible related jobs (800 or so) inspect the file and determine whether to use functional or func | 14:55 |
| *** jtomasek has joined #openstack-infra | 14:55 | |
| mordred | loverly | 14:55 |
| jeblair | mordred: i like your plan | 14:56 |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Add openstack-tox-with-sudo test job https://review.openstack.org/512653 | 14:56 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 14:56 |
| mtreinish | ianw, clarkb: hmm, it doesn't look like restarting the worker pulled in the worker changes | 14:56 |
| mtreinish | the names are still the same in the db | 14:57 |
| clarkb | mtreinish: I'll have a lool in a bit. Slpwly booting day now | 14:57 |
| clarkb | *look | 14:57 |
| jeblair | i think that message was clear :) | 14:57 |
| *** jtomasek has quit IRC | 14:58 | |
| *** janki has joined #openstack-infra | 14:58 | |
| *** jtomasek has joined #openstack-infra | 14:59 | |
| *** sdague has quit IRC | 14:59 | |
| AJaeger | any volunteer to +2A https://review.openstack.org/#/c/512647/ - fix by pabelanger to fix ansible in prep-zanata | 14:59 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 14:59 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Add /etc/ssl to trusted_ro_paths for zuul-executors https://review.openstack.org/512657 | 15:00 |
| pabelanger | AJaeger: mordred: fungi: ^ add /etc/ssl into brap as read-only | 15:00 |
| mtreinish | clarkb: cool, thanks | 15:00 |
| *** smarcet has joined #openstack-infra | 15:00 | |
| *** dhinesh has joined #openstack-infra | 15:01 | |
| pabelanger | AJaeger: mordred: fungi: we'll then need to restart executors | 15:01 |
| smcginnis | Is this still used in the zuulv3 world? https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L1221 | 15:02 |
| openstackgerrit | Merged openstack-infra/project-config master: Graphana: Add Zuul v3 executor graphs https://review.openstack.org/512451 | 15:02 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Fix build-deploy-guide https://review.openstack.org/512658 | 15:02 |
| AJaeger | pabelanger: thanks | 15:02 |
| fungi | smcginnis: nope | 15:03 |
| smcginnis | fungi: Is there a new equivalent? | 15:03 |
| *** kiennt26_ has joined #openstack-infra | 15:03 | |
| fungi | smcginnis: zuul v3 configuration is the new equivalent | 15:03 |
| smcginnis | fungi: Under zuul.d? | 15:04 |
| mpeterson | once things are stable, the deprecated things will be deleted by the infra team or do we need to submit patches to remove them? like Zuul<3 jobs | 15:05 |
| fungi | smcginnis: yeah, but where depends on which part of the v2 layout you're talking about. some things also moved into job definitions | 15:05 |
| *** dhinesh has quit IRC | 15:05 | |
| fungi | smcginnis: https://docs.openstack.org/infra/manual/zuulv3.html#how-jobs-are-defined-in-zuul-v3 talks about it a little | 15:05 |
| smcginnis | fungi: I'd like to do the similar functionality as there where jobs are skipped based on which files are modified. | 15:06 |
| fungi | smcginnis: you can do that in the job definition now, or override a job variant with files lists. i'll get you the right subsection of the docs for that | 15:07 |
| AJaeger | smcginnis: most of those list were automigrated | 15:07 |
| AJaeger | smcginnis: check files and irrelevant-files in project-config/zuul.d/projects.yaml and openstack-zuul-jobs/zuul.d/*.yaml | 15:08 |
| fungi | smcginnis: https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html#attr-job.files and https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html#attr-job.irrelevant-files | 15:08 |
| dhellmann | we have a new error in the tag-releases job. it looks like zuul-cloner is now failing because the cache for the repo we're cloning isn't present? http://logs.openstack.org/2b/2bf39ddf034397018c262c07e9d845d0235e2ee6/release-post/tag-releases/c838018/job-output.txt.gz#_2017-10-17_15_06_54_574799 | 15:08 |
| smcginnis | fungi, AJaeger: Thanks | 15:08 |
| dhellmann | do we need the releases repo to depend on every repo it's going to tag? or is there some more dynamic way to ensure we can clone the repo? | 15:09 |
| fungi | dhellmann: for zuul to provide you with git repos, it needs to know in advance (via the required-projects list) which ones you'll need | 15:09 |
| *** ccamacho has left #openstack-infra | 15:10 | |
| mtreinish | clarkb: actually looking at it a bit more some jobs I think it is applied and working | 15:10 |
| dhellmann | fungi : this is our script running zuul-cloner, though | 15:10 |
| mtreinish | I think I was just looking at older results | 15:10 |
| dhellmann | I thought zuul-cloner would say "oh, there's no cache, go to the server they specified" | 15:10 |
| fungi | dhellmann: yes, in v3 zuul-cloner is basically a no-op wrapper to copy the locally-provided repository from the filesystem in the expected location | 15:10 |
| dhellmann | :-( | 15:10 |
| dhellmann | so the releases repo now depends on 100s of repos? | 15:11 |
| fungi | and is deprecated in favor of just using the repos on disk directly now that you can guarantee all repos you declare you need are present | 15:11 |
| dhellmann | how long is it going to take to create a local mirror of 200+ repos for each release job? | 15:11 |
| jeblair | we may just need to switch to plain old git for this | 15:11 |
| dhellmann | maybe we should just -- yeah | 15:11 |
| fungi | dhellmann: it _requires_ checkouts of hundreds of repos... which it did before anyway right? we just move (or more likely duplicate) that list | 15:12 |
| dhellmann | fungi : it only needs the repos being tagged | 15:12 |
| dhellmann | the total set of possible repos is > 200 but the ones it needs at any given time might be == 1 | 15:12 |
| clarkb | mtreinish: gotcha | 15:12 |
| fungi | ahh, so this is a case for one job dynamically using on different repos in different runs | 15:12 |
| dhellmann | yes | 15:12 |
| fungi | s/using on/using/ | 15:12 |
| dhellmann | "tagging oslo" vs "tagging puppet" vs "tagging nova" | 15:12 |
| mtreinish | clarkb: it's kinda hard to tell actually, the timestamps make it hard to tell where the cutover is | 15:12 |
| jeblair | the only way to know which is to inspect the contents of the commit in the releases repo | 15:13 |
| dhellmann | right | 15:13 |
| fungi | dhellmann: and this job won't ever run in dependent pipelines where it needs speculative state of not-yet-merged changes for those repos, correct? | 15:13 |
| smcginnis | Right | 15:13 |
| pabelanger | mpeterson: the infra team will handle the clean up on zuulv2 in project-config | 15:13 |
| dhellmann | fungi : I'm not sure what that means exactly | 15:13 |
| dhellmann | "speculative state" | 15:13 |
| mtreinish | clarkb: like there is: http://status.openstack.org/openstack-health/#/job/openstack-tox-py35-nova but at the same time I'm seeing a lot of legacy- on the homepage | 15:13 |
| dhellmann | we don't tag things that haven't merged | 15:13 |
| dhellmann | we do run the validation job that way, though | 15:14 |
| smcginnis | No in-flight patches, if that's what you mean. | 15:14 |
| dhellmann | which makes me wonder why we didn't see this error in that job in the check queue | 15:14 |
| *** egonzalez has quit IRC | 15:14 | |
| dhellmann | we use zuul-cloner there, too | 15:14 |
| mpeterson | pabelanger: great! | 15:14 |
| smcginnis | dhellmann: Oh, good point. | 15:14 |
| mtreinish | clarkb: I might be seeing caching boundaries though and getting confused, I should just check the db directly | 15:14 |
| mordred | dhellmann, smcginnis, jeblair, fungi: this seems like a good case for just cloning from https://git.openstack.org/ | 15:14 |
| fungi | dhellmann: smcginnis: yeah, just making sure that you don't need zuul to provide, say, a checkout of openstack/puppet-vitrage which includes changes ahead of it in the gate pipeline, for example | 15:14 |
| clarkb | mtreinish: let me know if I can help | 15:14 |
| dhellmann | fungi : no, we were just using zuul-cloner before to take advantage of the local cache | 15:15 |
| fungi | dhellmann: smcginnis: or to depends-on a not-yet-merged change in openstack/puppet-vitrage | 15:15 |
| mordred | ah - nod. the local cache | 15:15 |
| dhellmann | and because it had a nice interface for falling back to master if the branch didn't exist | 15:15 |
| fungi | and actually use it in the job rather than just blocking on it merging first | 15:15 |
| dhellmann | fungi : no, this job runs post-merge, so depends-on doesn't apply | 15:15 |
| dhellmann | now the tox-validate job may need that, in the check queue | 15:15 |
| dhellmann | and it also uses zuul-cloner but it has been working fine | 15:15 |
| clarkb | fungi: dhellmann I think this is a weird case where you want the actual zuul clpner even om v3 | 15:16 |
| dhellmann | so maybe the queues are set up differently? | 15:16 |
| clarkb | basically just to update and checkout a repo based on local cache and some remote bht not worry about zuul refs | 15:16 |
| dhellmann | the original version of the release scripts used git directly, so we can go back to that | 15:16 |
| fungi | clarkb: noting that we wanted to look at getting rid of the local cache if that didn't result in too much additional work for the executors pushing the whole repo into place | 15:17 |
| dhellmann | however, I'm still confused why the check and release-post queues are behaving differently so I want to understand that before changing things | 15:17 |
| mordred | dhellmann: agree | 15:17 |
| fungi | so writing jobs which depend on that cache being present will make it harder for us to consider dropping it | 15:17 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Add /etc/ssl/certs to trusted_ro_paths for zuul-executors https://review.openstack.org/512657 | 15:17 |
| mtreinish | clarkb: yeah I guess it's working and I was just seeing stale data: http://paste.openstack.org/show/623851/ | 15:17 |
| *** markvoelker has quit IRC | 15:17 | |
| dhellmann | well, the job doesn't rely on the cache per se. it just used zuul-cloner, which now has a hard requirement for the cache | 15:17 |
| clarkb | fungi: yes though old zuul vloner used cache if it was present and cloned oyherwise | 15:18 |
| dhellmann | right | 15:18 |
| dhellmann | like I said, I'm ready to rewrite all of this to just use git directly, that's not a problem for me. but before I do that, I want to understand how the check job for tox-validate is working, because it also uses zuul-cloner. As does the release-tox-list-changes job | 15:18 |
| *** markvoelker has joined #openstack-infra | 15:18 | |
| dhellmann | I think that's the new name | 15:18 |
| fungi | dhellmann: have an example of a check job which used content from one of those repos? | 15:18 |
| dhellmann | the jobs on https://review.openstack.org/512586 | 15:19 |
| fungi | thanks | 15:19 |
| *** shiyaz has joined #openstack-infra | 15:19 | |
| *** LindaWang has quit IRC | 15:19 | |
| dhellmann | here's the zuul-cloner output: http://logs.openstack.org/86/512586/1/check/releases-tox-list-changes/5889069/job-output.txt.gz#_2017-10-17_11_19_46_071322 | 15:19 |
| openstackgerrit | Sean McGinnis proposed openstack-infra/openstack-zuul-jobs master: Skip openstack-tox-py35 jobs on release deliverables https://review.openstack.org/512663 | 15:19 |
| clarkb | new zuul cloner is basically a shim that just copies the zuulv3 pushed repo content to where you want it | 15:19 |
| dhellmann | oh, I wonder if we're pip installing zuul into our tox env | 15:19 |
| dhellmann | and since zuul v3 isn't released yet, we're getting 2 | 15:19 |
| clarkb | zuulv3 only pushes those repos for the current project and an required/sibling projects | 15:20 |
| dhellmann | yeah, that's it | 15:20 |
| *** iyamahat has joined #openstack-infra | 15:20 | |
| *** gongysh has joined #openstack-infra | 15:20 | |
| dhellmann | we have zuul in our requirements.txt that tox uses for that job | 15:20 |
| *** gongysh has quit IRC | 15:20 | |
| fungi | yeah, that's v2 cloner: http://logs.openstack.org/86/512586/1/check/openstack-tox-validate/20620d2/job-output.txt.gz#_2017-10-17_11_09_50_837127 | 15:20 |
| mordred | WELL ... | 15:20 |
| fungi | confirmed | 15:20 |
| mordred | v2 cloner is on the images unless we replace it | 15:21 |
| mordred | and we only replace it with the shim for legacy jobs | 15:21 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config master: Remove all OSA functional tests https://review.openstack.org/512666 | 15:21 |
| pabelanger | yah | 15:22 |
| pabelanger | which reminds me, should start staging patchs to remove that from DIB elements and WIP them | 15:22 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/openstack-zuul-jobs master: Remove legacy OSA role tests https://review.openstack.org/512635 | 15:22 |
| mordred | so since this job doesn't need any ZUUL_CHANGES or anything to work properly, it's just going to use v2 cloner - which means we should certainly make sure any scripts in release-tools or jenkins/scripts aren'tusing zuul-cloner anymore before we remove it from the images | 15:22 |
| *** markvoelker has quit IRC | 15:22 | |
| dhellmann | fungi, mordred, clarkb, jeblair : if we won't have a cache any more, I can just script the git commands we need for these jobs | 15:23 |
| mordred | pabelanger: yah ^^ as soon as we get that taken care of - otherwise we gwanna break folks | 15:23 |
| mordred | dhellmann: we do still have the cache - but medium-term we'd like to be able to get rid of it | 15:23 |
| dhellmann | ok, if I need to assume it's not present, that's fine, too | 15:23 |
| openstackgerrit | Nguyen Van Trung proposed openstack-infra/infra-manual master: Change http to https link https://review.openstack.org/512640 | 15:24 |
| jeblair | yeah, i think that's the way to go | 15:25 |
| mordred | pabelanger: weren't you talking about doing some pruning on dib cache? | 15:26 |
| pabelanger | mordred: yes, I did that yesterday | 15:26 |
| pabelanger | let me check logs | 15:26 |
| dhellmann | I'd like to take advantage of the git cache if it's present; what variable is being set to detect that now? or where is it on the node? | 15:26 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Update jobs for features/zuulv3 branch https://review.openstack.org/512642 | 15:27 |
| dhellmann | mostly because I have a cache locally | 15:27 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/openstack-zuul-jobs master: Remove legacy OSA role tests https://review.openstack.org/512635 | 15:27 |
| jeblair | dhellmann: /opt/git i believe | 15:27 |
| dhellmann | maybe I should fork zuul-cloner and release it separately | 15:27 |
| dhellmann | from v2 that is | 15:27 |
| AJaeger | pabelanger, mordred, could you review https://review.openstack.org/#/c/512658/ , please? My previous patch forgot to add the file ;( | 15:29 |
| *** psachin has joined #openstack-infra | 15:29 | |
| pabelanger | mordred: yah, centos-7 dib should have fresh git cache now | 15:30 |
| jeblair | dhellmann: i think at the end of the day, you probably don't need much more than an 'if [ -d /opt/zuul/...] ' | 15:30 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/openstack-zuul-jobs master: Remove legacy OSA role tests https://review.openstack.org/512635 | 15:30 |
| dhellmann | jeblair : yeah, I'll probably define some other variable for local use and then check both | 15:31 |
| dhellmann | or set the default to /opt/zuul/... | 15:31 |
| SamYaple | so my loci-publish-job in cinder post pipeline worked! there was a variable mixup (worked when run under project loci, not under cinder) but it used the secret correctly and everything else | 15:32 |
| AJaeger | odyssey4me: let's first merge the project-config change - and the nrecheck 512635. Since project-config is trusted, it does not take into account the dependent one... | 15:32 |
| AJaeger | SamYaple: woot! | 15:32 |
| odyssey4me | AJaeger ok, hang on - I want to finish both to make sure neither need changes | 15:33 |
| AJaeger | odyssey4me: you cannot ;( A depends-on from openstack-zuul-jobs to project-config will not work. Let's merge proejct-config, recheck to see, fix project-config etc... | 15:34 |
| AJaeger | odyssey4me: zuul will tell you the first error, so fix that - and then let's merge... | 15:35 |
| odyssey4me | AJaeger oh - it seems to be working quite nicely so far - telling me all sort of missing things | 15:35 |
| odyssey4me | but ok - lemme remove -w | 15:35 |
| AJaeger | mordred: could you +2A https://review.openstack.org/512647 as well, please? | 15:36 |
| AJaeger | fungi, mordred, could you review https://review.openstack.org/512657 again (system-config change by pabelanger so that https works from playbooks), please? | 15:36 |
| *** sree has quit IRC | 15:37 | |
| * AJaeger tries to get everything together for translation jobs... | 15:37 | |
| *** sree has joined #openstack-infra | 15:37 | |
| *** sree has quit IRC | 15:38 | |
| *** sree has joined #openstack-infra | 15:38 | |
| SamYaple | if a job is declared in merged in untrusted repo A, can you parent that job in untrusted repo B? | 15:38 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Harden exception handling around instance deletes https://review.openstack.org/512413 | 15:38 |
| *** sree has quit IRC | 15:39 | |
| *** ihrachys has joined #openstack-infra | 15:39 | |
| pabelanger | SamYaple: yes | 15:39 |
| *** sree has joined #openstack-infra | 15:39 | |
| SamYaple | pabelanger: i thought so | 15:40 |
| pabelanger | SamYaple: unless a job is marked final: true, you should be able to parent to any job | 15:42 |
| SamYaple | thought so. | 15:43 |
| SamYaple | running into some trouble | 15:43 |
| *** sree has quit IRC | 15:43 | |
| *** sdague has joined #openstack-infra | 15:44 | |
| AJaeger | pabelanger, fungi, regarding 512657: We do download stuff from service-types.openstack.org, see http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/roles/set-service-type-data-fact/tasks/main.yaml#n3 | 15:44 |
| SamYaple | pabelanger: https://github.com/openstack/openstack-ansible-tests/blob/master/zuul.d/jobs.yaml#L16 failing zuul on this patchset trying to use that job in a different repo https://review.openstack.org/#/c/512618/ | 15:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Several executor threading fixes https://review.openstack.org/512673 | 15:45 |
| jeblair | pabelanger, clarkb: the new graphs have shown me that the governor thread has died on all the executors ^ | 15:46 |
| jeblair | (i suspect it did not survive the scheduler restart due to loss of connectivity to geard) | 15:46 |
| *** e0ne_ has quit IRC | 15:46 | |
| fungi | AJaeger: i suppose it's a download in one sense... though i'd term that querying instead since we're just temporarily copying the result long enough to expose it in a variable | 15:46 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/openstack-zuul-jobs master: Remove legacy OSA role tests https://review.openstack.org/512635 | 15:47 |
| AJaeger | fungi: ;) Just wanted to point out one use case... | 15:47 |
| AJaeger | fungi: other use case is communicatoin with translation server - pushing to it and back down. But thta happens on the node itself. pabelanger will that work with your change? | 15:48 |
| *** yamamoto has joined #openstack-infra | 15:48 | |
| *** sdague has quit IRC | 15:48 | |
| fungi | AJaeger: yeah, that 512657 change is specifically about enabling the executor to communicate with https-based services, nothing to do with job nodes | 15:49 |
| pabelanger | SamYaple: okay, so there is one catch. and I think jeblair can confirm, we also depend on how projects are ordered in main.yaml, so if openstack/ansible-hardening is before openstack-ansible-tests zuul won't be able to see those jobs on reconfiguration. jeblair can you confim? | 15:49 |
| fungi | job nodes should already be able to do that much just fine | 15:50 |
| SamYaple | pabelanger: ouch! | 15:50 |
| SamYaple | what would be the workaround to that | 15:50 |
| SamYaple | odyssey4me mhayden see pabelanger's comment above | 15:50 |
| jeblair | pabelanger, SamYaple: yes, if job B inhertits from A, project A must be listed first in main.yaml | 15:51 |
| pabelanger | SamYaple: this is how we do it today: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/main.yaml#n8 | 15:51 |
| jeblair | pabelanger, SamYaple: it's on my list to fix as soon as the dust settles | 15:51 |
| jeblair | pabelanger, SamYaple: the fix for now is to reorder main.yaml | 15:51 |
| SamYaple | pabelanger: so a patch to zuul/main.yaml | 15:51 |
| SamYaple | awesome | 15:51 |
| SamYaple | i will pass it alone | 15:51 |
| SamYaple | along* | 15:51 |
| pabelanger | SamYaple: so, workaround would be to move common jobs for ansible above other projects, like we did for opentack-zuul-jobs and zuul-jobs | 15:51 |
| *** andreas_s has quit IRC | 15:52 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Fix ZUUL_REFNAME for translation jobs https://review.openstack.org/512675 | 15:52 |
| SamYaple | got it. easy peasy | 15:52 |
| openstackgerrit | Sean McGinnis proposed openstack-infra/project-config master: Skip openstack-tox-py35 jobs on release deliverables https://review.openstack.org/512676 | 15:52 |
| *** jascott1 has joined #openstack-infra | 15:52 | |
| AJaeger | jlk, pabelanger, mordred : https://review.openstack.org/512675 another translation fix | 15:52 |
| *** jascott1 has quit IRC | 15:53 | |
| jlk | looks | 15:53 |
| *** gouthamr has quit IRC | 15:54 | |
| *** yamamoto has quit IRC | 15:55 | |
| *** gouthamr_ has joined #openstack-infra | 15:55 | |
| AJaeger | jlk: thanks - fixing | 15:55 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Fix ZUUL_REFNAME for translation jobs https://review.openstack.org/512675 | 15:57 |
| AJaeger | jlk: ^ | 15:57 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config master: Swich load ordering for openstack-ansible-tests repo https://review.openstack.org/512678 | 15:57 |
| clarkb | scanning inode usage on the executors disk usage exceeds inode usage (by percentage) so that is looking good | 15:58 |
| odyssey4me | jeblair pabelanger ^ thanks SamYaple | 15:58 |
| pabelanger | clarkb: great | 15:58 |
| *** bh526r has quit IRC | 15:58 | |
| AJaeger | propose_update uses ZUUL_REFNAME - is there a replacement environment variable in v3? | 15:59 |
| *** owalsh_ is now known as owalsh | 15:59 | |
| pabelanger | jeblair: are you okay with us reordering main.yaml? or should we hold off for now? https://review.openstack.org/512678 | 15:59 |
| *** slaweq has joined #openstack-infra | 15:59 | |
| fungi | AJaeger: https://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/ansible/filter/zuul_filters.py?h=feature/zuulv3#n52 | 16:00 |
| jeblair | pabelanger: yep, no reason to hold off | 16:00 |
| fungi | AJaeger: for non-legacy jobs, you can set whatever envvar you want from 'ref' | 16:00 |
| *** pcaruana has quit IRC | 16:01 | |
| pabelanger | jeblair: great, thanks | 16:01 |
| AJaeger | fungi http://logs.openstack.org/periodic/git.openstack.org/openstack/requirements/stable/newton/propose-updates/a515bba/job-output.txt.gz#_2017-10-17_06_04_02_516270 | 16:01 |
| AJaeger | fungi: so, it's not set for that job... | 16:01 |
| jeblair | i started looking at a POST_FAILURE errors in the etherpad (currently line 199). i have realized that it is due to a bug in the executor that will cause us to send post_failure when we stop the executor, if we're running a post playbook when the job is aborted. | 16:02 |
| fungi | AJaeger: right, that's not a legacy job | 16:02 |
| AJaeger | fungi: yep | 16:02 |
| jeblair | the fix is easy, i'm going to try writing a test | 16:02 |
| AJaeger | fungi, so I need to pass in zuul.branch, correct? | 16:03 |
| jeblair | but we should *definitely* plan on having this change i'm writing in place before the next executor restarts, so that we only do this one more time | 16:03 |
| fungi | AJaeger: you can see it's available as the zuul.ref ansible variable though: http://logs.openstack.org/periodic/git.openstack.org/openstack/requirements/stable/newton/propose-updates/a515bba/zuul-info/inventory.yaml | 16:03 |
| openstackgerrit | Michael Johnson proposed openstack-infra/project-config master: Fix octavia-dashboard py27/35 jobs https://review.openstack.org/512439 | 16:03 |
| fungi | AJaeger: in legacy jobs, zuul.ref is being provided in $ZUUL_REFNAME and includes the refs/heads prefix, while zuul.branch does not | 16:03 |
| fungi | er, refs/heads/ | 16:04 |
| AJaeger | fungi, do you have an idea how to fix this and can give it a try, please? I need to be offline for the next hour or two and don't see the obvious fix directly... | 16:05 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Fix build-deploy-guide https://review.openstack.org/512658 | 16:05 |
| *** sdague has joined #openstack-infra | 16:05 | |
| openstackgerrit | Michael Johnson proposed openstack-infra/project-config master: Fix Octavia jobs to be voting again https://review.openstack.org/512444 | 16:05 |
| *** trown is now known as trown|lunch | 16:05 | |
| fungi | AJaeger: one option would be to adjust the role calling propose_update.sh to export that envvar before invocation, which can be done with the environment parameter i think. checking now | 16:06 |
| *** kiennt26_ has quit IRC | 16:07 | |
| *** vhosakot has joined #openstack-infra | 16:07 | |
| fungi | AJaeger: looks like we're already trying to do that by including the zuul_legacy_vars filter? http://git.openstack.org/cgit/openstack-infra/project-config/tree/playbooks/proposal/propose-updates.yaml | 16:08 |
| fungi | this is odd | 16:08 |
| clarkb | jeblair: makes sense | 16:08 |
| *** lucasagomes is now known as lucas-afk | 16:10 | |
| smcginnis | AJaeger: Hmm, I don't see what I'm doing differently here than the examples you pointed me at: https://review.openstack.org/#/c/512676/ | 16:10 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Switch neutron-classifier to openstack-python-jobs-neutron https://review.openstack.org/512683 | 16:11 |
| *** Apoorva has joined #openstack-infra | 16:11 | |
| pabelanger | AJaeger: clarkb: fungi: easy fix for neutron-classifier jobs | 16:11 |
| pabelanger | ^ | 16:11 |
| openstackgerrit | Sean McGinnis proposed openstack-infra/project-config master: Skip openstack-tox-py35 jobs on release deliverables https://review.openstack.org/512676 | 16:13 |
| clarkb | jeblair: there is a comment on the thread handling fix change that will need to be addressed, but otherwise I think we can get that one in for the next restart too | 16:14 |
| fungi | AJaeger: oh, i think we're not setting ZUUL_REFNAME in the filter because there's no newrev or oldrev! | 16:14 |
| jeblair | smcginnis: left comment | 16:14 |
| fungi | AJaeger: note the conditional here https://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/ansible/filter/zuul_filters.py?h=feature/zuulv3#n51 | 16:14 |
| fungi | AJaeger: is not satisfied by the vars here http://logs.openstack.org/periodic/git.openstack.org/openstack/requirements/stable/newton/propose-updates/a515bba/zuul-info/inventory.yaml | 16:15 |
| jeblair | clarkb: thx, i'll refresh that when i push up my current change | 16:15 |
| smcginnis | jeblair: Thanks | 16:15 |
| fungi | jeblair: do you think that's ^ an oversight in the filter? | 16:15 |
| *** jascott1 has joined #openstack-infra | 16:16 | |
| fungi | end result seems to be we don't provide $ZUUL_REFNAME for periodic jobs due to lack of oldrev and newrev | 16:16 |
| openstackgerrit | Sean McGinnis proposed openstack-infra/project-config master: Skip openstack-tox-py35 jobs on release deliverables https://review.openstack.org/512676 | 16:16 |
| *** dhinesh has joined #openstack-infra | 16:16 | |
| *** sileht has quit IRC | 16:17 | |
| clarkb | pabelanger jeblair: did the statsd stuff make it in yet? thinking it would be good to include that in the next restart if not already there | 16:18 |
| pabelanger | yah, I see some here: http://grafana.openstack.org/dashboard/db/zuul-status | 16:18 |
| jeblair | clarkb: yeah, it's all in. scheduler should be sending stats now. executors were until they hit the governor bug | 16:19 |
| jeblair | fungi: looking | 16:20 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid master: Fix Access Token reissue with Rotate Refresh Token policy active https://review.openstack.org/512685 | 16:20 |
| *** dhinesh has quit IRC | 16:21 | |
| pabelanger | clarkb: Shrews also said we might want to restart nodepool-launchers to pick up a fix | 16:21 |
| odyssey4me | we'd appreciate some reviews for https://review.openstack.org/#/c/512666/ and https://review.openstack.org/#/c/512678/ to help us get going to fix up our gates | 16:22 |
| pabelanger | odyssey4me: using topic:zuulv3-jobs and adding them to https://etherpad.openstack.org/p/zuulv3-issues also help, I've been trying to keep an eye on them | 16:23 |
| *** smarcet has quit IRC | 16:23 | |
| jeblair | fungi: i guess we could change line 51 to an "else" | 16:23 |
| odyssey4me | pabelanger noted - will use that topic for them | 16:24 |
| pabelanger | odyssey4me: great | 16:24 |
| odyssey4me | we'll be pushing through all our repositories and adding jobs over the next few hours | 16:24 |
| fungi | jeblair: i can do that... do we not want ZUUL_REFNAME set on change-based pipelines? | 16:24 |
| pabelanger | odyssey4me: exciting | 16:25 |
| odyssey4me | once we have things functional again we'll look at doing things better - right now our job is a simple like-for-like transfer, rather than using the shiny new facilities | 16:25 |
| jeblair | fungi: i don't think it was before, i think it was only set on non-change pipelines | 16:26 |
| *** jcoufal has quit IRC | 16:26 | |
| jeblair | fungi: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/launcher/gearman.py#n276 | 16:26 |
| *** sambetts is now known as sambetts|afk | 16:26 | |
| *** psachin has quit IRC | 16:27 | |
| clarkb | jeblair: that lines up with my memory | 16:27 |
| fungi | jeblair: and we want (or can assume will never happen) ZUUL_OLDREV and ZUUL_NEWREV set in all non-change-based builds even if both zuul['newrev'] and zuul['oldrev'] are missing? | 16:27 |
| *** salv-orl_ has joined #openstack-infra | 16:29 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove all OSA functional tests https://review.openstack.org/512666 | 16:29 |
| openstackgerrit | Merged openstack-infra/project-config master: Swich load ordering for openstack-ansible-tests repo https://review.openstack.org/512678 | 16:29 |
| *** martinkopec has quit IRC | 16:30 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul feature/zuulv3: Periodic refname, newrev and oldrev legacy vars https://review.openstack.org/512701 | 16:30 |
| odyssey4me | mhayden you around? the load ordering has been changed - try rechecking your patch now | 16:30 |
| fungi | jeblair: AJaeger: ^ | 16:30 |
| odyssey4me | oops - wrong window :p | 16:30 |
| fungi | jeblair: AJaeger: 512701 | 16:30 |
| jeblair | odyssey4me, mhayden, SamYaple: zuul needs a full reconfiguration for that change; it's driven by puppet, so could take a while | 16:31 |
| *** bhavik1 has joined #openstack-infra | 16:31 | |
| odyssey4me | jeblair ah ok, thanks - 20 mins or so IIRC | 16:32 |
| *** salv-orlando has quit IRC | 16:32 | |
| *** ralonsoh has quit IRC | 16:33 | |
| SamYaple | got it | 16:34 |
| *** yamahata has quit IRC | 16:34 | |
| *** bhavik1 has quit IRC | 16:34 | |
| *** iyamahat has quit IRC | 16:34 | |
| *** tosky has quit IRC | 16:34 | |
| *** gfidente is now known as gfidente|afk | 16:35 | |
| *** sileht has joined #openstack-infra | 16:36 | |
| gfidente|afk | fungi because we're basically tracking that plus more things in another pad, which is https://etherpad.openstack.org/p/tripleo-integration-squad-status | 16:37 |
| *** ykarel|away has quit IRC | 16:38 | |
| fungi | gfidente|afk: why not just clear out the contents of the pad and replace them with a link to the other in case people go looking at the defunct one? we tend to reserve pad deletions for emergencies like abuse or accidentally-disclosed sensitive date | 16:39 |
| clarkb | fungi: is params['ZUUL_REFNAME'] = zuul['ref'] valid in the periodic case? | 16:41 |
| clarkb | I worry that will throw and exception for not having the key in the dict | 16:41 |
| fungi | clarkb: see the example build log i pasted to the issues pad | 16:42 |
| fungi | it's there | 16:42 |
| fungi | though that does remind me of something else i should check... whether the _format_ matches | 16:43 |
| clarkb | fungi: I think ref being there may be a v3 behavior (so this is fine as change is on v3 branch) but ya checking the format probably a good idea in that case | 16:43 |
| *** jchhatbar has joined #openstack-infra | 16:44 | |
| fungi | clarkb: huh, now that i look, it wasn't set in v2 periodic jobs http://logs.openstack.org/periodic/barbican-propose-translation-update/be862fb/_zuul_ansible/vars.yaml | 16:44 |
| fungi | AJaeger: ^ are we sure it's an issue? | 16:44 |
| *** jchhatbar has quit IRC | 16:45 | |
| *** jchhatbar has joined #openstack-infra | 16:45 | |
| clarkb | fungi: ya we definitely didn't set it in v2 that is why we had to explicitly set the branch in periodic jobs | 16:45 |
| fungi | clarkb: i've marked that "fix" wip until i get a better idea of why that job is failing | 16:46 |
| *** janki has quit IRC | 16:47 | |
| clarkb | fungi: is it possible that the job migration assumed that refname would be set in v3 because periodic jobs have an explicit branch ref? | 16:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Several executor threading fixes https://review.openstack.org/512673 | 16:49 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Don't return POST_FAILURE when aborting post playbooks https://review.openstack.org/512712 | 16:49 |
| fungi | clarkb: the job is simply running the proposal script, which is unchanged from v2, so this is still likely some subtle change in envvar behaviors from v2 to v3-legacy | 16:51 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: WIP: shared ansible-linters jobs https://review.openstack.org/512715 | 16:51 |
| jeblair | fungi: i agree that the fix discussed earlier is looking doubtful | 16:52 |
| bnemec | So the coverage job output is kind of useless since we aren't actually capturing the output HTML: http://logs.openstack.org/38/511238/3/check/openstack-tox-cover/14b44bd/ | 16:52 |
| bnemec | Is this a known issue? | 16:52 |
| fungi | jeblair: i'm going to cross-check a v2 periodic and v3 legacy periodic build and see what's changed | 16:52 |
| jeblair | bnemec: doesn't look like it. want to add to 'issues with jobs' section of https://etherpad.openstack.org/p/zuulv3-issues ? | 16:52 |
| bnemec | jeblair: Yep, will do. | 16:53 |
| *** iyamahat has joined #openstack-infra | 16:53 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: WIP: shared ansible-linters jobs https://review.openstack.org/512715 | 16:55 |
| *** jchhatbar has quit IRC | 16:56 | |
| *** ssancheztrujillo has joined #openstack-infra | 16:56 | |
| clarkb | jeblair: I don't see where the test is creating the test_wait file in taht change | 16:57 |
| jeblair | clarkb: it never does | 16:58 |
| jeblair | clarkb: (since we're going to abort the playbook) | 16:58 |
| clarkb | oh right its an assert false on post end existing | 16:58 |
| jeblair | you could argue the pathname creation for such a file doesn't need to be so baroque, but i copied it from another test | 16:59 |
| jeblair | clarkb: ya. i did want to make sure that we never finished the playbook. | 16:59 |
| clarkb | jeblair: just one other comment then | 17:00 |
| clarkb | (left it on the change) | 17:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Don't return POST_FAILURE when aborting post playbooks https://review.openstack.org/512712 | 17:01 |
| *** tesseract has quit IRC | 17:02 | |
| *** slaweq has quit IRC | 17:04 | |
| *** trown|lunch is now known as trown | 17:08 | |
| *** derekh has quit IRC | 17:09 | |
| jeblair | i'm in and out from now until the infra meeting | 17:11 |
| *** yamahata has joined #openstack-infra | 17:11 | |
| *** ociuhandu has quit IRC | 17:11 | |
| *** weshay|ruck is now known as weshay|afk_see_r | 17:14 | |
| dhellmann | fungi, pabelanger, jeblair : I could use your eyes on this zuul-cloner replacement script when you have a few minutes: https://review.openstack.org/#/c/512751/ | 17:15 |
| fungi | okay, here's a bit of a confusion... http://logs.openstack.org/periodic/git.openstack.org/openstack/requirements/stable/newton/propose-updates/a515bba/ seems to be attempting to propose a constraints update against a stable branch | 17:16 |
| *** leyal has quit IRC | 17:17 | |
| odyssey4me | ok, a little advise from a pipeline standpoint - if I have https://git.openstack.org/cgit/openstack/openstack-ansible-tests/tree/zuul.d/project.yaml, I'm deliberately using a specific queue in gate... does it make any sense to do it for check too? | 17:18 |
| *** dtantsur is now known as dtantsur|afk | 17:19 | |
| *** jpena is now known as jpena|off | 17:19 | |
| *** leyal has joined #openstack-infra | 17:19 | |
| *** smarcet has joined #openstack-infra | 17:20 | |
| clarkb | fungi: I think its trying to propose a requirements update not a translation update? | 17:20 |
| fungi | clarkb: yeah, it's a requirements update according to the log | 17:21 |
| fungi | translation updates don't seem to use that script, based on tracing a v2 run | 17:21 |
| clarkb | is the v3 job 'propose-updates' supposed to propose translation updates or requirements updates? I guess that is the confusion? | 17:21 |
| pabelanger | odyssey4me: no, check is an idependent pipeline manager, so we don't need to form shared queues, unless using depends-On: https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html?highlight=pipeline#value-pipeline.manager.independent | 17:22 |
| fungi | looks like it should propose requirements updates, based on what i can see: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/propose_update.sh | 17:23 |
| *** rbrndt has quit IRC | 17:23 | |
| clarkb | fungi: yes the body of the job is definitely attempting to propose requirements updates based on the use of that propose_update.sh script | 17:23 |
| fungi | clarkb: translation updates seem to want http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/propose_translation_update.sh instead | 17:23 |
| clarkb | fungi: also that job ran against the requirements repo whichseems correct for requirements updates | 17:24 |
| odyssey4me | pabelanger thanks - I appreciate the quick response, and the detail :) | 17:24 |
| fungi | clarkb: yes, i don't think translation updates come into play here | 17:24 |
| fungi | this is a failing periodic job to update the constraints list in the requirements repo | 17:25 |
| *** ldnunes has quit IRC | 17:27 | |
| *** salv-orlando has joined #openstack-infra | 17:28 | |
| fungi | clarkb: mystery solved!!! http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/requirements.yaml#n288 | 17:28 |
| clarkb | fungi: aha so ya we explicitly set it there | 17:29 |
| clarkb | it is now available in v3 via your change (though the format will be slightly different I think) | 17:29 |
| clarkb | (or would be available if that change merges) | 17:29 |
| fungi | clarkb: well, i'm still uncertain that's the way to go | 17:29 |
| clarkb | I think it uses that var because post triggers the jobs too | 17:30 |
| fungi | basically that script was written to run in post and then we added that override in the periodic version of the job | 17:30 |
| clarkb | so your method might be a good approach to flatten behavior of post and periodic | 17:30 |
| fungi | to "emulate" the post pipeline | 17:30 |
| clarkb | ya | 17:31 |
| fungi | but i think that's a terrible downstream choice | 17:31 |
| *** ldnunes has joined #openstack-infra | 17:31 | |
| fungi | i think we should attempt to make zuul_legacy_vars as closely reproduce the variables present in v2 as possible | 17:31 |
| *** salv-orl_ has quit IRC | 17:32 | |
| *** jcoufal has joined #openstack-infra | 17:32 | |
| *** salv-orlando has quit IRC | 17:32 | |
| SamYaple | fungi: ++ | 17:32 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid master: Fix Access Token reissue with Rotate Refresh Token policy active https://review.openstack.org/512685 | 17:32 |
| SamYaple | would have been super helpful before haha | 17:32 |
| odyssey4me | I'm a little confused - according to the results in https://review.openstack.org/512635 there's a job definition missing, but it looks there to me. Maybe I'm just tired. Can another pair of eyes help out please? | 17:33 |
| fungi | SamYaple: well, they're already pretty close. the only differences i was able to find for a post pipeline run were the removal of 6 variables (which seem to be omitted in general, though we only list three of them as intentionally omitted in the comments), and the addition of two (ZUUL_BRANCH which used to exist for other pipelines but now we're adding it for periodic, and ZUUL_SHORT_PROJECT_NAME | 17:35 |
| fungi | which is entirely new) | 17:35 |
| AJaeger | fungi, thanks for digging further into REFNAME. I did already push https://review.openstack.org/512675 for translation jobs, but that might not be needed. | 17:35 |
| SamYaple | fungi: sure, but there were other bits i have to reconstruct. like docker registry mirror and package mirrors | 17:35 |
| dmsimard | clarkb: do we have a dev elk I could throw things at somehow ? Trying to wrap up the zuul v3 support for elastic-recheck and struggling to reproduce things reliably locally | 17:35 |
| fungi | SamYaple: those were not variables provided by zuul | 17:36 |
| clarkb | dmsimard: no, I've always just run the daemons locally | 17:36 |
| AJaeger | fungi: we could do something similar for propose_updates | 17:36 |
| fungi | SamYaple: they were (and still are) in a shell snippet in /etc, put there by nodepool | 17:36 |
| SamYaple | ah i understand your point | 17:36 |
| raissa | there's some centos jobs in manila failing intermittently because of epel it seems: http://logs.openstack.org/59/512559/2/check/manila-tempest-minimal-dsvm-lvm-centos-7/b8784b8/logs/devstack-early.txt.gz | 17:36 |
| SamYaple | yea thats fair | 17:36 |
| clarkb | dmsimard: you typically only need to run the chunk that you are workin on locally so for e-r just run that against production elasticsearch | 17:36 |
| dmsimard | clarkb: do you know what version of elk we're running ? Maybe I'm trying with stuff that is too recent ? | 17:36 |
| raissa | has anyone faced that? | 17:37 |
| clarkb | dmsimard: 1.7.6 iirc, should be available via the api too | 17:37 |
| armax | hi folks, I am trying to understand the current status of periodic job runs | 17:37 |
| pabelanger | raissa: a) what are you installing from epel? b) we do have afs mirrors for epel, you likely want to be using that | 17:37 |
| armax | can anyone point me in the right direction, if possible? Thanks! | 17:37 |
| dmsimard | clarkb: I basically want to test what we discussed the other day, either specify a 'type' jenkins or zuul when submitting the gearman job or default it to jenkins and then only set zuul for v3 | 17:37 |
| pabelanger | raissa: c) i think epel repo is installed, but disabled (properly configured) so you should be able to just yum install --enable-repo=epel foo | 17:38 |
| dmsimard | clarkb: so I'm basically running a elk instance locally with https://github.com/openstack-infra/system-config/blob/master/modules/openstack_project/templates/logstash/indexer.conf.erb (had to install a couple plugins like input-tcp, multiline, etc.) | 17:38 |
| clarkb | dmsimard: so for that you should only need the log submission worker and logstash | 17:39 |
| *** ociuhandu has joined #openstack-infra | 17:39 | |
| dmsimard | and then I found this convenient piece just to send raw data for test purposes https://gist.githubusercontent.com/jgoodall/6323951/raw/2f033e16bcee80830f398b1bf7752895a4139fe9/sendMessageToLogstash.py | 17:39 |
| clarkb | dmsimard: execute the log submission worker with some dummy data and have it talk to local logstash with stdout output | 17:39 |
| armax | when looking at http://logs.openstack.org/periodic everything seem to have stopped as of Oct 15 | 17:40 |
| fungi | AJaeger: i suppose we could alter 512675 to rely on $ZUUL_BRANCH when $ZUUL_REFNAME is missing, but i'm sort of conflicted on whether we should be providing $ZUUL_BRANCH in the legacy_zuul_vars filter for periodic builds, since we _didn't_ set it in v2 periodic environments | 17:40 |
| clarkb | dmsimard: but I've basically done subsets of services like ^ to iterate on this stuff, its pretty easy to run just logstash locally or just $otherthing for dev | 17:40 |
| clarkb | hardest part is editing the config probably | 17:40 |
| *** ijw has joined #openstack-infra | 17:41 | |
| *** rwsu has quit IRC | 17:41 | |
| pabelanger | armax: look to http://logs.openstack.org/periodic/git.openstack.org/openstack/ for new log path | 17:41 |
| dmsimard | clarkb: bah I'm just running this in a docker thing, it's fairly easy to edit and stuff. | 17:41 |
| clarkb | dmsimard: you needing to install plugins for logstash makes me think your logstash may be much newwer too fwiw | 17:42 |
| dmsimard | clarkb: yeah it's probably whatever is latest :/ | 17:43 |
| clarkb | (I gave up following upstream closely because every release was brokem and backward incompatible so now we tend to do big jumps when software goes eol) | 17:43 |
| dmsimard | 5.6.2 | 17:43 |
| raissa | pabelanger: it's during _install_epel_and_rdo at the beginning of running stack.sh "Could not retrieve mirrorlist http://mirrors.fedoraproject.org/mirrorlist?repo=epel-7&arch=x86_64" | 17:43 |
| dmsimard | with new kibana and stuff | 17:43 |
| fungi | armax: periodic jobs are now in fully-qualified repository paths, since zuul v3 knows them by sitename/repository/branch/job for disambiguation: | 17:43 |
| fungi | armax: for example, http://logs.openstack.org/periodic/git.openstack.org/openstack/requirements/master/legacy-propose-requirements-constraints-master/ | 17:43 |
| dmsimard | https://hub.docker.com/r/sebp/elk/ goes as low as 2.2.0 | 17:44 |
| clarkb | dmsimard: ya I think thats much newer. My suggestion would be isolate the services you actually need to test and only use those. In this case no kibana, no elasticsearch | 17:44 |
| pabelanger | raissa: right, we sholdn't be using mirrorlist from fedoraproject.org, because it isn't realiable | 17:44 |
| tbarron | pabelanger: raissa w.r.t. "c) " (installed but disabled) I don't think we are installing it at all and devstack build fails altogether | 17:44 |
| clarkb | dmsimard: you can check the puppet manifests to see what we are running. I dont recall | 17:44 |
| tbarron | pabelanger: raissa "Cannot find a valid baseurl for repo: epel-bootstrap/x86_64 | 17:44 |
| tbarron | Could not retrieve mirrorlist http://mirrors.fedoraproject.org/mirrorlist?repo=epel" ... | 17:44 |
| pabelanger | tbarron: raissa: see http://logs.openstack.org/59/512559/2/check/manila-tempest-minimal-dsvm-lvm-centos-7/b8784b8/job-output.txt.gz#_2017-10-17_12_28_58_351162 | 17:45 |
| pabelanger | that is where we setup epel.repo | 17:45 |
| pabelanger | but it is disabled by default | 17:45 |
| pabelanger | so, if your job needs it, you can enable it | 17:45 |
| dmsimard | clarkb: thanks, I'll figure something out. | 17:45 |
| *** ijw has quit IRC | 17:45 | |
| pabelanger | tbarron: raissa: however, I'm curious what you need from epel | 17:45 |
| clarkb | dmsimard: but also I think just having a config option in e-r would likely be simplest | 17:46 |
| dmsimard | clarkb: a toggle between v2 and v3 ? | 17:46 |
| clarkb | yes | 17:46 |
| dmsimard | clarkb: fwiw having a log of 'type: jenkins' sort of bothers me but it's not really important | 17:46 |
| dmsimard | let me see if I can put a toggle somewhere. | 17:47 |
| clarkb | dmsimard: we can just delete that if anyone cares | 17:47 |
| tbarron | pabelanger: raissa it isn't that we think we need it, we just see this error in devstack-early.txt.gz: https://paste.fedoraproject.org/paste/VMd6jddktTymNfVRpddWRQ | 17:47 |
| clarkb | change it to "CI" or something | 17:47 |
| *** dhinesh has joined #openstack-infra | 17:47 | |
| tbarron | pabelanger: raissa when we hit the intermittent devtack didn't set up go look at the relevant logs failures | 17:47 |
| *** ijw has joined #openstack-infra | 17:48 | |
| openstackgerrit | Shachar Snapiri proposed openstack-infra/project-config master: Remove dragonflow jobs that were moved to the project itself https://review.openstack.org/512761 | 17:48 |
| tbarron | pabelanger: raissa weire dead:beef:cafe:fed3 ipv6 address | 17:48 |
| dmsimard | clarkb: there's probably no value in setting a field in the first place if it's always going to be the same thing | 17:48 |
| tbarron | *wierd | 17:48 |
| pabelanger | tbarron: raissa: okay, if you don't need it, we should be setting SKIP_EPEL_INSTALL=true for those jobs | 17:48 |
| clarkb | dmsimard: its there so that we could index other data too | 17:48 |
| clarkb | dmsimard: without needing to drastically change or migrate existing data | 17:49 |
| pabelanger | tbarron: yes, epel repos are unreliable at best, this is my each regional mirror has http://mirror.dfw.rax.openstack.org/epel/ | 17:49 |
| tbarron | pabelanger: k, I guess we never set that for those jobs before | 17:49 |
| *** ijw has quit IRC | 17:50 | |
| raissa | pabelanger: I'll try that | 17:50 |
| *** ijw has joined #openstack-infra | 17:50 | |
| *** dfflanders has quit IRC | 17:50 | |
| pabelanger | tbarron: raissa; so, if epel is not needed, set SKIP_EPL_INSTAL=true, if you do need epel, still set SKIP_EPEL_INSTALL=true, and run yum --enablerepo epel, which should be setup to use our AFS mirrors | 17:50 |
| jeblair | dhellmann, fungi: commens on 512751 | 17:50 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Fix deploy-guide publish https://review.openstack.org/512765 | 17:51 |
| clarkb | dmsimard: the issue is you can't update data in ES you can only delete and add. So was possibly a little overoptimization to avoid needing to do anything drastic. But then we had to trim the retention period to so short it likely doesn't matter much anyways | 17:51 |
| AJaeger | mordred: could you review this, please? ^ I'm sure it's wrong... (5127659 | 17:51 |
| AJaeger | I mean: 512765 | 17:51 |
| AJaeger | fungi: or pass in ZUUL_REFNAME for this single job | 17:52 |
| *** gouthamr_ is now known as gouthamr | 17:52 | |
| raissa | pabelanger: ok, thanks | 17:52 |
| dmsimard | clarkb: would you be open to the idea of adding a 'zuul_version' key in data_source ? https://github.com/openstack-infra/puppet-elastic_recheck/blob/master/templates/elastic-recheck.conf.erb#L16 We can run with that in elastic-recheck after. | 17:53 |
| dmsimard | I suppose e-r could be rewritten to be far more generic and be used outside the context of zuul and openstack but here we are | 17:53 |
| clarkb | dmsimard: it can be run that way if you ignore the irc bot bits iirc. But yes it has long been a hard coded mess :) | 17:54 |
| clarkb | dmsimard: ya I would make it an optional value with a default to v2 so that anyone else running it continues to work | 17:54 |
| armax | pabelanger, fungi: excellent, thank you! I figured the path might have changed, thanks for the lead | 17:54 |
| clarkb | then we can set it to v3 | 17:54 |
| jeblair | dmsimard: what's the switch for? | 17:55 |
| jeblair | (sorry, i can't find that answer in scrollback) | 17:56 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: WIP: shared ansible-linters jobs https://review.openstack.org/512715 | 17:56 |
| clarkb | jeblair: we renamed the console log file and changed the end of file marker | 17:56 |
| fungi | armax: though the example one i gave you also stops at the 15th... because that particular job got replaced. http://logs.openstack.org/periodic/git.openstack.org/openstack/neutron/stable/pike/legacy-periodic-neutron-dsvm-functional/ is perhaps a better example | 17:56 |
| clarkb | jeblair: so console.html is now job-output.txt and we don't end job-output.txt with data bout completion | 17:56 |
| dmsimard | jeblair: elastic-recheck has several hardcoded bits such as the files that are expected/supposed to be there when a job has finished logging, see https://review.openstack.org/#/c/509313/ | 17:56 |
| jeblair | clarkb, dmsimard: should we add a special tag to the console log? | 17:56 |
| armax | fungi: I think I found this one logs.openstack.org/periodic/git.openstack.org/openstack/networking-midonet/master/legacy-periodic-py35-with-neutron-lib-master/e9caff9/job-output.txt.gz | 17:56 |
| jeblair | then change the queries to look for the tag instead of the fn? | 17:57 |
| armax | that is more relevant to what I am looking for | 17:57 |
| clarkb | jeblair: that wouldn't be backward compatible either though because v2 didn't set a special tag | 17:57 |
| armax | the job itself fails right now, but I think I know why and I’ll have to go back and fix it in zuul.d | 17:57 |
| dmsimard | we set 'console' as a tag regardless of v2 and v3 | 17:57 |
| clarkb | jeblair: and we don't end the files with the same string. I think it is easiest if we just switch off of it | 17:57 |
| clarkb | er switch off of a config flag | 17:57 |
| dmsimard | but the thing is, as far as I can understand, e-r checks for some 'required' files | 17:57 |
| *** slaweq has joined #openstack-infra | 17:57 | |
| armax | at least the error message is adamant, I can | 17:57 |
| armax | :) | 17:57 |
| armax | http://logs.openstack.org/periodic/git.openstack.org/openstack/networking-midonet/master/legacy-periodic-py35-with-neutron-lib-master/e9caff9/job-output.txt.gz#_2017-10-17_06_08_26_859792 | 17:58 |
| clarkb | dmsimard: yes e-r looks for the "console.html" beacuse every job gets one. And it looks for the Zuul finished or whatever that string is to know elasticsearch has the complete console log indexed | 17:58 |
| jeblair | dmsimard: oh, so it's not about the queries | 17:58 |
| clarkb | jeblair: they are special administrative queries to keep track of when things have been indexed | 17:58 |
| clarkb | I think we'll also have to assume the file is indexed if any of it is present in elasticsearch with v3 rather than looking for the EOF string | 17:59 |
| clarkb | since there doesn't appear to be an EOF string in v3 console logs | 17:59 |
| dmsimard | jeblair: there's some changes involved in the queries, yes | 17:59 |
| jeblair | ok. that makes more sense. i was mostly thinking about the filesystem queries and how they should be version agnostic. | 17:59 |
| pabelanger | armax: look to the openstack-tox-*-neutron project-templates, you likey can start using them | 17:59 |
| clarkb | jeblair: ya those should be ok | 17:59 |
| jeblair | i think a flag for internal server stuff makes sense. | 17:59 |
| dmsimard | clarkb: there isn't because logs end up truncated anyway, but that's part of why I sent https://review.openstack.org/#/c/509254/ | 17:59 |
| armax | pabelanger: will do | 18:00 |
| clarkb | dmsimard: ah ok so we can at least look for a post playbook completion | 18:00 |
| dmsimard | clarkb: so we could more or less rely on something like "POST-RUN <last role that is logged>" which is still meh | 18:00 |
| *** baoli has quit IRC | 18:00 | |
| dmsimard | POST-RUN START: [trusted : git.openstack.org/openstack-infra/project-config/playbooks/base/post-logs@master] | 18:01 |
| dmsimard | Is probably the last thing that ends up being reliably logged | 18:01 |
| *** baoli has joined #openstack-infra | 18:01 | |
| jeblair | (and of course will be different for a few jobs) | 18:01 |
| jeblair | but not ones we care much about in e-r | 18:02 |
| *** rwsu has joined #openstack-infra | 18:02 | |
| clarkb | ya could also just key off of POST-RUN START and call that good enough | 18:02 |
| clarkb | its mostly a sanity check in e-r that it can go look for logs at this point | 18:02 |
| *** slaweq has quit IRC | 18:02 | |
| jeblair | clarkb: i think some devstack post-runs may take a while | 18:02 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul feature/zuulv3: Be explicit about legacy var additions/omissions https://review.openstack.org/512769 | 18:02 |
| *** Apoorva_ has joined #openstack-infra | 18:02 | |
| clarkb | jeblair: but they will be complete by the time es indexes them right? since we submit the logstash jobs after logs are copied | 18:03 |
| clarkb | so getting within a couple hundred lines of file EOF is probably good enough (TM) | 18:03 |
| jeblair | clarkb: oh yeah. i guess it's volume of log lines after that line | 18:03 |
| jeblair | "post-run start [trusted" might be a good choice too | 18:03 |
| clarkb | ya | 18:03 |
| jeblair | sort of splits the difference :) | 18:04 |
| dmsimard | sure | 18:04 |
| openstackgerrit | Shachar Snapiri proposed openstack-infra/project-config master: Remove dragonflow jobs that were moved to the project itself https://review.openstack.org/512761 | 18:04 |
| dmsimard | there's bound to be a trusted post job somewhere in there | 18:04 |
| dmsimard | post run* | 18:04 |
| * jeblair afk again until meeting | 18:04 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Fix REFNAME in propose_update.sh https://review.openstack.org/512770 | 18:04 |
| AJaeger | fungi: what to you think ^ | 18:04 |
| *** panda|rover is now known as panda|rover|off | 18:05 | |
| *** Apoorva has quit IRC | 18:05 | |
| *** jamesmcarthur has joined #openstack-infra | 18:06 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Fix ZUUL_REFNAME for translation jobs https://review.openstack.org/512675 | 18:08 |
| AJaeger | and I don't think we need this, ^, will WIP for now | 18:08 |
| dhellmann | jeblair, fungi : comments addressed | 18:08 |
| *** jamesmcarthur has quit IRC | 18:09 | |
| *** jamesmcarthur has joined #openstack-infra | 18:09 | |
| * clarkb preps for meeting, if you have anything to add get it on the agenda (or ping me) | 18:09 | |
| fungi | clarkb: AJaeger: 512770 raises an interesting meeting topic: when do we make the call that it's okay to start altering scripts used by v2 jobs in ways which will break if we roll back to zuul v2? | 18:10 |
| *** jamesmcarthur has quit IRC | 18:10 | |
| fungi | i think that's probably just a subtopic item under the zuul v3 priority efforts discussion though, so probably not needed formally on the agenda | 18:11 |
| *** jamesmcarthur has joined #openstack-infra | 18:11 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Fix REFNAME in propose_update.sh https://review.openstack.org/512770 | 18:12 |
| AJaeger | fungi: I made it compatible ^ | 18:12 |
| fungi | AJaeger: cool, kicks that can down the road (but probably a good item for discussion nonetheless) | 18:13 |
| AJaeger | fungi: sure, we can discuss ;) | 18:13 |
| AJaeger | this was an easy change - if it's correct ;) | 18:13 |
| dmsimard | clarkb: meeting: new core reviewers ? | 18:14 |
| *** CrayZee has joined #openstack-infra | 18:18 | |
| AJaeger | argh, lint hates me ;( | 18:18 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Fix REFNAME in propose_update.sh https://review.openstack.org/512770 | 18:18 |
| clarkb | fungi: ya its a good question, but we are keeping copies of those aren't we? or are we just executing what is in the jenkins scripts bin dir? | 18:18 |
| fungi | clarkb: we copy the "v2" slave scripts into the node rather than using the ones baked onto the disk images | 18:19 |
| dmsimard | There's a lot of things that aren't forward/backward compat, some of which don't live in project-config and inside project repositories | 18:19 |
| fungi | dmsimard: yep, already came up last rollback where people had made non-backward-compatible changes to scripts in their projects which their v2 jobs had been calling | 18:20 |
| CrayZee | AJaeger: after removing my tasks from project-config, am I expected to remove their definitions and playbooks from openstack-zuul-jobs? | 18:20 |
| AJaeger | to fix translations, could I get reviews on https://review.openstack.org/#/c/512529/ and https://review.openstack.org/#/c/512657/ , please? | 18:20 |
| fungi | just trying to avoid inflictnig that same pain on ourselves. just because other projects don't make attempts at planning for a potential second rollback doesn't mean we should necessarily do the same | 18:20 |
| dmsimard | oh, sure. | 18:20 |
| pabelanger | clarkb: fungi: yah, once we know we are not rolling back, I think one of the first changes for elements we push up is removal of embedded jenkins scripts and user | 18:21 |
| AJaeger | CrayZee: Yes - as documented in https://docs.openstack.org/infra/manual/zuulv3.html#moving-legacy-jobs-to-projects | 18:21 |
| CrayZee | Thanks! | 18:21 |
| pabelanger | clarkb: fungi: then move jenkins/script contents into playbooks | 18:21 |
| fungi | pabelanger: yep, and removing a ton of v2 stuff in project-config | 18:21 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Do not rely on openstack-puppet-modules in Queens and beyond https://review.openstack.org/512783 | 18:22 |
| openstackgerrit | Shachar Snapiri proposed openstack-infra/openstack-zuul-jobs master: Remove legacy dragonflow jobs https://review.openstack.org/512785 | 18:23 |
| openstackgerrit | Shachar Snapiri proposed openstack-infra/project-config master: Remove dragonflow jobs that were moved to the project itself https://review.openstack.org/512761 | 18:24 |
| *** jcoufal_ has joined #openstack-infra | 18:24 | |
| pabelanger | up to 172 changes in check pipeline, things looking great | 18:26 |
| AJaeger | indeed! I'm happy how Zuulv3 is pushing through these! well done, jeblair and team! | 18:26 |
| *** jcoufal has quit IRC | 18:26 | |
| *** baoli has quit IRC | 18:27 | |
| *** dhinesh has quit IRC | 18:28 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul feature/zuulv3: Be explicit about legacy var additions/omissions https://review.openstack.org/512769 | 18:29 |
| *** rbrndt has joined #openstack-infra | 18:30 | |
| *** rwsu has quit IRC | 18:32 | |
| clarkb | woo having local network problems. I've switched off desktop to laptop so should be able to do meeting just fine | 18:33 |
| clarkb | I am getting packet loss between two hosts on the same switch :/ time to replace that hardware I guess | 18:33 |
| *** hemna_ has quit IRC | 18:34 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Move the common PTI jobs for releases back to project-config https://review.openstack.org/512788 | 18:35 |
| *** amoralej is now known as amoralej|off | 18:35 | |
| AJaeger | smcginnis: https://review.openstack.org/512787 and 512788 go together, that puts releases on current best practices again. | 18:35 |
| *** baoli has joined #openstack-infra | 18:36 | |
| *** smarcet has quit IRC | 18:36 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Several executor threading fixes https://review.openstack.org/512673 | 18:37 |
| *** electrofelix has quit IRC | 18:39 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Don't return POST_FAILURE when aborting post playbooks https://review.openstack.org/512712 | 18:39 |
| *** rloo has joined #openstack-infra | 18:40 | |
| fungi | clarkb: you sure it's not a reflection issue, e.g. arp/broadcast flood? | 18:42 |
| fungi | could be somethnig choking the switch | 18:42 |
| *** Apoorva_ has quit IRC | 18:43 | |
| clarkb | fungi: I don't think so, there are only two hosts on the switch and the rest of the network is happy | 18:43 |
| *** Apoorva has joined #openstack-infra | 18:43 | |
| clarkb | its also older consumer hardware | 18:43 |
| fungi | ahh | 18:44 |
| * fungi recommends connecting the remaining empty switchports to each other, for improved throughput ;) | 18:44 | |
| openstackgerrit | Merged openstack-infra/infra-specs master: The Gerrit 2.13 upgrade is implemented https://review.openstack.org/509257 | 18:44 |
| clarkb | heh | 18:44 |
| *** rwsu has joined #openstack-infra | 18:45 | |
| openstackgerrit | Merged openstack-infra/infra-manual master: Fix typo in zuulv3 https://review.openstack.org/512614 | 18:45 |
| fungi | was helping my brother troubleshoot a flood in his office a few years ago. they had voip phones with a built-in two-port switch (well, technically three-port with one of them internally connected to the phone) so that you could plug your computer into the phone as a passthrough... this is pretty common... well, some person in a random office which had two network jacks plumbed back to the central | 18:46 |
| fungi | switching backplane had plugged his phone into both of them | 18:46 |
| fungi | and of course, no spanning tree or portguard or useful mitigations in the cheap switching they'd invested in | 18:48 |
| smcginnis | fungi: I've seen the same. | 18:48 |
| fungi | to make matters worse, they had a layer-2 wan link to servers in a datacenter, so basically that whole line ended up choked | 18:48 |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: Publish keystone loci images to DockerHub https://review.openstack.org/512793 | 18:49 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/elastic-recheck master: Add support for Zuulv3-specific parameters in elastic-recheck https://review.openstack.org/509313 | 18:49 |
| dmsimard | clarkb, jeblair: ^ took a slightly different approach which should probably work https://review.openstack.org/#/c/509313/3/elastic_recheck/elasticRecheck.py | 18:50 |
| AJaeger | fungi, could you review https://review.openstack.org/#/c/512529/ and https://review.openstack.org/#/c/512647/ to fix translation scripts, please? | 18:50 |
| AJaeger | clarkb: could you review https://review.openstack.org/#/c/512657/ so that https access work for our zuul executors, please? | 18:51 |
| clarkb | AJaeger: done | 18:53 |
| AJaeger | thanks clarkb and fungi ! | 18:54 |
| SamYaple | does the post pipeline have a lower priority than other pipelines? is that why it gets backed up so far | 18:55 |
| AJaeger | SamYaple: yes, exactly | 18:56 |
| SamYaple | got it | 18:56 |
| *** gfidente|afk is now known as gfidente | 18:56 | |
| SamYaple | ive never paid attention before.... now i am haha | 18:56 |
| AJaeger | SamYaple: check project-config/zuul.d/pipelines.yaml for configuration | 18:56 |
| SamYaple | thanks AJaeger | 18:57 |
| *** rbrndt has quit IRC | 18:57 | |
| fungi | AJaeger: want to un-wip your 512675 fix? | 18:57 |
| dmsimard | SamYaple: fwiw this is available in the zuul config: https://github.com/openstack-infra/project-config/blob/master/zuul.d/pipelines.yaml see: 'precedence' | 18:57 |
| AJaeger | pabelanger, mordred - will https://review.openstack.org/#/c/512765/1/playbooks/publish/deploy-guide.yaml work ? | 18:57 |
| AJaeger | fungi: I'm not sure ^ | 18:57 |
| dmsimard | SamYaple: there's 3 levels of priority, "high", "normal" and "low" -- the highest of which will get nodes first, the second will not get a node until all high precedence jobs have been attributed a node and so on | 18:58 |
| AJaeger | fungi, mordred, pabelanger - http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/roles/prepare-docs-for-afs/tasks/main.yaml#n3 sets the variable as fact, so I doubt I can override it - can I? | 18:58 |
| clarkb | meeting time | 18:59 |
| dmsimard | AJaeger: facts can be overriden by a set_fact that happens later, or through an extra-var | 18:59 |
| SamYaple | dmsimard: those precedences look reasonable. release has a high precendence, so thats all that really matters | 18:59 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: tripleo/paunch: run legacy-tripleo-ci-centos-7-nonha-multinode-oooq https://review.openstack.org/512795 | 18:59 |
| AJaeger | dmsimard: so, is https://review.openstack.org/#/c/512765 correct? | 18:59 |
| dmsimard | AJaeger: a simple var (say, in inventory) will get overridden by a set_fact | 18:59 |
| dmsimard | looking | 18:59 |
| *** sileht has quit IRC | 19:00 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: WIP: shared ansible-linters jobs https://review.openstack.org/512715 | 19:00 |
| dmsimard | AJaeger: hm, missing context about what the other roles do, I'll need to dig more into it. Meeting now. | 19:00 |
| fungi | AJaeger: i was asking about un-wiping 512675 not 512765 (though the numbers are strikingly similar) | 19:00 |
| *** sileht has joined #openstack-infra | 19:00 | |
| AJaeger | fungi: Ah! I can... | 19:01 |
| openstackgerrit | Merged openstack-infra/project-config master: Use proper script for translations https://review.openstack.org/512529 | 19:01 |
| openstackgerrit | Merged openstack-infra/project-config master: Fix type with java_cert in prep-zanata https://review.openstack.org/512647 | 19:01 |
| *** slaweq has joined #openstack-infra | 19:02 | |
| otherwiseguy | Anyone available for second +2s on https://review.openstack.org/#/c/512565/2 and I'm guessing https://review.openstack.org/#/c/512565/2 will too. networking-ovn gate-breaking fixes. | 19:03 |
| *** camunoz has quit IRC | 19:04 | |
| AJaeger | dmsimard: I want to set that variable and override thus the value in main.yaml. | 19:04 |
| *** sileht has quit IRC | 19:04 | |
| otherwiseguy | (didn't mean to hit enter yet, getting liason +1s) | 19:04 |
| *** sileht has joined #openstack-infra | 19:04 | |
| *** dave-mccowan has quit IRC | 19:04 | |
| pabelanger | +3 | 19:04 |
| otherwiseguy | thanks! | 19:05 |
| dmsimard | AJaeger: in terms of variable precedence, you setting the variable in the playbook would take precedence if it would be set in the role's defaults/main.yml, however, it would be overwritten first thing by the set_fact that occurs in https://github.com/openstack-infra/openstack-zuul-jobs/blob/master/roles/prepare-docs-for-afs/tasks/main.yaml | 19:05 |
| Shrews | jeblair: Is the zuul comment on https://review.openstack.org/512642 its way of saying "your referenced jobs don't exist in my world yet" because the changes on the master branch (referenced in depends-on) have not yet merged? | 19:06 |
| dmsimard | AJaeger: maybe what you want to do is to set the var in defaults/main.yaml of the role instead and remove the set_fact altogether ? Then, someone can override it at a job level. | 19:07 |
| dmsimard | AJaeger: but again, I'm missing a bit of context since I'm not familiar with this workflow. | 19:07 |
| AJaeger | dmsimard: thanks, I'll look further into it... | 19:08 |
| dmsimard | AJaeger: left a comment about what I mentioned for posterity | 19:09 |
| *** hemna_ has joined #openstack-infra | 19:10 | |
| openstackgerrit | Merged openstack-infra/project-config master: Networking-ovn: Use publish-openstack-sphinx-docs-neutron https://review.openstack.org/512565 | 19:10 |
| openstackgerrit | Merged openstack-infra/project-config master: Fix networking-ovn openstack-tox-cover job https://review.openstack.org/512347 | 19:10 |
| *** jamesdenton has quit IRC | 19:13 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add /etc/ssl/certs to trusted_ro_paths for zuul-executors https://review.openstack.org/512657 | 19:13 |
| AJaeger | dmsimard: thanks | 19:15 |
| *** vhosakot has quit IRC | 19:15 | |
| AJaeger | pabelanger: 512657 is merged - will you restart executors later, please? | 19:15 |
| *** camunoz has joined #openstack-infra | 19:16 | |
| pabelanger | AJaeger: i think we want to confirm with jeblair and clarkb first what else we want to land in our restart | 19:16 |
| AJaeger | pabelanger: sure... | 19:18 |
| *** sileht has quit IRC | 19:18 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/openstack-zuul-jobs master: Add missing required-project for tripleo-buildimage-overcloud-full https://review.openstack.org/512800 | 19:21 |
| openstackgerrit | gordon chung proposed openstack-infra/project-config master: remove legacy-grenade-dsvm-ceilometer https://review.openstack.org/512801 | 19:21 |
| EmilienM | dmsimard, matbu ^ I think it will do the job, (thanks david) | 19:21 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: prepare-docs-for-afs: Allow reuse https://review.openstack.org/512802 | 19:21 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Fix deploy-guide publish https://review.openstack.org/512765 | 19:21 |
| pabelanger | EmilienM: comment | 19:23 |
| AJaeger | dmsimard: updated, thanks for help ^ | 19:23 |
| *** slaweq has quit IRC | 19:24 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/openstack-zuul-jobs master: Add missing required-project for buildimage jobs https://review.openstack.org/512800 | 19:25 |
| EmilienM | pabelanger: nice catch. Done ^ | 19:25 |
| armax | fungi, pabelanger: one other related question I have is: where would the new graphite stats_counts would be? | 19:26 |
| *** markvoelker has joined #openstack-infra | 19:26 | |
| armax | now that the job is called legacy-periodic-py35-with-neutron-lib-master consistently for the projects involved I wonder where I should look | 19:28 |
| pabelanger | armax: see https://review.openstack.org/512451/ for example on new counters | 19:29 |
| *** sileht has joined #openstack-infra | 19:29 | |
| armax | excellent! | 19:29 |
| *** dave-mccowan has joined #openstack-infra | 19:29 | |
| armax | thanks | 19:30 |
| dmsimard | AJaeger: +1 with comment | 19:33 |
| *** slaweq has joined #openstack-infra | 19:38 | |
| AJaeger | dmsimard: thanks! Looking at other files in that directory, I would leave as is - commented. | 19:39 |
| *** slaweq has quit IRC | 19:40 | |
| *** slaweq has joined #openstack-infra | 19:41 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: WIP: shared ansible-linters jobs https://review.openstack.org/512715 | 19:42 |
| *** weshay|afk_see_r is now known as weshay|ruck | 19:42 | |
| *** jamesmcarthur has quit IRC | 19:44 | |
| AJaeger | mordred: could you review https://review.openstack.org/512802 and https://review.openstack.org/512765 , please? | 19:45 |
| Shrews | pabelanger: any idea what this error is? http://logs.openstack.org/37/512637/5/check/tox-py35/0324599/job-output.txt.gz#_2017-10-17_19_19_47_203407 | 19:47 |
| *** MasterOfBugs has joined #openstack-infra | 19:47 | |
| *** pramodrj07 has joined #openstack-infra | 19:47 | |
| pabelanger | Shrews: I've see that a few times from tox, but seems to be more a warning then error | 19:48 |
| pabelanger | since tox passes | 19:48 |
| pabelanger | but http://logs.openstack.org/37/512637/5/check/tox-py35/0324599/job-output.txt.gz#_2017-10-17_19_19_57_472443 is an error | 19:49 |
| Shrews | pabelanger: oh, hrm | 19:49 |
| Shrews | pabelanger: that one seems just as confusing | 19:49 |
| pabelanger | looks like syntax error | 19:49 |
| pabelanger | SyntaxError: Missing parentheses in call to 'print' | 19:50 |
| Shrews | pabelanger: hrm. how on earth was that ever passing py35 test before then? | 19:50 |
| pabelanger | Oh, so the code is merged | 19:51 |
| pabelanger | how odd | 19:51 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 19:51 |
| Shrews | pabelanger: yeah | 19:51 |
| Shrews | but ^^ should fix the print call | 19:51 |
| pabelanger | going to see how we merged that | 19:52 |
| *** e0ne has joined #openstack-infra | 19:52 | |
| Shrews | pabelanger: oh! i see | 19:52 |
| *** kjackal_ has quit IRC | 19:53 | |
| openstackgerrit | Tom Barron proposed openstack-infra/openstack-zuul-jobs master: Collect output from coverage job https://review.openstack.org/512808 | 19:53 |
| Shrews | pabelanger: we only run the dsvm tests on master with py35, but the unit tests only used py27 | 19:53 |
| Shrews | pabelanger: which i think is probably an oversight? | 19:53 |
| * Shrews used https://review.openstack.org/501793 as a reference to verify that | 19:54 | |
| pabelanger | Ah, yes | 19:54 |
| pabelanger | so, we could keep that and add branch: feature/zuulv3 for now | 19:54 |
| pabelanger | then once we merge master, remove it | 19:54 |
| pabelanger | your clal | 19:54 |
| pabelanger | call* | 19:54 |
| *** jamesmcarthur has joined #openstack-infra | 19:55 | |
| Shrews | i'll see if that's the only py3 error. if there are many, i'll do that | 19:57 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 19:59 |
| dmsimard | frickler: what do you think of https://review.openstack.org/#/c/509313/2..3/elastic_recheck/elasticRecheck.py ? | 19:59 |
| Shrews | yup, there's gonna be more than few :( | 19:59 |
| openstackgerrit | Thomas Morin proposed openstack-infra/project-config master: n8g-bgpvpn: make triple-o job check/nv https://review.openstack.org/512332 | 19:59 |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/project-config master: Zuul-v3: add required projects for neutron-lib periodic jobs https://review.openstack.org/512811 | 19:59 |
| *** jtomasek has quit IRC | 20:00 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 20:01 |
| *** armax has quit IRC | 20:01 | |
| pabelanger | Shrews: maybe keep it non-voting | 20:01 |
| Shrews | pabelanger: i just used the branch matcher, as you suggested. hopefully the merge back to master will have all the cleanups | 20:02 |
| openstackgerrit | Ken Giusti proposed openstack-infra/project-config master: Restore cover test to oslo.messaging https://review.openstack.org/512813 | 20:02 |
| pabelanger | Shrews: +1 | 20:02 |
| Shrews | ugh, would be nice if i named the branch correctly | 20:03 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 20:03 |
| *** slaweq has quit IRC | 20:04 | |
| AJaeger | Shrews: https://docs.openstack.org/infra/manual/zuulv3.html#what-not-to-convert - just the legacy stuff, please | 20:04 |
| Shrews | AJaeger: those jobs are not being defined in the project | 20:07 |
| Shrews | AJaeger: nor are the referenced templates | 20:07 |
| *** jtomasek has joined #openstack-infra | 20:07 | |
| AJaeger | Shrews: those jobs should stay in project-config | 20:08 |
| AJaeger | please read the link I gave | 20:08 |
| *** rbrndt has joined #openstack-infra | 20:08 | |
| AJaeger | we don't want everything moving over | 20:08 |
| Shrews | AJaeger: technically, nodepool isn't an "official" openstack project | 20:09 |
| pabelanger | Oh, i thought we did? | 20:10 |
| *** rbrndt has quit IRC | 20:10 | |
| Shrews | I thought the goal was to eventually remove the project definition out of project-config and have it in-repo? | 20:10 |
| clarkb | as a heads up I have a dentist appointment today so will be disappearing for that and unsure if I will return. Probably depends on how many drugs they give me | 20:10 |
| AJaeger | Shrews: it is - it's listed in governance repo | 20:10 |
| dmsimard | nodepool is in governance ? really ? | 20:11 |
| fungi | clarkb: you mean if the drugs are really good you might forget to avoid us? | 20:11 |
| AJaeger | Shrews: and system-requires by policy always stay in project-config | 20:11 |
| pabelanger | AJaeger: does that mean we'd need to change http://git.openstack.org/cgit/openstack-infra/zuul/tree/.zuul.yaml?h=feature/zuulv3#n44 ? | 20:11 |
| clarkb | fungi: something like that | 20:11 |
| dmsimard | AJaeger: nodepool is in governance but zuul isn't ? is that an anomaly ? I thought only projects shipped as part of openstack (i.e, nova, glance, keystone) were part of governance | 20:12 |
| pabelanger | I haven't been following, but how would we stop projects from adding what Shrews did in 512637? | 20:12 |
| AJaeger | pabelanger: that's special IMHO since it's on a feature branch. But once that's master, yes, I would move most back to us | 20:12 |
| *** vhosakot has joined #openstack-infra | 20:12 | |
| AJaeger | pabelanger: I review when jobs get removed in project-config ;) | 20:12 |
| AJaeger | dmsimard: zuul is in governance as well | 20:13 |
| *** smatzek has quit IRC | 20:13 | |
| Shrews | AJaeger: that's not how i read that document. i've been reading it as "the system-requires template definition must remain in project-config", not "references to system-requires must also remain in project-config" | 20:13 |
| *** e0ne has quit IRC | 20:13 | |
| dmsimard | AJaeger: oh damn you're right | 20:13 |
| *** rbrndt has joined #openstack-infra | 20:13 | |
| pabelanger | okay, I haven't been following much, but seems like if we are keeping things in project-config, it would still mean more things to review | 20:13 |
| AJaeger | dmsimard: https://governance.openstack.org/tc/reference/projects/infrastructure.html | 20:13 |
| *** e0ne has joined #openstack-infra | 20:13 | |
| Shrews | happy to do it the other way, but i'm not seeing the reason why i can't reference it from my in-repo project definition | 20:14 |
| *** camunoz has quit IRC | 20:14 | |
| dmsimard | AJaeger: yeah, I see it, I wasn't looking at the right place. I guess it makes sense, even though it is not "shipped" | 20:14 |
| AJaeger | Shrews: then let'S fix the wording. The discussion we had here a couple of days ago that we want those in project-config so that we can easy do OpenStack wide changes | 20:14 |
| *** smatzek has joined #openstack-infra | 20:14 | |
| pabelanger | Shrews: yah, I am not to be honest | 20:15 |
| pabelanger | not sure* | 20:15 |
| *** ldnunes has quit IRC | 20:15 | |
| AJaeger | Shrews: technically you can do whatever you want - this is policy | 20:15 |
| dmsimard | AJaeger: openstack wide changes would imply changing the template | 20:15 |
| *** markvoelker_ has joined #openstack-infra | 20:15 | |
| dmsimard | AJaeger: which is still achieved by keeping the project layout in their own projects | 20:15 |
| pabelanger | AJaeger: when you have a moment, can you share link to discussion? | 20:15 |
| AJaeger | dmsimard: they would, yes.... | 20:15 |
| AJaeger | pabelanger: let me search eavesdrop... | 20:16 |
| Shrews | AJaeger: ah, ok. i guess that makes sense. and yes, we should fix that wording | 20:16 |
| dmsimard | AJaeger: however, the problem is that if the project layout is in their repository, we don't control whether or not system-required is part of their definition | 20:16 |
| *** chlong has quit IRC | 20:16 | |
| jeblair | system-required is going to be in project-config for every project | 20:16 |
| jeblair | so there is no point to additing it in-repo | 20:16 |
| dmsimard | jeblair: right, but what I mean is that there's nothing preventing me from not including that template in my project jobs | 20:17 |
| *** jcoufal_ has quit IRC | 20:17 | |
| jeblair | dmsimard: yep. but there's no point to doing so, so we should ask people not to. | 20:17 |
| dmsimard | jeblair: the only way to enforce it is to keep the project layouts inside project-config | 20:17 |
| dmsimard | or put it as part of the base job which can arguably be "avoided" as well | 20:17 |
| Shrews | jeblair: yep. my point of confusion was in thinking we should be striving to remove the project definition out of project-config. AJaeger has explained why we don't actually want that. | 20:17 |
| jeblair | dmsimard: yep. it's a brave new world of shared responsibility. :) | 20:18 |
| dmsimard | trusting people? this is madness | 20:18 |
| jeblair | Shrews: ++ | 20:18 |
| dmsimard | that was a joke/sarcasm by the way | 20:18 |
| dmsimard | (sorry) | 20:18 |
| *** markvoelker has quit IRC | 20:18 | |
| * jeblair laughs nervously | 20:19 | |
| *** smatzek has quit IRC | 20:19 | |
| SamYaple | haha | 20:19 |
| jeblair | i totally knew that was a joke | 20:19 |
| dmsimard | yeah, just figured it was non-obvious without voice intonation | 20:19 |
| *** e0ne has quit IRC | 20:19 | |
| SamYaple | ive never heard anyone use "this is madness" in a serious way | 20:19 |
| SamYaple | out side of a bond movie | 20:20 |
| AJaeger | previous discussion starts at http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2017-10-05.log.html#t2017-10-05T09:07:43 | 20:20 |
| jeblair | SamYaple: 'as a bond villian' is exactly how i read everyone in irc | 20:20 |
| dmsimard | hah | 20:20 |
| pabelanger | AJaeger: tyty | 20:20 |
| pabelanger | I'll re-read this evening | 20:20 |
| fungi | SamYaple: occasionally when a song by the band "madness" comes on the radio, i will blurt out "this is madness" | 20:20 |
| SamYaple | lol thats pretty good | 20:21 |
| pabelanger | is there a line between tox-py35 / openstack-tox-py35 in project-config? | 20:21 |
| pabelanger | since on is zuul-jobs and other openstack-zuul-jobs | 20:21 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 20:21 |
| jeblair | i'm going to tag zuul-sphinx so we can start to use project-template docs | 20:21 |
| pabelanger | jeblair: clarkb: AJaeger was asking before if we are ready to restart zuul executors? | 20:22 |
| pabelanger | I was unsure | 20:22 |
| clarkb | all my work there was done last night. I think jblairs changes merged? | 20:22 |
| AJaeger | pabelanger: what we discsused is "PTI stays in project-config". And both tox-py35 and openstack-tox-py35 implement the PTI | 20:22 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config master: Add linters job to all OSA role jobs https://review.openstack.org/512816 | 20:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: WIP: shared ansible-linters jobs https://review.openstack.org/512715 | 20:23 |
| jeblair | pabelanger, clarkb: yes, i think we're ready for executor restart -- after the zuul-sphinx jobs finish. | 20:23 |
| pabelanger | AJaeger: kk, that helps | 20:24 |
| jeblair | we're going to emit some stray post_failures, but hopefully this'll be the last time. | 20:24 |
| *** ijw has quit IRC | 20:24 | |
| *** camunoz has joined #openstack-infra | 20:25 | |
| jeblair | pabelanger: shall i perform the restarts, or were you planning on it? | 20:25 |
| * AJaeger left a few more changes to review in the etherpad, would be nice to get those in - as well as some of the other reviews if people have time. | 20:26 | |
| *** jtomasek has quit IRC | 20:28 | |
| jeblair | pabelanger, clarkb, fungi: http://logs.openstack.org/71/71220534fd0a76c1c26839c8a808de0d106bc490/release/release-openstack-python/17d65ef/job-output.txt.gz#_2017-10-17_20_30_03_841723 | 20:32 |
| jeblair | dhellmann: ^ | 20:32 |
| *** kgiusti has left #openstack-infra | 20:33 | |
| *** smarcet has joined #openstack-infra | 20:35 | |
| *** camunoz has quit IRC | 20:35 | |
| jeblair | after digging -- i *think* we should change our setup to omit the repository_url | 20:36 |
| jeblair | i'm looking to see if our role supports that | 20:36 |
| *** armax has joined #openstack-infra | 20:36 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 20:36 |
| jeblair | it does not | 20:37 |
| jeblair | okay, so i think we want 2 change: 1) remove the default pypi_repository_url from upload-pypi. then remove the repository_url from our production pypi secret. | 20:37 |
| jeblair | er, sorry -- that was both changes. i forgot to number the second | 20:38 |
| *** tosky has joined #openstack-infra | 20:38 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/project-config master: Do not run tox-py35 on nodepool master branch https://review.openstack.org/512851 | 20:40 |
| Shrews | AJaeger: is that what you're suggesting? ^^^^ | 20:40 |
| Shrews | That's something we'll have to remove after merging the feature branch back to master | 20:41 |
| pabelanger | jeblair: odd, they list repository:https://pypi.python.org/pypi being good in the docs | 20:41 |
| jeblair | Shrews, AJaeger: i'd personally prefer project-pipeline variants like that to stay in the project repos. | 20:42 |
| *** xarses has quit IRC | 20:42 | |
| *** smarcet has quit IRC | 20:43 | |
| Shrews | jeblair: as in here? https://review.openstack.org/#/c/512637/10/.zuul.yaml | 20:43 |
| jeblair | Shrews: yeah -- so just adding the 2 lines needed for tox-py35 to that change | 20:43 |
| Shrews | jeblair: k, that's what i had. i'll bring it back and abandon the project-config change. thx | 20:44 |
| jeblair | Shrews: not the templates though | 20:44 |
| ianw | jeblair / pabelanger / dmsimard: i couldn't find any examples of extracting src_dir from zuul.projects. is it a matter of {{ zuul.projects | select_attr('project_name') | map('src_dir') }} ? of course i can just guess it, but seemed better to take if from ansible if possible? | 20:45 |
| Shrews | jeblair: yeah, that PS did not have the templates | 20:45 |
| dmsimard | ianw: what do you want to do ? | 20:45 |
| jeblair | Shrews: looks like i disagree with AJaeger on that, based on his comments in ps10 | 20:45 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: WIP: shared ansible-linters jobs https://review.openstack.org/512715 | 20:46 |
| ianw | dmsimard: in line 26 https://review.openstack.org/#/c/512450/9/playbooks/dib-functests/run.yaml i want the path to the requirements.txt in the requirements project | 20:46 |
| Shrews | jeblair: yep | 20:46 |
| ianw | dmsimard: hard-coding works, but it just feels a bit ugly | 20:46 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 20:47 |
| dmsimard | ianw: sec | 20:47 |
| *** dave-mccowan has quit IRC | 20:48 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 20:49 |
| pabelanger | ianw: have you see the tox-siblings role? I would suggest looking into that, might be able to leverage some of that for requirements handling | 20:49 |
| dhellmann | jeblair : that sounds like the right changes to make | 20:50 |
| fungi | jeblair: oh, i thought we were already using pypi.org/legacy (lemme see where that change ended up) | 20:51 |
| pabelanger | fungi: just for testpypi | 20:51 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Update jobs for features/zuulv3 branch https://review.openstack.org/512642 | 20:51 |
| *** ijw has joined #openstack-infra | 20:52 | |
| ianw | pabelanger: yeah that was the closest i found, but it doesn't exactly extract one property of the list -- projects: "{{ zuul.projects | selectattr('required') | list }}" | 20:52 |
| fungi | pabelanger: we should have used something like dev.pypi.org/legacy for testpypi, i thought | 20:52 |
| pabelanger | fungi: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/secrets.yaml#n42 | 20:53 |
| pabelanger | ianw: tox-siblings does some of that magic now, mordred wrote that | 20:53 |
| fungi | yeah, http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/templates/pypirc.erb | 20:53 |
| *** xarses has joined #openstack-infra | 20:53 | |
| dmsimard | ianw, pabelanger: We could avoid this kind of hack altogether if we adjusted the 'projects' structure from the inventory just a bit | 20:53 |
| pabelanger | fungi: jeblair: so, if we use https://upload.pypi.org/legacy/ that is like zuulv2 | 20:54 |
| pabelanger | and less of a change | 20:54 |
| ianw | dmsimard: yeah, i was hoping it was a dict keyed, that would be easy | 20:54 |
| jeblair | pabelanger: if i'm fixing it, i'm *fixing* it. :) | 20:54 |
| dmsimard | ianw: yeah, exactly | 20:54 |
| pabelanger | jeblair: ack | 20:54 |
| dmsimard | ianw: however at this point, this is a breaking change | 20:54 |
| fungi | so i guess when the new pypi upload jobs were written, the existing pypi upload url was not actually used | 20:55 |
| pabelanger | no, I followed example at https://packaging.python.org/guides/migrating-to-pypi-org/#uploading | 20:55 |
| pabelanger | then, this is the first time we've run a job on pypi | 20:56 |
| fungi | pabelanger: first mistake: assuming the python packaging howto keeps up with changes to the pypa's infrastructure ;) | 20:56 |
| pabelanger | fungi: indeed | 20:56 |
| dmsimard | jeblair: should we consider keying the list of dicts in projects ? http://paste.openstack.org/raw/623890/ | 20:56 |
| *** david-lyle has quit IRC | 20:57 | |
| dmsimard | jeblair: use case being primarily (for the time being) to easily retrieve the src_dir | 20:57 |
| jeblair | i remember having a discussion about this | 20:57 |
| ianw | i think the key is hard if you consider different providers (github etc) and potential conflicts? | 20:58 |
| jeblair | i'm in the middle of the pypi thing and don't have time to dig it up | 20:58 |
| jeblair | if anyone has a link, that'd be great | 20:58 |
| dmsimard | sure, not an emergency, I wasn't part of that discussion I think. | 20:58 |
| jeblair | i believe it reached a conclusion, which is why i think it's important to find it. :) | 20:59 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: WIP: shared ansible-linters jobs https://review.openstack.org/512715 | 20:59 |
| *** bobh has quit IRC | 20:59 | |
| ianw | pabelanger: ahh, tox-siblings drops into python, which i want to avoid :) | 21:00 |
| openstackgerrit | Miguel Lavalle proposed openstack-infra/irc-meetings master: New time for Neutron drivers meeting, odd weeks https://review.openstack.org/512856 | 21:00 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: WIP: shared ansible-linters jobs https://review.openstack.org/512715 | 21:00 |
| *** ijw has quit IRC | 21:01 | |
| odyssey4me | anyone available to take a peek at another job change quickly for us? https://review.openstack.org/512816 | 21:01 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: New Location public endpoints https://review.openstack.org/512857 | 21:01 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: New Location public endpoints https://review.openstack.org/512857 | 21:02 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Marketplace API https://review.openstack.org/498102 | 21:04 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: import the clone_repo.sh script from the releases repo https://review.openstack.org/512859 | 21:05 |
| dhellmann | fungi, jeblair, pabelanger, mordred : ^^ should update the tag-releases job to not use zuul-cloner. There's some reasoning in the commit message about the temporary duplication of the script. | 21:06 |
| dirk | pabelanger: did you see the comment about the opensuse 422 mirroring data removal? | 21:06 |
| *** rhallisey has quit IRC | 21:06 | |
| *** trown is now known as trown|outtypewww | 21:07 | |
| *** mat128 has quit IRC | 21:07 | |
| *** sdague has quit IRC | 21:08 | |
| pabelanger | dirk: I did, I'll make a note to look into that shortly | 21:08 |
| *** gouthamr has quit IRC | 21:08 | |
| inc0 | hey guys, I think our zuulv3 initial patch is ready, can someone look at it to see if we did something stupid in it? https://review.openstack.org/#/c/508759/ | 21:11 |
| inc0 | thank you | 21:12 |
| *** david-lyle has joined #openstack-infra | 21:13 | |
| dmsimard | ianw: yeah super ugly, not sure I know of a good way -- this works but not ideal http://paste.openstack.org/raw/623891/ | 21:13 |
| *** xarses has quit IRC | 21:14 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Marketplace API https://review.openstack.org/498102 | 21:15 |
| *** edmondsw has quit IRC | 21:16 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: upload-pypi: make repository url optional https://review.openstack.org/512862 | 21:16 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Remove repository url from pypi secret https://review.openstack.org/512863 | 21:16 |
| jeblair | dhellmann, pabelanger, fungi: ^ | 21:16 |
| dmsimard | ianw: Anything we do will be a variation of that, either in jinja (using a filter that ultimately does that) or in python I think, there's no way to find the value you're interested in without iterating through the list | 21:18 |
| *** chlong has joined #openstack-infra | 21:18 | |
| *** edmondsw_ has joined #openstack-infra | 21:18 | |
| *** dhill_ has quit IRC | 21:18 | |
| *** dprince has quit IRC | 21:19 | |
| *** gfidente has quit IRC | 21:20 | |
| *** tosky has quit IRC | 21:20 | |
| openstackgerrit | Mike Perez proposed openstack-infra/project-config master: Set contributor-guide location to /contributors https://review.openstack.org/512865 | 21:21 |
| EmilienM | pabelanger, dmsimard : can you please get https://review.openstack.org/#/c/512800/ ? we're stuck now. Thanks | 21:21 |
| jeblair | dmsimard: http://eavesdrop.openstack.org/irclogs/%23zuul/%23zuul.2017-10-03.log.html#t2017-10-03T21:01:17 | 21:21 |
| EmilienM | pabelanger, dmsimard: also a really quick one https://review.openstack.org/#/c/512795/ | 21:22 |
| dmsimard | EmilienM: all I can do is +1 :/ | 21:22 |
| EmilienM | ah I thought you had +2 | 21:22 |
| jeblair | probably when clarkb has ingested sustenance | 21:22 |
| *** edmondsw_ has quit IRC | 21:22 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul feature/zuulv3: Add a projects_by_name view of zuul.projects https://review.openstack.org/512868 | 21:24 |
| dmsimard | jeblair: https://review.openstack.org/#/c/509294/ is reasonable | 21:24 |
| dmsimard | ianw: ^ see that review | 21:24 |
| *** gildub has joined #openstack-infra | 21:24 | |
| *** dhill_ has joined #openstack-infra | 21:24 | |
| ianw | dmsimard: huh, yeah and probably better than adding another dict, thanks jeblair :) | 21:25 |
| jeblair | yeah. i'd be okay with that, or flipping it around so that by default it's a dict and we iterate over values when needed. | 21:25 |
| *** thorst has quit IRC | 21:25 | |
| jeblair | ianw: yeah, we definitely shouldn't have both, but i'd be on board switching from list to dict if that's better | 21:25 |
| dmsimard | jeblair: I don't see a strong argument for keeping a list, it sounds like the filter is a workaround because we're not using keys | 21:25 |
| *** calbers has quit IRC | 21:25 | |
| *** thorst has joined #openstack-infra | 21:26 | |
| ianw | jeblair: it would be backwards incompat though? | 21:26 |
| jeblair | right now, i think there are very few places treating it as a list, so should be easy to fix | 21:26 |
| dmsimard | probably places like cached repos and such | 21:26 |
| jeblair | dmsimard: that's the only thing that comes to mind | 21:26 |
| ianw | jeblair: is the name safe for a key? | 21:26 |
| jeblair | ianw: canonical name (git.openstack.org/openstack/nova) | 21:26 |
| jeblair | i think if we switch, we'll have to find the uses and make them handle both forms temporarily | 21:27 |
| SamYaple | we now successfully push loci-cinder to dockerhub each time a commit lands in openstack/loci *or* openstack/cinder! | 21:28 |
| *** xarses has joined #openstack-infra | 21:28 | |
| dmsimard | SamYaple: how many container images does loci maintain ? | 21:28 |
| SamYaple | spent a few hours waiting for the post queue to empty, but success! | 21:28 |
| jeblair | mirror-workspace-git-repos, tox-siblings, use-cached-repos, and the playbooks/release/pre.yaml playbook | 21:28 |
| jeblair | ianw, dmsimard: ^ those are the current usages | 21:29 |
| SamYaple | dmsimard: right now, 8 or so projects | 21:29 |
| SamYaple | dmsimard: that number is about to explode since the structure makes it very easy to add new projects | 21:29 |
| jeblair | SamYaple: i have a plan to speed up the post pipeline soonish | 21:29 |
| *** jamesmcarthur has quit IRC | 21:29 | |
| SamYaple | dmsimard: we hope to have all the big-tent projects done by Queens, which should be easy | 21:29 |
| dmsimard | SamYaple: we'll want to be on the look out for rate limiting/blacklisting from dockerhub if every project (kolla, tripleo, loci, etc.) end up publishing to dockerhub | 21:29 |
| jeblair | SamYaple: (by not running intermediate post jobs, only the most recent, when it gets backed up) | 21:30 |
| SamYaple | jeblair: that would be good | 21:30 |
| *** thorst has quit IRC | 21:30 | |
| dmsimard | SamYaple: mordred inc0 and I discussed different opportunities to be efficient in our container things | 21:30 |
| SamYaple | dmsimard: im not sure kolla has plans to push to dockerhub, but the loci images are ~40MB per image and docker said they do not rate limit the public repos | 21:31 |
| SamYaple | worst case, we are fine switching to release pipelines for tags and such | 21:31 |
| *** tosky has joined #openstack-infra | 21:31 | |
| *** jamesmca_ has joined #openstack-infra | 21:32 | |
| jeblair | ianw, dmsimard: i assume there's a jinja filter to do dict.values() ? | 21:32 |
| SamYaple | jeblair: would it be possible to give quotas per pipeline? not quite sure how the scheduling works, but ti would be nice for no pipeline to completely stop running jobs | 21:32 |
| dmsimard | jeblair: no filter, just .items() iirc: for key, value in dict.items() | 21:33 |
| jeblair | dmsimard: you can do that in a jinja template? | 21:33 |
| ianw | jeblair / dmsimard : ok ... so are mordred's proposed filters a work-around, or a solution? so it would be {{ zuul.projects | zuul_projects_by_name | ???.src_dir }} (not sure get attribute in there? | 21:33 |
| jeblair | SamYaple: it's priority based, so gate starves check starves post. i think a lot of openstack devs are happy with that arrangement. | 21:34 |
| dmsimard | jeblair: https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/templates/osa_interfaces_multinode.cfg.j2#L2 | 21:34 |
| *** jamesmca_ has quit IRC | 21:34 | |
| jeblair | dmsimard: neat. | 21:34 |
| SamYaple | jeblair: you can do alot of python-y things in jinja2. including functions | 21:35 |
| *** gildub has quit IRC | 21:35 | |
| jeblair | ianw: i lean toward thinking dict is the best thing. slightly more annoying to implement, but i'm convinced it's the better interface. | 21:36 |
| ianw | maybe it's {{ zuul.projects | zuul_projects_by_name | extract('project_name') | extract('src_dir') }} ? | 21:36 |
| dhellmann | jeblair, pabelanger, mordred : if I can get https://review.openstack.org/#/c/512859/ in today I can have one more try at tag-releases before EOD. We'd like to get that working by q-1 on thursday. | 21:36 |
| jeblair | dhellmann: you'll need my twine fix too, right? | 21:36 |
| dhellmann | jeblair : probably, although I don't know if we'll even get that far. Seems like this game of whack-a-mole has them all lined up one after another... | 21:37 |
| dmsimard | jeblair: +1, handling forward/backward compat in current roles will be tricky (especially since it can't be reliably tested) | 21:37 |
| dmsimard | I have to step away for dinner, be back later | 21:37 |
| mgagne | just a reminder that we will perform an upgrade/maintenance on inap-mtl01 in ~20m, API will be unavailable | 21:38 |
| jeblair | dhellmann: hopefully i whacked a mole two steps ahead :) | 21:38 |
| dhellmann | jeblair : I would like to put my whacking day stick up soon ;-) | 21:38 |
| ianw | jeblair: ok, i'll see what i can come up with ... would you consider having this "projects_by_name" for an intermediate period, we change the jobs using "projects" one-by-one to that test they work with it, then switch it back? | 21:38 |
| jeblair | ianw: yeah, it's more changes but it's safe. another option is adding filters that take list or dict as input then produce dict output: add filter, use filter, switch format, stop using filter, drop filter. | 21:40 |
| jeblair | ianw: whichever of those you prefer | 21:40 |
| ianw | jeblair: ok, will think and play around with it, thanks | 21:41 |
| openstackgerrit | Mike Perez proposed openstack-infra/project-config master: Create a playbook for publishing contributor-guide https://review.openstack.org/512865 | 21:42 |
| openstackgerrit | Mike Perez proposed openstack-infra/project-config master: Set contributor-guide location to /contributors https://review.openstack.org/512871 | 21:42 |
| *** salv-orlando has joined #openstack-infra | 21:42 | |
| jeblair | dhellmann: one comment on 512859. would you prefer me to +2 or -1 because of that? | 21:43 |
| dhellmann | jeblair : I don't mind the redundancy for now. I'll clean that up in a second patch. | 21:44 |
| clarkb | jeblair: what will happen after I finish eating? | 21:44 |
| *** salv-orlando has quit IRC | 21:44 | |
| *** salv-orlando has joined #openstack-infra | 21:44 | |
| * clarkb goes to modify core groups | 21:44 | |
| jeblair | dhellmann: okay. +3. i do highly recommend it; it's surprisingly expensive, especially on certain repos. :) | 21:44 |
| jeblair | clarkb: that. | 21:45 |
| clarkb | ah ok | 21:45 |
| clarkb | I also had to do some thinking about home network design because bad switch | 21:45 |
| clarkb | frickler jlk mnaser dmsimard core access has been added you may need to refresh gerrit so that it draws the shiny +2's | 21:47 |
| jlk | woo, thanks! | 21:47 |
| jeblair | (or reload changes in gertty!) | 21:47 |
| jeblair | clarkb: 512862 and 863 could use a push | 21:48 |
| clarkb | jeblair: looking | 21:48 |
| openstackgerrit | Merged openstack-infra/project-config master: import the clone_repo.sh script from the releases repo https://review.openstack.org/512859 | 21:48 |
| jeblair | if no one objects, i'm going to restart the executors now | 21:48 |
| *** jklare has quit IRC | 21:48 | |
| jeblair | i'm actually tempted to just stop all of them briefly, then restart them all; that will reduce the amount of mis-scheduling. | 21:49 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: WIP: shared ansible-linters jobs https://review.openstack.org/512715 | 21:49 |
| *** erlon has quit IRC | 21:50 | |
| *** thorst has joined #openstack-infra | 21:50 | |
| SamYaple | jeblair: re: post pipeline optomization. my only worry would be if a project merges fast enough and the new jobs append to the post pipeline that none of those jobs would run. if the new job repleaced the old job in the queue, then it would be fine | 21:50 |
| *** edmondsw has joined #openstack-infra | 21:51 | |
| jeblair | SamYaple: yeah, i got that covered. the plan is to have no more than one item from the same project-branch in the pipeline. once the jobs for an item start, that one stays, another gets appended but doesn't start. if new ones come in, they replace it. it only starts when the item ahead finishes. | 21:52 |
| SamYaple | perfect | 21:52 |
| pabelanger | mgagne: ++ | 21:52 |
| *** jklare has joined #openstack-infra | 21:53 | |
| jeblair | in zuul speak, it's like a dependent pipeline with a window size of 1, but with replacement (that's the new part). | 21:53 |
| pabelanger | yah, that should help cut down on some resources too | 21:53 |
| jeblair | okay, i'm going to restart executors now | 21:53 |
| pabelanger | jeblair: ty | 21:54 |
| *** thorst has quit IRC | 21:54 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: remove redundant git operation https://review.openstack.org/512880 | 21:54 |
| dhellmann | jeblair : ^^ | 21:54 |
| EmilienM | pabelanger, jeblair, dmsimard : ty for your quick reviews | 21:55 |
| *** edmondsw has quit IRC | 21:55 | |
| *** esberglu has quit IRC | 21:55 | |
| pabelanger | EmilienM: be sure to also add them to https://etherpad.openstack.org/p/zuulv3-issues and topic:zuulv3-jobs | 21:56 |
| *** dhinesh has joined #openstack-infra | 21:56 | |
| *** esberglu has joined #openstack-infra | 21:56 | |
| inc0 | SamYaple: kolla plans to push to dockerhub, just not straight to dockerhub | 21:56 |
| inc0 | although it might not be too horrible if we do it nightly | 21:57 |
| SamYaple | inc0: would you be doing just tags or more often? | 21:57 |
| EmilienM | pabelanger: doing it now | 21:57 |
| inc0 | SamYaple: every night | 21:57 |
| SamYaple | irc lag says possible nightly, got it | 21:57 |
| *** ssancheztrujillo has quit IRC | 21:57 | |
| inc0 | or something frequent like that, no less than once a week for sure | 21:58 |
| *** jklare has quit IRC | 21:58 | |
| *** Apoorva_ has joined #openstack-infra | 21:59 | |
| SamYaple | we may switch to just release pipelines if the dockerhub traffic is to much (with the exception of requirements) | 22:00 |
| clarkb | I've approved the use unbound change | 22:00 |
| SamYaple | then do a periodic job of some kind | 22:00 |
| *** esberglu has quit IRC | 22:00 | |
| clarkb | ianw: ^ fyi | 22:01 |
| inc0 | part of TC resolution is that images has to be fresh, so we need to do it often. That being said, we can limit images to most important, do like once a week and in quietest time | 22:01 |
| SamYaple | to be hoenst, the post-pipeline optimization you were talking about jeblair will act as a sort of throttle to the dockerhub pushes anyway | 22:01 |
| jeblair | yeah | 22:02 |
| inc0 | no, no post pipeline...it was meant to be cron | 22:02 |
| SamYaple | inc0: i was refering to loci | 22:03 |
| *** Apoorva has quit IRC | 22:03 | |
| pabelanger | I think once we get our own infra docker registry, we can discuss out of band mirror to docker-hub too, incase jobs start getting rate limited some how | 22:03 |
| *** rbrndt has quit IRC | 22:03 | |
| SamYaple | that seems like the best option yea | 22:03 |
| SamYaple | when we get there | 22:03 |
| inc0 | we need images to be downloadable | 22:04 |
| CrayZee | Hi Infra, I have a patch https://review.openstack.org/#/c/508761/ which passed all the gate tests, and still gets -2 from zuul for some reason. Any idea why? | 22:04 |
| dirk | question - what do I need to do to have a new job show up https://review.openstack.org/#/c/512492/ ? | 22:05 |
| dirk | here I want to have a cross-nova-func check job, but it doesn't run | 22:05 |
| inc0 | but yeah I'd love to have infra registry, but my goal is dockerhub/quay.io | 22:05 |
| jeblair | executors started | 22:05 |
| clarkb | CrayZee: look at the gerrit comments from zuul (toggle ci if you have to) but jeblair just restarted executors ^ which marked your jobs aborted which should've rerun them | 22:05 |
| pabelanger | jeblair: great | 22:05 |
| clarkb | jeblair: see https://review.openstack.org/#/c/508761/ that may be a bug in the retry logic? | 22:06 |
| jeblair | clarkb: i'll look | 22:06 |
| *** rbrndt has joined #openstack-infra | 22:06 | |
| clarkb | dirk: does files: need to be a list value? | 22:07 |
| clarkb | (I don't actually know) | 22:08 |
| pabelanger | okay, EOD for me. camp fire on the beach tonight :D | 22:08 |
| *** gildub has joined #openstack-infra | 22:08 | |
| CrayZee | clarkb: I am not sure what you mean by "marked your jobs aborted which should've rerun them" | 22:09 |
| clarkb | dirk: https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html#attr-job.files says either singleton or list is acceptable so there goes that idea | 22:10 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Add missing required-project for buildimage jobs https://review.openstack.org/512800 | 22:10 |
| clarkb | CrayZee: in the comment from Zuul on the change two of the jobs have a status of ABORTED. This happened I think becuase jeblair restarted the executors which should automatically rerun those jobs rather than reporting them as aborted | 22:10 |
| *** baoli has quit IRC | 22:10 | |
| dirk | clarkb: hmm. tried that, doesn't change it | 22:11 |
| dirk | does the project dict need to be in the toplevel .zuul.yaml maybe in order to be picked up? | 22:11 |
| CrayZee | clarkb: sorry for the noob question, but what do I do in such a case? | 22:12 |
| clarkb | dirk: I don't think so I think .zuul.d is trated as equivalent, I'm not sure why that is happening | 22:12 |
| clarkb | CrayZee: you can recheck it to get the jobs to run again. Jeblair is looking into why it was reported that way in the first place | 22:13 |
| *** tpsilva has quit IRC | 22:13 | |
| clarkb | dirk: could also be a copy problem? | 22:14 |
| clarkb | though the nme for this job changed | 22:14 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Don't set the build result to ABORTED https://review.openstack.org/512883 | 22:14 |
| jeblair | clarkb: ^ | 22:14 |
| jeblair | that, at least, is only a scheduler restart to fix. so we should land and restart the scheduler before our next executor restart. | 22:15 |
| jeblair | CrayZee: thanks for the report, you found a bug! :) | 22:15 |
| *** ijw has joined #openstack-infra | 22:16 | |
| clarkb | I've got to head to the dentist now | 22:16 |
| clarkb | dirk: jeblair may be able to help you with that config thing | 22:16 |
| CrayZee | clarkb: thanks for your assistance | 22:17 |
| jeblair | dirk: looking | 22:17 |
| CrayZee | jeblair: Great :) | 22:17 |
| *** felipemonteiro__ has quit IRC | 22:17 | |
| thingee | hi infra, i think me initially building /contributor-guide for publish with the /contributor-guide 301 to /doc-contrib-guide is tripping some tests http://logs.openstack.org/59/512659/1/check/build-tox-manuals-checkbuild/dc92ba0/job-output.txt.gz#_2017-10-17_20_56_12_471615 | 22:18 |
| thingee | would these two commits alone help, or does the artifact have to be removed first? https://review.openstack.org/#/q/topic:contributor-guide+status:open | 22:19 |
| *** ijw has quit IRC | 22:20 | |
| dirk | hah! | 22:20 |
| dirk | deleting .zuul.yaml helped | 22:21 |
| jeblair | dirk: it looks like cross-nova-func is in the check pipeline... | 22:21 |
| dirk | so it is either .zuul.yaml, or if it does not exist, then .zuul.d/* ? | 22:21 |
| dirk | jeblair: yes, but only in revision ,9 the previous one didn't have it | 22:21 |
| jeblair | dirk: yes, .zuul.yaml takes precedence. once we have the ability to propogate warnings up, we should be able to have zuul report that. | 22:22 |
| dirk | ok, that wasn't obvious | 22:22 |
| jeblair | i agree | 22:22 |
| dirk | ok, next thing.. the current legacy jobs are running tox in openstack/requirements, not in the cross job | 22:22 |
| CrayZee | jeblair: any chance you could review https://review.openstack.org/#/c/512761/ to help move it forward? | 22:23 |
| openstackgerrit | Dean Troyer proposed openstack-infra/project-config master: Remove legacy jobs from python-openstackclient https://review.openstack.org/512888 | 22:23 |
| dirk | the idea is that they *also* git checkout the other repo, and then run tox in *there* with the local upper-constraints.txt file | 22:23 |
| *** jamesmcarthur has joined #openstack-infra | 22:23 | |
| *** boden has quit IRC | 22:23 | |
| jeblair | dirk: do you have a link to an example build? | 22:24 |
| dirk | you can see that e.g. here : http://logs.openstack.org/02/511202/8/check/legacy-cross-glance-python27/5da11d7/testr_results.html.gz | 22:24 |
| dirk | that one should have the glance unit tests, but it has the requirements unit tests | 22:24 |
| dirk | I wonder if we can handle that via required-projects somehow | 22:24 |
| *** jamesmcarthur has quit IRC | 22:25 | |
| *** Adri2000 has quit IRC | 22:25 | |
| *** jamesmcarthur has joined #openstack-infra | 22:26 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: upload-pypi: make repository url optional https://review.openstack.org/512862 | 22:27 |
| *** jamesmcarthur has quit IRC | 22:27 | |
| jeblair | dirk: the migration script made an assumption which is not true for the cross jobs. | 22:33 |
| jeblair | dirk: i'll fix them real quick | 22:33 |
| *** rlandy is now known as rlandy|brb | 22:34 | |
| *** jklare has joined #openstack-infra | 22:35 | |
| *** xarses has quit IRC | 22:36 | |
| *** dave-mccowan has joined #openstack-infra | 22:36 | |
| *** jascott1 has quit IRC | 22:36 | |
| dirk | jeblair: hmm, so I see in the logs the ansible step "setup legacy workspace" which copies from src/ the requirements repo and then runs tox inside that one. how can I tell it to copy the other one instead? | 22:36 |
| *** lbragstad has quit IRC | 22:36 | |
| openstackgerrit | James E. Blair proposed openstack-infra/openstack-zuul-jobs master: Fix requirements cross-test jobs https://review.openstack.org/512893 | 22:38 |
| jeblair | dirk: ^ that should fix those | 22:38 |
| *** jcoufal has joined #openstack-infra | 22:39 | |
| dirk | jeblair: aha! | 22:39 |
| openstackgerrit | Merged openstack-infra/project-config master: Revert "Revert "Add configure-unbound to base job"" https://review.openstack.org/512153 | 22:40 |
| jeblair | dirk: i would recommend that we merge that to fix things, then, rather than importing those into the requirements repo, build a new set of jobs from scratch. i think these will benefit from a redesign for the new features zuul v3 provides, and i think that will be easier on its own. | 22:40 |
| *** jklare has quit IRC | 22:40 | |
| jeblair | dirk: and yeah, for the new structure, i'd suggest adding the other project as a required project, and just having the job's playbook inspect the zuul.projects variable to see what the "other" project is. i can help with the syntax for that when we get there. | 22:42 |
| dirk | jeblair: yeah, avoiding the repetitiveness would be good here | 22:43 |
| dirk | jeblair: so I'm testing that in https://review.openstack.org/#/c/512492/11/playbooks/cross-nova-func/run.yaml | 22:45 |
| *** salv-orlando has quit IRC | 22:45 | |
| *** salv-orlando has joined #openstack-infra | 22:46 | |
| *** jklare has joined #openstack-infra | 22:46 | |
| *** rbrndt has quit IRC | 22:50 | |
| jeblair | dirk: i think we may have some things that may make this easier... let me see what i can put together real quick | 22:50 |
| *** oanson has quit IRC | 22:50 | |
| *** salv-orlando has quit IRC | 22:51 | |
| *** oanson has joined #openstack-infra | 22:52 | |
| dmsimard | jeblair: Looked into zuulv3-issues but didn't see anything. Is there a known issue for depends-on not merging after their dep merged ? https://review.openstack.org/#/c/512863/ was not enqueued after https://review.openstack.org/#/c/512862/ merged. | 22:53 |
| dirk | jeblair: I"m playing with this in https://review.openstack.org/#/c/512897/ | 22:53 |
| dmsimard | jeblair: I wanted to wait for your ack before issuing a recheck | 22:53 |
| dirk | jeblair: but its EOD for me today.. I'll read the backlog tomorrow | 22:53 |
| jeblair | dmsimard: they probably don't share a queue | 22:54 |
| dmsimard | jeblair: queue or pipeline ? | 22:54 |
| jeblair | dmsimard: i re-workflowed | 22:54 |
| jeblair | dmsimard: queue | 22:54 |
| mgagne | maintenance on inap-mtl01 is over, let me know if there is any problem/issue | 22:54 |
| dmsimard | hm, that is new vocabulary to me, what does it stand for ? | 22:54 |
| dmsimard | mgagne: thanks | 22:54 |
| jeblair | dmsimard: shared change queue. it's the actual "queue" part of the pipeline. items line up behind each other in queues. some projects share queues, others don't. | 22:55 |
| jeblair | dmsimard: some openstack projects share a queue; pupet projects share a different one. tripleo, etc. | 22:56 |
| dmsimard | oh, right, okay | 22:56 |
| jeblair | dmsimard: the status page shows them to you too ("Queue: integrated") | 22:56 |
| *** lin_yang has joined #openstack-infra | 22:56 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove dragonflow jobs that were moved to the project itself https://review.openstack.org/512761 | 22:56 |
| jeblair | dmsimard: in order for a change to be pulled in behind a dependent, they need to share a queue. otherwise the first has to land, then the second can enqueue. | 22:57 |
| dmsimard | the enqueue was automatically done before, though, did it not ? | 22:57 |
| jeblair | dmsimard: there *is* a zuul trigger that should let us pull in the second change after the first has landed, but i don't think that's turned on in openstack-infra, for some reason. | 22:57 |
| jeblair | dmsimard: this was the behavior before -- only shared queues was automatic | 22:58 |
| jeblair | again, i think only because no one got around to adding the zuul trigger | 22:58 |
| dmsimard | okay, I'll keep that in mind | 22:59 |
| SamYaple | what would be the minium amount of resources needed to have a new cloud region if someone where to want to donate some resources to infra? | 22:59 |
| jeblair | SamYaple: https://docs.openstack.org/infra/system-config/contribute-cloud.html | 23:00 |
| SamYaple | oh boy. docs for this and everything | 23:00 |
| SamYaple | you are always on the ball jeblair | 23:00 |
| SamYaple | im liking you more and more without little talks | 23:00 |
| SamYaple | me ask questions you provide immediate answer | 23:00 |
| SamYaple | its a great relationship | 23:00 |
| jeblair | i'm one of those automatic fortune tellers, but i spit out urls | 23:01 |
| * fungi often feels like meat-based search engine | 23:01 | |
| SamYaple | im going to make a jeblair bot for some of my channels | 23:01 |
| SamYaple | fungi: you are just a chemical based database | 23:01 |
| SamYaple | at least to me | 23:02 |
| jeblair | SamYaple: but let me know if you need more info. infra-root folks are always particularly happy to talk about that subject. :) | 23:02 |
| SamYaple | jeblair: im looking at the minimum specs now | 23:02 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove repository url from pypi secret https://review.openstack.org/512863 | 23:02 |
| SamYaple | theres a chance i may be getting a good bit of infra to throw around at my discretion, so im just building a solid case for htat | 23:02 |
| SamYaple | "good bit" is of course relative | 23:03 |
| *** calbers has joined #openstack-infra | 23:03 | |
| SamYaple | so it looks like 1TB ram and 1000 cores might the entry point | 23:04 |
| SamYaple | cores/threads | 23:04 |
| fungi | and call it 10tb spinning rust under there too | 23:05 |
| SamYaple | well thats easy. i have 100TB laying around at my house | 23:06 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Use sample environments for OVB configuration https://review.openstack.org/494643 | 23:06 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Support deploying extra nodes in te-broker https://review.openstack.org/512899 | 23:06 |
| SamYaple | well ive got 300GB ram too | 23:06 |
| *** rloo has quit IRC | 23:06 | |
| SamYaple | and symetric gigabit.... | 23:06 |
| * SamYaple ponders | 23:06 | |
| fungi | the internet connectivity/addressing is the harder part to come by (though if you have an isp who will route you at least a /64 of ipv6, that's mostly met too) | 23:07 |
| openstackgerrit | Merged openstack-infra/project-config master: Switch neutron-classifier to openstack-python-jobs-neutron https://review.openstack.org/512683 | 23:07 |
| *** tosky has quit IRC | 23:07 | |
| SamYaple | fungi: no ipv6 unfortunately | 23:08 |
| SamYaple | without a tunnel at least | 23:08 |
| fungi | otherwise, 100 public ipv4 addresses are getting tough to come by most places these days | 23:08 |
| openstackgerrit | Merged openstack-infra/project-config master: remove redundant git operation https://review.openstack.org/512880 | 23:09 |
| SamYaple | dont some of the clouds just assign private addresses? | 23:09 |
| SamYaple | or does those require floating ips | 23:09 |
| SamYaple | let me rephrase, i *know* some clouds only provide private, do those also need public floatings | 23:10 |
| fungi | we don't have a means of connecting to them without directly attached global addresses (one of v4 or v6) or 1:1 nat (floating-ip) | 23:10 |
| SamYaple | got it. no problem. just getting the requirements together | 23:10 |
| *** mlavalle has quit IRC | 23:10 | |
| fungi | we dealt with osic not providing global ipv4 addresses, because they had global routed ipv6 | 23:11 |
| *** Apoorva_ has quit IRC | 23:11 | |
| SamYaple | cool | 23:11 |
| *** Apoorva has joined #openstack-infra | 23:11 | |
| *** rlandy|brb is now known as rlandy | 23:11 | |
| jeblair | remote: https://review.openstack.org/512901 Add a v3-native cross-check job | 23:14 |
| jeblair | dirk: ^ | 23:14 |
| jeblair | mordred: ^ | 23:14 |
| *** calbers has quit IRC | 23:15 | |
| *** jklare has quit IRC | 23:15 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Update bindep role defaults to be more sane https://review.openstack.org/512639 | 23:15 |
| *** gildub has quit IRC | 23:16 | |
| *** ijw has joined #openstack-infra | 23:17 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Don't set the build result to ABORTED https://review.openstack.org/512883 | 23:19 |
| *** ijw has quit IRC | 23:22 | |
| *** jklare has joined #openstack-infra | 23:22 | |
| *** wolverineav has quit IRC | 23:23 | |
| *** gmann_afk is now known as gmann | 23:24 | |
| *** esberglu has joined #openstack-infra | 23:31 | |
| *** hongbin has quit IRC | 23:32 | |
| *** bobh has joined #openstack-infra | 23:33 | |
| *** claudiub has quit IRC | 23:36 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: Create openstack-zuul-jobs-linters jobs https://review.openstack.org/512715 | 23:40 |
| *** jklare has quit IRC | 23:40 | |
| pabelanger | mordred: jeblair: jlk: AJaeger: ^first steps to cross repo ansible-linters job for all our playbooks and roles in our 3 openstack-infra projects | 23:40 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: Create openstack-zuul-jobs-linters jobs https://review.openstack.org/512715 | 23:41 |
| *** jklare has joined #openstack-infra | 23:42 | |
| *** bobh has quit IRC | 23:44 | |
| *** salv-orlando has joined #openstack-infra | 23:46 | |
| *** gildub has joined #openstack-infra | 23:50 | |
| *** salv-orlando has quit IRC | 23:52 | |
| openstackgerrit | John L. Villalovos proposed openstack-dev/hacking master: autospec check: Allow 'wraps' to be an alternative to autospec https://review.openstack.org/512902 | 23:52 |
| pabelanger | jeblair: dmsimard: a shared queue for openstack-zuul-jobs, project-config and zuul-jobs doesn't sound too bad to me | 23:56 |
| pabelanger | in fact, once we have a shared linters jobs, we'd get it for free right? | 23:57 |
| *** ijw has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!