| *** hwoarang has joined #openstack-infra | 00:01 | |
| *** weshay|ruck is now known as weshay_PTO | 00:28 | |
| *** markvoelker has joined #openstack-infra | 00:33 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: connections: always register the scheduler https://review.openstack.org/637444 | 00:37 |
|---|---|---|
| *** rkukura has quit IRC | 00:39 | |
| *** markvoelker has quit IRC | 01:07 | |
| *** wolverineav has joined #openstack-infra | 01:15 | |
| *** ruffian_sheep has joined #openstack-infra | 01:17 | |
| *** wolverineav has quit IRC | 01:19 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: connections: always register the scheduler https://review.openstack.org/637444 | 01:20 |
| *** dave-mccowan has joined #openstack-infra | 01:21 | |
| *** wolverineav has joined #openstack-infra | 01:35 | |
| *** wolverineav has quit IRC | 01:38 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: connections: remove unused getSchema procedure https://review.openstack.org/637445 | 01:46 |
| *** sdake has joined #openstack-infra | 01:59 | |
| *** yikun has joined #openstack-infra | 01:59 | |
| *** markvoelker has joined #openstack-infra | 02:04 | |
| *** whoami-rajat has joined #openstack-infra | 02:13 | |
| *** sdake has quit IRC | 02:18 | |
| *** sdake has joined #openstack-infra | 02:22 | |
| *** mgoddard has quit IRC | 02:32 | |
| *** mgoddard has joined #openstack-infra | 02:34 | |
| *** markvoelker has quit IRC | 02:38 | |
| *** sdake has quit IRC | 02:44 | |
| *** hwoarang has quit IRC | 02:49 | |
| *** hwoarang has joined #openstack-infra | 02:51 | |
| *** psachin has joined #openstack-infra | 02:51 | |
| *** hwoarang has quit IRC | 02:56 | |
| *** hwoarang has joined #openstack-infra | 03:00 | |
| *** wolverineav has joined #openstack-infra | 03:05 | |
| *** hwoarang has quit IRC | 03:05 | |
| *** hwoarang has joined #openstack-infra | 03:06 | |
| *** wolverineav has quit IRC | 03:09 | |
| *** hongbin has joined #openstack-infra | 03:10 | |
| *** ykarel|away has joined #openstack-infra | 03:10 | |
| *** janki has joined #openstack-infra | 03:21 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: fixtures: use default _cleanUp procedure https://review.openstack.org/637449 | 03:23 |
| *** markvoelker has joined #openstack-infra | 03:34 | |
| *** ricolin has joined #openstack-infra | 03:38 | |
| *** udesale has joined #openstack-infra | 03:44 | |
| *** ykarel|away is now known as ykarel | 04:00 | |
| *** markvoelker has quit IRC | 04:06 | |
| *** ramishra has joined #openstack-infra | 04:07 | |
| *** dave-mccowan has quit IRC | 04:07 | |
| *** dave-mccowan has joined #openstack-infra | 04:09 | |
| *** verdurin has quit IRC | 04:14 | |
| *** hwoarang has quit IRC | 04:15 | |
| *** agopi has quit IRC | 04:15 | |
| *** hwoarang has joined #openstack-infra | 04:17 | |
| *** verdurin has joined #openstack-infra | 04:26 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: config: add playbooks to job.toDict() https://review.openstack.org/621343 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Add API endpoint to get frozen jobs https://review.openstack.org/607077 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Get executor job params https://review.openstack.org/607078 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Separate out executor server from runner https://review.openstack.org/607079 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: implement prep-workspace https://review.openstack.org/607082 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: web: add /connections route https://review.openstack.org/631703 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: use connections endpoint for prepare-workspace https://review.openstack.org/631704 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add execute sub-command https://review.openstack.org/630944 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add support for depends-on https://review.openstack.org/632064 | 04:37 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add quick-start integration test https://review.openstack.org/635701 | 04:37 |
| *** lpetrut has joined #openstack-infra | 04:43 | |
| *** hwoarang has quit IRC | 04:46 | |
| *** hwoarang has joined #openstack-infra | 04:47 | |
| *** ykarel has quit IRC | 04:58 | |
| *** markvoelker has joined #openstack-infra | 05:03 | |
| *** lpetrut has quit IRC | 05:10 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [dnm] letsencrypt rax dns prototype implementation https://review.openstack.org/637456 | 05:14 |
| *** auristor has quit IRC | 05:17 | |
| *** ykarel has joined #openstack-infra | 05:19 | |
| *** auristor has joined #openstack-infra | 05:22 | |
| *** markvoelker has quit IRC | 05:37 | |
| *** hongbin has quit IRC | 05:42 | |
| *** tkajinam_ has joined #openstack-infra | 05:54 | |
| *** tkajinam has quit IRC | 05:57 | |
| *** jbadiapa has joined #openstack-infra | 06:02 | |
| *** chandankumar is now known as chkumar|ruck | 06:17 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 06:20 |
| *** hwoarang has quit IRC | 06:23 | |
| *** hwoarang has joined #openstack-infra | 06:29 | |
| *** markvoelker has joined #openstack-infra | 06:34 | |
| *** sdake has joined #openstack-infra | 06:38 | |
| *** e0ne has joined #openstack-infra | 06:41 | |
| *** kjackal has quit IRC | 06:41 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add /connections route https://review.openstack.org/631703 | 06:46 |
| *** e0ne has quit IRC | 06:47 | |
| *** jtomasek has joined #openstack-infra | 06:50 | |
| *** xek has joined #openstack-infra | 06:52 | |
| *** AJaeger has quit IRC | 06:55 | |
| *** AJaeger has joined #openstack-infra | 07:01 | |
| *** yboaron_ has quit IRC | 07:02 | |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Add new merger job to get role definitions from a repository https://review.openstack.org/637181 | 07:04 |
| *** markvoelker has quit IRC | 07:06 | |
| *** tkajinam_ has quit IRC | 07:10 | |
| *** slaweq has joined #openstack-infra | 07:10 | |
| *** tkajinam_ has joined #openstack-infra | 07:12 | |
| *** hwoarang has quit IRC | 07:12 | |
| *** tkajinam__ has joined #openstack-infra | 07:14 | |
| *** hwoarang has joined #openstack-infra | 07:16 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/infra-specs master: letsencrypt spec https://review.openstack.org/587283 | 07:17 |
| *** tkajinam_ has quit IRC | 07:17 | |
| *** aojea has joined #openstack-infra | 07:19 | |
| *** apetrich has joined #openstack-infra | 07:19 | |
| *** kjackal has joined #openstack-infra | 07:22 | |
| *** quiquell|off is now known as quiquell|rover | 07:23 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 07:24 |
| *** jtomasek has quit IRC | 07:27 | |
| *** dpawlik has joined #openstack-infra | 07:27 | |
| *** jtomasek has joined #openstack-infra | 07:27 | |
| *** sdake has quit IRC | 07:30 | |
| *** xek has quit IRC | 07:33 | |
| *** e0ne has joined #openstack-infra | 07:35 | |
| *** quiquell|rover is now known as quiquell|rover|b | 07:36 | |
| *** e0ne has quit IRC | 07:38 | |
| *** pgaxatte has joined #openstack-infra | 07:40 | |
| *** dtantsur|afk is now known as dtantsur | 07:42 | |
| *** pcaruana has joined #openstack-infra | 07:43 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 07:48 |
| *** sshnaidm|off is now known as sshnaidm | 07:56 | |
| *** ykarel is now known as ykarel|lunch | 07:58 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 08:02 |
| *** yboaron_ has joined #openstack-infra | 08:02 | |
| *** markvoelker has joined #openstack-infra | 08:03 | |
| *** kopecmartin|off is now known as kopecmartin | 08:04 | |
| *** wolverineav has joined #openstack-infra | 08:05 | |
| *** quiquell|rover|b is now known as quiquell|rovee | 08:05 | |
| *** quiquell|rovee is now known as quiquell|rover | 08:05 | |
| *** yboaron_ has quit IRC | 08:06 | |
| *** yboaron_ has joined #openstack-infra | 08:07 | |
| *** panda|off is now known as panda | 08:09 | |
| *** ginopc has joined #openstack-infra | 08:09 | |
| *** rascasoft has joined #openstack-infra | 08:09 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 08:12 |
| *** egonzalez has quit IRC | 08:15 | |
| *** rpittau has joined #openstack-infra | 08:17 | |
| *** tkajinam__ has quit IRC | 08:28 | |
| *** gfidente has joined #openstack-infra | 08:29 | |
| *** tosky has joined #openstack-infra | 08:33 | |
| *** gfidente has quit IRC | 08:36 | |
| *** egonzalez has joined #openstack-infra | 08:36 | |
| *** markvoelker has quit IRC | 08:37 | |
| *** xek has joined #openstack-infra | 08:37 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 08:43 |
| *** rossella_s has quit IRC | 08:46 | |
| *** jpena|off is now known as jpena | 08:52 | |
| *** ruffian_sheep has quit IRC | 08:52 | |
| *** jpich has joined #openstack-infra | 08:57 | |
| *** aojea has quit IRC | 08:57 | |
| *** ykarel|lunch is now known as ykarel | 08:59 | |
| *** rossella_s has joined #openstack-infra | 09:00 | |
| *** pcaruana|afk| has joined #openstack-infra | 09:01 | |
| *** pcaruana has quit IRC | 09:02 | |
| *** ramishra has quit IRC | 09:05 | |
| *** e0ne has joined #openstack-infra | 09:08 | |
| *** ramishra has joined #openstack-infra | 09:09 | |
| *** roman_g has joined #openstack-infra | 09:11 | |
| *** ccamacho has joined #openstack-infra | 09:13 | |
| *** sdake has joined #openstack-infra | 09:15 | |
| *** kjackal has quit IRC | 09:17 | |
| *** kjackal has joined #openstack-infra | 09:18 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 09:20 |
| *** gfidente has joined #openstack-infra | 09:21 | |
| *** ramishra_ has joined #openstack-infra | 09:21 | |
| *** ramishra has quit IRC | 09:23 | |
| *** markvoelker has joined #openstack-infra | 09:34 | |
| *** derekh has joined #openstack-infra | 09:37 | |
| *** trident has joined #openstack-infra | 09:38 | |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: wip: Jaeger tracing PoC https://review.openstack.org/637505 | 09:42 |
| openstackgerrit | Simon Westphahl proposed openstack-infra/nodepool master: wip: Jaeger tracing PoC https://review.openstack.org/637506 | 09:42 |
| *** electrofelix has joined #openstack-infra | 09:42 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: amqp: add basic trigger https://review.openstack.org/637458 | 09:47 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool master: WIP: Second try to fix single cloud config reload https://review.openstack.org/608695 | 09:50 |
| *** pcaruana|afk| has quit IRC | 10:01 | |
| *** ramishra_ has quit IRC | 10:06 | |
| *** gfidente has quit IRC | 10:06 | |
| *** pcaruana has joined #openstack-infra | 10:07 | |
| *** markvoelker has quit IRC | 10:07 | |
| *** ramishra has joined #openstack-infra | 10:09 | |
| *** xek has quit IRC | 10:09 | |
| *** ociuhandu has joined #openstack-infra | 10:14 | |
| *** gfidente has joined #openstack-infra | 10:14 | |
| *** xek has joined #openstack-infra | 10:18 | |
| *** ociuhandu has quit IRC | 10:20 | |
| *** xek has quit IRC | 10:24 | |
| *** xek has joined #openstack-infra | 10:27 | |
| *** ociuhandu has joined #openstack-infra | 10:30 | |
| openstackgerrit | Jakub Bielecki proposed openstack-infra/nodepool master: doc bugfix for static provider https://review.openstack.org/637518 | 10:32 |
| *** xek has quit IRC | 10:34 | |
| *** tobias-urdin has joined #openstack-infra | 10:40 | |
| openstackgerrit | ladjadj proposed openstack-infra/grafyaml master: Add capability to use grafana folderid parameter https://review.openstack.org/637520 | 10:40 |
| *** cgoncalves has quit IRC | 10:52 | |
| *** cgoncalves has joined #openstack-infra | 10:56 | |
| openstackgerrit | ladjadj proposed openstack-infra/grafyaml master: Add capability to use grafana folderid parameter https://review.openstack.org/637520 | 10:56 |
| *** wolverineav has quit IRC | 11:03 | |
| *** markvoelker has joined #openstack-infra | 11:04 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: URLTrigger driver time based - artifact change jobs triggering driver https://review.openstack.org/635567 | 11:05 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: URLTrigger driver time based - artifact change jobs triggering driver https://review.openstack.org/635567 | 11:06 |
| *** luizbag has joined #openstack-infra | 11:08 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: URLTrigger driver time based - artifact change jobs triggering driver https://review.openstack.org/635567 | 11:13 |
| *** sdake has quit IRC | 11:16 | |
| *** sdake has joined #openstack-infra | 11:20 | |
| *** udesale has quit IRC | 11:33 | |
| *** udesale has joined #openstack-infra | 11:34 | |
| *** markvoelker has quit IRC | 11:37 | |
| *** jpena is now known as jpena|brb | 11:42 | |
| *** janki has quit IRC | 11:49 | |
| *** janki has joined #openstack-infra | 11:54 | |
| *** ginopc has quit IRC | 12:10 | |
| *** apetrich has quit IRC | 12:11 | |
| *** sdake has quit IRC | 12:12 | |
| *** sdake has joined #openstack-infra | 12:16 | |
| *** ginopc has joined #openstack-infra | 12:20 | |
| *** kaiokmo has joined #openstack-infra | 12:25 | |
| *** apetrich has joined #openstack-infra | 12:25 | |
| *** ginopc has quit IRC | 12:28 | |
| *** ginopc has joined #openstack-infra | 12:29 | |
| *** apetrich has quit IRC | 12:30 | |
| *** xek has joined #openstack-infra | 12:40 | |
| *** xek_ has joined #openstack-infra | 12:47 | |
| *** xek has quit IRC | 12:47 | |
| *** apetrich has joined #openstack-infra | 12:48 | |
| *** jcoufal has joined #openstack-infra | 12:49 | |
| *** rlandy has joined #openstack-infra | 12:54 | |
| openstackgerrit | ladjadj proposed openstack-infra/grafyaml master: Add capability to use grafana folderid parameter https://review.openstack.org/637520 | 12:56 |
| *** xek__ has joined #openstack-infra | 12:59 | |
| *** xek_ has quit IRC | 13:01 | |
| *** rh-jelabarre has joined #openstack-infra | 13:01 | |
| *** smarcet has joined #openstack-infra | 13:04 | |
| *** xek__ has quit IRC | 13:06 | |
| *** xek__ has joined #openstack-infra | 13:06 | |
| *** trown|outtypewww is now known as trown | 13:06 | |
| *** janki has quit IRC | 13:06 | |
| *** eharney has joined #openstack-infra | 13:20 | |
| *** gfidente has quit IRC | 13:22 | |
| openstackgerrit | ladjadj proposed openstack-infra/grafyaml master: Add capability to use grafana folderid parameter https://review.openstack.org/637520 | 13:22 |
| *** gfidente has joined #openstack-infra | 13:23 | |
| *** priteau has joined #openstack-infra | 13:26 | |
| *** jpena|brb is now known as jpena | 13:27 | |
| *** sdake has quit IRC | 13:28 | |
| *** quiquell|rover is now known as quique|rover|eat | 13:28 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/system-config master: Updated OpenStackId Production Node https://review.openstack.org/637541 | 13:31 |
| *** rh-jelabarre has quit IRC | 13:37 | |
| *** rh-jelabarre has joined #openstack-infra | 13:37 | |
| *** kaisers has quit IRC | 13:42 | |
| *** priteau has quit IRC | 13:45 | |
| *** kaisers has joined #openstack-infra | 13:49 | |
| *** fried_rice is now known as efried | 13:50 | |
| *** mriedem has joined #openstack-infra | 13:50 | |
| *** sdake has joined #openstack-infra | 13:56 | |
| openstackgerrit | ladjadj proposed openstack-infra/grafyaml master: Add capability to use grafana folderid parameter https://review.openstack.org/637520 | 13:58 |
| *** agopi has joined #openstack-infra | 13:59 | |
| *** priteau has joined #openstack-infra | 14:02 | |
| *** yboaron_ has quit IRC | 14:04 | |
| *** yboaron_ has joined #openstack-infra | 14:04 | |
| *** quique|rover|eat is now known as quiquell|rover | 14:09 | |
| *** jroll has quit IRC | 14:13 | |
| *** ykarel is now known as ykarel|away | 14:14 | |
| *** jroll has joined #openstack-infra | 14:14 | |
| *** ykarel|away is now known as ykarel | 14:15 | |
| *** gfidente has quit IRC | 14:22 | |
| *** rkukura has joined #openstack-infra | 14:23 | |
| *** eharney has quit IRC | 14:23 | |
| *** psachin has quit IRC | 14:28 | |
| *** chkumar|ruck is now known as chandankumar | 14:28 | |
| *** gfidente has joined #openstack-infra | 14:32 | |
| *** bnemec-pto is now known as bnemec | 14:37 | |
| *** ricolin_ has joined #openstack-infra | 14:40 | |
| *** ricolin has quit IRC | 14:42 | |
| *** ykarel is now known as ykarel|away | 14:42 | |
| *** rh-jelabarre has quit IRC | 14:43 | |
| *** priteau has quit IRC | 14:47 | |
| *** rh-jelabarre has joined #openstack-infra | 14:48 | |
| *** ykarel|away has quit IRC | 14:52 | |
| *** aojea has joined #openstack-infra | 14:53 | |
| *** kgiusti has joined #openstack-infra | 14:55 | |
| *** aojea has quit IRC | 14:57 | |
| *** dave-mccowan has joined #openstack-infra | 14:58 | |
| *** iurygregory has joined #openstack-infra | 14:58 | |
| *** sdake has quit IRC | 14:58 | |
| *** priteau has joined #openstack-infra | 14:59 | |
| *** janki has joined #openstack-infra | 15:00 | |
| iurygregory | have anyone saw an error from LIBS_FROM_GIT: http://logs.openstack.org/69/636169/11/check/ironic-tempest-dsvm-ironic-inspector-pike/be70a42/job-output.txt.gz#_2019-02-18_13_27_13_465763 ? | 15:00 |
| *** rh-jelabarre has quit IRC | 15:01 | |
| *** dave-mccowan has quit IRC | 15:03 | |
| fungi | iurygregory: i haven't seen that before, but you might have more luck inquiring in #openstack-qa if you haven't already | 15:03 |
| iurygregory | fungi, ty =D | 15:04 |
| fungi | looks like according to `pip list` these packages weren't installed from git checkouts: ironic-tempest-plugin ironic-python-agent requirements tempest placement cinder devstack | 15:05 |
| *** aojea has joined #openstack-infra | 15:06 | |
| iurygregory | but before it shows that they were found strange | 15:07 |
| iurygregory | LOOP [use-cached-repos : Find locally cached git repos] | 15:08 |
| fungi | that lib_installed_from_git function looks fragile since it's parsing `pip list` stdout | 15:08 |
| fungi | and looking for lines mentioning those packages to contain a comma and a space and then more characters | 15:08 |
| fungi | i have a feeling a future pip minor release could break it completely | 15:09 |
| *** quiquell|rover is now known as quiquell|off | 15:11 | |
| iurygregory | D: | 15:12 |
| iurygregory | the function is very different in master | 15:12 |
| *** munimeha1 has joined #openstack-infra | 15:13 | |
| *** ricolin_ has quit IRC | 15:16 | |
| *** ricolin_ has joined #openstack-infra | 15:17 | |
| *** priteau has quit IRC | 15:18 | |
| *** rh-jelabarre has joined #openstack-infra | 15:19 | |
| openstackgerrit | Merged openstack-infra/zuul master: Make UnsafeTag self registering https://review.openstack.org/637023 | 15:20 |
| *** yboaron_ has quit IRC | 15:26 | |
| *** rh-jelabarre has quit IRC | 15:28 | |
| *** rh-jelabarre has joined #openstack-infra | 15:31 | |
| *** smarcet has quit IRC | 15:34 | |
| fungi | infra-root: ovh has sent us a notification of a ceph outage which occurred in gra1 on january 27. dmesg does show some i/o errors on /dev/sda from that timeframe but it doesn't look like the rootfs mounted from a partition on it got marked read-only. i'm tempted not to bother rebooting that instance unless we see evidence of a problem, but can certainly be convinced otherwise | 15:35 |
| *** rh-jelabarre has quit IRC | 15:36 | |
| *** rh-jelabarre has joined #openstack-infra | 15:36 | |
| smcginnis | fungi: Maybe worth rebooting now to make sure there's not a problem? Versus a reboot happening somewhere down the line, something going wrong, and not having the context to tie it to this event? | 15:39 |
| fungi | maybe, however it's been over 3 weeks already and this is the first we've even found out it happened | 15:41 |
| *** udesale has quit IRC | 15:41 | |
| *** e0ne has quit IRC | 15:42 | |
| fungi | #status log according to ovh, there was a ceph outage which affected the rootfs for our gra1 mirror there on 2018-01-27 between 11:20 and 16:20 utc | 15:42 |
| openstackstatus | fungi: finished logging | 15:42 |
| fungi | that was a sunday, which is probably why we didn't notice | 15:43 |
| fungi | job volume is usually almost nonexistent for us at that time of week | 15:43 |
| *** rh-jelabarre has quit IRC | 15:44 | |
| *** rh-jelabarre has joined #openstack-infra | 15:45 | |
| *** sdake has joined #openstack-infra | 15:45 | |
| *** e0ne has joined #openstack-infra | 15:48 | |
| *** ekultails has joined #openstack-infra | 15:49 | |
| *** ykarel|away has joined #openstack-infra | 15:50 | |
| *** rh-jelabarre has quit IRC | 15:50 | |
| *** ykarel|away is now known as ykarel | 15:51 | |
| *** rh-jelabarre has joined #openstack-infra | 15:56 | |
| *** jamesmcarthur has joined #openstack-infra | 16:01 | |
| *** smarcet has joined #openstack-infra | 16:04 | |
| *** janki has quit IRC | 16:08 | |
| *** pcaruana has quit IRC | 16:10 | |
| *** ramishra has quit IRC | 16:13 | |
| *** rh-jelabarre has quit IRC | 16:13 | |
| *** ykarel is now known as ykarel|away | 16:13 | |
| fungi | probably also what caused us to not end up with a broken rootfs after the outage concluded | 16:16 |
| *** ramishra has joined #openstack-infra | 16:17 | |
| *** kjackal has quit IRC | 16:18 | |
| *** kjackal_v2 has joined #openstack-infra | 16:18 | |
| *** rh-jelabarre has joined #openstack-infra | 16:18 | |
| *** kjackal_v2 has quit IRC | 16:24 | |
| *** kjackal has joined #openstack-infra | 16:25 | |
| openstackgerrit | Jan Kundrát proposed openstack-infra/git-review master: Support usernames that contain '@' and ssh Git URLs https://review.openstack.org/428700 | 16:26 |
| *** smarcet has quit IRC | 16:30 | |
| *** rh-jelabarre has quit IRC | 16:30 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: PBX is running on xenial now, not trusty https://review.openstack.org/637580 | 16:30 |
| clarkb | er I should update the inventory file too | 16:31 |
| * clarkb makes new patchset | 16:31 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: PBX is running on xenial now, not trusty https://review.openstack.org/637580 | 16:33 |
| *** rh-jelabarre has joined #openstack-infra | 16:33 | |
| *** e0ne has quit IRC | 16:34 | |
| clarkb | #status log Deleted Trusty pbx.openstack.org (038e80f5-15aa-4f69-8c6c-0f43b3587778) as new Xenial pbx01.opendev.org is up and running | 16:37 |
| openstackstatus | clarkb: finished logging | 16:37 |
| clarkb | infra-root https://review.openstack.org/637580 is additional cleanup for pbx.o.o removal. DNS is already pointed over to the new stuff so I think we are done once that gets in | 16:38 |
| *** eharney has joined #openstack-infra | 16:40 | |
| *** ramishra has quit IRC | 16:42 | |
| clarkb | infra-root (and mordred in particular) are we ready to delete the old puppetmaster? | 16:46 |
| clarkb | I think we've kept it around out of caution, but its been a while and I have not logged in or needed the old server in months | 16:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Deploy gitea with docker-compose https://review.openstack.org/637330 | 16:46 |
| corvus | clarkb: 'yes' from me. | 16:46 |
| corvus | clarkb, mordred: when we're all around, let's talk about gitea/k8s/docker. | 16:47 |
| clarkb | ++ | 16:48 |
| *** emccormick has quit IRC | 16:50 | |
| *** kjackal has quit IRC | 16:52 | |
| *** pgaxatte has quit IRC | 16:57 | |
| *** panda is now known as panda|off | 16:58 | |
| *** rh-jelabarre has quit IRC | 16:59 | |
| roman_g | Hello team. Do you know if there was any discussing I could have a look at regarding readthdocs job failures? https://zuul.openstack.org/builds?job_name=trigger-readthedocs-webhook | 17:00 |
| roman_g | *discussion | 17:00 |
| *** ricolin_ has quit IRC | 17:00 | |
| *** sdake has quit IRC | 17:00 | |
| clarkb | roman_g: yes it is a known issue with the rtd api | 17:00 |
| clarkb | ianw filed a bug with them. Let me see if I can find it | 17:01 |
| *** rh-jelabarre has joined #openstack-infra | 17:02 | |
| fungi | infra-root: related, cleanup from the storyboard xenial replacement work is 637388 and the next step (production replacement) for openstackid xenial upgrade is 637541 | 17:02 |
| fungi | clarkb: and i'm +2 on old puppetmaster deletion as well | 17:03 |
| *** aojea has quit IRC | 17:04 | |
| roman_g | clarkb: https://github.com/rtfd/readthedocs.org/issues/4986 - this one? | 17:04 |
| *** sdake has joined #openstack-infra | 17:05 | |
| corvus | fungi, clarkb: could you please review https://review.openstack.org/637386 and https://review.openstack.org/637387 -- next steps in docker registry work. | 17:05 |
| clarkb | roman_g: yes that is it | 17:05 |
| fungi | corvus: thanks! on it | 17:05 |
| roman_g | clarkb: Thank you. I've left a comment there, asking if there is anything I can help with. | 17:06 |
| roman_g | ianw: Thanks for your work on RTD issue ^^^ . | 17:06 |
| * clarkb opens all the changes | 17:10 | |
| *** rh-jelabarre has quit IRC | 17:11 | |
| *** rh-jelabarre has joined #openstack-infra | 17:12 | |
| *** kjackal has joined #openstack-infra | 17:12 | |
| clarkb | corvus: one question on https://review.openstack.org/#/c/637387/1 | 17:14 |
| clarkb | fungi: oh cacti thats a piece I need to update in my pbx cleanup change too | 17:14 |
| clarkb | fungi: reviewing your change is pointing out all the problems with mine :) | 17:14 |
| *** smarcet has joined #openstack-infra | 17:15 | |
| corvus | clarkb: answered | 17:15 |
| fungi | clarkb: yeah, i still need to set up backups too | 17:15 |
| *** ricolin has joined #openstack-infra | 17:16 | |
| *** rpittau has quit IRC | 17:16 | |
| *** rh-jelabarre has quit IRC | 17:16 | |
| *** ginopc has quit IRC | 17:19 | |
| *** rpittau has joined #openstack-infra | 17:19 | |
| *** rh-jelabarre has joined #openstack-infra | 17:19 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: pbx.openstack.org cleanup https://review.openstack.org/637580 | 17:20 |
| clarkb | corvus: fungi ^ is hopefully a bit more complete now | 17:20 |
| *** rh-jelabarre has quit IRC | 17:24 | |
| *** rh-jelabarre has joined #openstack-infra | 17:25 | |
| *** rpittau has quit IRC | 17:26 | |
| *** rh-jelabarre has quit IRC | 17:29 | |
| *** rh-jelabarre has joined #openstack-infra | 17:30 | |
| *** smarcet has quit IRC | 17:32 | |
| *** rh-jelabarre has quit IRC | 17:34 | |
| *** rh-jelabarre has joined #openstack-infra | 17:35 | |
| clarkb | fungi: re cleaning up the regexes we should probably do a pass through those in a month or two and bulk update things that are missed | 17:39 |
| *** kjackal has quit IRC | 17:39 | |
| *** smarcet has joined #openstack-infra | 17:40 | |
| openstackgerrit | Merged openstack-infra/system-config master: Fix htpasswd creation for registry https://review.openstack.org/637386 | 17:42 |
| fungi | yeah, feel free to self-approve that one | 17:43 |
| fungi | just trying to fix them up as i go, if i remember to | 17:43 |
| *** hwoarang has quit IRC | 17:47 | |
| *** hwoarang has joined #openstack-infra | 17:47 | |
| *** kjackal has joined #openstack-infra | 17:50 | |
| *** jpich has quit IRC | 17:50 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Update docker image roles https://review.openstack.org/637387 | 17:51 |
| *** ricolin has quit IRC | 17:52 | |
| openstackgerrit | Merged openstack-infra/system-config master: Updated OpenStackId Production Node https://review.openstack.org/637541 | 17:53 |
| *** priteau has joined #openstack-infra | 17:53 | |
| openstackgerrit | Merged openstack-infra/zuul master: Add spec for multi ansible version support https://review.openstack.org/623927 | 17:56 |
| *** kopecmartin is now known as kopecmartin|off | 17:56 | |
| *** gtmanfred has quit IRC | 17:59 | |
| *** gtmanfred has joined #openstack-infra | 18:00 | |
| *** trown is now known as trown|lunch | 18:01 | |
| *** derekh has quit IRC | 18:02 | |
| *** Vadmacs has joined #openstack-infra | 18:06 | |
| *** jpena is now known as jpena|off | 18:09 | |
| *** rh-jelabarre has quit IRC | 18:15 | |
| *** rh-jelabarre has joined #openstack-infra | 18:15 | |
| *** ykarel|away has quit IRC | 18:16 | |
| *** ykarel|away has joined #openstack-infra | 18:17 | |
| *** xek__ has quit IRC | 18:18 | |
| openstackgerrit | Merged openstack-infra/zuul master: web: prevent status update loop in background https://review.openstack.org/636343 | 18:18 |
| *** xek__ has joined #openstack-infra | 18:19 | |
| *** xek__ has quit IRC | 18:19 | |
| *** xek__ has joined #openstack-infra | 18:20 | |
| *** rh-jelabarre has quit IRC | 18:20 | |
| *** gfidente is now known as gfidente|afk | 18:20 | |
| *** rh-jelabarre has joined #openstack-infra | 18:20 | |
| *** ykarel|away has quit IRC | 18:22 | |
| *** xek__ has quit IRC | 18:24 | |
| *** xek__ has joined #openstack-infra | 18:24 | |
| *** e0ne has joined #openstack-infra | 18:24 | |
| *** ociuhandu_ has joined #openstack-infra | 18:24 | |
| *** ociuhandu has quit IRC | 18:28 | |
| *** ociuhandu_ has quit IRC | 18:29 | |
| *** sdake has quit IRC | 18:29 | |
| *** xek__ has quit IRC | 18:30 | |
| *** sdake_ has joined #openstack-infra | 18:30 | |
| openstackgerrit | Merged openstack-infra/system-config master: pbx.openstack.org cleanup https://review.openstack.org/637580 | 18:32 |
| *** smarcet has quit IRC | 18:36 | |
| openstackgerrit | Jan Kundrát proposed openstack-infra/git-review master: Support usernames that contain '@' and ssh Git URLs https://review.openstack.org/428700 | 18:37 |
| clarkb | corvus: I've got gitea on the infra agenda for tomorrow if we don't get to it before then we can talk then and if we do talk before then we can recap | 18:38 |
| corvus | kk | 18:38 |
| *** rh-jelabarre has quit IRC | 18:39 | |
| *** rh-jelabarre has joined #openstack-infra | 18:39 | |
| *** ociuhandu has joined #openstack-infra | 18:40 | |
| *** ociuhandu has quit IRC | 18:45 | |
| fungi | gotta go grab grub | 18:46 |
| *** dtantsur is now known as dtantsur|afk | 18:47 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Load buildset_registry in intermediate pull https://review.openstack.org/637603 | 18:48 |
| corvus | fungi, clarkb: ^ fix for the next registry error | 18:48 |
| *** rh-jelabarre has quit IRC | 18:49 | |
| *** rh-jelabarre has joined #openstack-infra | 18:50 | |
| openstackgerrit | Merged openstack-infra/zuul master: connections: always register the scheduler https://review.openstack.org/637444 | 18:54 |
| *** rh-jelabarre has quit IRC | 18:54 | |
| clarkb | corvus: is there already an artifact at that point? | 18:55 |
| clarkb | corvus: we run buildset registry in job A then job B runs and starts with pull, I guess the way we tell B about A's registry if via zuul return so this should work | 18:55 |
| *** rh-jelabarre has joined #openstack-infra | 18:56 | |
| openstackgerrit | Merged openstack-infra/zuul master: connections: remove unused getSchema procedure https://review.openstack.org/637445 | 18:58 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Rename project to project_name in getPullBySha https://review.openstack.org/637218 | 19:01 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Test GithubShaCache https://review.openstack.org/637228 | 19:01 |
| *** rh-jelabarre has quit IRC | 19:05 | |
| *** rh-jelabarre has joined #openstack-infra | 19:06 | |
| *** trown|lunch is now known as trown | 19:06 | |
| *** electrofelix has quit IRC | 19:15 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Remember scroll offset of lanes when refreshing board view https://review.openstack.org/637607 | 19:15 |
| *** rh-jelabarre has quit IRC | 19:16 | |
| *** rh-jelabarre has joined #openstack-infra | 19:17 | |
| *** jamesmcarthur has quit IRC | 19:19 | |
| openstackgerrit | Doug Wiegley proposed openstack/gertty master: Add ctrl-v/meta-v bindings for page-up/page-down https://review.openstack.org/637610 | 19:24 |
| *** Vadmacs has quit IRC | 19:25 | |
| *** priteau has quit IRC | 19:25 | |
| *** rh-jelabarre has quit IRC | 19:26 | |
| *** rh-jelabarre has joined #openstack-infra | 19:27 | |
| *** e0ne has quit IRC | 19:30 | |
| *** smarcet has joined #openstack-infra | 19:32 | |
| *** rh-jelabarre has quit IRC | 19:32 | |
| TheJulia | Has there been any discussion, anywhere... if possibly enabling gophercloud to trigger a job defined in ironic's zuul config? | 19:32 |
| *** rh-jelabarre has joined #openstack-infra | 19:32 | |
| *** jamesmcarthur has joined #openstack-infra | 19:33 | |
| *** stbenjam has joined #openstack-infra | 19:33 | |
| mordred | TheJulia: you mean running an ironic job when someone pushes up a gophercloud change? or just using gophercloud in ironic jobs? | 19:35 |
| *** hongbin has joined #openstack-infra | 19:36 | |
| mordred | TheJulia: openlab are running gophercloud changes, so it should be possible for them to pull in job config from the ironic repo and re-use it - although IIRC in practice there are still a few roadblocks for that to 100% work (due to dependency chains) | 19:36 |
| *** e0ne has joined #openstack-infra | 19:37 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Fixed TimeZoneEntity https://review.openstack.org/637612 | 19:38 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Fixed TimeZoneEntity https://review.openstack.org/637612 | 19:39 |
| openstackgerrit | Gorka Eguileor proposed openstack-infra/project-config master: Add cinderlib project https://review.openstack.org/637613 | 19:40 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Switch to LRU + TTL based sha to PR cache https://review.openstack.org/637615 | 19:40 |
| TheJulia | mordred: kind of thinking both. Mainly report back to gophercloud, but I see value in ironic for us to be able to test the support being written in gophercloud for ironic | 19:40 |
| clarkb | fwiw I don't think we should add any github based projects to zuul until the situation in the change I just pushed is addressed, but otherwise we should be able to do both? I think that falls under our third party testing rule and if you have that integration that opens the door to depends on and all that | 19:42 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Load buildset_registry in intermediate pull https://review.openstack.org/637603 | 19:43 |
| *** hongbin has quit IRC | 19:43 | |
| TheJulia | clarkb: ack, thinking more long term anyway :) | 19:43 |
| openstackgerrit | Gorka Eguileor proposed openstack-infra/project-config master: Add cinderlib project https://review.openstack.org/637613 | 19:49 |
| *** zaneb has quit IRC | 19:50 | |
| *** sdake_ has quit IRC | 19:54 | |
| *** wolverineav has joined #openstack-infra | 19:58 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Switch to LRU based sha to PR cache https://review.openstack.org/637615 | 19:59 |
| fungi | okay, returned from grub-getting so catching back up on registry things | 20:02 |
| *** mriedem has quit IRC | 20:03 | |
| *** mriedem has joined #openstack-infra | 20:04 | |
| fungi | acting as a third party to test gophercloud pull requests don't break ironic sounds awesome | 20:05 |
| fungi | but will check out clarkb' | 20:05 |
| fungi | s lru implementation first | 20:05 |
| *** wolverineav has quit IRC | 20:06 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Speakers/Moderators Refactoring https://review.openstack.org/637620 | 20:09 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Speakers/Moderators Refactoring https://review.openstack.org/637620 | 20:10 |
| fungi | caught up and getting to work launching the new openstackid01.openstack.org node | 20:12 |
| fungi | looks like i need some regex/glob updates first | 20:12 |
| *** smarcet has quit IRC | 20:15 | |
| fungi | oh, actually i guess i don't | 20:15 |
| fungi | past me was smarter than present me | 20:15 |
| fungi | this does not bode well for my future | 20:15 |
| *** wolverineav has joined #openstack-infra | 20:16 | |
| *** luizbag has quit IRC | 20:19 | |
| *** rh-jelabarre has quit IRC | 20:21 | |
| *** rh-jelabarre has joined #openstack-infra | 20:22 | |
| *** whoami-rajat has quit IRC | 20:23 | |
| *** rh-jelabarre has quit IRC | 20:26 | |
| *** rh-jelabarre has joined #openstack-infra | 20:26 | |
| *** eharney has quit IRC | 20:29 | |
| mnaser | openstackid01.openstack.org | 20:30 |
| mnaser | oops misclick | 20:30 |
| *** rh-jelabarre has quit IRC | 20:31 | |
| *** rh-jelabarre has joined #openstack-infra | 20:32 | |
| fungi | it will exist soon (hopefully in minutes) serving the openstackid.org site | 20:34 |
| *** rh-jelabarre has quit IRC | 20:37 | |
| *** rh-jelabarre has joined #openstack-infra | 20:37 | |
| *** agopi has quit IRC | 20:37 | |
| *** eernst has joined #openstack-infra | 20:39 | |
| ianw | roman_g: yes, it's kind of frustrating how long it's gone on :/ however zuul recently merged features to pass secrets "up" I think, which would make it significantly easier to use the private token approach | 20:41 |
| ianw | i should probably escalate that on my todo list ... | 20:41 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [dnm] letsencrypt rax dns prototype implementation https://review.openstack.org/637456 | 20:55 |
| *** smarcet has joined #openstack-infra | 20:59 | |
| *** rh-jelabarre has quit IRC | 21:08 | |
| *** rh-jelabarre has joined #openstack-infra | 21:08 | |
| *** eernst has quit IRC | 21:11 | |
| *** zaneb has joined #openstack-infra | 21:12 | |
| corvus | clarkb: mordred is mostly afk today -- so let's regroup tomorrow morning on gitea | 21:21 |
| *** rh-jelabarre has quit IRC | 21:22 | |
| *** rh-jelabarre has joined #openstack-infra | 21:23 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Add new openstackid01 host to inventory and cacti https://review.openstack.org/637633 | 21:24 |
| fungi | smarcet: ^ is the replacement production openstackid server | 21:24 |
| smarcet | fungi: thx u | 21:24 |
| fungi | it'll get more thoroughly puppeted once that change to add it to our inventory merges | 21:24 |
| *** ijw has joined #openstack-infra | 21:27 | |
| *** rh-jelabarre has quit IRC | 21:27 | |
| *** rh-jelabarre has joined #openstack-infra | 21:28 | |
| *** wolverineav has quit IRC | 21:29 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/infra-specs master: letsencrypt spec https://review.openstack.org/587283 | 21:30 |
| *** ijw has quit IRC | 21:32 | |
| *** ijw has joined #openstack-infra | 21:32 | |
| *** rh-jelabarre has quit IRC | 21:32 | |
| *** ijw has quit IRC | 21:32 | |
| *** rh-jelabarre has joined #openstack-infra | 21:33 | |
| *** ijw has joined #openstack-infra | 21:34 | |
| *** rh-jelabarre has quit IRC | 21:34 | |
| *** zaneb has quit IRC | 21:37 | |
| *** rcernin has joined #openstack-infra | 21:40 | |
| *** jtomasek has quit IRC | 21:40 | |
| *** eernst has joined #openstack-infra | 21:43 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [dnm] letsencrypt rax dns prototype implementation https://review.openstack.org/637456 | 21:49 |
| clarkb | corvus: ok | 21:49 |
| *** eernst has quit IRC | 21:50 | |
| *** smarcet has quit IRC | 21:52 | |
| ianw | clarkb / corvus / fungi / anyone : letsencrypt spec is ready for review https://review.openstack.org/587283 (could keep fiddling wording all day). i'm happy enough with the prototypes that, as written, it's actually implementable | 21:53 |
| clarkb | cool, I got it on the meeting agenda with updated text noting it was updated. I do need to reread it though | 21:53 |
| clarkb | maybe I should just go ahead and do that | 21:54 |
| corvus | ianw: i just finished up reading and posting some comments | 21:54 |
| *** smarcet has joined #openstack-infra | 21:56 | |
| ianw | thanks | 21:57 |
| *** smarcet has quit IRC | 21:57 | |
| clarkb | does apache config language have conditional section imports based on files existing or not (trying to understand how setting that up conditionally would look) | 22:01 |
| corvus | clarkb: if not, self-signed would probably be fine? | 22:01 |
| clarkb | https://httpd.apache.org/docs/2.4/mod/core.html#iffile TIL | 22:02 |
| corvus | neat :) | 22:02 |
| *** trown is now known as trown|outtypewww | 22:02 | |
| fungi | apache can do *anything* | 22:02 |
| corvus | fungi: well, i read clarkb's question as ending in an implicit "without using mod_perl" :) | 22:02 |
| corvus | which is the best way to read most apache questions | 22:03 |
| * fungi tries to remember to ask apache for stock tips | 22:03 | |
| *** e0ne has quit IRC | 22:03 | |
| *** e0ne has joined #openstack-infra | 22:04 | |
| *** kjackal has quit IRC | 22:05 | |
| fungi | corvus: worth mentioning, unless we expect to send packets for an already established tcp socket to a different server, it's okay of each server in the load-balanced pool has a different cert for the same hostname(s) | 22:05 |
| clarkb | fungi: does validating a new cert in letsencrypt update a revocation list for the old certs? | 22:05 |
| corvus | fungi: that is a good point -- i feel certain that's something we can live with for the current git LB setup. i *suspect* that's also okay for the gitea/k8s setup, but i'm not 100% sure about that, having discovered surprising new behavior there on Friday. :) | 22:07 |
| fungi | clarkb: i can't say for certain, but i hope not as that would invalidate a whole set of assumptions on how people use x.509 certificates | 22:07 |
| corvus | clarkb: i'm *pretty sure* i remember reading something about how they concluded revocation lists are terrible and don't want to use them, and that's why lifetimes are short. | 22:08 |
| fungi | yeah, that's also a big part of it for letsencrypt in particular | 22:08 |
| corvus | right, they==letsencrypt | 22:08 |
| fungi | though their reasoning was mostly around client implementations generally not implementing standardized (ocsp or similar) crl retrieval mechanisms | 22:09 |
| *** kgiusti has quit IRC | 22:09 | |
| clarkb | ya clients don't actually check/update the revocation lists | 22:09 |
| fungi | not that the idea of a crl is necessarily bad, just that most client-side implementations have little incentive to do it because it's additional network connections and user-facing delays for some silly security which is almost never necessary | 22:10 |
| fungi | (except when it is, of course) | 22:10 |
| clarkb | re DNS I do like that we can bootstrap things on the cert side without needing a fully deployed instance (or even half deployed in the case of iffile ) | 22:11 |
| fungi | there's also the somewhat legitimate argument that a trivial mitm can block crl checks and allow clients they're compromising to continue using revoked certs | 22:11 |
| clarkb | so I boot a new storyboard and before I update dns to point at that server I can have a fully configured host | 22:11 |
| clarkb | then do the dns switchover for the service itself | 22:11 |
| clarkb | (whereas dns has to point at the new server before the switch for letsencrypt to validate aiui | 22:12 |
| fungi | clarkb: the same could be achieved by starting with a snakeoil cert and configuring certbot to overwrite that | 22:12 |
| fungi | (using http-based validation) | 22:12 |
| clarkb | fungi: no this is a different concern | 22:12 |
| fungi | oh, got it, before we have a record for the server in dns at all | 22:13 |
| clarkb | the way letsencrypt knows you own the domain is by HTTP GETting a known file | 22:13 |
| clarkb | yes | 22:13 |
| clarkb | so in your case of replacing storyboard servers we'd have to bootstrap the cert or take a longer outage | 22:13 |
| clarkb | still doable, just less nice imo | 22:13 |
| clarkb | with the DNS validation we can fully bootstrap the server before updating DNS to point at the server | 22:13 |
| fungi | this is a good point. if we want to avoid copying a cert to a replacement server which is not yet reachable at that name, dns-based validation does have an up-side | 22:14 |
| corvus | agreed | 22:14 |
| corvus | i hope someone is writing these rebuttals to my points in review comments on the spec, i don't want to have to argue with myself :) | 22:16 |
| clarkb | I haven't yet but will go do so now | 22:16 |
| corvus | that leaves me with the question as to whether having this hybrid centralized/distributed approach is worth the extra complexity. basically, that gives us the ability to use dns validation on ephemeral certs only held on the hosts, but it's fairly complicated. if we had precious certs, the system could be simpler (as described in the openstack.org section). | 22:18 |
| fungi | i can rebut, just wanted to think through a rebuttal out loud first | 22:20 |
| corvus | yes, it's been good. i just noticed nothing posted yet and wanted to make sure it didn't end here :) | 22:21 |
| *** sdake has joined #openstack-infra | 22:26 | |
| *** sdake has quit IRC | 22:27 | |
| clarkb | fungi: let me know if you want me to add the notes about being able to validate multiple certs for the same domain(s) | 22:30 |
| fungi | i just did | 22:30 |
| ianw | corvus: even with the precious certs held on bridge.o.o, if we stick with dns based validation we still have the swizzle of bridge.o.o having to update the dns entries on the dns server. | 22:31 |
| ianw | so that's boiling down to "does the token come from the remote host, or bridge.o.o" | 22:31 |
| corvus | ianw: yeah, i guess i was thinking that having 2 hosts involved (bridge/dns) instead of 3 (bridge/host/dns) would be simpler, but maybe it's not that big of a deal because ansible makes it easy enough? | 22:33 |
| *** sdake has joined #openstack-infra | 22:33 | |
| ianw | corvus: yeah, I think it's just a "with_inventory_hostnames" -> https://review.openstack.org/#/c/636759/23/playbooks/roles/letsencrypt-install-txt-record/tasks/main.yaml | 22:33 |
| ianw | each host can just set the token in a known var in it's host variables, like -> https://review.openstack.org/#/c/636759/23/playbooks/roles/letsencrypt-request-certs/tasks/main.yaml | 22:34 |
| corvus | ianw: i haven't internalized what 636759 is doing yet -- but will it be possible to run a smaller playbook? | 22:38 |
| corvus | ie, right now, we're doing everything from base.yaml, but at some point, we need to stop doing that and just have, for example a playbook which just manages the elasticsearch cluster. | 22:38 |
| corvus | but are those letsencrypt roles something that has to run for every host? and if so, are they still compatible with a playbook that just manages one or a small set of hosts? | 22:39 |
| ianw | corvus: so the basic pattern in 636759 is 1) host starts ACME and generates TXT record, puts it in host var 2) zone.db is updated on adns1 3) host now finishes auth and writes out keys | 22:41 |
| corvus | ianw: so those same 3 steps could be applied just to a single host | 22:41 |
| ianw | i think it is compatible, but you *might* end up being a bit racy | 22:41 |
| corvus | ianw: maybe we put some lock files around letsencrytp-install-txt-record ? | 22:41 |
| corvus | so that only one dns update happens at once? | 22:42 |
| ianw | if, two hosts decided to renew at exactly the same time. but a bit of flock could sort that out i think | 22:42 |
| ianw | right ... it's much simpler to just overwrite the zone.db file. i guess it would involve one more step of removing the lock, after the key is issued | 22:44 |
| *** eharney has joined #openstack-infra | 22:45 | |
| clarkb | we can add a fair bit of randomness to renewal cycle if we renew early enough. We should still have a lock but chances of contention would be low | 22:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: DNM: sample letsencrypt playbook for small service https://review.openstack.org/637645 | 22:46 |
| ianw | right, we're talking every few months | 22:46 |
| corvus | ianw: ^ that's an example of what i'm thinking about | 22:46 |
| corvus | basically, can we do that -- i think the thing that worries me is that i've seen roles reference the 'letsencrypt' group | 22:46 |
| *** sdake has quit IRC | 22:46 | |
| corvus | but i think our "add servers to groups and run everything from base.yaml" is getting out of hand -- i'd like to be able to just add a letsecrypt role to a server | 22:48 |
| corvus | (or 4 letsencrypt roles, that's fine :) | 22:48 |
| ianw | corvus: well the prototype as written gets the key configuration from the group variable -> https://review.openstack.org/#/c/636759/23/playbooks/group_vars/letsencrypt.yaml | 22:48 |
| ianw | i figured on one host, you'd potentially want many different certificates | 22:49 |
| *** sdake has joined #openstack-infra | 22:49 | |
| clarkb | we should be able to define those on host vars too right? then resolution of vars will sort it out | 22:49 |
| corvus | ianw: maybe we could make that just a "letencrypt_certs" hostvar? then we don't need to have a central list, we could add that variable to individual hostvars files... | 22:49 |
| corvus | (same way we have what iptables ports need to be opened on a host in its hostvars file, we'd have what certs need to be requested) | 22:50 |
| ianw | i don't see a problem switching to that. my first thought was it was nice to have a big list of certs right in front of you, but all that matters is the host knows what certs to request | 22:51 |
| corvus | oh, back to the central vs distributed question -- how do we handle testing if certs are not precious? for example, we have self-signed certs as hostvars for the registry: http://git.openstack.org/cgit/openstack-infra/system-config/tree/playbooks/zuul/templates/group_vars/registry.yaml.j2 | 22:54 |
| corvus | that means we can deploy the registry in a ci job complete with cert -- but how do we do that if we expect the registry to get it cert on startup? | 22:55 |
| clarkb | corvus: wrap the include role for letsencrypt in a when registry_tls_cert not defined? | 22:55 |
| clarkb | corvus: then we can separately test that role against their test server. Or configure it to talk to the test server in the test suite instead of my when condition? | 22:56 |
| ianw | there could be an alternative role used for testing that instead of creating a real key creates a self-signed certificate in the same location as letsencrypt would. anything connecting to it for testing would also then have to be in testing mode accepting self-signed certs | 22:58 |
| corvus | if we wrap it in a conditional, then we won't test the interaction of the letsencrypt role with the rest of the system (eg, is the registry configured to read the certs from the correct location?) until we run it in prod, so there's a hole in our testing... | 22:58 |
| *** eharney has quit IRC | 22:58 | |
| corvus | the alternative testing role that makes the cert (or having the real role switch on testing) sounds better as it's a bit less of a hole in test coverage | 22:59 |
| ianw | or, instead of running the actual renew in -> https://review.openstack.org/#/c/636759/23/playbooks/roles/letsencrypt-create-certs/tasks/main.yaml | 22:59 |
| corvus | if we can use letsencrypt's test server, maybe that's the best? does that do any validation? | 23:00 |
| ianw | it could generate self-signed certs. just flip the behaviour based on a var | 23:00 |
| ianw | corvus: the staging servers still validate, you just don't get a real cert | 23:00 |
| ianw | and you don't hit failure rate limits, etc | 23:00 |
| *** tkajinam has joined #openstack-infra | 23:00 | |
| corvus | okay, validation is probably too hard since it's dns based and i don't think we'll be delegating a zone to ephemeral ci nodes :) | 23:01 |
| corvus | so self-signed triggered by var may be the best | 23:01 |
| *** mriedem has quit IRC | 23:03 | |
| *** dklyle has quit IRC | 23:03 | |
| ianw | we *could* ... it would certainly fall out easier with designate or some DNS-as-an-api type thing. then you wouldn't worry about races, serial numbers etc | 23:03 |
| corvus | i don't think we could do that in a check job | 23:03 |
| ianw | but for immediate purposes, i think we could do self-signed | 23:04 |
| *** rascasoft has quit IRC | 23:06 | |
| ianw | oh reading it you've still got limits on le staging; 60 failed validations an hour. i could DOS that just trying to figure things out :) | 23:08 |
| openstackgerrit | Gorka Eguileor proposed openstack-infra/project-config master: Add cinderlib project https://review.openstack.org/637613 | 23:09 |
| ianw | ... it's a standalone docker image to run your own version of Let's Encrypt easily ... | 23:10 |
| corvus | ianw: that's... that's a really cool idea :) | 23:11 |
| *** munimeha1 has quit IRC | 23:12 | |
| *** e0ne has quit IRC | 23:12 | |
| clarkb | zbr: for https://review.openstack.org/#/c/634438/1 we normally just add people via chanserv directly | 23:20 |
| *** e0ne has joined #openstack-infra | 23:23 | |
| *** sdake has quit IRC | 23:24 | |
| *** sdake has joined #openstack-infra | 23:26 | |
| clarkb | fungi can we approve https://review.openstack.org/#/c/636574/1/gerrit/projects.yaml or does that need a coordinated cut over? | 23:27 |
| *** e0ne has quit IRC | 23:30 | |
| *** ociuhandu has joined #openstack-infra | 23:30 | |
| clarkb | fungi: also any idea what the expected relationship between SIG repos and sigs is/will be? https://review.openstack.org/#/c/637125/4 | 23:31 |
| *** sdake has quit IRC | 23:32 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Fix build-docker-image when using buildset_registry https://review.openstack.org/637650 | 23:33 |
| corvus | fungi, clarkb: ^ sorry, there was an error in an earlier change | 23:33 |
| *** jcoufal has quit IRC | 23:34 | |
| clarkb | ah right. If that was shell it would be ok (maybe I'd have to double check wht that quoting type does) | 23:34 |
| corvus | it folds onto one line | 23:34 |
| corvus | so yeah, the other fix would be to change it to "shell: |" | 23:35 |
| *** ociuhandu has quit IRC | 23:35 | |
| corvus | instead of "command: >-" | 23:35 |
| clarkb | re earlier ironic question, https://review.openstack.org/#/c/636093/1 is similar. I think in both cases we would need to confirm with the upstream they want our third party input (and we should fix the github perforamnce issure and get that deployed first). Also be extra clear we aren't gating or main CI for that project (as noted in that email thread I wrote a little while back) and instead are a | 23:36 |
| clarkb | third party input | 23:36 |
| openstackgerrit | Merged openstack-infra/zuul master: Re-use the github PR object when fetching reviews https://review.openstack.org/636705 | 23:36 |
| openstackgerrit | Merged openstack-infra/zuul master: Add comment about extra issues request https://review.openstack.org/636706 | 23:36 |
| *** gfidente|afk has quit IRC | 23:37 | |
| corvus | clarkb: https://governance.openstack.org/tc/resolutions/20180215-third-party-check.html is a handy link if you need to share it | 23:37 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Don't request PR issue data https://review.openstack.org/636728 | 23:40 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Don't request PR issue data https://review.openstack.org/636728 | 23:41 |
| clarkb | corvus: thanks | 23:41 |
| corvus | clarkb: also.... technically.... 636093 won't increase our volume -- adding the github app will, and that's independent of 636093. it's unclear if the goal is to have gnocchi trigger zuul (eg, third-party-check), or just have telementry be able to use required-projects and depends-on with gnocchi. if it's the latter, then they don't have to add the app. but if you still want to hold that change | 23:41 |
| corvus | until lru is in out of an abundance of caution, that wfm. | 23:41 |
| clarkb | ya I think we are really close to fixing the sha to pr mapping so may as well try and get that in first. Also good point re depends on or adding the app | 23:42 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove rtd_web_hook from networking-ansible https://review.openstack.org/636651 | 23:43 |
| *** rascasoft has joined #openstack-infra | 23:44 | |
| *** agopi has joined #openstack-infra | 23:47 | |
| clarkb | somehow X has decided to consume an entire cpu core. /me performs a long overdue reboot | 23:48 |
| *** rascasoft has quit IRC | 23:50 | |
| *** smarcet has joined #openstack-infra | 23:50 | |
| *** smarcet has quit IRC | 23:56 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [dnm] letsencrypt prototype implementation https://review.openstack.org/636759 | 23:58 |
| *** spsurya has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!