Wednesday, 2018-07-25

*** bdodd has quit IRC00:03
*** wolverineav has quit IRC00:06
*** wolverineav has joined #openstack-infra00:07
*** felipemonteiro has joined #openstack-infra00:08
*** slaweq has joined #openstack-infra00:11
*** hongbin has joined #openstack-infra00:12
*** wolverineav has quit IRC00:14
*** hamzy_ has joined #openstack-infra00:14
pabelangerianw: I think you added hwe kernel for zuul-executor, ze11.o.o might be of interest to you: http://grafana.openstack.org/dashboard/db/zuul-status for some reason, we seem to be getting much better performance on that server for builds.  Only thing I noticed was a much newer kernel.  But could also be better compute node too.00:14
pabelangergoing to read though kernel logs and see if something changed, but sent from 4.13 to 4.15 it seems00:14
pabelangereg: almost twice the jobs, with 1/2 of the memory usage for ansible00:15
pabelangerwe'll, not 1/2, maybe 30%00:15
*** slaweq has quit IRC00:16
ianwhrm, is all the pti etc stuff turned on?00:16
openstackgerritMerged openstack-infra/system-config master: Turn on the future parser for groups.o.o  https://review.openstack.org/58463800:16
clarkboh if its an entirely new kernel version that may explain it a lot of the page table stuff is much more performant on newer kernels00:17
ianwyeah, it probably is time to cycle them over00:18
*** felipemonteiro has quit IRC00:19
ianwIIRC HWE switch was based on empirical testing that it OOM'd less00:19
clarkbyup00:19
ianwso if emperical testing shows 4.15 working better, then seems as good a reason as any :)00:19
clarkbbecause it didn't treat file caches as used memory or similar00:19
*** bdodd has joined #openstack-infra00:20
*** shrircha has quit IRC00:20
*** shrircha has joined #openstack-infra00:21
*** rfolco has quit IRC00:21
*** rfolco has joined #openstack-infra00:22
ianwprobably a bad time to cycle through them and do it right now, but put it in meeting for next week?00:22
*** gongysh has quit IRC00:24
*** wolverineav has joined #openstack-infra00:26
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: WIP: Add upload-logs-swift role  https://review.openstack.org/58454100:27
*** wolverineav has quit IRC00:28
ianwCould not parse line "HijxEUeWHhFJDIVRGUf0Ef0nrmBv fungi-openstack-2015" at /root/.ssh/authorized_keys:2700:30
ianwhttp://logs.openstack.org/92/585292/2/check/infra-puppet-apply-4-centos-7/7ce14b4/applytest/puppetapplytest27.final.out.FAILED00:30
ianwthat's a weird one00:30
cmurphythat's been happening a lot :/ it used to be pleia2's key that tripped it up00:30
ianwit's weird that it would only sometimes happen?00:32
*** wolverineav has joined #openstack-infra00:32
cmurphyit's really weird00:32
cmurphybut seems to be mostly on centos00:32
cmurphyit's happened before on puppet 3 but seems to be more common on puppet 400:32
ianwcmurphy: if you're here for a second -- https://review.openstack.org/#/c/585291/ ... is nodejs-legacy package being brought in?00:33
ianwoh, and thanks a ton for working on it all :)00:33
*** felipemonteiro has joined #openstack-infra00:33
cmurphylet me see if i wrote down the reason in another patch00:33
*** dave-mcc_ has quit IRC00:35
*** dklyle_ has quit IRC00:36
cmurphyianw: i'm not sure off the top of my head whether it's nodejs-legacy or just nodejs, but i think it comes down to an ordering issue - if the package already lays down /usr/bin/node then puppet won't try to make the symlink, but if it makes the symlink first then update-alternatives gets broken00:36
*** dklyle has joined #openstack-infra00:36
cmurphyhttps://review.openstack.org/#/c/582759/00:36
*** wolverineav has quit IRC00:36
cmurphyalso the maintainers realized the symlink was a bad idea and removed it in later versions of that module00:37
ianwcmurphy: yeah, i'm not really concerned ... just AIUI /usr/bin/node link must have been coming from nodejs-legacy package, and i'm not sure if that's just because of luck that it comes in, or by design00:37
cmurphynot sure either00:37
pabelangerianw: clarkb: yah, new kernel bump. will be interesting to upgrade another zuul-executor, maybe once things calm down a little, to see if that results in same improvement00:38
pabelangercmurphy: ianw: I want to say it is a race with glean, the SSH key error. And always only happens on centos00:39
*** wolverineav has joined #openstack-infra00:40
ianwpabelanger: in that glean has actually written out an invalid authorized_keys file?00:41
*** agopi has quit IRC00:42
*** wolverineav has quit IRC00:45
ianwdef safe_open(*args, **kwargs): ... i wonder if this ... isn't safe00:47
*** yamamoto has quit IRC00:49
*** wolverineav has joined #openstack-infra00:49
*** yamamoto has joined #openstack-infra00:50
*** yamamoto has quit IRC00:52
*** rlandy has joined #openstack-infra00:52
*** wolverineav has quit IRC00:54
pabelangerianw: or puppet is reading the same time as glean00:56
*** felipemonteiro has quit IRC00:57
ianwpabelanger: i don't feel like that could be happening ... it's so long after boot by the time puppet gets to it?00:58
clarkbya my theory at one time was glean writing invalid file due to possible metadata corruption/incompleteness00:59
clarkbI have/had a change somewhere to cat the ssh authorized  keys file before puppet runs to check00:59
ianwi do think safe_open() violates the united nations international convention on functions doing what their name says, as it doesn't close the file.  if that can result in this is questionable00:59
openstackgerritIan Wienand proposed openstack-infra/glean master: Close file in safe_open  https://review.openstack.org/58557401:02
pabelangerianw: any idea what happen to time to ready graph at http://grafana.openstack.org/dashboard/db/nodepool01:02
pabelangerianw: seems different since granfa update01:03
*** wolverineav has joined #openstack-infra01:03
ianwpabelanger: hrm, yeah it has options to continue the line.  but usually when i've seen that, it's because we've blasted a whole bunch of data at statsd and some of it got lost.  using statsd pipelines helps with that as it bunches it into bigger packets01:04
pabelangerlooks like we are not connecting points anymore01:04
ianwwhere's that stat generated?01:04
pabelangerfrom nodepool01:05
pabelangerstats.timers.nodepool.launch.provider.*.ready.mean01:05
ianwhttps://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/stats.py#n82 is my first suspect01:07
ianwany time there's a "for key: statsd.incr()" type thing, depending on the number of keys the packets tend to get lost01:08
*** wolverineav has quit IRC01:08
pabelangernot sure, if we go back to http://grafana.openstack.org/d/rZtIH5Imz/nodepool?panelId=18&fullscreen&orgId=1&from=1460950556899&to=1469129795241 (2016) same issue. I don't remember seeing gaps in the graph before01:09
*** slaweq has joined #openstack-infra01:11
pabelangerianw: Hmm, where did https://git.openstack.org/cgit/openstack-infra/project-config/tree/grafana/nodepool.yaml go01:12
pabelangerianw: deleted in https://review.openstack.org/#/c/577045/ by mistake?01:13
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Use pipelines for stats keys  https://review.openstack.org/58557701:15
ianwpabelanger: hrm, may have been01:15
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Add back grafana/nodepool.yaml  https://review.openstack.org/58557801:16
pabelangerianw: ^01:16
*** slaweq has quit IRC01:16
ianwyeah sorry, i think that just slipped in when i created those new pages01:16
pabelangernp01:17
gmannmriedem: ack. i remember they were not taking too long when we marked slow tests but yes we can re-iterate those01:17
gmannmriedem: did you open bug ?01:17
pabelangerI wonder if some default in grafana changed, and grafyaml is out of sync01:17
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Use pipelines for stats keys  https://review.openstack.org/58557701:18
ianwpabelanger: the old one is still there -> http://grafana01.openstack.org/dashboard/db/nodepool?panelId=15&fullscreen01:19
ianwi think it's connecting the dots for the missing bits01:20
pabelangeryah, that looks better01:20
pabelangernullPointMode01:21
pabelangerI think that is the setting01:21
*** wolverineav has joined #openstack-infra01:21
pabelangerseem to be set to connected in old grafana01:21
pabelangerand missing on new01:21
*** wolverineav has quit IRC01:22
ianwnull as zero might be correct here?01:22
pabelangeryah, null is nothing happened01:23
pabelangeroh01:23
pabelangeryah, maybe that is the new default01:23
ianwi think it's defaulting to null as "not shown"01:24
ianwwhat's the y-axis, seconds?01:24
*** rlandy has quit IRC01:24
pabelangeryah01:24
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Set nullPointMode for Time to Ready panel  https://review.openstack.org/58558001:25
pabelangerianw: ^if correct, that should restore it01:25
pabelangerbut don't have local instance to test with01:25
ianwpabelanger: http://grafana.openstack.org/d/ADm3HOKmz/nodepool-copy?orgId=1 i quickly made a copy01:26
pabelangerianw: how did you do that?01:27
ianwthrough the magic of logging in :)01:27
pabelangerah01:27
pabelangerianw: try changing nullPointMode to connected01:28
*** wolverineav has joined #openstack-infra01:28
ianwpabelanger: yeah, reload and it has that now.01:29
pabelangerthere we go01:29
pabelangercool01:29
*** agopi has joined #openstack-infra01:29
ianwi'm not really sure that's the right mode for it.  we could also do bars01:30
*** links has joined #openstack-infra01:30
pabelangerI like that view personally, as you get the average when you slide your mouse over the lines. otherwise, we get a lot of zero01:30
ianwload it now and it's bars01:31
pabelangerharder to read, IMO01:31
pabelangerbut having the connected null point, it is easier to see when nothing was launched in cloud, usually meaning something delayed the node come coming online01:32
pabelangerat least, how I read it01:32
*** wolverineav has quit IRC01:32
*** wolverineav has joined #openstack-infra01:34
*** dave-mccowan has joined #openstack-infra01:35
ianwpableanger; that job runtime down the bottom isn't going to work either, right?  get rid of it?01:35
*** bobh has joined #openstack-infra01:36
pabelangeryah, we can delete that too now01:36
*** jiapei has joined #openstack-infra01:38
openstackgerritIan Wienand proposed openstack-infra/project-config master: More slight enhancements to nodepool graph  https://review.openstack.org/58558401:38
*** felipemonteiro has joined #openstack-infra01:39
ianwpabelanger: follow-on for all that ^  i removed that nodepool copy dashboard too01:39
*** wolverineav has quit IRC01:39
*** iyamahat__ has joined #openstack-infra01:40
ianwi'll just approve this trivial stuff and make sure it puppets ok01:40
*** wolverineav has joined #openstack-infra01:41
pabelangerack01:41
*** dingyichen has joined #openstack-infra01:42
*** iyamahat_ has quit IRC01:43
*** yamahata has quit IRC01:43
openstackgerritMerged openstack-infra/project-config master: Add back grafana/nodepool.yaml  https://review.openstack.org/58557801:47
ianwi was thinking that grafyaml should maybe just pass through .json files unaltered, so people could interactively create nice dashboards but we still commit them as usual01:48
*** iyamahat__ has quit IRC01:48
ianwcreating the dashboard, then "downconverting" it to yaml is a bit of a pain01:49
*** wolverineav has quit IRC01:49
*** felipemonteiro has quit IRC01:52
openstackgerritMerged openstack-infra/project-config master: Set nullPointMode for Time to Ready panel  https://review.openstack.org/58558001:52
*** bobh has quit IRC01:55
*** gyee has quit IRC01:55
*** iyamahat has joined #openstack-infra01:57
*** zhangfei has joined #openstack-infra01:58
clarkbI'm not in a spot to debug this further (dinner time), but keystone auth problems to vexxhost have stopped our ansible puppet coron01:59
clarkb*cron01:59
*** wolverineav has joined #openstack-infra01:59
clarkbI've put groups.o.o in the emergency file as a result because I won't be able to watch the puppet future parser stuff happen later tonight02:00
openstackgerritMerged openstack-infra/project-config master: More slight enhancements to nodepool graph  https://review.openstack.org/58558402:00
ianw(https://auth.vexxhost.net/auth/tokens): The resource could not be found. (HTTP 404)02:00
*** wolverineav has quit IRC02:04
ianwif i make the url https://auth.vexxhost.net/v3 it works02:04
*** felipemonteiro has joined #openstack-infra02:10
*** slaweq has joined #openstack-infra02:11
openstackgerritIan Wienand proposed openstack-infra/system-config master: Update puppetlabs-postgresql to 4.8.0  https://review.openstack.org/55899502:11
openstackgerritIan Wienand proposed openstack-infra/system-config master: Make ask/ask-staging numeric  https://review.openstack.org/55899202:11
openstackgerritIan Wienand proposed openstack-infra/system-config master: Minor fork puppet-solr  https://review.openstack.org/55917802:11
*** yamahata has joined #openstack-infra02:16
*** slaweq has quit IRC02:16
*** adriant has quit IRC02:16
*** wolverineav has joined #openstack-infra02:17
*** adriant has joined #openstack-infra02:19
*** roman_g has quit IRC02:23
*** wolverineav has quit IRC02:23
*** ramishra has joined #openstack-infra02:27
*** wolverineav has joined #openstack-infra02:29
*** dave-mccowan has quit IRC02:29
*** dave-mccowan has joined #openstack-infra02:30
*** gongysh has joined #openstack-infra02:30
*** s-shiono has joined #openstack-infra02:32
*** wolverineav has quit IRC02:33
*** larainema has quit IRC02:34
*** zeus has quit IRC02:35
*** StevenK has quit IRC02:35
*** StevenK has joined #openstack-infra02:35
*** rh-jelabarre has quit IRC02:36
*** zeus has joined #openstack-infra02:39
*** zeus is now known as Guest1421702:40
openstackgerritMerged openstack-infra/puppet-iptables master: Ensure firewalld package is absent, not purged  https://review.openstack.org/58144602:41
openstackgerritMerged openstack-infra/puppet-iptables master: Ensure iptables service is running  https://review.openstack.org/58144702:41
openstackgerritMerged openstack-infra/puppet-iptables master: Explicitly set selinux seltype for rules link  https://review.openstack.org/58144802:41
*** rh-jelabarre has joined #openstack-infra02:42
*** felipemonteiro has quit IRC02:43
*** jamesmcarthur has joined #openstack-infra02:48
*** seongsoocho has joined #openstack-infra02:52
*** felipemonteiro has joined #openstack-infra02:54
openstackgerritMerged openstack-infra/system-config master: Skip epel mirror if already setup  https://review.openstack.org/58505602:56
*** zhangfei has quit IRC02:58
*** felipemonteiro has quit IRC03:03
*** jamesmcarthur has quit IRC03:09
*** dave-mcc_ has joined #openstack-infra03:09
*** slaweq has joined #openstack-infra03:11
mnaserclarkb, ianw thats weird, i thought we fixed that?!03:12
*** jamesmcarthur has joined #openstack-infra03:12
mnaseri wonder why it keeps breaking...03:12
*** dave-mccowan has quit IRC03:12
*** dave-mcc_ has quit IRC03:12
*** armaan has joined #openstack-infra03:13
ianwmnaser: it's still doing it.  i actually wonder why shade doesn't seem to use the v3 endpoint?03:15
mnasermordred had a theory about all of that but i dont remember the exact details03:15
mnasernothing changed on our side03:15
*** slaweq has quit IRC03:15
ianwdo you want me to trace it or something?03:16
*** jamesmcarthur has quit IRC03:16
*** janki has joined #openstack-infra03:17
*** wolverineav has joined #openstack-infra03:17
*** jamesmcarthur has joined #openstack-infra03:18
*** wolverineav has quit IRC03:22
*** zhurong has joined #openstack-infra03:27
*** jamesmcarthur has quit IRC03:38
*** jamesmcarthur has joined #openstack-infra03:40
*** wolverineav has joined #openstack-infra03:40
*** wolverineav has quit IRC03:48
mnaserianw: if you could, it's a bit beyond me right now why it's doing that, maybe there was an openstacksdk/osc release03:49
*** wolverineav has joined #openstack-infra03:50
*** lpetrut has joined #openstack-infra03:52
*** wolverineav has quit IRC03:55
ianwhrm, well it chats to auth.vexxhost.net and decides "Auth plugin v3password selected"03:57
ianwMaking authentication request to https://auth.vexxhost.net/auth/tokens03:58
ianwStarting new HTTPS connection (1): auth.vexxhost.net:44303:58
ianwhttps://auth.vexxhost.net:443 "POST /auth/tokens HTTP/1.1" 404 13403:58
*** rosmaita has quit IRC03:59
*** psachin` has joined #openstack-infra04:00
ianwhrm, yeah, so this seems to be the openstackclient installed on puppetmaster.  if it do it in a virtualenv with a current one it works04:02
*** eernst has joined #openstack-infra04:02
*** psachin`` has joined #openstack-infra04:05
ianwwell durrhttps://docs.openstack.org/releasenotes/openstacksdk/unreleased.html#bug-fixes04:06
*** mschuppert has joined #openstack-infra04:06
ianw#status log upgraded openstacksdk to 0.17.0 on puppetmaster for to resolve vexxhost issues (see 0.17.0 release notes currently @ https://docs.openstack.org/releasenotes/openstacksdk/unreleased.html#bug-fixes)04:07
*** psachin` has quit IRC04:07
openstackstatusianw: finished logging04:07
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an OpenShift resource provider  https://review.openstack.org/57066704:07
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: executor: add support for generic build resource  https://review.openstack.org/57066804:07
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-base-jobs master: Add base openshift job  https://review.openstack.org/57066904:07
*** slaweq has joined #openstack-infra04:11
*** wolverineav has joined #openstack-infra04:15
*** slaweq has quit IRC04:16
*** armaan has quit IRC04:19
*** armaan has joined #openstack-infra04:20
*** wolverineav has quit IRC04:20
*** wolverineav has joined #openstack-infra04:22
*** larainema has joined #openstack-infra04:26
*** wolverineav has quit IRC04:27
*** zhurong has quit IRC04:27
*** wolverineav has joined #openstack-infra04:31
*** r-daneel has joined #openstack-infra04:37
*** udesale has joined #openstack-infra04:39
*** wolverineav has quit IRC04:40
*** hongbin has quit IRC04:40
*** jamesmcarthur has quit IRC04:41
*** lpetrut has quit IRC04:42
*** udesale has quit IRC04:42
*** udesale has joined #openstack-infra04:43
*** ykarel has joined #openstack-infra04:46
*** wolverineav has joined #openstack-infra04:49
*** yamamoto has joined #openstack-infra04:50
*** wolverineav has quit IRC04:54
*** gongysh has quit IRC04:55
*** armaan has quit IRC04:56
*** armaan has joined #openstack-infra04:56
*** hongbin has joined #openstack-infra04:58
*** pcichy has quit IRC04:58
*** pcichy has joined #openstack-infra04:58
*** hongbin has quit IRC05:00
*** kjackal has joined #openstack-infra05:03
*** wolverineav has joined #openstack-infra05:04
*** dtruong2 has joined #openstack-infra05:07
*** slaweq has joined #openstack-infra05:11
*** wolverineav has quit IRC05:12
*** udesale_ has joined #openstack-infra05:13
*** agopi has quit IRC05:14
*** udesale has quit IRC05:15
*** slaweq has quit IRC05:15
*** armaan has quit IRC05:17
*** armaan has joined #openstack-infra05:18
*** agopi has joined #openstack-infra05:18
*** jiapei has quit IRC05:18
*** graphene has quit IRC05:19
*** udesale_ has quit IRC05:19
*** udesale has joined #openstack-infra05:20
*** graphene has joined #openstack-infra05:20
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Use pipelines for stats keys  https://review.openstack.org/58557705:23
*** gongysh has joined #openstack-infra05:27
*** auristor has quit IRC05:28
*** auristor has joined #openstack-infra05:30
*** armaan has quit IRC05:32
*** armaan has joined #openstack-infra05:32
*** eernst has quit IRC05:36
*** quiquell has joined #openstack-infra05:36
*** wolverineav has joined #openstack-infra05:38
*** wolverineav has quit IRC05:42
*** wolverineav has joined #openstack-infra05:52
*** dhajare has joined #openstack-infra05:52
*** felipemonteiro has joined #openstack-infra05:52
*** wolverineav has quit IRC05:58
*** wolverineav has joined #openstack-infra06:00
*** yamahata has quit IRC06:02
*** yamahata has joined #openstack-infra06:03
*** armaan has quit IRC06:04
*** annp has joined #openstack-infra06:05
*** armaan has joined #openstack-infra06:05
*** luzC has quit IRC06:07
*** luzC has joined #openstack-infra06:07
*** wolverineav has quit IRC06:08
*** ccamacho has joined #openstack-infra06:09
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add /{tenant}/pipelines route  https://review.openstack.org/54152106:11
*** slaweq has joined #openstack-infra06:11
*** cshastri has joined #openstack-infra06:14
*** slaweq has quit IRC06:16
*** felipemonteiro has quit IRC06:16
*** graphene has quit IRC06:20
*** armaan has quit IRC06:22
*** graphene has joined #openstack-infra06:22
*** armaan has joined #openstack-infra06:22
*** psachin`` has quit IRC06:24
*** lpetrut has joined #openstack-infra06:26
*** kjackal has quit IRC06:30
*** pgadiya has joined #openstack-infra06:30
*** pgadiya has quit IRC06:31
*** pcaruana has joined #openstack-infra06:33
*** dtruong2 has quit IRC06:34
*** wolverineav has joined #openstack-infra06:36
*** quiquell is now known as quiquell|bbl06:37
*** felipemonteiro has joined #openstack-infra06:37
*** wolverineav has quit IRC06:41
*** jamesmcarthur has joined #openstack-infra06:42
*** wolverineav has joined #openstack-infra06:42
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Use pipelines for stats keys  https://review.openstack.org/58557706:46
*** wolverineav has quit IRC06:46
*** MatheusGabriel has joined #openstack-infra06:48
gmannmriedem: replied on ML on slow tests. collecting the data here - https://ethercalc.openstack.org/dorupfz6s9qt06:48
*** MatheusGabriel has quit IRC06:49
openstackgerritMerged openstack-infra/project-config master: Fix typo in ansible-role-openstack-operations project  https://review.openstack.org/58554306:56
*** graphene has quit IRC06:59
*** rcernin has quit IRC06:59
*** graphene has joined #openstack-infra07:00
*** yolanda_ has quit IRC07:00
*** quiquell|bbl is now known as quiquell07:01
*** adrianreza has joined #openstack-infra07:05
*** pguimaraes has joined #openstack-infra07:05
*** armaan has quit IRC07:09
*** armaan has joined #openstack-infra07:10
*** slaweq has joined #openstack-infra07:11
*** zhangfei has joined #openstack-infra07:14
*** shardy has joined #openstack-infra07:14
*** armaan has quit IRC07:14
*** shardy has quit IRC07:15
*** slaweq has quit IRC07:16
*** tesseract has joined #openstack-infra07:16
*** slaweq has joined #openstack-infra07:16
*** gfidente has joined #openstack-infra07:18
*** gfidente has joined #openstack-infra07:18
*** kjackal has joined #openstack-infra07:19
*** yolanda has joined #openstack-infra07:22
*** florianf has joined #openstack-infra07:22
*** shardy has joined #openstack-infra07:25
*** ihar has quit IRC07:25
*** wolverineav has joined #openstack-infra07:27
*** dtantsur|afk is now known as dtantsur07:28
*** jtomasek has joined #openstack-infra07:31
*** wolverineav has quit IRC07:32
*** wolverineav has joined #openstack-infra07:32
*** ykarel is now known as ykarel|lunch07:34
*** lpetrut_ has joined #openstack-infra07:35
*** wolverineav has quit IRC07:37
*** lpetrut has quit IRC07:38
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an OpenShift resource provider  https://review.openstack.org/57066707:40
*** zhangfei has quit IRC07:40
*** ihar has joined #openstack-infra07:41
*** jpich has joined #openstack-infra07:47
*** rossella_s has quit IRC07:51
*** rossella_s has joined #openstack-infra07:53
*** felipemonteiro has quit IRC08:01
*** vivsoni_ has joined #openstack-infra08:02
*** tosky has joined #openstack-infra08:02
*** vivsoni has quit IRC08:02
*** alexchadin has joined #openstack-infra08:04
*** armaan has joined #openstack-infra08:05
*** florianf has quit IRC08:16
*** florianf has joined #openstack-infra08:16
*** rossella_s has quit IRC08:17
*** rossella_s has joined #openstack-infra08:20
*** ykarel|lunch is now known as ykarel08:21
*** zhangfei has joined #openstack-infra08:21
*** lucas-afk is now known as lucasagomes08:22
*** roman_g has joined #openstack-infra08:28
*** rcernin has joined #openstack-infra08:29
*** ykarel is now known as ykarel|mtg08:34
*** florianf has quit IRC08:35
*** pbourke has quit IRC08:37
*** wolverineav has joined #openstack-infra08:38
*** pbourke has joined #openstack-infra08:39
*** dingyichen has quit IRC08:40
*** wolverineav has quit IRC08:43
*** shardy has quit IRC08:44
*** shardy has joined #openstack-infra08:44
*** armaan has quit IRC08:46
*** armaan has joined #openstack-infra08:47
*** openstackgerrit has quit IRC08:48
*** wolverineav has joined #openstack-infra08:50
*** armaan has quit IRC08:51
*** wolverineav has quit IRC08:55
*** panda|off is now known as panda08:57
*** zoli is now known as zoli|lunch08:59
*** Guest56850 is now known as honza09:01
*** s-shiono has quit IRC09:01
*** rossella_s has quit IRC09:11
*** armaan has joined #openstack-infra09:11
*** rossella_s has joined #openstack-infra09:12
*** notmyname has quit IRC09:16
*** pcichy has quit IRC09:16
*** chason has quit IRC09:16
*** notmyname has joined #openstack-infra09:17
*** pcichy has joined #openstack-infra09:17
*** chason has joined #openstack-infra09:17
*** arxcruz|off is now known as arxcruz09:22
*** armaan has quit IRC09:26
*** armaan has joined #openstack-infra09:26
*** armaan has quit IRC09:30
*** openstackgerrit has joined #openstack-infra09:30
openstackgerritColleen Murphy proposed openstack-infra/system-config master: Add beaker tests for firehose  https://review.openstack.org/58569109:30
*** d0ugal has joined #openstack-infra09:36
*** d0ugal has quit IRC09:36
*** d0ugal has joined #openstack-infra09:36
*** lifeless has quit IRC09:37
*** ykarel|mtg is now known as ykarel09:38
openstackgerritColleen Murphy proposed openstack-infra/puppet-ansible master: Fix template variables for puppet 4  https://review.openstack.org/58569209:39
openstackgerritMerged openstack-infra/system-config master: Add beaker tests for openstack_project::server  https://review.openstack.org/58098509:44
*** shardy has quit IRC09:45
*** wolverineav has joined #openstack-infra09:46
*** e0ne has joined #openstack-infra09:48
*** Tahvok has quit IRC09:50
*** shardy has joined #openstack-infra09:50
*** wolverineav has quit IRC09:51
*** Tahvok has joined #openstack-infra09:51
*** zoli|lunch is now known as zoli09:53
*** wolverineav has joined #openstack-infra09:53
*** e0ne has quit IRC09:57
*** wolverineav has quit IRC09:59
*** alexchadin has quit IRC09:59
*** chuck_ has joined #openstack-infra09:59
*** e0ne has joined #openstack-infra10:00
*** zul has quit IRC10:01
*** vipul has quit IRC10:01
*** vipul has joined #openstack-infra10:05
*** alexchadin has joined #openstack-infra10:08
*** chason has quit IRC10:15
*** yamamoto has quit IRC10:16
*** chason has joined #openstack-infra10:17
*** e0ne has quit IRC10:17
*** chason[m] has joined #openstack-infra10:17
*** e0ne has joined #openstack-infra10:17
*** e0ne has quit IRC10:17
*** e0ne has joined #openstack-infra10:18
*** e0ne has quit IRC10:19
*** pguimaraes has quit IRC10:21
*** d0ugal has quit IRC10:21
*** wolverineav has joined #openstack-infra10:22
*** jamesden_ has joined #openstack-infra10:22
*** kukacz_ has quit IRC10:23
*** rossella_s has quit IRC10:23
*** kukacz_ has joined #openstack-infra10:23
*** lifeless has joined #openstack-infra10:23
*** rossella_s has joined #openstack-infra10:25
*** e0ne has joined #openstack-infra10:25
*** wolverineav has quit IRC10:26
*** wolverineav has joined #openstack-infra10:28
*** e0ne has quit IRC10:29
*** sshnaidm|ruck is now known as sshnaidm|afk10:29
*** kjackal has quit IRC10:30
*** rossella_s has quit IRC10:31
*** wolverineav has quit IRC10:32
*** e0ne has joined #openstack-infra10:33
*** rossella_s has joined #openstack-infra10:33
*** wolverineav has joined #openstack-infra10:33
*** alexchad_ has joined #openstack-infra10:35
*** e0ne has quit IRC10:36
*** alexchad_ has quit IRC10:36
*** alexchadin has quit IRC10:36
*** wolverineav has quit IRC10:37
*** yamamoto has joined #openstack-infra10:38
*** wolverineav has joined #openstack-infra10:39
*** gongysh has quit IRC10:41
*** e0ne has joined #openstack-infra10:43
*** wolverineav has quit IRC10:43
*** e0ne has quit IRC10:44
*** armaan has joined #openstack-infra10:45
*** wolverineav has joined #openstack-infra10:46
*** edmondsw has joined #openstack-infra10:49
*** e0ne has joined #openstack-infra10:49
*** e0ne has quit IRC10:50
*** wolverineav has quit IRC10:50
*** wolverineav has joined #openstack-infra10:52
*** e0ne has joined #openstack-infra10:52
*** ykarel is now known as ykarel|mtg10:52
*** e0ne has quit IRC10:52
*** Guest23519 has joined #openstack-infra10:52
*** ykarel|mtg has quit IRC10:52
Guest23519Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/10:53
Guest23519or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/10:53
Guest23519Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate10:53
Guest23519Voice your opinions at https://webchat.freenode.net/?channels=#freenode10:53
*** Guest23519 has quit IRC10:55
*** armaan has quit IRC10:55
*** ykarel has joined #openstack-infra10:56
*** armaan has joined #openstack-infra10:56
*** armaan has quit IRC10:56
*** wolverin_ has joined #openstack-infra10:59
*** wolverineav has quit IRC11:00
*** sshnaidm|afk is now known as sshnaidm|ruck11:00
*** wolverin_ has quit IRC11:01
*** wolverineav has joined #openstack-infra11:01
*** wolverin_ has joined #openstack-infra11:03
*** wolverineav has quit IRC11:03
*** armaan has joined #openstack-infra11:07
*** wolverin_ has quit IRC11:08
*** alexchadin has joined #openstack-infra11:09
*** alexchadin has quit IRC11:15
openstackgerritColleen Murphy proposed openstack-infra/system-config master: Add beaker tests for firehose  https://review.openstack.org/58569111:18
*** rcernin has quit IRC11:21
*** zhangfei has quit IRC11:26
*** panda is now known as panda|lunch11:26
*** armaan has quit IRC11:27
*** armaan has joined #openstack-infra11:27
*** armaan has quit IRC11:29
*** armaan has joined #openstack-infra11:30
openstackgerritMerged openstack-infra/system-config master: Stop installing puppet with puppet  https://review.openstack.org/58159611:31
*** tpsilva has joined #openstack-infra11:33
*** alexchadin has joined #openstack-infra11:37
*** ccamacho has quit IRC11:38
*** yamamoto has quit IRC11:41
*** yamamoto has joined #openstack-infra11:43
*** ccamacho has joined #openstack-infra11:44
*** kjackal has joined #openstack-infra11:45
*** christel| has joined #openstack-infra11:45
christel|Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/11:45
christel|or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/11:45
christel|Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate11:45
christel|Voice your opinions at https://webchat.freenode.net/?channels=#freenode11:45
*** christel| has quit IRC11:47
*** kjackal has quit IRC11:52
*** kjackal has joined #openstack-infra11:53
*** ykarel has quit IRC11:56
*** kgiusti has joined #openstack-infra11:58
*** trown|outtypewww is now known as trown12:02
*** armaan has quit IRC12:03
*** shardy has quit IRC12:03
*** armaan has joined #openstack-infra12:04
*** jcoufal has joined #openstack-infra12:06
*** armaan has quit IRC12:06
*** e0ne has joined #openstack-infra12:06
*** armaan has joined #openstack-infra12:07
*** wolverineav has joined #openstack-infra12:08
*** armaan has quit IRC12:11
*** wolverineav has quit IRC12:12
*** wolverineav has joined #openstack-infra12:12
*** shardy has joined #openstack-infra12:17
*** lpetrut has joined #openstack-infra12:18
*** lpetrut_ has quit IRC12:18
*** shardy has quit IRC12:18
*** neiloy has joined #openstack-infra12:22
*** shardy has joined #openstack-infra12:22
*** larainema has quit IRC12:24
*** ykarel has joined #openstack-infra12:25
*** dave-mccowan has joined #openstack-infra12:25
*** jaosorior has quit IRC12:27
*** pguimaraes has joined #openstack-infra12:27
*** camunoz has joined #openstack-infra12:30
*** mriedem has quit IRC12:39
*** mriedem has joined #openstack-infra12:41
*** rlandy has joined #openstack-infra12:43
*** jaosorior has joined #openstack-infra12:50
*** panda|lunch is now known as panda12:53
*** eharney has quit IRC12:56
*** r-daneel has quit IRC12:57
*** pcichy has quit IRC12:59
*** rh-jelabarre has quit IRC13:04
*** ssbarnea has quit IRC13:05
*** wolverineav has quit IRC13:06
*** pcichy has joined #openstack-infra13:07
*** Tahvok has quit IRC13:08
*** agopi is now known as agopi|brb13:08
*** wolverineav has joined #openstack-infra13:09
*** rh-jelabarre has joined #openstack-infra13:11
*** Tahvok has joined #openstack-infra13:11
*** rosmaita has joined #openstack-infra13:12
*** agopi|brb has quit IRC13:13
*** armaan has joined #openstack-infra13:18
*** boden has joined #openstack-infra13:21
*** tanhengyeow has joined #openstack-infra13:22
*** boden has quit IRC13:22
*** tanhengyeow has quit IRC13:22
openstackgerritTobias Henkel proposed openstack-infra/zuul master: WIP: Support job pause  https://review.openstack.org/58538913:24
*** armaan has quit IRC13:25
*** graphene has quit IRC13:26
*** samouy| has joined #openstack-infra13:26
samouy|Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/13:26
samouy|or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/13:26
samouy|Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate13:27
samouy|Voice your opinions at https://webchat.freenode.net/?channels=#freenode13:27
*** Guest79333| has joined #openstack-infra13:27
Guest79333|Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/13:27
Guest79333|or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/13:27
Guest79333|Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate13:27
Guest79333|Voice your opinions at https://webchat.freenode.net/?channels=#freenode13:27
*** d0ugal has joined #openstack-infra13:27
*** ykarel has quit IRC13:28
*** graphene has joined #openstack-infra13:28
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Add query for nova functional evacuate anti-affinity bug 1783565  https://review.openstack.org/58574513:28
openstackbug 1783565 in OpenStack Compute (nova) "ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host intermittently fails with "Instance compute service state on host2 expected to be down, but it was up."" [Undecided,New] https://launchpad.net/bugs/178356513:28
*** cmurphy is now known as cmurphy|vacation13:29
*** samouy| has quit IRC13:30
*** Guest79333| has quit IRC13:30
*** armyriad| has joined #openstack-infra13:31
armyriad|Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/13:31
armyriad|or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/13:31
armyriad|Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate13:31
armyriad|Voice your opinions at https://webchat.freenode.net/?channels=#freenode13:31
Shrewssomeone is bored. and sad.13:32
*** yamamoto has quit IRC13:32
*** yamamoto has joined #openstack-infra13:32
*** yamamoto has quit IRC13:32
*** pblaho has joined #openstack-infra13:33
*** dave-mcc_ has joined #openstack-infra13:33
*** ChanServ sets mode: +b armyriad|!*@*13:34
*** armyriad| was kicked by ChanServ (Banned: spamming is off-topic)13:34
*** dave-mccowan has quit IRC13:34
*** dave-mccowan has joined #openstack-infra13:35
*** efried_pto is now known as efried13:36
*** dave-mcc_ has quit IRC13:38
*** rpioso|afk is now known as rpioso13:40
*** agopi has joined #openstack-infra13:41
*** graphene has quit IRC13:42
*** graphene has joined #openstack-infra13:43
*** graphene has quit IRC13:44
*** d0ugal has quit IRC13:45
*** graphene has joined #openstack-infra13:46
*** zhipeng has joined #openstack-infra13:46
*** jchhatbar has joined #openstack-infra13:53
*** graphene has quit IRC13:53
*** graphene has joined #openstack-infra13:55
*** graphene has quit IRC13:55
*** r-daneel has joined #openstack-infra13:55
*** janki has quit IRC13:56
*** graphene has joined #openstack-infra13:56
*** jamesmcarthur has quit IRC13:59
*** gongysh has joined #openstack-infra14:00
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Update query for bug 1770179  https://review.openstack.org/58575614:01
openstackbug 1770179 in Cinder "GroupSnapshotsV319Test.test_reset_group_snapshot_status intermittently fails waiting to reset status to 'creating'" [High,Confirmed] https://launchpad.net/bugs/177017914:01
*** jamesmcarthur has joined #openstack-infra14:06
*** felipemonteiro has joined #openstack-infra14:08
*** michagogo| has joined #openstack-infra14:09
michagogo|Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate14:09
michagogo|Voice your opinions at https://webchat.freenode.net/?channels=#freenode14:10
*** michagogo| has quit IRC14:10
AJaegernot again ;(14:10
*** yamamoto has joined #openstack-infra14:10
* TheJulia sighs14:12
*** hamzy_ has quit IRC14:13
*** iyamahat has quit IRC14:15
*** armaan has joined #openstack-infra14:17
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Fix zuul reporting build failure with only non-voting jobs  https://review.openstack.org/58499014:17
*** hongbin has joined #openstack-infra14:17
*** yamamoto has quit IRC14:18
*** ssbarnea has joined #openstack-infra14:19
*** armaan_ has joined #openstack-infra14:19
*** armaan has quit IRC14:21
*** bdodd has quit IRC14:22
*** yamamoto has joined #openstack-infra14:25
*** eharney has joined #openstack-infra14:25
*** eernst has joined #openstack-infra14:27
*** linuxmodder| has joined #openstack-infra14:28
*** linuxmodder| has quit IRC14:30
*** armaan_ has quit IRC14:31
*** armaan has joined #openstack-infra14:32
*** armaan has quit IRC14:32
*** links has quit IRC14:33
*** felipemonteiro has quit IRC14:33
*** ginopc has joined #openstack-infra14:36
*** jamesmcarthur has quit IRC14:36
*** yamamoto has quit IRC14:37
*** armaan has joined #openstack-infra14:37
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Updated permissions for endpoint summit/all  https://review.openstack.org/58576414:38
*** hamzy_ has joined #openstack-infra14:38
*** hamzy_ is now known as hamzy14:38
openstackgerritMerged openstack-infra/openstackid-resources master: Updated permissions for endpoint summit/all  https://review.openstack.org/58576414:39
*** hongbin has quit IRC14:40
*** graphene has quit IRC14:41
*** armaan has quit IRC14:42
*** graphene has joined #openstack-infra14:42
*** boden has joined #openstack-infra14:43
openstackgerritMerged openstack-infra/elastic-recheck master: Add query for nova functional evacuate anti-affinity bug 1783565  https://review.openstack.org/58574514:45
openstackbug 1783565 in OpenStack Compute (nova) "ServerGroupTestV21.test_evacuate_with_anti_affinity_no_valid_host intermittently fails with "Instance compute service state on host2 expected to be down, but it was up."" [Undecided,New] https://launchpad.net/bugs/178356514:45
*** udesale has quit IRC14:48
*** gongysh has quit IRC14:48
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an OpenShift resource provider  https://review.openstack.org/57066714:48
*** jamesmcarthur has joined #openstack-infra14:48
*** jamesmcarthur has quit IRC14:49
*** jamesmcarthur has joined #openstack-infra14:49
*** jrist has quit IRC14:49
*** ykarel has joined #openstack-infra14:50
*** pcaruana has quit IRC14:52
*** dhajare has quit IRC14:53
*** lpetrut has quit IRC14:54
*** ykarel is now known as ykarel|away14:55
*** mriedem is now known as mriedem_away14:57
stephenfinAfternoon. Anything we can do about the random spam we're seeing on #openstack-nova?14:58
*** NyanCat_ has joined #openstack-infra14:59
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Fix zuul reporting build failure with only non-voting jobs  https://review.openstack.org/58499014:59
*** yamamoto has joined #openstack-infra14:59
*** NyanCat_ has quit IRC15:00
clarkbstephenfin: ops can ban people but tey appear to be coming from different IPs and using different nicks so may be a losing battle15:00
stephenfinclarkb: Ah, guess we'll just wait it out, in that case. Thanks for the info15:00
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-base-jobs master: Add base openshift job  https://review.openstack.org/57066915:01
*** graphene has quit IRC15:01
*** armaan has joined #openstack-infra15:02
*** graphene has joined #openstack-infra15:03
*** dhajare has joined #openstack-infra15:04
pabelangerjohn_studarus: clarkb: seem to be getting a few exceptions in packethost today, when we removed the compute node did we reduce max-servers?15:07
openstackgerritPaul Belanger proposed openstack-infra/nodepool master: Use hostname for launch exception  https://review.openstack.org/58578115:07
clarkbpabelanger: we did not but are oversubscribed anyway so not sure how big a difference it makes (memory is probably the thing to consider?)15:07
*** r-daneel has quit IRC15:07
*** r-daneel has joined #openstack-infra15:08
*** bandini has quit IRC15:09
*** StephenS28 has joined #openstack-infra15:09
StephenS28Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/15:09
StephenS28or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/15:09
StephenS28Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate15:09
StephenS28<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>15:09
StephenS28This message was brought to you by Private Internet Access15:09
*** ChanServ sets mode: +b StephenS28!*@*15:10
*** StephenS28 was kicked by ChanServ (Banned: spamming)15:10
*** bandini has joined #openstack-infra15:10
*** pguimaraes has quit IRC15:11
*** graphene has quit IRC15:13
*** pguimaraes has joined #openstack-infra15:14
*** armaan has quit IRC15:14
*** graphene has joined #openstack-infra15:14
*** armaan has joined #openstack-infra15:14
*** armaan has quit IRC15:14
*** jrist has joined #openstack-infra15:15
*** jrist has quit IRC15:15
*** jrist has joined #openstack-infra15:15
*** cshastri has quit IRC15:16
*** r-daneel has quit IRC15:19
*** nlsun5 has joined #openstack-infra15:22
nlsun5Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/15:22
nlsun5or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/15:23
nlsun5Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate15:23
nlsun5<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>15:23
nlsun5This message was brought to you by Private Internet Access15:23
*** emilsp25 has joined #openstack-infra15:23
emilsp25Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/15:23
emilsp25or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/15:23
emilsp25Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate15:23
emilsp25<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>15:23
emilsp25This message was brought to you by Private Internet Access15:23
*** larainema has joined #openstack-infra15:24
*** emilsp25 has quit IRC15:24
*** zhipeng has quit IRC15:25
*** graphene has quit IRC15:25
*** nlsun5 has quit IRC15:26
*** graphene has joined #openstack-infra15:27
*** r-daneel has joined #openstack-infra15:27
clarkbrax-iad mirror appears unhappy15:28
*** gyee has joined #openstack-infra15:29
*** armaan has joined #openstack-infra15:29
silehthi guys, there is a way to have a tox py37 target on gate ?15:29
*** erlon_ has joined #openstack-infra15:29
clarkbssh is listening but auth negotiation hangs15:30
pabelangersileht: fedora should have python3715:31
clarkbI'm going to reboot it with nova api15:31
*** quiquell has quit IRC15:33
clarkbpabelanger: not on 28, it is 3.615:33
pabelangerclarkb: yah, think 3.6 is default, but python37 has an rpm15:33
openstackgerritOliver Walsh proposed openstack/diskimage-builder master: modprobe DIB_MODPROBE_BLACKLIST should be optional  https://review.openstack.org/58578415:33
clarkbah, well if that is the case fedora package search doesn't seem to pull it up15:34
clarkbit is 3.7 on rawhide though15:34
silehtOk I will try with the fedora rpm15:35
clarkbsileht: one option is to use pyenv but no existing tox jobs do that that I know of15:35
silehtthanks15:35
pabelangeryah, I haven't tried python37 myself15:35
pabelangerbut do see it in yum locally15:35
openstackgerritOliver Walsh proposed openstack/diskimage-builder master: Add DIB element to blacklist nouveau  https://review.openstack.org/57177715:36
EmilienMpabelanger: can you make me +o on #tripleo please15:36
EmilienMwe have some spammers15:37
clarkbrax iad mirror seems to be working again after a reboot15:37
clarkblogstash says it was the only unhappy mirror15:38
clarkbEmilienM: they seem to be all over and coming with different IPs and nicks, not sure how effective bans will be but you can try15:38
pabelangerEmilienM: done15:38
clarkblet us know if it helps15:39
pabelangeryah, I've see freenode staff already banning some too15:39
clarkbEmilienM: also you already had op in that channel looks like15:40
clarkbEmilienM               +AVefortv (admin) [modified 43w 1d 8h ago]15:40
EmilienMclarkb: thank you15:41
EmilienMpabelanger: ^ ty15:41
pabelangerEmilienM: clarkb: https://docs.openstack.org/infra/system-config/irc.html for commands! :)15:42
*** eernst has quit IRC15:43
EmilienMthanks15:43
Shrewsspeaking of that, i noticed lots of folks with ops for #-infra that have moved on (or i just don't recognize)15:44
Shrewsperhaps that should be cleaned up15:44
*** agopi is now known as agopi|food15:46
*** kei-ichi has joined #openstack-infra15:46
*** sshnaidm|ruck is now known as sshnaidm15:46
*** panda is now known as panda|rover15:46
*** rfolco is now known as rfolco|ruck15:46
clarkbya our default ops list configured by accessbot could probably be trimmed (and possibly add some new volunteers15:47
pleia2Shrews: you can propose a patch against project-config in accessbot/channels.yaml15:47
*** alexchadin has quit IRC15:48
pleia2(and yes, I can probably be removed :))15:49
*** alexchadin has joined #openstack-infra15:50
*** yamahata has quit IRC15:50
*** eernst has joined #openstack-infra15:51
*** Guest48767 has joined #openstack-infra15:53
Guest48767Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/15:53
Guest48767or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/15:53
Guest48767Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate15:53
Guest48767<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>15:53
Guest48767This message was brought to you by Private Internet Access15:53
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Set image_format to raw for packethost  https://review.openstack.org/58580315:53
*** ChanServ sets mode: +b Guest48767!*@*15:53
*** Guest48767 was kicked by ChanServ (Banned: spamming)15:53
openstackgerritDavid Shrewsbury proposed openstack-infra/project-config master: Cleanup IRC access  https://review.openstack.org/58580415:54
pabelangerclarkb: john_studarus: ^ is a follow up to discussion yesterday about packethost and using raw images in nodepool.15:54
*** yamamoto has quit IRC15:55
*** sshnaidm is now known as sshnaidm|bbl15:56
*** high_fiver has joined #openstack-infra15:56
high_fiverHey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/15:56
high_fiveror maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/15:56
high_fiverRead what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate15:57
high_fiver<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>15:57
high_fiverThis message was brought to you by Private Internet Access15:57
*** graphene has quit IRC15:57
*** high_fiver has quit IRC15:57
*** panda|rover is now known as panda|rover|off15:57
*** dtruong2 has joined #openstack-infra15:57
*** armaan has quit IRC15:58
*** graphene has joined #openstack-infra15:58
*** dave-mccowan has quit IRC15:58
*** armaan has joined #openstack-infra15:58
*** armaan has quit IRC15:59
*** dave-mccowan has joined #openstack-infra16:00
*** armaan has joined #openstack-infra16:00
openstackgerritMerged openstack-infra/elastic-recheck master: Update query for bug 1770179  https://review.openstack.org/58575616:00
openstackbug 1770179 in Cinder "GroupSnapshotsV319Test.test_reset_group_snapshot_status intermittently fails waiting to reset status to 'creating'" [High,Confirmed] https://launchpad.net/bugs/177017916:00
*** submain has joined #openstack-infra16:03
submainHey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/16:03
submainor maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/16:03
submainRead what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate16:03
submain<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>16:03
submainThis message was brought to you by Private Internet Access16:03
*** submain has quit IRC16:03
pabelangerout in force today16:03
*** submain23 has joined #openstack-infra16:03
submain23Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/16:04
submain23or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/16:04
submain23Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate16:04
pabelangerShrews: mind reviewing 58580316:04
submain23<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>16:04
submain23This message was brought to you by Private Internet Access16:04
*** submain23 has quit IRC16:04
Shrewspabelanger: done16:05
clarkblooks like tumbleweed is due to have python3 soon too, the factory package was updated, now just has to get through testing16:05
clarkber python3716:05
*** ginux has joined #openstack-infra16:05
*** ginopc has quit IRC16:07
*** ginux is now known as ginopc16:07
*** Tengu has joined #openstack-infra16:07
tristanCclarkb: prometheanfire is about to send an OSSA to openstack-announce, would you have the permission to white-list his address?16:08
*** graphene has quit IRC16:08
*** myoung is now known as myoung|lunch16:08
clarkbtristanC: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-announce says ttx is the list admin16:08
clarkbgenerally we try to not override or go around existing service admins/managers.16:08
*** pguimaraes has quit IRC16:09
tristanCclarkb: yes but he seems afk since a few days already16:09
clarkbya I think he is traveling16:09
prometheanfireclarkb: it'd by my gentoo address16:09
*** graphene has joined #openstack-infra16:09
pabelangerShrews: thanks!16:09
clarkbprometheanfire: tristanC I can probably figure out how to send it via command line on the host let me take a look16:10
*** iyamahat has joined #openstack-infra16:11
tristanCclarkb: thank you16:12
clarkbhrm docs don't seem to indicate you can moderate fro mcommand line. I would have to add myself as admin then do it from webui?16:12
clarkbcorvus: ^ you may know16:12
*** ccamacho has quit IRC16:12
prometheanfirenot sure if troll, can someone (who's been idling in -requirements) kick (or kb) Martha?16:15
prometheanfirecan provide logs if needed16:15
*** armaan has quit IRC16:16
*** armaan has joined #openstack-infra16:16
clarkbprometheanfire: is it the blog by freenode staff stuff?16:16
*** agopi|food is now known as agopi16:17
prometheanfirepossibly, let me check the blog16:17
*** ykarel|away has quit IRC16:18
*** ykarel|away has joined #openstack-infra16:18
prometheanfireI can't find what you are talking about16:18
prometheanfireit's just a bunch of nonsense, like someone ranting at nothing16:19
*** rosmaita has quit IRC16:19
clarkbya thats different16:19
clarkbprometheanfire: fwiw you do have op on that channel16:20
* prometheanfire goes to remember how to kb someone16:20
prometheanfireclarkb: thanks :P16:20
clarkbprometheanfire: https://docs.openstack.org/infra/system-config/irc.html#banning-disruptive-users may be helpful16:21
prometheanfirecool16:21
*** ForexTrader has joined #openstack-infra16:21
ForexTraderHey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/16:21
ForexTraderor maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/16:22
ForexTraderRead what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate16:22
*** efried is now known as efried_rollin16:22
ForexTrader<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>16:22
ForexTraderThis message was brought to you by Private Internet Access16:22
*** rosmaita has joined #openstack-infra16:22
prometheanfireclarkb: seemed to work16:22
*** rorx2 has joined #openstack-infra16:24
rorx2Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/16:24
*** chkumar|rover is now known as chandankumar16:24
rorx2or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/16:24
rorx2Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate16:25
*** ForexTrader has quit IRC16:25
rorx2<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>16:25
rorx2This message was brought to you by Private Internet Access16:25
persiaNote:  the most recent traffic reached about 150Hz between GIMPnet and Matrix about 15 hours ago.  If someone can stomp it fast, that might be ideal.16:25
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: Add upload-logs-swift role  https://review.openstack.org/58454116:26
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: Add a python unit test framework  https://review.openstack.org/58581316:26
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: Restore zuul_swift_upload.py script  https://review.openstack.org/58581416:26
*** alexchadin has quit IRC16:26
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: DNM: Run tox-py35 job  https://review.openstack.org/58581516:26
*** gfidente has quit IRC16:26
*** rorx2 has quit IRC16:26
*** alexchadin has joined #openstack-infra16:26
*** alexchadin has quit IRC16:27
*** dhajare has quit IRC16:27
clarkbpersia: it is coming from different IPs not sure we'll be able to stomp it ourselves?16:27
*** alexchadin has joined #openstack-infra16:27
*** jpich has quit IRC16:27
*** alexchadin has quit IRC16:27
*** alexchadin has joined #openstack-infra16:28
*** zoli is now known as zoli|gone16:28
*** alexchadin has quit IRC16:28
persiaclarkb: Understood.  I don't currently have standing with staff, so more want to raise it isn't necessarily a one-off message if no action is taken.16:28
*** yamahata has joined #openstack-infra16:28
*** alexchadin has joined #openstack-infra16:29
persiaIf it's been reported, presumably people prepared to deal (and with both standing and access) are sorting it.16:29
*** alexchadin has quit IRC16:29
*** bobh has joined #openstack-infra16:29
clarkbpersia: freenode staff have been kicking people16:30
clarkbwe could require voice temporarily and voice everyone, but that seems problematic too16:31
clarkb(new users won't be able to talk)16:31
prometheanfireiirc I'm a registered contact with them, but just for gentoo stuff16:31
pabelangerwhat was the command dmsimard ran last time, we had a bot attack. Was it users needed to be register with freenode?16:32
*** ChanServ sets mode: +o corvus16:32
dmsimardyeah, it prevented people unregistered with nickserv from joining the channels16:32
pabelangerhttps://specs.openstack.org/openstack-infra/infra-specs/specs/irc.html is how we discussed making it better, but haven't started work on it yet16:32
persiaIf staff is on it, I think no action is necessary.  I remember staff being quick and effective when I was a contact.  I only raised it because that particular text ended up causing gateway traffic issues for other networks last night.16:32
*** Sigyn has joined #openstack-infra16:32
prometheanfirewe mark our channels +cnt for requiring registration16:32
persiaI think the +t is unrelated (as I believe that only controls who can set /topic)16:33
prometheanfireprobably, I just copypasta'd it16:33
*** ginopc has quit IRC16:33
prometheanfireit looks like reqs is already cnt anyway16:34
prometheanfiremaybe it's cnrt (the r being registered?)16:34
corvusi have invited Sigyn to the channel to help16:34
*** yamamoto has joined #openstack-infra16:34
corvusany other channels to which we should invite her?16:34
prometheanfireya, r Prevents users who are not identified to services from joining the channel.16:34
clarkb'submain (~submain@207.38.164.133) has quit (Killed (Sigyn (Spam is off topic on freenode.)))' they seem to be on top of it or at least trying16:35
clarkbcorvus: tripleo, nova, and I'm sure others have been affected too16:35
pabelangercorvus: ah, okay. I've ask tripleo not to, but mostly wanted to first run it by here16:35
pabelangerfirst*16:35
corvuspabelanger: i don't understand, can you rephrase?16:35
clarkbcorvus: re prometheanfire's list moderation requiest, looks like `MAILMAN_SITE_DIR=/srv/mailman/openstack /usr/lib/mailman/bin/change_pw -l openstack-announce@lists.openstack.org -p somesecret` is my option for letting an email through if ttx is afk?16:36
clarkbcorvus: pabelanger do you mean forcing registered users only?16:36
pabelangercorvus: sorry, tripleo did invite Sigyn to their channel about 30mins ago, but asked them not, which out first checking here. Mostly wanted to make sure we were okay with other bots operating on channels first16:36
pabelangerugh16:37
pabelangers/which/with/16:37
corvuspabelanger: i still don't understand the words you are saying.  "who asked them not"  what does that mean?16:37
pabelangeryes, not doing good this moring. I asked tripleo not to invite sigyn for their channel.16:38
corvusoh, ok16:38
pabelangerbut will indicate they can16:38
corvusi think that's fine.  Sigyn is run by freenode and is rather special16:38
pabelangerack16:38
corvusbit like having ChanServ around, if you ask me16:38
pabelangerrelaying message now16:38
clarkbcorvus: but before I got changing the list admin password on ttx I figured I would double check there wasn't another option there16:39
corvusi think it's helpful both for the people in channel (as she is rather fast), and also helpful for freenode, as I imagine she gets more data that way16:39
corvusclarkb: you could use the mailman site password16:39
corvusclarkb: should be recorded in the password store16:39
clarkbcorvus: ah and I can use that to log into the list and moderate it? eg it checks both list and site passwords?16:40
corvusclarkb: lets you log in as admin for any list16:40
corvusyep16:40
clarkbTIL16:40
clarkbthat is better than changing the pw on ttx too16:40
pabelangerclarkb: I wasn't sure if that is what we did temporarily last bot spamming (registering)16:40
corvusyeah, though not as "fun"16:41
*** e0ne has quit IRC16:41
clarkbpabelanger: ya, I'm wary about that (or requiring voice) simply because so many of our users are unregistered and come and go (so may not get voice)16:41
*** dtruong2 has quit IRC16:42
*** iyamahat has quit IRC16:42
*** ramishra has quit IRC16:42
*** corvus sets mode: -o corvus16:42
*** iyamahat has joined #openstack-infra16:42
*** yamamoto has quit IRC16:43
pabelangerclarkb: agree, mostly wanted to understand again what dmsimard set last time16:43
*** pk1210 has joined #openstack-infra16:43
pk1210Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/16:43
*** pk1210 has quit IRC16:43
corvusSigyn joined this channel and nova immediately.  tripleo reeds a staff review.16:45
corvuss/reeds/needs/16:46
*** bobh has quit IRC16:46
clarkbprometheanfire: hsa the email been sent and is waiting moderation now?16:48
clarkbprometheanfire: if not let me know when that happens and I'll take the site passwd route to let it through16:49
*** iyamahat has quit IRC16:49
*** iyamahat has joined #openstack-infra16:49
prometheanfireno16:50
prometheanfirewaiting on the doc job to finish on https://review.openstack.org/#/c/585805/16:50
*** graphene has quit IRC16:52
*** iyamahat has quit IRC16:53
*** graphene has joined #openstack-infra16:53
*** tesseract has quit IRC16:53
*** iyamahat has joined #openstack-infra16:54
*** kjackal has quit IRC16:54
*** chuck_ is now known as zul16:55
*** trown is now known as trown|lunch16:55
*** kjackal has joined #openstack-infra16:55
*** alexchadin has joined #openstack-infra17:00
clarkbI've removed groups.o.o from the emergency file now and am waiting for it to run puppet with the puppet4 parser17:00
*** graphene has quit IRC17:00
*** graphene has joined #openstack-infra17:02
*** alexchadin has quit IRC17:06
*** Nil_ has quit IRC17:06
openstackgerritJames E. Blair proposed openstack-infra/project-config master: Run tox-py35 on changes to zuul-jobs  https://review.openstack.org/58582917:09
openstackgerritJames E. Blair proposed openstack-infra/project-config master: Add a base job to test storing logs in swift  https://review.openstack.org/58583017:09
corvusclarkb, Shrews: ^17:09
Shrews*nod*17:09
*** ykarel|away has quit IRC17:11
*** dtantsur is now known as dtantsur|afk17:13
*** iyamahat has quit IRC17:13
*** iyamahat has joined #openstack-infra17:14
*** Xlbrag_21 has joined #openstack-infra17:15
openstackgerritPaul Belanger proposed openstack-infra/nodepool master: Use hostname for launch exception  https://review.openstack.org/58578117:15
Xlbrag_21Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/17:15
*** Xlbrag_21 has quit IRC17:15
*** sshnaidm|bbl is now known as sshnaidm17:16
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: Add upload-logs-swift role  https://review.openstack.org/58454117:16
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: DNM: Run tox-py35 job  https://review.openstack.org/58581517:16
*** Sheraf has joined #openstack-infra17:19
clarkbok groups.o.o ran with new puppet parser. The config file updated removing the redundant cron_key entry as expected17:19
SherafHey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/17:19
*** Sheraf has quit IRC17:19
*** leachim69 has joined #openstack-infra17:23
*** high_fiver has joined #openstack-infra17:23
high_fiverHey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/17:23
*** high_fiver has quit IRC17:23
*** annieslmaos8 has joined #openstack-infra17:23
annieslmaos8Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/17:23
*** annieslmaos8 has quit IRC17:23
*** shardy has quit IRC17:24
*** leachim69 has quit IRC17:25
*** iyamahat has quit IRC17:33
*** iyamahat has joined #openstack-infra17:34
*** jamesmcarthur has quit IRC17:38
*** alexchadin has joined #openstack-infra17:41
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Default to 0o022 for python-daemon  https://review.openstack.org/58506817:42
*** felipemonteiro has joined #openstack-infra17:45
*** alexchadin has quit IRC17:46
*** Frosty has joined #openstack-infra17:47
FrostyHey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/17:47
Frostyor maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/17:47
FrostyRead what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate17:47
*** Frosty has quit IRC17:47
openstackgerritMonty Taylor proposed openstack-infra/system-config master: WIP Move base activities into ansible  https://review.openstack.org/58583617:48
mordredclarkb: there's a very first stab at starting that work17:48
*** rh-jelabarre has quit IRC17:50
*** rh-jelabarre has joined #openstack-infra17:51
*** trown|lunch is now known as trown17:53
pabelangermordred: exciting!17:53
*** ykarel|away has joined #openstack-infra17:53
*** myoung|lunch is now known as myoung17:53
mordredpabelanger: yeah 0- it wasn't super bad to write - main challenge is thinking through sequence vs. dag :)17:55
* mordred afks for a little bit17:55
*** e0ne has joined #openstack-infra17:57
*** jamesmcarthur has joined #openstack-infra17:59
*** jamesmcarthur has quit IRC17:59
*** jamesmcarthur has joined #openstack-infra17:59
openstackgerritMerged openstack-infra/zuul master: Fix zuul reporting build failure with only non-voting jobs  https://review.openstack.org/58499018:00
pabelangerGem::InstallError: net-telnet requires Ruby version >= 2.3.0.18:04
pabelangerlooks like new puppet jobs failing18:04
pabelangerguess we need to cap18:04
*** jamesmcarthur has quit IRC18:04
pabelangeryah, 0.2.0 released 7 hours ago18:05
clarkbwe've had to do that for a few gem libs18:05
pabelangerhttps://github.com/ruby/net-telnet/releases18:05
clarkbshould be in the helper repo that installs everything else iirc18:05
*** markvoelker_ has joined #openstack-infra18:05
prometheanfireclarkb: sent to the list btw18:05
clarkbprometheanfire: ok, looking for it now18:05
*** markvoelker has quit IRC18:06
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Added endpoint Create My Speaker  https://review.openstack.org/58583718:07
openstackgerritPaul Belanger proposed openstack-infra/puppet-openstack_infra_spec_helper master: Cap net-telnet < 0.2.0 for ruby 1.9 support  https://review.openstack.org/58583818:08
*** pguimaraes has joined #openstack-infra18:08
*** cottongin2325 has joined #openstack-infra18:08
cottongin2325Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/18:08
*** cottongin2325 has quit IRC18:08
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Set image_format to raw for packethost  https://review.openstack.org/58580318:09
*** rcernin has joined #openstack-infra18:09
clarkbprometheanfire: done18:10
prometheanfirety18:10
*** iyamahat has quit IRC18:10
openstackgerritMerged openstack-infra/openstackid-resources master: Added endpoint Create My Speaker  https://review.openstack.org/58583718:11
*** thevdude17 has joined #openstack-infra18:17
thevdude17Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/18:17
thevdude17or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/18:17
*** thevdude17 has quit IRC18:17
*** dave-mcc_ has joined #openstack-infra18:19
*** ykarel|away has quit IRC18:19
*** e0ne has quit IRC18:20
*** dave-mccowan has quit IRC18:20
pabelangerclarkb: corvus: with 584990 merged for zuul, should we consider restarting all zuul services to pick up latest release before a potential tag?18:22
*** camunoz has quit IRC18:22
*** rodrigod` is now known as rodrigods18:22
*** jamesmcarthur has joined #openstack-infra18:22
*** rodrigods has quit IRC18:22
*** rodrigods has joined #openstack-infra18:22
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: Add upload-logs-swift role  https://review.openstack.org/58454118:23
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: DNM: Run tox-py35 job  https://review.openstack.org/58581518:23
pabelangerze11.o.o is also still crushing it for builds :)18:23
corvuspabelanger: yes18:24
pabelangerokay, I can help here after I get some food18:24
*** mriedem_away is now known as mriedem18:28
*** quiquell has joined #openstack-infra18:29
*** jamesmcarthur has quit IRC18:34
*** jamesmcarthur has joined #openstack-infra18:37
*** quiquell has quit IRC18:39
*** e0ne has joined #openstack-infra18:43
*** dtruong_ has joined #openstack-infra18:45
*** cloe26 has joined #openstack-infra18:48
cloe26Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/18:48
*** cloe26 has quit IRC18:48
*** tomaw has joined #openstack-infra18:48
*** dtruong has quit IRC18:49
*** e0ne has quit IRC18:50
*** labviking has joined #openstack-infra18:50
*** labviking has quit IRC18:51
*** kazsh has quit IRC18:52
*** kazsh has joined #openstack-infra18:52
*** graphene has quit IRC18:53
*** rcernin has quit IRC18:53
*** kota_ has quit IRC18:54
*** ktsuyuzaki has joined #openstack-infra18:54
*** jamesmcarthur has quit IRC18:54
*** graphene has joined #openstack-infra18:54
*** jamesmcarthur has joined #openstack-infra18:56
clarkbpabelanger: corvus I too must go do the lunch thing and pick up CSA veggies. The one thing I'd be careful with restarting is that we have some very long queues18:59
*** Welcome has joined #openstack-infra19:03
WelcomeHey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/19:03
*** Welcome has quit IRC19:03
*** dklyle has quit IRC19:03
*** graphene has quit IRC19:06
*** graphene has joined #openstack-infra19:07
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: Add upload-logs-swift role  https://review.openstack.org/58454119:07
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: DNM: Run tox-py35 job  https://review.openstack.org/58581519:07
*** e0ne has joined #openstack-infra19:07
*** rcernin has joined #openstack-infra19:09
*** d0ugal has joined #openstack-infra19:11
*** myoung is now known as myoung|bbl19:12
*** graphene has quit IRC19:13
*** graphene has joined #openstack-infra19:14
openstackgerritMerged openstack-infra/python-storyboardclient master: Switch to using stestr  https://review.openstack.org/57147719:15
*** rcernin has quit IRC19:16
pabelangerclarkb: yah, time of year I guess19:17
pabelangerI'm ready when others are19:17
openstackgerritTobias Henkel proposed openstack-infra/zuul master: WIP: Support job pause  https://review.openstack.org/58538919:23
*** eharney has quit IRC19:26
*** graphene has quit IRC19:26
*** graphene has joined #openstack-infra19:28
prometheanfirehow often are docs updated? https://security.openstack.org/19:35
pabelangerclarkb: corvus: just check with release team (smcginnis) and preference would be to wait until after the deadline crunch is over. However, if we think we should land it, they are willing to hold off on releases for the moment19:36
pabelangers/land it/restart zuul19:36
*** efried_rollin is now known as efried19:38
*** alexchadin has joined #openstack-infra19:42
openstackgerritMerged openstack-infra/nodepool master: Use hostname for launch exception  https://review.openstack.org/58578119:43
*** jamesmcarthur has quit IRC19:44
*** jamesmcarthur has joined #openstack-infra19:44
*** linuxdaemon23 has joined #openstack-infra19:46
linuxdaemon23Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/19:46
linuxdaemon23or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/19:46
*** linuxdaemon23 has quit IRC19:46
*** d0ugal has quit IRC19:46
*** alexchadin has quit IRC19:47
*** erlon_ has quit IRC19:49
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: Add a dequeue command to zuul client  https://review.openstack.org/9503519:51
*** felipemonteiro_ has joined #openstack-infra19:52
*** slaweq has quit IRC19:55
*** felipemonteiro has quit IRC19:56
corvuspabelanger: i'm in no rush19:59
*** slaweq has joined #openstack-infra20:00
edmondswcan someone tell me why https://review.openstack.org/#/c/582404/ won't merge though it has zuul +2 ?20:06
*** rosmaita has quit IRC20:09
*** rosmaita has joined #openstack-infra20:10
*** jchhatbar has quit IRC20:11
*** MrElendig15 has joined #openstack-infra20:13
MrElendig15Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/20:13
MrElendig15or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/20:13
*** MrElendig15 has quit IRC20:13
*** bobh has joined #openstack-infra20:16
*** alexchadin has joined #openstack-infra20:17
*** alexchadin has quit IRC20:17
clarkbedmondsw: its parent is an old patchset of another change and old patchsets can never merge so its child cant either20:21
*** jamesde__ has joined #openstack-infra20:22
*** jamesden_ has quit IRC20:22
edmondswclarkb ah, so it needs a rebase... but shouldn't this have been flagged somehow in the review?20:23
openstackgerritDavid Shrewsbury proposed openstack-infra/puppet-nodepool master: Update nodepool launcher logging  https://review.openstack.org/58587620:25
*** edmondsw has quit IRC20:28
pabelanger#status log enqueued 585839 into gate to help fix tripleo queue20:28
openstackstatuspabelanger: finished logging20:28
*** edmondsw has joined #openstack-infra20:28
clarkbedmondsw: I think it usually puts a little orange dot next to the parent sha1 to indicate it is out of date. I wonder if that goes away if the parent change merges20:31
clarkbnot that its a very visible indicator either way20:31
*** jamesmcarthur has quit IRC20:31
*** jamesmcarthur has joined #openstack-infra20:32
*** edmondsw has quit IRC20:32
Shrewspabelanger: so for v3 world, builder-logging.conf seems to get generated from system-config, and launcher-logging.conf from puppet-nodepool20:33
*** auristor has quit IRC20:34
clarkbpabelanger: smcginnis we actually appear to have caught up on the job backlog while I had lunch. Only ~500 node requests in flight instead of ~1.5k. Maybe we can restart when smcginnis EODs if that is an indication that release work is done for a while?20:36
*** jamesmcarthur has quit IRC20:36
smcginnisclarkb: I suppose later would be fine.20:38
smcginnisclarkb: tonyb is just coming online, but if we know not to approve anything for awhile that should be OK.20:38
*** wolverineav has quit IRC20:38
smcginnisHonestly still feel a little uncomfortable making zuul changes in the middle of the deadline week, but I can keep my fingers crossed.20:39
*** akhilaki has joined #openstack-infra20:40
Shrewsclarkb: pabelanger: so, i'm no puppet expert, but is http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp#n1044 overriding the builder config with system-config's version over what is given in http://git.openstack.org/cgit/openstack-infra/puppet-nodepool/tree/manifests/builder.pp#n24 (from puppet-nodepool)?20:41
*** jcoufal has quit IRC20:41
*** yamamoto has joined #openstack-infra20:42
clarkbShrews: it was when we used nodepool.o.o for nodepool things (now it is just zookeeper), the nl and nb node defs below owuld be what the current servers get and I don't see them overriding20:43
mordredsmcginnis: I think one of my life goals is to reach the point where we can make frequent zuul changes during a deadline week and you never notice20:43
smcginnismordred: I like that goal.20:43
smcginnisCall me a little gun shy yet though. ;)20:43
Shrewsclarkb: oh, 1st link was wrong. should be http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp#n118120:43
*** agopi is now known as agopi|brb20:44
clarkbShrews: yup that looks like it, but I don't see it for the launchers only the builders20:44
mordredsmcginnis: oh - me too - and we still have *known* todo list tasks before we can do that- we don't even know what we don't know yet- so it **probably** won't be next cycle20:44
clarkbwhich makes sense since we do the image specific log configs for image builds20:44
openstackgerritMerged openstack-infra/zuul-jobs master: Add a python unit test framework  https://review.openstack.org/58581320:45
Shrewsclarkb: ok, this is likely the source of (my) confusion. i'll see about cleaning this up tomorrow20:45
clarkbcorvus: outlook.com (and apparently cern.ch as a result) do not like lists.o.o 'Access denied, banned sending IP [50.56.173.222]. To request removal from this list please visit https://sender.office.com/ and follow the directions.' I'm checking if we are on spamhaus for some reason20:45
smcginnismordred: Still a good goal to shoot for. The CI/CD platform that does its own CI/CD.20:45
clarkbI think it likely they are mad about how we violate dmarc because we run mailman20:45
mordredclarkb: what do we violate?20:46
clarkbya spamhaus says we are good. Should I bother going their their whitelist request form link above?20:46
clarkbmordred: we send email for domains not our own (its how mailman and mailing lists work) and that violates what dmarc intends to guard again20:46
clarkb*against20:46
mordredah. that20:47
mordredthe whole thing where gmail and yahoo got together and broke email20:47
clarkbya that20:47
clarkbexcept gmail still works oddly enough but yahoo and outlook do not20:47
*** yamamoto has quit IRC20:47
clarkbnewer mailman lets us spoof things just for endpoints that enforce dmarc20:48
clarkbwhich is probably as reasonable a compromise we can reach but I don't think we can do anything like that with the version we are currnetly running20:48
mordredthe ornery part of me wants to not do anything about it and tell anyone who is upset to go get an email provider that isn't contributing to the breaking of the internet20:48
*** agopi|brb has quit IRC20:48
corvusclarkb: whoah wait20:48
clarkbmordred: ya thats sort of how I feel about it too20:48
corvusdmarc is way different than an ip being banned20:48
clarkbcorvus: yes, I know, but I think microsoft may have baneed us due to dmarc violations (unfortunately they don't say why we have been banned and spamhaus says we are good)20:49
*** gongysh has joined #openstack-infra20:50
clarkbwe can go through their delist process but that apperas to be per email-address:ip-address tuple which could get annoying fast20:50
corvusmordred: that's not an ornery reply.  that's actually what we're supposed to be doing.  dmarc is a way of saying "this domain is only used to send notices from banks, not real email".  there's no onus on anyone to help a domain violate that intention.20:50
corvusclarkb: why do you think dmarc is involved?20:51
*** e0ne has quit IRC20:51
clarkbcorvus: only because I know microsoft has rejected email from our list servers due to dmarc recently (specifically kata list)20:51
mordredcorvus: I bet with a little bit of effort I could make it a more ornery reply  :)20:51
clarkband considering spamhaus is happy with us I'm leaping to conclusions20:52
openstackgerritMerged openstack-infra/zuul master: Default to 0o022 for python-daemon  https://review.openstack.org/58506820:52
corvusclarkb: did someone complain?20:52
clarkbcorvus: yes tim bell20:53
clarkb(he reached out ot foundation and not infra)20:53
*** e0ne has joined #openstack-infra20:53
corvusclarkb: i'm happy to be cc'd on an email thread with tim bell and his site mail admins to help sort it out20:53
clarkbcorvus: wfm I'll get foundation to loop you in20:53
*** ktsuyuzaki is now known as kota_20:54
corvuswe don't have enough information to act on at the moment, so the best we can say is "we attempted to deliver mail to your site but your site rejected it"20:54
*** graphene has quit IRC20:54
*** hamzy has quit IRC20:55
*** graphene has joined #openstack-infra20:56
openstackgerritMerged openstack-infra/project-config master: Import ansible-role-tripleo-cookiecutter  https://review.openstack.org/58142720:57
*** john_studarus has quit IRC20:57
*** e0ne has quit IRC20:58
*** cloe23 has joined #openstack-infra20:58
cloe23Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/20:58
cloe23or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/20:58
*** cloe23 has quit IRC20:58
*** jamesde__ has quit IRC20:59
*** auristor has joined #openstack-infra21:00
*** jamesden_ has joined #openstack-infra21:01
*** owalsh_ has joined #openstack-infra21:02
*** gongysh has quit IRC21:04
*** openstackgerrit has quit IRC21:04
*** owalsh has quit IRC21:05
*** wolverineav has joined #openstack-infra21:07
tonybsmcginnis, clarkb: we have 2 changes going through the gate pipeline once they (and the post-release jobs are done) I think we're ok to restart zuul from the release team's POV IIUC smcginnis's comments21:10
clarkbtonyb: sounds good, thank you for letting us know21:10
tonybclarkb: all good pabelanger popped into the release channel to check ... I probably shoudl have added him to the previous comment ;p21:12
*** rh-jelabarre has quit IRC21:12
*** mriedem is now known as mriedem_afk21:18
*** openstackgerrit has joined #openstack-infra21:19
openstackgerritMerged openstack-infra/storyboard master: change default python 3 env in tox to 3.5  https://review.openstack.org/57453621:19
*** bobh has quit IRC21:21
*** graphene has quit IRC21:26
openstackgerritMerged openstack-infra/zuul master: Add remote tests for synchronize  https://review.openstack.org/58465521:26
*** graphene has joined #openstack-infra21:27
*** martyn_h has joined #openstack-infra21:28
martyn_hHey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/21:29
*** martyn_h has quit IRC21:29
pabelangertonyb: will defer to clarkb now, I have a soccer game here shortly, but will be able to help in a few hours.21:31
*** bobh has joined #openstack-infra21:31
clarkbI can help restart things at tail end of my day, would be good if another root is around too21:32
clarkbcheck queue is almost half as long as it was and post queue is dropping too  so we are moving stuff through now21:33
pabelangerclarkb: yah, I should be back by then to help21:34
clarkbanyone else able to review https://review.openstack.org/#/c/585838/1 to fix our puppet gates? thank you pabelanger for soeting that out21:35
pabelangerclarkb: np, https://review.openstack.org//585803/ shows it worked too21:36
clarkbyup that was how I found it :)21:36
pabelangeryay21:37
*** dklyle has joined #openstack-infra21:38
*** rfolco|ruck is now known as rfolco|ruck|off21:39
*** graphene has quit IRC21:40
*** edmondsw has joined #openstack-infra21:40
*** graphene has joined #openstack-infra21:41
*** edmondsw has quit IRC21:45
*** rook has quit IRC21:48
*** felipemonteiro_ has quit IRC21:48
*** rook has joined #openstack-infra21:49
*** akhilaki has quit IRC21:50
*** rook is now known as Guest5546021:50
*** agopi|brb has joined #openstack-infra21:54
*** boden has quit IRC21:54
*** yamamoto has joined #openstack-infra21:55
*** bobh has quit IRC21:58
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Support complex data structures as secrets  https://review.openstack.org/58589221:58
ianwpabelanger / clarkb : is there a link to a failure?22:01
clarkbianw: http://logs.openstack.org/03/585803/1/check/openstackci-beaker-ubuntu-trusty/9d0f93e/job-output.txt.gz#_2018-07-25_17_44_00_23679622:02
ianwoh ok it's for ruby syntax issues22:02
*** NvpkD1y7Ez has joined #openstack-infra22:02
NvpkD1y7EzHey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/22:02
NvpkD1y7Ezor maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/22:02
*** NvpkD1y7Ez has quit IRC22:02
*** wolverineav has quit IRC22:03
*** neiloy has quit IRC22:03
*** agopi|brb is now known as agopi22:07
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Support complex data structures as secrets  https://review.openstack.org/58589222:08
openstackgerritMerged openstack-infra/puppet-openstack_infra_spec_helper master: Cap net-telnet < 0.2.0 for ruby 1.9 support  https://review.openstack.org/58583822:12
*** austin-sun has joined #openstack-infra22:16
*** pguimaraes has quit IRC22:18
*** eandersson has quit IRC22:22
*** eandersson has joined #openstack-infra22:24
*** spiffxp has quit IRC22:32
*** mtreinish has quit IRC22:33
*** spiffxp has joined #openstack-infra22:33
*** mtreinish has joined #openstack-infra22:36
*** sshnaidm is now known as sshnaidm|afk22:38
*** tpsilva has quit IRC22:40
*** kjackal has quit IRC22:41
clarkbpabelanger: I've approved the packethost change to raw images. I'm not sure if nodepool will notice until we make a nodepool.yaml change or restart the launcher as its in clouds.yaml (something to keep in mind if you don't see it take effect right away)22:43
openstackgerritMerged openstack-infra/storyboard master: fix tox python3 overrides  https://review.openstack.org/57435222:44
*** Guest55460 is now known as rook22:52
*** rook is now known as Guest6919022:52
*** Guest69190 is now known as rook--22:53
*** jamesmcarthur has joined #openstack-infra22:56
*** myoung|bbl is now known as myoung23:00
*** Looking has joined #openstack-infra23:03
LookingHey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/23:03
Lookingor maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/23:03
*** Looking has quit IRC23:03
*** eernst has quit IRC23:03
*** mschuppert has quit IRC23:03
openstackgerritMerged openstack-infra/system-config master: Set image_format to raw for packethost  https://review.openstack.org/58580323:06
*** jamesmcarthur has quit IRC23:07
*** wolverineav has joined #openstack-infra23:10
*** rwsu has joined #openstack-infra23:13
*** wolverineav has quit IRC23:14
*** jamesmcarthur has joined #openstack-infra23:15
clarkbpabelanger: corvus I think things have quieted down a bunch at this point and I believe tonyb's release changes are done23:21
*** kgiusti has left #openstack-infra23:21
clarkbdo we want to go ahead with some restarts?23:21
*** akhilaki has joined #openstack-infra23:21
tonybclarkb: +1 from a release-POV23:24
*** bobh has joined #openstack-infra23:25
*** tosky has quit IRC23:26
*** r-daneel has quit IRC23:28
*** yamamoto has quit IRC23:29
*** edmondsw has joined #openstack-infra23:29
clarkb/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/primitives/constant_time.py:26: CryptographyDeprecationWarning: Support for your Python version is deprecated. The next version of cryptography will remove support. Please upgrade to a 2.7.x release that supports hmac.compare_digest as soon as possible.23:32
clarkbI'm not sure how I feel about that23:32
*** yamamoto has joined #openstack-infra23:33
*** edmondsw has quit IRC23:33
clarkbzuul==3.1.1.dev166  # git sha cee121c is installed everywhere which looks right23:37
clarkb(I'm not keen on doing this if everyone else has called it a day though, since smcginnis is already worried :) )23:37
*** bobh has quit IRC23:38
*** bobh has joined #openstack-infra23:43
*** jamesmcarthur has quit IRC23:46
*** bobh has quit IRC23:47
*** jangutter has quit IRC23:48
*** rlandy has quit IRC23:48
*** bairdmich14 has joined #openstack-infra23:48
bairdmich14Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/23:48
*** bairdmich14 has quit IRC23:48
*** bobh has joined #openstack-infra23:53
*** bobh has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!