Friday, 2018-08-24

*** rlandy|brb is now known as rlandy00:01
*** longkb has joined #openstack-infra00:07
*** jamesmcarthur has joined #openstack-infra00:07
*** sthussey has quit IRC00:09
*** jamesmcarthur has quit IRC00:14
*** jamesmcarthur has joined #openstack-infra00:16
*** hamzy has quit IRC00:18
*** hamzy has joined #openstack-infra00:19
*** jamesmcarthur has quit IRC00:23
*** r-daneel has quit IRC00:28
*** r-daneel has joined #openstack-infra00:28
*** jamesmcarthur has joined #openstack-infra00:30
*** jamesmcarthur has quit IRC00:33
*** rlandy is now known as rlandy|bbl00:34
*** gyee has quit IRC00:44
*** hongbin has quit IRC00:45
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: Add guidelines on variable naming  https://review.openstack.org/59592000:59
*** ChanServ has quit IRC01:00
openstackgerritDavid Moreau Simard proposed openstack-infra/nodepool master: Implement an Amazon EC2 driver  https://review.openstack.org/53555801:00
*** ChanServ has joined #openstack-infra01:03
*** barjavel.freenode.net sets mode: +o ChanServ01:03
*** mriedem_afk has quit IRC01:05
*** r-daneel has quit IRC01:05
*** jamesmcarthur has joined #openstack-infra01:11
*** felipemonteiro has quit IRC01:12
*** ramishra has joined #openstack-infra01:15
*** dbecker has quit IRC01:15
*** jamesmcarthur has quit IRC01:15
*** felipemonteiro has joined #openstack-infra01:18
*** jamesmcarthur has joined #openstack-infra01:32
prometheanfirecan someone confirm that the job never kicked or something? https://review.openstack.org/#/c/595354/01:34
corvusprometheanfire: which job?01:35
prometheanfirethe one I linked01:35
prometheanfirewell, I didn't see anything kicked for that review at least01:36
corvusprometheanfire: oh you're wondering why the change wasn't enqueued01:36
prometheanfireya01:36
corvusprometheanfire: it depends on https://review.openstack.org/59535001:36
prometheanfireoh, lol, thanks01:36
corvusnp01:36
*** felipemonteiro has quit IRC01:37
*** jd_ has quit IRC01:46
*** pabelanger has quit IRC01:46
*** raginbajin has quit IRC01:46
*** jd_ has joined #openstack-infra01:47
*** jamesmcarthur_ has joined #openstack-infra01:47
*** mmedvede has quit IRC01:48
*** ericyoung has quit IRC01:48
*** raginbajin has joined #openstack-infra01:49
*** ericyoung has joined #openstack-infra01:49
*** masayukig has quit IRC01:49
*** dansmith has quit IRC01:49
*** jamesmcarthur has quit IRC01:50
openstackgerritIan Wienand proposed openstack-infra/zuul-jobs master: Add guidelines on variable naming  https://review.openstack.org/59592001:51
*** dansmith has joined #openstack-infra01:51
*** jamesmcarthur_ has quit IRC01:52
*** threestrands has quit IRC02:08
*** jamesmcarthur has joined #openstack-infra02:11
*** hongbin has joined #openstack-infra02:21
*** dmsimard has quit IRC02:22
*** dmsimard has joined #openstack-infra02:22
openstackgerritFilippo Inzaghi proposed openstack-dev/pbr master: import zuul job settings from project-config  https://review.openstack.org/59598302:24
openstackgerritFilippo Inzaghi proposed openstack-dev/pbr master: switch documentation job to new PTI  https://review.openstack.org/59598402:24
openstackgerritFilippo Inzaghi proposed openstack-dev/pbr master: add lib-forward-testing-python3 test job  https://review.openstack.org/59598502:24
*** gcb_ has quit IRC02:24
*** hamzy has quit IRC02:28
*** panda has quit IRC02:28
*** sweston has quit IRC02:28
*** ianychoi has quit IRC02:28
*** rlandy|bbl has quit IRC02:28
*** nicolasbock has quit IRC02:28
*** jlvillal has quit IRC02:28
*** dims has quit IRC02:28
*** mtreinish has quit IRC02:28
*** mnaser has quit IRC02:28
*** TheJulia has quit IRC02:28
*** zxiiro has quit IRC02:28
*** chandankumar has quit IRC02:28
*** adriant has quit IRC02:28
*** niska has quit IRC02:28
*** egonzalez has quit IRC02:28
*** slaweq has quit IRC02:28
*** Miouge has quit IRC02:28
*** mordred has quit IRC02:28
*** aluria has quit IRC02:28
*** sshnaidm|off has quit IRC02:28
*** auristor has quit IRC02:28
*** rtjure has quit IRC02:28
*** hamzy has joined #openstack-infra02:30
*** panda has joined #openstack-infra02:30
*** sweston has joined #openstack-infra02:30
*** ianychoi has joined #openstack-infra02:30
*** rlandy|bbl has joined #openstack-infra02:30
*** nicolasbock has joined #openstack-infra02:30
*** mtreinish has joined #openstack-infra02:30
*** dims has joined #openstack-infra02:30
*** jlvillal has joined #openstack-infra02:30
*** mnaser has joined #openstack-infra02:30
*** TheJulia has joined #openstack-infra02:30
*** zxiiro has joined #openstack-infra02:30
*** chandankumar has joined #openstack-infra02:30
*** adriant has joined #openstack-infra02:30
*** aluria has joined #openstack-infra02:30
*** niska has joined #openstack-infra02:30
*** egonzalez has joined #openstack-infra02:30
*** slaweq has joined #openstack-infra02:30
*** Miouge has joined #openstack-infra02:30
*** mordred has joined #openstack-infra02:30
*** sshnaidm|off has joined #openstack-infra02:30
*** auristor has joined #openstack-infra02:30
*** rtjure has joined #openstack-infra02:30
openstackgerritDavid Moreau Simard proposed openstack-infra/nodepool master: Implement an Amazon EC2 driver  https://review.openstack.org/53555802:30
*** openstack has joined #openstack-infra13:20
*** ChanServ sets mode: +o openstack13:20
fungi#status log updated chat.freenode.net entry in eavesdrop.o.o's /etc/hosts file from no-longer-active 195.154.200.232 to 82.96.96.11 (kornbluth.freenode.net) and restarted the openstack-meetbot service13:20
fungiShrews: ^ there13:20
Shrewsfungi: oh. why do we set that in /etc/hosts?13:20
fungiit's a workaround for supybot/twisted-python lacking support for combining ssl connections with hostnames which resolve to ipv6 addresses13:21
fungithere is no v4-only dns round-robin name for freenode or we'd just set that in the config13:21
Shrewsah13:21
*** openstackstatus has joined #openstack-infra13:22
*** ChanServ sets mode: +v openstackstatus13:22
Shrewsdon't see that documented. useful info13:22
*** openstackstatus has quit IRC13:22
*** openstackstatus has joined #openstack-infra13:24
*** ChanServ sets mode: +v openstackstatus13:24
fungi#status log updated chat.freenode.net entry in eavesdrop.o.o's /etc/hosts file from no-longer-active 195.154.200.232 to 82.96.96.11 (kornbluth.freenode.net) and restarted the openstack-meetbot and statusbot services13:27
openstackstatusfungi: finished logging13:27
openstackgerritMerged openstack-infra/project-config master: stx-specs: Add new repo for specifications  https://review.openstack.org/59267613:27
*** felipemonteiro has joined #openstack-infra13:29
AJaegerconfig-core, https://review.openstack.org/596281 is needed for rocky  periodic testing, please review13:29
*** eharney has joined #openstack-infra13:30
*** antonym_ is now known as antonym13:43
*** pabelanger has joined #openstack-infra13:55
*** jaosorior has joined #openstack-infra14:02
*** r-daneel has joined #openstack-infra14:04
*** eumel8 has quit IRC14:07
openstackgerritDavid Shrewsbury proposed openstack-infra/system-config master: Add NTP service to base-server  https://review.openstack.org/59531214:08
fungiShrews: the hope is that with the upgrade to limnoria that problem will go away14:08
*** jamesmcarthur has joined #openstack-infra14:09
fungiat least that's a hope of mine14:09
samueldmqhi - how can I add a meeting bot back to a channel?14:15
samueldmqit's gone for the outreachy channel :(14:15
*** felipemonteiro has quit IRC14:16
*** efried_afk is now known as efried14:16
cmurphysamueldmq: it should be back now, fungi restarted it a few minutes ago14:18
samueldmqcmurphy: awesome, yes it's back thanks!14:18
fungisamueldmq: yeah, logs say it joined #openstack-outreachy at 13:26:0614:19
fungi(utc)14:19
dtroyerI need to have Saul Wold (sgw@linux.intel.com) and/or myself added to bootstrap the new starlingx-specs-core group in Gerrit…14:19
fungisamueldmq: so ~45 minutes ago14:20
fungidtroyer: happy to, just a moment14:20
fungidtroyer: i added saul as the initial member since his account was the owner of the change which added that group14:22
dtroyerfungi: great, thanks!14:22
samueldmqfungi: awesome,, thanks!14:23
*** efried is now known as senhor_granhular14:24
*** pbourke has quit IRC14:26
*** pbourke has joined #openstack-infra14:28
corvusfungi, Shrews: good morning, would you please review and approve https://review.openstack.org/595905 ?14:28
corvusor https://review.openstack.org/59537814:28
Shrewscorvus: is that a stipulation for me having a good morning???14:29
Shrews:)14:29
corvusShrews: yes. because if we don't merge the base job stuff we're all going to have a bad day :)14:29
fungilooking14:30
* fungi is already not going to space today14:30
Shrewscorvus: oh, i looked at that this morning. the only difference is the var names, yes?14:30
corvusShrews: yes.  i think it is important that roles in system-config have prefixed var names.  i don't think it is important for zuul-jobs, but ianw thinks it is.14:31
*** hashar has quit IRC14:31
*** jaosorior has quit IRC14:33
corvusvery few roles in zuul-jobs obey that (because for most of them, they aren't behaving as roles, they're behaving as reusable functions)14:33
*** hashar has joined #openstack-infra14:34
openstackgerritNate Johnston proposed openstack/gertty master: Add ability to set lock file in config  https://review.openstack.org/59631814:35
corvusanyway, i need someone who cares to pick one of those so we can merge it and move on :)14:35
Shrewshrm, zuul-jobs is designed to be used by other repos where role vars may conflict with roles outside z-j, yeah? i admit my role var scoping knowledge is lacking here14:36
AJaegercorvus, Shrews, fungi, for the variable names, ianw pushed a proposal as policy, see https://review.openstack.org/#/c/595920/14:37
*** senhor_granhular is now known as fried_rice14:37
* Shrews reads14:37
corvusShrews: yeah, most of the time vars get set in a global scope, so you have to be careful14:37
corvusi'd say our current policy is "think about it" :)14:37
corvusfor instance, lots of roles in zuul-jobs use "zuul_work_root" as a variable.  it's intended to be used globally and would be inappropriate to prefix.14:38
*** viks__ has quit IRC14:38
AJaegercorvus: agreed on zuul_work_root - that must be shared ;)14:39
corvusfor another example, zuul_site_upload_logs should definitely not be "zuul_swift_site_upload_logs" -- the same var should affect all of the upload roles14:39
*** dpawlik_ has quit IRC14:39
corvusbut it would be bad to have a variable named, say "enabled".  that's not very specific14:39
corvusthe bindep role has prefixed names14:40
*** njohnston has joined #openstack-infra14:40
corvusprobably the safest thing to do is generally prefix things, except where we expect them to hold values that are used by multiple roles14:41
openstackgerritChristophe Sauthier proposed openstack-infra/project-config master: Add publish-to-pypi job to cloudkitty-tempest-plugin  https://review.openstack.org/59632914:41
AJaegeranybody wants to +2A the change above for Rocky release? I just gave my +2...14:42
Shrewscorvus: i mean, i lean toward prefixing just out of fear. your 'dest' var in that inventory role seems pretty generic14:43
Shrewsthough i don't have a specific example in my head atm where that breaks something14:43
pabelangermorning, config-core could you please add https://review.openstack.org/593725/ to your pipeline, a new pbrx job from Shrews to start publishing container images14:44
*** ociuhandu has joined #openstack-infra14:44
corvusShrews: okay, let's go with prefixed14:44
fungicorvus: how would you characterize the practical differences between 595905 and 595378?14:45
fungiskimming them they look like they both do basically the same things14:45
corvusfungi: they are exactly the same except the variable names14:46
fungi(and seem to share a lot of code in common)14:46
fungiaha, okay i didn't know if there were other subtle behavioral differences i wasn't picking up on14:46
fungii didn't bother to directly diff the patches14:46
fungiprefixed does seem safer to me, i agree14:47
corvus905 is how i would write a role that i intended to be used as a "role" as the concept was originally designed.  for example, adding a "web-server" role to a host.  378 is how i would write a function (ie, a reusable chunk of code) in ansible by abusing the role concept since ansible basically only has "role" as the unit of code reuse.14:47
openstackgerritJames E. Blair proposed openstack-infra/zuul-jobs master: write-inventory: fix typo in README  https://review.openstack.org/59635514:48
corvusfungi, Shrews: then if you would go ahead and approve 905 and 596355 i'll update the base-test job series14:49
fungithanks!14:50
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add system-config-run-base job  https://review.openstack.org/59525214:51
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Don't create "admin" group  https://review.openstack.org/59545414:51
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add iptables role  https://review.openstack.org/59397314:51
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Transfer iptables rules to ansible  https://review.openstack.org/59434014:51
openstackgerritJames E. Blair proposed openstack-infra/system-config master: iptables: factor out snmp rules  https://review.openstack.org/59443714:51
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add a testinfra test for iptables  https://review.openstack.org/59590014:51
corvusthose were all green earlier -- the only change i made was to switch to the other write-inventory patch.  it should be safe to go ahead and review those14:52
*** jlvillal is now known as jlviva-viva14:52
corvus(i'd like to land them asap since there are changes to the base playbook, that way we don't have to stay late today watching them)14:53
*** ociuhandu has quit IRC14:53
*** ykarel is now known as ykarel|away14:55
Shrewsooh, pytest-xdist is new to me14:57
Shrewsnice14:57
*** pcaruana has quit IRC14:59
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add system-config-run-base to gate  https://review.openstack.org/59636215:00
*** trown has joined #openstack-infra15:02
*** r-daneel has quit IRC15:04
*** r-daneel has joined #openstack-infra15:06
*** ssbarnea|ruck has quit IRC15:07
*** Bhujay has quit IRC15:08
AJaegerpabelanger: +2A on 59372515:09
pabelangerAJaeger: danke!15:09
openstackgerritMerged openstack-infra/zuul-jobs master: Add write-inventory role  https://review.openstack.org/59590515:09
AJaegerconfig-core, anybody wants to +2A https://review.openstack.org/596329 for Rocky release, please?15:09
AJaegerspeaking about policy, infra-root, could you check https://review.openstack.org/596268 ? This is a result of discussion between corvus and me yesterday on what to leave in project-config - specially integrated-gate setting.15:10
*** r-daneel_ has joined #openstack-infra15:18
mnaserAJaeger: done15:18
AJaegerthanks, mnaser15:19
*** r-daneel has quit IRC15:19
*** r-daneel_ is now known as r-daneel15:19
corvusfungi: can you look at 595252 and children please?15:20
corvusShrews: mind if i reparent your ntp and timezone patches on the stack?15:21
*** nicolasbock has joined #openstack-infra15:21
ShrewsDo it15:21
Shrewsafk right now15:21
fungicorvus: yep, looking. slowed by release team meeting15:22
corvusfungi: thx15:22
openstackgerritMerged openstack-infra/project-config master: Add a job to push pbrx container images  https://review.openstack.org/59372515:23
*** radez has joined #openstack-infra15:23
*** ramishra has quit IRC15:25
openstackgerritMerged openstack-infra/project-config master: Add publish-to-pypi job to cloudkitty-tempest-plugin  https://review.openstack.org/59632915:27
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add NTP service to base-server  https://review.openstack.org/59531215:27
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add timezone role  https://review.openstack.org/59454215:27
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add testinfra test for ntp  https://review.openstack.org/59638215:27
fungihttps://review.openstack.org/595454 feels like a regression to me15:29
pabelangerfungi: why not have the group as sudo on those things?15:30
corvusfungi: yeah, i don't feel strongly about that.  i'd be okay keeping admin or changing it to sudo15:30
*** ykarel|away has quit IRC15:30
*** anteaya has joined #openstack-infra15:31
corvuswow, someone broke the base playbook out from under the new base-test job15:31
corvusmy new patchsets are getting -1s from zuul, because the exim role can't be found15:32
corvusi suspect https://review.openstack.org/590753 somehow15:32
corvusto be honest, i fall the other way on the split15:33
corvusi think roles in system-config should be private by default, unless we want to share them15:33
pabelangeryah, I was under the impress that is what we decided15:34
pabelangerimpression*15:34
corvusi believe ianw thinks the opposite -- that they should be public by default unless we need to keep them private15:34
pabelangerI could get behind them public, but think that would mean moving them into their own project in openstack-infra. Which, i think for now, was too much work15:35
pabelangerI'd be recultant to have something depend on system-config for a top level role15:36
*** shardy has quit IRC15:37
*** aojea has quit IRC15:37
corvusat the very least it would have been nice to hold that change until after we had the test in place15:38
openstackgerritMerged openstack-infra/zuul-jobs master: write-inventory: fix typo in README  https://review.openstack.org/59635515:38
pabelangercorvus: yah, I think revert and propse again after the testing is place is also fine solution15:39
pabelangeragree getting testing in place will be helpful15:39
fungihappy to approve a revert if someone has a moment to propose it15:39
corvusi'm looking into the current state to figure whether it's easier to revert or fix15:40
pabelangerI suspect we need to setup ANSIBLE_ROLES_PATH variable, since role isn't next to playbook now15:45
corvuspabelanger: i'd like to be more like production than that15:46
corvusi'm looking at setting up /opt/system-config which is what's in ansible.cfg15:46
pabelangeractually, how would we ensure exim is in the role path on brige.o.o, isn't it also failing now?15:47
pabelangerah, I understand now15:47
corvusyeah, production isn't broken (if it were, i would revert)15:49
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add system-config-run-base job  https://review.openstack.org/59525215:50
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Don't create "admin" group  https://review.openstack.org/59545415:50
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add iptables role  https://review.openstack.org/59397315:50
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Transfer iptables rules to ansible  https://review.openstack.org/59434015:50
openstackgerritJames E. Blair proposed openstack-infra/system-config master: iptables: factor out snmp rules  https://review.openstack.org/59443715:50
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add a testinfra test for iptables  https://review.openstack.org/59590015:50
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add system-config-run-base to gate  https://review.openstack.org/59636215:50
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add NTP service to base-server  https://review.openstack.org/59531215:50
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add timezone role  https://review.openstack.org/59454215:50
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add testinfra test for ntp  https://review.openstack.org/59638215:50
corvusfungi, pabelanger, Shrews: ^ i updated 595252 to clone /opt/system-config from the zuul workspace, so the roles should be checked out into the appropriate location (including speculative changes)15:51
*** jpena is now known as jpena|off15:53
fungithanks!15:53
*** SteffanW has joined #openstack-infra15:53
*** Bhujay has joined #openstack-infra16:01
*** dtantsur is now known as dtantsur|afk16:01
pabelanger+216:02
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add snmpd role and add it to base  https://review.openstack.org/59396916:06
AJaegerpabelanger: did https://review.openstack.org/595229 help? That's the /home/zuul/src directory change16:08
AJaegerpabelanger: want to abandon https://review.openstack.org/#/c/594569/ ? Did you push up an alternative approach?16:09
corvusShrews: have you started on unbound?16:11
*** Dobroslaw has quit IRC16:11
pabelangerAJaeger: not sure, we need to land base-test change to validate16:11
pabelangerAJaeger: if it works, I propose we update use-cache-dirs role to first create the directory16:11
pabelangerAJaeger: but right now, it is more a guess that needs to be tested16:12
AJaegerpabelanger: anything I shall review?16:12
pabelangerbecause, I don't see the 777 directories in rdoproject16:12
pabelangerAJaeger: https://review.openstack.org/595229/ needs a review and +3, then we can do a quick test16:12
fungicorvus: i think we need to add an extra package for debian/ubuntu in the iptables role (see comments on 593973)16:13
AJaegerpabelanger: +2 from me16:13
fungipretty sure it's just going to no-op unless the i'm simply missing where the additional package is being added16:13
corvusfungi: huh.  testinfra should have validated that it worked.16:14
fungicorvus: indeed, and i found what i'd overlooked16:14
fungisorry16:14
AJaegerconfig-core, https://review.openstack.org/#/c/586949 updates zuul-base-jobs for ansible 2.5 testing - any volunteer to review, please?16:14
corvusfungi: oh good, what's the answer?16:14
*** ginopc has quit IRC16:15
*** electrofelix has quit IRC16:15
fungioh, wait, that's only on trusty it's being installed16:16
*** ijw has joined #openstack-infra16:16
pabelangerfungi: no, I think your comment might be correct. Is it possible testinfra isn't failing because our images already have the iptables-persistent package installed already?16:17
*** jamesmcarthur has quit IRC16:17
*** r-daneel_ has joined #openstack-infra16:18
fungithat's what i'm attempting to ascertain16:19
*** udesale has quit IRC16:19
*** gyee has joined #openstack-infra16:19
fungithough first i'm trying to find where in testinfra tests the loaded rules are checked16:19
*** r-daneel has quit IRC16:20
*** r-daneel_ is now known as r-daneel16:20
corvusfungi: its a followup change:16:20
corvushttps://review.openstack.org/59590016:20
fungi'splains why i didn't see it in the job log for that change, thanks ;)16:20
fungiwhat job runs those? is it not system-config-run-base?16:22
corvusfungi: it is system-config-run-base16:22
fungidoesn't look like zuul ran system-config-run-base against 59590016:22
fungiat least not the patchset before your rebase just now16:23
corvushrm, bad files stanza?16:23
openstackgerritMerged openstack-infra/zuul-base-jobs master: Update test-requirements ansible version  https://review.openstack.org/58694916:23
corvusyeah, let me redo that16:24
*** larainema has quit IRC16:24
corvusi'll insert a change above iptables to add testinfra to files, preserving the status of the first 2 changes in the stack16:26
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add system-config-run-base to gate  https://review.openstack.org/59636216:28
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add iptables role  https://review.openstack.org/59397316:28
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Transfer iptables rules to ansible  https://review.openstack.org/59434016:28
openstackgerritJames E. Blair proposed openstack-infra/system-config master: iptables: factor out snmp rules  https://review.openstack.org/59443716:28
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add a testinfra test for iptables  https://review.openstack.org/59590016:28
corvusi squashed it with the add-to-gate change and moved it up16:30
*** jpich has quit IRC16:31
fungithanks, i still suspect pabelanger may be right that we have iptables-persistent (and netfilter-persistent) already preinstalled on our images16:31
corvusfungi: should we just go ahead and add that package to the change, or are you curious about whether it passes without it?16:32
corvusfungi: i tend to think that regardless of the test result, we would probably add the package anyway, so maybe we should?16:33
fungiyeah, it should be necessary from everything i can see16:33
fungibasically the behavior switch from trusty to xenial was 1. the iptables-persistent package was split into iptables-persistent (which carries the iptables load/dump scripts) and netfilter-persistent (which contains the initscript/systemd unit), 2. the service name was changed from iptables-persistent to netfilter-persistent16:35
corvusfungi: oh, you know what, our puppet module just installs the iptables-persistent package in all cases16:36
corvusfungi: does that get netfilter-persistent on >trusty?16:36
fungiyes16:36
corvusso i'll just do that then :)16:36
fungiiptables-persistent depends-on netfilter-persistent16:36
corvuswhich is what i was trying to do in the first place but apparently got cross-eyed16:37
fungiunfortunately the role there wants to conflate the package name and the service name16:37
corvusfungi: well, they're separate for that reason, i just put the wrong name in16:37
fungiso ultimately if we install the iptables-persistent package on all releases but call the iptables-persistent initscript on <=trusty and the netfilter-persistent initscript on >=xenial it should work16:37
fungithere should be no need to explicitly install the netfilter-persistent package since it's a dependency of iptables-persistent16:38
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add iptables role  https://review.openstack.org/59397316:38
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Transfer iptables rules to ansible  https://review.openstack.org/59434016:38
openstackgerritJames E. Blair proposed openstack-infra/system-config master: iptables: factor out snmp rules  https://review.openstack.org/59443716:38
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add a testinfra test for iptables  https://review.openstack.org/59590016:38
corvusfungi: that should do it16:38
fungiawesome!16:38
*** fried_rice is now known as fried_rolls16:39
fungiyep, that looks like it should do what we want16:40
*** anteaya has quit IRC16:42
AJaegercorvus: could you check what I wrote in the infra-manual regarding shared queues, please? https://review.openstack.org/596268 That's based on yesterday's discussion16:44
corvusAJaeger: that looks right (modulo a tiny but important nit i left as an inline comment) if we want to go that direction.16:46
*** anteaya has joined #openstack-infra16:46
openstackgerritJeremy Stanley proposed openstack-infra/puppet-meetbot master: Wait to join channels until identified by NickServ  https://review.openstack.org/59639816:48
fungiokay, finally got a chance to look at options for the missing channels problem16:48
fungihopefully that should fix it, but we'll want to wait until there are no meetings running and then babysit to make sure it actually rejoins16:49
*** xavinux has joined #openstack-infra16:49
openstackgerritAndreas Jaeger proposed openstack-infra/infra-manual master: Document shared queue and policy for integrated-gate  https://review.openstack.org/59626816:49
AJaegercorvus: updated ^16:49
AJaegerfungi, pabelanger, what do you think of the documented policy above?16:50
AJaegerBasically: "queue: integrated" stays in project-config, teams can add other queues in-repo16:50
AJaegeralternative would be that all "queue: " configuration stay in project-config.16:51
*** xavinux has quit IRC16:53
Shrewscorvus: i have not started on either unbound or unattended-upgrades16:53
corvusShrews: cool, i'll start on unbound then16:54
fungiAJaeger: as an interim policy it makes sense to at least have that centralized somewhere so there can be a specific team who gets to decide which projects join that queue. eventually i have a feeling we'll want to delegate review of openstack community policies like that to a specific team so might not be in the main zuul config repo indefinitely16:54
fungiit seems like we're probably going to want a place for communities (openstack, zuul, airship, starlingx, kata, et cetera) to centrally control their own job policy decisions independent of the group of people we have reviewing the main zuul configuration16:56
corvusfungi: those communities should be tenants and each one should have its own project-config repo16:57
corvusright now, we only have one community so we only have one repo16:57
AJaegerfungi: yes, that's definitely an option going forward. Do you argue for *all* queues or only integegrated?16:57
fungicorvus: but the repo which has the list of tenants in it will be separate at that point, right?16:57
*** hashar is now known as hasharAway16:58
corvusfungi: yes, winterscale will control main.yaml.  but queue assignment is in the regular zuul configuration, not main.yaml.16:58
fungiAJaeger: i think it can be the case for any shared queue which needs centralized oversight. but for example if there's a tripleo shared queue then maybe somewhere under the control of the tripleo team in that case?16:58
*** holser_ has quit IRC16:59
corvusfungi: so when we go multi-tenant, main.yaml and nodepool will go to winterscale, but the rest of project-config will be managed by the openstack project16:59
fungiyep, that sounds great to me16:59
AJaegerfungi: I documented that cross-team should be in project-config - so far we only have integrated.17:00
AJaegerso, tripleo could do this on their own repos - unless they want to include oslo - for example17:00
fungiAJaeger: sounds fine to me. i'll take a look at the documentation change in a bit17:00
AJaegerthanks, fungi.17:01
*** mriedem is now known as mriedem_afk17:02
*** trown is now known as trown|lunch17:06
*** armax has quit IRC17:07
*** ijw has quit IRC17:08
openstackgerritMerged openstack-dev/pbr master: Support subdirectory in the url  https://review.openstack.org/59006617:08
Shrewsfungi: confused on https://review.openstack.org/593973 ... you think that's gtg now?17:09
corvusShrews: let's hold off on +3ing that until we have results from testinfra17:10
Shrewsfrom https://review.openstack.org/595900, yeah?17:10
corvuswe should probably treat those 4 patches as one17:10
corvusShrews: yep that one17:10
Shrewsk, just wanted to make sure17:10
Shrewslooks like zuul just -1'd17:10
fungiShrews: yeah, i think it should be good now, but i agree we want to make sure the job runs and tests it17:10
fungithere was a bit of a hole where zuul wasn't running the job for the change that added the test to exercise it17:11
corvuswith a -1 from zuul, i'm likely to need to rework those -- should i keep them as 4 patches or squash them?17:11
corvus(i don't mind either way, whatever's easy for reviewing)17:12
fungihaving the tests in the change which also introduces the role might be a little cleaner17:12
Shrewsyah17:12
fungibut i'm fine reviewing them however is easier for you to go at it17:13
corvusyep, but i think it'd be tricky to only partially squash, so i think the good options are either 4 changes or 1.  one of those is the one that transfers the rules which is difficult to read.17:14
corvusbut if we just squash them now, we'll have a new baseline and can just do patchset diffs after that17:15
fungifull squash is fine by me17:16
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add iptables role  https://review.openstack.org/59397317:18
*** mkoprowski has joined #openstack-infra17:18
corvusthat's the current state squashed with no other changes17:18
corvusso that's our baseline to compare for whatever i do to fix it17:18
corvusfungi: https://review.openstack.org/596362 is safe to merge (and a good idea) if you want to +317:19
*** diablo_rojo has joined #openstack-infra17:20
*** dpawlik has joined #openstack-infra17:20
*** dpawlik has quit IRC17:25
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add iptables role  https://review.openstack.org/59397317:28
corvusfungi, Shrews: how about we actually use the role?17:28
Shrewsthat's crazy talk17:29
fungiheh17:32
corvusi am really confused by this: http://logs.openstack.org/69/593969/3/check/puppet-beaker-rspec-infra-centos-7-system-config/94379c3/job-output.txt.gz#_2018-08-24_16_40_17_13331717:33
corvusthat is consistently failing because exim isn't running.  but that change has nothing to do with exim17:34
corvusif the exim module is "broken", why wouldn't that job fail on any other change?17:34
corvus(for instance, the entire stack we've been working on)17:34
AJaegerconfig-core, this change adds rocky periodic jobs - could you review, please? https://review.openstack.org/59628117:36
*** tosky has quit IRC17:36
AJaegerconfig-core, what do you think of pabelanger's idea to get rid of word-writeable /home/zuul/src with https://review.openstack.org/595229 ? Shall we try?17:38
corvusAJaeger, pabelanger: even if that corrects ~/src, won't the subdirs under that still be o+w ?17:42
*** anteaya has quit IRC17:43
pabelangerat this point, there is nothing under ~/src, we are just going to per create it. Looking at logs, the only thing I can thing, is for some reason git mirror is creating them with 777, but I don't really know why that would be17:43
pabelangerlike I said, this is mostly a guess right now to help rule out things17:43
corvuspabelanger: yeah, i'm just wondering what the next step is depending on the outcome17:44
*** anteaya has joined #openstack-infra17:44
pabelangerif it works, i think adding that into use-cache-dirs was my thinking. If it doesn't, I don't really know where to go next.  Accept maybe try to start to reproduce locally17:45
pabelangerwe don't see this issue in rdoproject, but we are just using prepare-workspace role17:45
corvuspabelanger: do you think that pre-creating src will be enough to change the permissions of all the subdirs under src?17:46
pabelangerthat's my current thought yes17:46
pabelangerbut really not sure if it will do anything17:46
*** fried_rolls is now known as fried_rice17:47
*** eharney has quit IRC17:53
AJaegerso, merge - test - revert or party? ;)17:53
pabelangeractually17:54
pabelangerwe could write a test job and parent to base-minimal, then use prepare-workspace to see if we have the same issue17:54
pabelangerbase-minimal doesn't include use-cache-repos role17:55
pabelangerlet me try that is afternoon17:55
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add iptables role  https://review.openstack.org/59397318:02
openstackgerritJeremy Stanley proposed openstack-infra/system-config master: Add kata-discuss ML to replace kata-dev  https://review.openstack.org/59641918:06
*** olivierb_ has joined #openstack-infra18:07
*** olivierb has quit IRC18:09
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add NTP service to base-server  https://review.openstack.org/59531218:09
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add timezone role  https://review.openstack.org/59454218:09
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add testinfra test for ntp  https://review.openstack.org/59638218:09
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add snmpd role and add it to base  https://review.openstack.org/59396918:09
*** anteaya has quit IRC18:10
openstackgerritJeremy Stanley proposed openstack-infra/system-config master: Add kata-discuss ML to replace kata-dev  https://review.openstack.org/59641918:10
*** trown|lunch is now known as trown18:16
corvuslimestone does not appear to be operating normally18:20
corvushttp://grafana.openstack.org/d/WFOSH5Siz/nodepool-limestone?orgId=1&from=now-24h&to=now18:20
corvusit seems to be accumulating deleted servers18:21
openstackgerritJames E. Blair proposed openstack-infra/project-config master: Temporarily disable limestone  https://review.openstack.org/59642318:22
corvusfungi: ^18:22
*** hasharAway is now known as hasharDinner18:23
mnasercc logan- ^ ?18:24
logan-odd.. im not seeing the deleted servers in the cloud itself, just currently running job instances18:24
corvuslogan-: f8b79e38-06b0-4d71-a34c-5d9136943710 is an instance it just tried and failed to delete18:25
*** armax has joined #openstack-infra18:25
logan-http://paste.openstack.org/raw/728766/18:26
corvuswow, um, all the delete attempts appear to be for that instance18:27
corvusShrews: ^ something is weird about this18:27
*** HenryG has quit IRC18:28
logan-running instances: http://paste.openstack.org/raw/728767/18:28
corvusi'm going to WIP my disable patch because i'm not sure that's the way to proceed right now18:29
*** HenryG has joined #openstack-infra18:29
Shrewslooking18:30
corvusi need to run right now; i can help look into this in about 1-1.5 hours18:31
*** rlandy is now known as rlandy|brb18:32
openstackgerritJeremy Stanley proposed openstack-infra/system-config master: Add kata-discuss ML to replace kata-dev  https://review.openstack.org/59641918:33
ShrewsLooks like we had an error creating that server. The zk node for it was marked failed. We eventually delete the zk node (no instance id associated with it). Then we find instance f8b79e38-06b0-4d71-a34c-5d9136943710 as a leaked node and try to delete it (so it eventually got created??)18:38
*** jcoufal has quit IRC18:41
*** mriedem_afk is now known as mriedem18:45
*** florianf has quit IRC18:47
*** SteffanW has quit IRC18:47
*** eharney has joined #openstack-infra18:49
*** rlandy|brb is now known as rlandy18:50
*** armax_ has joined #openstack-infra18:52
*** armax has quit IRC18:52
*** armax_ is now known as armax18:52
Shrewscorvus: i think we may need this fix: https://review.openstack.org/576288 for starters18:53
Shrewscorvus: why nodepool keeps finding that instance as leaked is a puzzler if it truly does not exist18:53
*** rossella_s has quit IRC18:55
*** rossella_s has joined #openstack-infra18:59
Shrewscorvus: logan-: $ openstack --os-cloud limestone server list | grep "f8b79e38-06b0-4d71-a34c-5d913694371"19:01
Shrews| f8b79e38-06b0-4d71-a34c-5d9136943710 | centos-7-limestone-regionone-0001519765      | BUILD  |                                                                  | centos-7-1535040910      | nodepool.0 |19:01
Shrewsso it is there19:01
openstackgerritDean Troyer proposed openstack-infra/project-config master: Add pushMerge for starlingx-release  https://review.openstack.org/59642619:02
logan-oh i see it now in the top of that running instances list i pasted19:02
logan-let me reset the state of it19:03
Shrewslogan-: or delete it19:03
logan-nova won't let me reset-state or delete it heh19:04
*** Bhujay has quit IRC19:04
AJaegerdtroyer: why do you need pushMerge?19:05
Shrewslogan-: fun19:05
*** hwoarang has quit IRC19:05
fungidtroyer: feature branches i guess?19:07
*** bdodd has joined #openstack-infra19:07
AJaegerfungi: yeah, that'S what the commit message says... Should hae checked first19:08
*** rossella_s has quit IRC19:09
dtroyerAJaeger: on the specs repo maybe we don't19:12
*** rossella_s has joined #openstack-infra19:12
dtroyerI'm sorting out the docs repo-style setup for the first time, so I don't have a problem taking that off, it seemed simpler to make them the same for this bit19:13
AJaegerdtroyer: I'm all for adding what you need know - and add later as needed...19:13
AJaegerdtroyer: far less risk of making a mistake ;)19:13
AJaeger"need now" I mean19:14
dtroyerAJaeger: true that… even for a limited size team19:14
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Add query for nova file not found func test failure 1788922  https://review.openstack.org/59643119:14
openstackgerritDean Troyer proposed openstack-infra/project-config master: Add pushMerge for starlingx-release  https://review.openstack.org/59642619:15
dtroyerAJaeger: ^^^ better?19:15
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Add query for nova file not found func test failure bug 1788922  https://review.openstack.org/59643119:15
openstackbug 1788922 in OpenStack Compute (nova) "SRIOVServersTest.test_create_server_with_VF intermittently fails due to "FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/src/git.openstack.org/openstack/nova/instances/.directio.test'"" [Medium,Confirmed] https://launchpad.net/bugs/178892219:15
AJaegerdtroyer: do you need pushMerge and create at all for specs? That's a different use than OpenStack does - there all repos are branchless and untagged. The other file looks good19:17
dtroyerAJaeger: I suppose not…again I was thinking this would be like docs, and IIUC the docs will be branched.  We'll skip it for now then19:19
openstackgerritDean Troyer proposed openstack-infra/project-config master: Add pushMerge for starlingx-release  https://review.openstack.org/59642619:20
dtroyerI'll let the docs folk sort it out19:20
*** ade_lee has joined #openstack-infra19:20
AJaegerdtroyer: happy to review later if needed...19:21
AJaegerdtroyer: +219:21
dtroyerAJaeger: thanks… now to head down the publishing path for these :)  stx-docs mostly though…19:22
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: zk: skip node already being deleted in cleanup leaked instance task  https://review.openstack.org/57628819:22
AJaegerdtroyer: where will those get published? did infra create a docs site for sterlingx already?19:25
dtroyernot yet, that's part of what I need to do19:25
dtroyerhey were in a hurry to get the repo right away so we did that much, then nothing for a while, now hurry again :)  you know how that goes19:26
AJaegerfun ;)19:27
*** rossella_s has quit IRC19:32
*** apetrich has quit IRC19:32
*** rossella_s has joined #openstack-infra19:34
AJaegernice, airship repos have a third-party CI that reports - which is not reachable ;( Must be an internal system :(19:39
*** dpawlik has joined #openstack-infra19:39
*** mriedem has quit IRC19:40
*** dpawlik has quit IRC19:44
*** mriedem has joined #openstack-infra19:48
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Remove unneeded noop-jobs from airship  https://review.openstack.org/59644919:48
openstackgerritMerged openstack-infra/elastic-recheck master: Add query for nova file not found func test failure bug 1788922  https://review.openstack.org/59643119:49
openstackbug 1788922 in OpenStack Compute (nova) "SRIOVServersTest.test_create_server_with_VF intermittently fails due to "FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/src/git.openstack.org/openstack/nova/instances/.directio.test'"" [Medium,Confirmed] https://launchpad.net/bugs/178892219:49
corvusAJaeger: we do have a policy about that19:49
AJaegerDo we have a contact for airship?19:50
AJaegerfound it on the wiki...19:52
openstackgerritMerged openstack-infra/system-config master: Add system-config-run-base job  https://review.openstack.org/59525219:57
openstackgerritMerged openstack-infra/system-config master: Don't create "admin" group  https://review.openstack.org/59545419:57
openstackgerritMerged openstack-infra/system-config master: Add system-config-run-base to gate  https://review.openstack.org/59636219:57
* AJaeger will send an email and CC openstack-infra19:59
*** rossella_s has quit IRC20:01
AJaegerat least I cleaned up the extra noops and added venv to tox.ini for airship now after noticing that their post jobs fail ;( https://review.openstack.org/596449  removes the extra noops from project-config20:03
*** rossella_s has joined #openstack-infra20:04
*** olivierb has joined #openstack-infra20:06
*** olivierb_ has quit IRC20:07
*** fried_rice is now known as efried_afk20:13
openstackgerritAndreas Jaeger proposed openstack-infra/system-config master: Cleanup docs formatting  https://review.openstack.org/59645720:14
*** efried_afk is now known as fried_rice20:14
*** olivierb_ has joined #openstack-infra20:16
*** olivierb has quit IRC20:17
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Add publish-to-pypi for barbican-tempest-plugin  https://review.openstack.org/59645820:17
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Add publish-to-pypi for blazar-tempest-plugin  https://review.openstack.org/59645920:17
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Add publish-to-pypi for ec2api-tempest-plugin  https://review.openstack.org/59646020:18
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Add publish-to-pypi for magnum-tempest-plugin  https://review.openstack.org/59646120:18
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Add publish-to-pypi for mistral-tempest-plugin  https://review.openstack.org/59646220:18
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Add publish-to-pypi for senlin-tempest-plugin  https://review.openstack.org/59646320:18
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Add publish-to-pypi for telemetry-tempest-plugin  https://review.openstack.org/59646420:18
AJaegersmcginnis: did you create all these on pypi?20:18
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Add publish-to-pypi for trove-tempest-plugin  https://review.openstack.org/59646520:19
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Add publish-to-pypi for zaqar-tempest-plugin  https://review.openstack.org/59646620:19
AJaegersmcginnis: https://pypi.org/project/barbican-tempest-plugin/ gives a 404 - please double check20:19
smcginnisAJaeger: Yep, I got them all registered first and set up openstackci.20:19
smcginnisAJaeger: That one is there, but I had the bright idea of "oh, we don't need this initial version"20:19
smcginnisAJaeger: So it's registered and there, but can't be found because there are no versions.20:20
smcginnisAJaeger: I can do an image paste to show the proper setup if that would be better.20:20
AJaegersmcginnis: if it fails, you'll notice - I'll trust you on that one ;)20:20
AJaegerwhat about https://pypi.org/project/mistral-tempest-plugin/ ?20:21
smcginnisI will add this to it - https://pasteboard.co/HAHD94p.png20:21
*** njohnston has left #openstack-infra20:21
smcginnisAJaeger: That one is odd in that the repo is named mistral-tempest-plugin, but the package name is mistral-tempest-tests20:22
AJaegerargh ;/20:22
smcginnisYeah, I thought of trying to get them to fix the package name first, but given there's been no response to several notifications that this work needed to be done and I'm assuming release team will have to force the release, I didn't want to have to wait on that.20:23
openstackgerritwes hayutin proposed openstack-infra/openstack-zuul-jobs master: wip: remove call to legacy copy jenkins role  https://review.openstack.org/59646920:25
corvusShrews: back.  anything i should do?20:25
AJaegersmcginnis: argh ;( Thanks20:26
AJaegerconfig-core, smcginnis just pushed https://review.openstack.org/#/q/topic:tempest-plugin+status:open - those all look fine. anybody to 2A to help smcginnis get rocky out of the door, please?20:26
*** armax_ has joined #openstack-infra20:26
smcginnisAJaeger: Thanks!20:26
fungicorvus: i just checked back, looks like we're merged up to the iptables role change and that one's failing a bunch of jobs. i haven't started to dig into logs for thos eyet20:26
*** armax has quit IRC20:28
*** armax_ is now known as armax20:28
corvusfungi: i see the next problem in the iptables patch.  i'll fix it then hold the next job so i can iterate faster on failure20:29
AJaegerconfig-core, https://review.openstack.org/596449 and https://review.openstack.org/592576 remove some noop-jobs, could you review, please?20:30
mnaserAJaeger: smcginnis publish-to-pypi tempest stuff lgtm20:31
smcginnismnaser, AJaeger: Thank you both for making that so easy.20:31
* mnaser adds to the beers owed counter :)20:32
mnaser:P20:32
AJaegerthanks, mnaser !20:32
smcginnis:)20:32
mnasernp20:32
AJaegermnaser: one more? https://review.openstack.org/592671, please ;)20:33
*** hasharDinner is now known as hashar20:33
* AJaeger waves good bye20:33
smcginnisAJaeger: o/20:33
corvusShrews, logan-: my understanding is we need https://review.openstack.org/576288 to fix the nodepool behavior which is causing us to create a bunch of "fake" zk nodes to try to delete the same stuck instance over and over.  when that is fixed, we would be in a state where we just have one stuck zk node for one stuck instance.  then it's up to logan- to find a way to convince openstack to actually20:33
corvusdelete that.  does that all sound right?20:33
Shrewscorvus: yep.20:35
Shrewscorvus: my fix to tristanC's patch failed. lemme look at that20:36
*** diablo_rojo has quit IRC20:36
*** olivierb_ has quit IRC20:38
corvusShrews: i also left a suggestion for a small change on that.  lemme know if you need me to look into the errors.  i'll keep tending the iptables, etc, fires in the mean time.20:38
*** rossella_s has quit IRC20:41
*** olivierb has joined #openstack-infra20:41
*** rossella_s has joined #openstack-infra20:42
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add iptables role  https://review.openstack.org/59397320:42
Shrewscorvus: oh, good find20:42
openstackgerritMerged openstack-infra/project-config master: Add publish-to-pypi for barbican-tempest-plugin  https://review.openstack.org/59645820:43
openstackgerritMerged openstack-infra/project-config master: Add publish-to-pypi for ec2api-tempest-plugin  https://review.openstack.org/59646020:43
pabelangercorvus: left a comment on 59397320:44
pabelangerI'm off to grab some food now20:45
mnaserAJaeger: +2, later! :)20:46
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add iptables role  https://review.openstack.org/59397320:46
corvuspabelanger: thx20:46
*** kgiusti has left #openstack-infra20:48
openstackgerritMerged openstack-infra/project-config master: Add publish-to-pypi for magnum-tempest-plugin  https://review.openstack.org/59646120:49
openstackgerritMerged openstack-infra/project-config master: Add publish-to-pypi for senlin-tempest-plugin  https://review.openstack.org/59646320:49
openstackgerritMerged openstack-infra/project-config master: Add publish-to-pypi for blazar-tempest-plugin  https://review.openstack.org/59645920:49
openstackgerritMerged openstack-infra/project-config master: Add publish-to-pypi for telemetry-tempest-plugin  https://review.openstack.org/59646420:49
notmynamewhere is lib-forward-testing-python3 defined?20:53
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: zk: skip node already being deleted in cleanup leaked instance task  https://review.openstack.org/57628820:56
Shrewscorvus: i think ^ will work now (and addresses your comment)20:57
*** onovy has joined #openstack-infra20:59
openstackgerritMerged openstack-infra/project-config master: Add publish-to-pypi for trove-tempest-plugin  https://review.openstack.org/59646521:06
openstackgerritMerged openstack-infra/project-config master: Add publish-to-pypi for zaqar-tempest-plugin  https://review.openstack.org/59646621:06
openstackgerritMerged openstack-infra/project-config master: Drop noop job of ui-cookiecutter  https://review.openstack.org/59257621:06
openstackgerritMerged openstack-infra/project-config master: Add publish-to-pypi for mistral-tempest-plugin  https://review.openstack.org/59646221:06
openstackgerritMerged openstack-infra/project-config master: Move afsmon publish-to-pypi here  https://review.openstack.org/59267121:06
corvusnotmyname: http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/project-templates.yaml#n121621:08
*** trown is now known as trown|outtypewww21:08
notmynamethanks21:08
notmynamecorvus: lib-forward-testing isn't in that same file?21:09
corvusnotmyname: http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/zuul-legacy-project-templates.yaml#n7621:09
Shrewscorvus: fyi, i did begin to look at unattended-upgrades until i got distracted (after feeling a bit lost). i did find this (https://github.com/jnv/ansible-role-unattended-upgrades) that maybe we can take advantage of21:12
*** hashar has quit IRC21:13
corvusShrews: cool, i think it's fine (preferable even) to use existing roles if they work21:13
corvusi don't think we're doing anything too weird with unattended upgrades21:13
*** rossella_s has quit IRC21:19
notmynamecorvus: so lib-forward-testing is really legacy-tempest-dsvm-neutron-src (although it runs more than just neutron?) and lib-forward-testing-python3 is actually tempest-full-py3 (but I can't find that one defined anywhere)21:19
*** rossella_s has joined #openstack-infra21:20
notmynamecontext: someone has proposed using lib-forward-testing-python3 to swiftclient, so I'm trying to figure out what it does21:20
notmynamecorvus: do you have any hints or tips on tracing through job definitions to find what's where? so far, I've got 3 repos to look through (project, zuul_jobs, and project_config) and I don't know how any of they "layer"21:23
corvusnotmyname: i'm not really familiar with those jobs :(  i assume the reference to neutron just means "uses neutron instead of n-net".  that would make them roughly equivalent in purpose i think.  i do not understand how lib-forward-testing-python3 has anything to do with libraries or forward testing.21:23
notmynameoh21:24
corvusnotmyname: we have changes in flight to make that easier in zuul, but until they land, codesearch.openstack.org is probably the most effective way.21:24
notmynamewhere can I go for more insight?21:24
corvusnotmyname: we may want to chat with dhellmann when he's back around next week21:25
notmynamewell, even once I find definitions, how do they work togehter? I don't understand how zuul works that way. what if there ist he same job defined in both the project and zuul_jobs21:25
corvusnotmyname: a job can only be defined in one repo (same for a project-template)21:26
notmynameto be frank, I don't believe you ;-)21:26
corvusnotmyname: https://zuul-ci.org/docs/zuul/user/config.html#configuration-loading  and  https://zuul-ci.org/docs/zuul/user/config.html#job if you want some bedtime reading :)21:27
*** boden has quit IRC21:27
notmynameok21:27
corvusnotmyname: heh, well, i mean, there was a bug once where that was broken, but i think it's been true for a while :)21:27
*** diablo_rojo has joined #openstack-infra21:29
notmynamewell for example, swift has had `legacy-tempest-dsvm-neutron-full` for a long time (ignoring the rabbit trail of "neutron" again). it's defined in zuul_jobs. but we've also got https://review.openstack.org/#/c/596210/ proposed (and https://review.openstack.org/#/c/595316/) that define it in swift's repo. for the patch 595316, it's seems to be additive maybe?21:29
notmyname...additive in that it doesn't redefine it, but it adds ignored files21:30
notmynamebut it's still defined in both places21:30
notmynameand we (swift team, but mostly timburke and I) are all just guessing at how these combine. but things generally seem to be going in the direction of "define everything in the project repo", and we really like that and think it will be a *lot* simpler to understand21:31
openstackgerritSaul Wold proposed openstack-infra/project-config master: stx-specs: Fix typo on label-Workflow  https://review.openstack.org/59649021:32
fungicorvus: notmyname: yes, the naming of the tempest-dsvm-neutron job (in the long-pre-zuulv3 days) was meant to disambiguate it from the even earlier tempest-dsvm job which used nova network (the job names weren't quite that, but you get the idea)21:32
corvusnotmyname: you're technically correct, which is the best kind of correct.  :)  internally in zuul, that is a "project-pipeline job variant" which is a form of job definition.  but it only applies to swift because it's attached to swift's project-pipeline configuration.  but it's probably easier to think of things in programming terms and consider that more like the "invocation" of the job.21:33
funginotmyname: basically additive, yes. in zuul terminology that's a job variant, not a job definition21:33
fungier, corvus also just said that while i was still looking at scrollback21:34
notmynameisn't the goal to remove the definition from zuul_jobs and have them only defined in the project?21:34
notmynameor is it that jobs are defined in zuul_jobs and invoked in the project? is that the future goal?21:35
fungiwell, not the zuul-jobs repo for sure. that's basically the zuul project's stdlib of jobs. from the openstack-zuul-jobs repo yes some of them, but jobs used by multiple projects we may still want to define there21:36
corvusnotmyname: partly, but also, to enable cross-project collaboration.  so swift might have a job defined in its repo, which inherits from the tempest job defined in tempest, which inherits from devstack defined in devstack, and so on...21:36
fungithe lists of what jobs are being run for projects is a big chunk of what's in the current push to move into projects. it makes only a meager amount of sense to centralize that information since it's per-project info anyway21:37
corvusthe lib-forward-testing templates (or something like them), should really probably be in the tempest repo.21:38
fungibasically put the job definitions themselves in repos under the control of the people best positioned to maintain them21:39
*** EvilienM is now known as EmilienM21:39
fungiso a swift functional job (which might be run against changes to both swift and python-swiftclient) could make sense to go in the swift repo21:39
notmynameoh, interesting. lib-forward-testing-python3 is actually tempest-full-py3 which *is* defined in tempest, but it explicitly disables swift (http://git.openstack.org/cgit/openstack/tempest/tree/.zuul.yaml#n72), so it seems it doesn't have any business being run on swift project patches21:41
funginotmyname: at least not until it stops disabling swift (which it does currently because it can't run swift under python3)21:41
*** elod has quit IRC21:42
corvus(if you want to see the WIP job pages, it's here: http://logs.openstack.org/56/591156/2/check/zuul-build-dashboard/a1d106d/npm/html/job.html?job_name=tempest-full-py3 )21:43
fungior, rather, because it couldn't at the time the job was designed anyway. i haven't kept up with the latest in the swift python3ing21:43
notmynamefungi: we're still working on the "end of T" timeline21:43
corvus(that's not quite ready to land yet, but it should give you an idea.  you can click the parent link and follow the trail up.  when you get to devstack, you'll see all the different branch variants)21:43
funginotmyname: cool, that rings a bell from discussions at the last forum21:43
notmynamecorvus: interesting. does "WIP job pages" mean that these are jobs that are proposed?21:45
fungidraft renderings of the zuul dashboard built from changes to zuul which haven't merged yet21:45
notmynameah ok21:45
fungiso you're looking at future zuul under review, being rendered by current zuul21:46
fungior, rather, built in a job run by zuul21:46
corvusyeah.  it's hitting the live api endpoint, just the web app itself is under review.  those pages have some rough edges yet to polish before we merge21:46
fungii wonder how much of https://docs.openstack.org/infra/system-config/third_party.html#requirements is relevant to third-party ci systems reporting on projects which are not official openstack projects. a lot of that came out of things the neutron team expected from third-party ci systems testing neutron (and later other openstack projects) but a lot of it seems less relevant to ci systems operated by21:59
fungithe projects they're reporting on21:59
notmynamecorvus: fungi: thanks for the info :-)21:59
corvusfungi: i'm curious which of those (other than openstack-specific log file names) shouldn't apply.  they were written to set a baseline expectation and set some consistent behavior to make things easier for developers22:01
corvusfungi: for instance, if you're asking because of AJaeger's email, that came about because there's a ci system leaving comments with inacessible urls.  i would think that's something that would be considered undesirable in all cases.22:03
fungii think it's less that they shouldn't apply, and more that when it's the same developers running that ci system they may be iterating on tuning and improving it through pressure from their own team anyway. just wondering if we need to confront them with policy when we spot an infraction which wasn't brought to us by the people reviewing the changes it reports on22:04
openstackgerritMerged openstack-infra/nodepool master: zk: skip node already being deleted in cleanup leaked instance task  https://review.openstack.org/57628822:04
corvusfungi: well, i rather suspect it may not have occurred to them that the url is inacessible22:04
fungiright, more that "these are some things you should be aware of" than "you must comply with this policy" i guess22:05
corvuswe may disagree on that.  i think that's the policy :)22:05
fungithough i also suppose given anyone might want to review any changes for any project we're hosting, consistency in what to expect from other ci systems reporting on those changes would be nice to have22:05
*** olivierb has quit IRC22:06
corvusclearly, for winterscale, the policy should be revised.  but i think we should keep the core of it intact.22:06
fungii do think most of it is applicable22:06
fungithings like e-mailing openstack-dev asking to be whitelisted need revisiting22:06
corvusyes :)22:06
*** eharney has quit IRC22:07
fungii was mainly rereading it as i hadn't looked at it in a while, and noticing at least a handful of what we'd likely consider openstackisms22:07
corvusbut "logs must be publicly acessible", "have contact info", and "don't report on projects without asking" should be universal :)22:07
fungiyep. the multiple comments thing and the must support the recheck command seem like a maybe22:08
fungirecheck is certainly useful. avoiding leaving unnecessary comments reduces load on our systems too22:08
fungibut we present a lot of those things as a must when some of them seem more like a should22:09
corvusleaving multiple comments is sort of a bad idea since you only get one vote.  so if someone is doing that to run two systems on one account, that person may not be aware of the negative implications22:09
corvusi'd suggest we keep the multiple-comments one just on the basis of preventing people from foot-gunning.  or asking us why the vote is wrong.  :)22:10
corvusi have no strong feelings on recheck outside of openstack22:11
fungiyes, i'm really just thinking how some of it might be reworded to separate out things which you need to do because we'll turn you off if you don't comply and things which you need to address because, well, you've got a broken configuration22:11
corvusthere's plenty of room for that i agree22:11
fungii'll mull it over22:13
fungiwe'll have a lot of documentation to revisit in light of winterscale22:13
*** xavinux has joined #openstack-infra22:14
xavinuxHi people, I'm Javier form Argentina. New to the group and interested in contributing  to the infra team22:17
*** apetrich has joined #openstack-infra22:17
xavinuxNow reading the information about the infrastructure project22:17
corvusfungi: "/etc/init.d/netfilter-persistent reload" exits 0 on failure.  :(22:19
corvusxavinux: welcome!  let us know if you have questions22:19
pabelangercorvus: AJaeger: it looks like git is creating the work dir in use-cached-repos as 0777: https://github.com/git/git/blob/master/builtin/clone.c#L988 but first creating the /home/zuul/src folder, git seems to use umask: http://logs.openstack.org/21/596421/8/check/test2/9d2967e/job-output.txt.gz#_2018-08-24_22_13_42_04961022:21
xavinuxcorvus: Thanks!22:21
pabelangercorvus: AJaeger: so fix does seem to be to update use-cached-repo to first create the ~/src directory, then git clone22:22
corvuspabelanger: won't that just fix ~/src?  what about ~/src/git.openstack.org?22:23
fungicorvus: hrm, that's a mightily disappointing discovery. i wonder if we should call the /usr/sbin/netfilter-persistent script directly instead22:23
fungiwelcome xavinux!22:23
corvusfungi: that exits 122:23
pabelangercorvus: ~/src/git.openstack.org will get 0775: http://logs.openstack.org/21/596421/8/check/test2/9d2967e/job-output.txt.gz#_2018-08-24_22_13_42_93613122:24
pabelangerI am looking to see why it isn't 075522:24
pabelangerbut might continue testing over the weekend22:24
fungiso `/usr/sbin/netfilter-persistent start` to reload i suppose?22:25
fungilooks like that's what the initscript runs anyway22:25
corvusyeah, i'll switch to that22:25
fungigood catch22:25
corvusi'll need to do something different on trusty, presumably22:26
fungilooking22:26
*** tpsilva has quit IRC22:28
fungi`iptables-restore < /etc/iptables/rules.v4 && ip6tables-restore < /etc/iptables/rules.v6`22:28
pabelangercorvus: does service iptables-persistent reload work on trusty?22:28
pabelangerand service netfilter-persistent on xenial+22:28
pabelangerthat's how I read puppet-iptables22:28
fungipabelanger: the service definition (at least the sysvinit script version) exits 0 on failure, which is what brought this up22:29
fungiso calling `/usr/sbin/netfilter-persistent start` on xenial and bionic should work around that22:29
pabelangerodd22:29
corvusfungi: maybe i'll just use the init script on trusty22:30
fungithat seems reasonable22:30
fungihoping we don't end up maintaining the trusty compat code for too much longer22:31
*** rlandy has quit IRC22:33
corvusthis role is not as finished as i thought it was when i picked it up :)22:34
*** pbourke has quit IRC22:35
*** rh-jelabarre has quit IRC22:37
*** pbourke has joined #openstack-infra22:37
*** rossella_s has quit IRC22:40
*** rossella_s has joined #openstack-infra22:43
*** armax has quit IRC22:45
*** xavinux has quit IRC22:52
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add iptables role  https://review.openstack.org/59397322:52
*** gyee has quit IRC22:52
corvusokay, that works and passes testinfra in manual testing22:52
notmynameif I recheck something that's currently in the gate queue, will it be immediately re-enqueued?23:09
corvusnotmyname: it will stay in gate.  i think it will also be added to check.  that may mean that if it fails gate it will have already started the process to go through check and gate again.23:11
mnaserall these cool infra playbooks23:11
mnasers/playbooks/roles/23:12
mnaserwe have a bunch of stuff internally that are similar, so i'll look into seeing if i can help move them out to something we can both consume23:12
*** jamesmcarthur has joined #openstack-infra23:18
*** slagle has joined #openstack-infra23:19
*** jamesmcarthur has quit IRC23:23
*** r-daneel has quit IRC23:24
fungiacceptance tests task for puppet-beaker-rspec-infra-centos-7-system-config seems remarkably unhappy on the most recent patchset of 59397323:30
corvusfungi: yeah, it's the same problem that hit the snmp change23:30
corvusexim isn't running23:30
corvusneither of those changes had anything to do with exim23:31
corvusthe only thing i can see that's related is that they added a second role that runs after the exim role23:31
fungiyeah, "expected Service "exim" to be running"23:31
fungihuh23:31
fungiwas it just centos-7 failing the snmp change too?23:32
corvusi have two diverging thoughts on this: 1) i want to understand what has changed to cause the test to fail in this way.  2) we shouldn't actually test that exim is running after the base playbook, because that's one of the services that we shouldn't force to be running anyway.23:32
corvusfungi: yep23:32
corvusand it's reliable/consistent23:32
corvusso ultimately, i'd like to just remove the rspec test for the exim service running since i think that's the right thing to do.  but if we do that now, we won't understand this strange behavior (which could be an ansible problem)23:33
fungiyep, and could crop up for other services we actually want running23:33
*** diablo_rojo has quit IRC23:36
corvusfungi: http://paste.openstack.org/show/728779/23:46
corvusthe thing that captures my eye there is that the output of that is all about iptables.  not exim.23:46
corvusit looks a lot like ansible has gotten its handlers mixed up.23:46
corvusoh, i have a theory...23:47
corvusi think we're seeing ansible's variable binding/scoping in action23:48
corvusthe handler runs at the end of the playbook.  and it's late-binding variables.  so whereas normally all the tasks in a role can assume that the variables were set by the vars/main.yaml file in that role, handlers can not.23:49
corvusthe service name is a variable that's used by the exim restart handler, and it's overidden by the role which runs after exim23:49
corvusoh, maybe we should flush handlers here23:53
*** sthussey has quit IRC23:55
fungiahh, is it a variable scoping problem we should be addressing with more namespacing in that case?23:55
fungivariable name collision between the service handlers, specifically?23:55
*** tristanC has quit IRC23:56
corvusfungi: namespacing is an option, but these are variables which are intended only to be used within the role.  i would find it extra frustrating to have to make *all* variables globally unique.  i think, if it works, it will be safer to flush the handlers at the end of the role tasks.  there's no reason why the exim restart needs to wait until all the other things are finished anyway.23:57
corvus(another option, short of namespacing, would be not to use variables in handlers; a little more verbose, but pretty easy to follow)23:57
corvusyay, it looks like flush_handlers works23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!