Wednesday, 2015-12-09

*** _denisra has joined #openstack-infra00:00
*** mattoliverau has quit IRC00:00
anteayanotmorgan: nicely said00:00
*** Daisy has quit IRC00:01
purpHey, all. Anyone aware of etherpad issues?00:02
notmorganpurp: what kind of issues?00:03
purpnotmorgan: Can't load at all.00:03
purpFrom California or Boston.00:03
notmorganpurp: hm. it's loading ok for me here in PDX00:03
*** dimtruck is now known as zz_dimtruck00:03
*** yuanying has quit IRC00:04
anteayaisn't loading for me yet in Canada00:04
notmorganpurp: which provider (CA / BOS)? comcast here00:04
anteayawas earlier today00:04
anteayafinally rendered00:04
anteayabut took a while00:04
purpI've got ATT. Comcast in BOS.00:05
purpAnd now working.00:05
purpGot a proxy error in the meantime.00:05
*** yuanying has joined #openstack-infra00:05
*** Sukhdev has quit IRC00:05
anteayatraffic spike00:05
clarkb(db backups)00:06
anteayapurp: timing for the win00:06
anteayaI always forget the backup timing00:06
*** pahuang has quit IRC00:06
*** aeng has quit IRC00:06
*** dewsday has quit IRC00:06
*** sflanigan has quit IRC00:07
notmorganclarkb: ah00:07
openstackgerritHaomeng,Wang proposed openstack-infra/project-config: skip ironic client *dsvm* gates for doc patches
*** dewsday has joined #openstack-infra00:08
*** sflanigan has joined #openstack-infra00:08
*** sflanigan has joined #openstack-infra00:08
*** otter768 has joined #openstack-infra00:08
*** Hink has quit IRC00:08
*** hdd has quit IRC00:09
*** Hink has joined #openstack-infra00:09
*** Sukhdev has joined #openstack-infra00:09
*** yamamoto has joined #openstack-infra00:09
*** matt6434 has joined #openstack-infra00:12
*** Apoorva has joined #openstack-infra00:12
*** eil397 has quit IRC00:13
*** rbrndt has quit IRC00:14
openstackgerritClaudiu Belu proposed openstack/requirements: Bumps os-win version to 0.0.7
*** jamesmcarthur has quit IRC00:15
*** matt6434 has quit IRC00:17
*** aeng has joined #openstack-infra00:18
*** salv-orl_ has joined #openstack-infra00:18
*** aeng has quit IRC00:18
*** aeng has joined #openstack-infra00:18
*** akshai has quit IRC00:18
*** otter768 has quit IRC00:19
*** alex_xu has quit IRC00:19
*** Hink has quit IRC00:19
*** pahuang has joined #openstack-infra00:20
*** otter768 has joined #openstack-infra00:20
ianwarrrgh f23 puppet, why you fail now!00:20
*** salv-orlando has quit IRC00:20
*** alex_xu has joined #openstack-infra00:21
*** yamamoto has quit IRC00:21
*** cschwede has quit IRC00:21
*** bdemers_ has joined #openstack-infra00:21
*** EricGonczer_ has joined #openstack-infra00:21
*** psanchez has quit IRC00:21
*** Hink has joined #openstack-infra00:22
*** migi has quit IRC00:22
*** darvon has quit IRC00:22
*** cschwede has joined #openstack-infra00:22
*** _denisra has quit IRC00:22
*** zul has quit IRC00:22
*** eil397 has joined #openstack-infra00:22
*** darvon has joined #openstack-infra00:23
*** mordred has quit IRC00:23
*** zul has joined #openstack-infra00:23
*** migi has joined #openstack-infra00:23
*** mordred has joined #openstack-infra00:23
*** psanchez has joined #openstack-infra00:23
*** _denisra has joined #openstack-infra00:23
openstackgerritHaomeng,Wang proposed openstack-infra/project-config: skip ironic client dsvm and pypy gates for doc patches
*** bdemers has quit IRC00:24
*** bdemers_ is now known as bdemers00:24
*** otter768 has quit IRC00:24
ianwnibalizer: <- cfacter might be the issue ... why did i not see this on local runs00:25
*** gyee has joined #openstack-infra00:25
*** sdake has quit IRC00:25
*** sridhar_ram has quit IRC00:25
*** matt6434 has joined #openstack-infra00:25
*** matt6434 is now known as mattoliverau00:27
*** weshay has quit IRC00:27
ianwyou know you're in trouble when the top match for fedora and cfacter is a story "Sarah Jessica Parker rocks a stylish fedora ..."00:27
*** chlong has joined #openstack-infra00:28
*** gokrokve has joined #openstack-infra00:28
*** ashleighfarnham has quit IRC00:29
openstackgerritClaudiu Belu proposed openstack/requirements: Bumps os-win version to 0.0.7
anteayaha ha ha00:29
mgagnetrue, first page, last result for me00:29
pleia2the only meeting scheduled for the near future was canceled, so if someone wants to look at this patch to finish the ceilometer channel move, now's a good time :)
anteayaand noone I know of rocks a fedora more stylishly than jeblair00:30
*** mkoderer has quit IRC00:30
*** FallenPegasus is now known as MarkAtwood00:30
anteayapleia2: +100:31
clarkbpleia2: done00:31
anteayait's all I have to offer there00:31
pleia2clarkb: thanks00:31
anteayaoh good00:31
anteayais there any limit on how many times a project can go back and forth between the same two names?00:31
pleia2I don't think it's come up00:32
pleia2but in this case I think it's because it's no longer just ceilometer, there are a few telemetry projects in the mix00:32
*** yuanying has quit IRC00:32
*** sridhar_ram has joined #openstack-infra00:32
*** ZZelle_ has quit IRC00:33
*** yuanying has joined #openstack-infra00:34
*** pvaneck has quit IRC00:36
*** mkoderer has joined #openstack-infra00:37
openstackgerritJames E. Blair proposed openstack-infra/zuul: Add tenants
openstackgerritJames E. Blair proposed openstack-infra/zuul: Disable most tests
*** _nadya_ has joined #openstack-infra00:38
ianwnibalizer / pliea2 : so this cfacter change merged last night my time,  -- which is why all my f23 testing yesterday my time didn't pick it up as an issue00:39
jeblairmost of the tests actually still work, but every single one of them needs configuration changes, and that will make our eyes bleed, so i'm deferring it for now00:39
ianwnibalizer / pliea2 : so, what are you thinking for distros like f23 that don't have puppetlabs branches for stuff?  skip?00:39
*** Sukhdev has quit IRC00:40
jeblair(long term answer: stop using puppet to configure test nodes)00:40
openstackgerritMerged openstack-infra/system-config: Rename Ceilometer to Telemetry
pleia2ianw: ah, boo, what is f23 using? native packages?00:41
clarkbianw: maybe we should just revert and rely on facter facter00:41
pleia2this is what I was worried about00:41
pabelangerclarkb: ++00:41
ianwpliea2: yep, native packages.  puppetlabs is usually a long lag on new fedoras00:41
pleia2I think it's revert then00:41
jeblairrevert is probably a good enough solution until the long-term stop using puppet on test nodes (only use dib)00:41
jeblair(i honestly don't think it's that far out once the rest of the bindep/dib stuff is done)00:42
*** dewsday has quit IRC00:42
*** gokrokve has quit IRC00:43
*** _nadya_ has quit IRC00:43
*** tsg has quit IRC00:43
*** gongysh_ has joined #openstack-infra00:44
openstackgerritIan Wienand proposed openstack-infra/system-config: Revert "Add cfacter to puppet3 nodes"
*** gokrokve has joined #openstack-infra00:44
*** yuanying has quit IRC00:44
ianw^ ok, that's the revert option00:44
ianwi don't mind doing a skip for f23, but it just seems it will break again if support for it is enabled00:45
*** tonytan4ever has quit IRC00:45
* pleia2 nods00:45
pabelangerwe'll need to a logic to ensure cfacter is absent actually, since puppet processed that code00:45
pabelangerotherwise, manual deletes all around00:45
clarkbpabelanger: I don't think we need to remove cfacter if its there00:45
clarkbor do we?00:45
*** Sukhdev has joined #openstack-infra00:46
pabelangerunsure, its now and unmanaged application00:46
pabelangerwell, will become unmanaged00:46
pleia2there is that00:47
*** yuanying has joined #openstack-infra00:47
ianwso add an ensure absent?00:47
pabelangerand now, I step away00:47
clarkbI doubt we need it00:48
clarkbunless we really want the package gone00:48
*** gokrokve has quit IRC00:49
ianwi feel like since you've got to manually fiddle config files to use it, probably can leave it, but happy to change too00:50
*** openstackstatus has quit IRC00:50
*** openstack has joined #openstack-infra00:54
nibalizeris there a cfacter package in f23?00:54
nibalizeri heard there was a puppet4 package00:54
nibalizeranyways we can revert the cfacter change00:55
nibalizeror just put some "not on fedora" logic in00:55
pleia2ianw: was suggesting the opposite (opt in, rather than opt out)00:57
ianwi don't care, i just want my build to work :)  i was so close!00:57
pleia2indeed :)00:57
nibalizerhah sorry!00:57
nibalizerwhere is fedora on puppet4 and cfacter packaging?00:58
ianwwell it's 4.2.1 i guess00:58
*** gokrokve has joined #openstack-infra00:58
ianwf22 shipping python4 was one of the big hold ups getting that working00:58
nibalizeralso i am on phone00:58
ianwpuppet4 even00:58
nibalizeris there a cfacter package?00:59
ianwnot that i can see01:00
nibalizerhow are you installing puppet 3?01:00
*** sridhar_ram has joined #openstack-infra01:01
lifelesspython 4?01:01
ianwwe're not for fedora, just using the native 4 packages (see prior note about having to modify stuff for puppet4 during f22 cycle)01:01
lifelessianw: wat^ ?01:01
lifelessianw: oh, puppet ;)01:01
ianwlifeless: typo, i meant puppet01:01
jrollpython 4 could be so fun though!01:01
crinklei think cfacter == facter 3.0 == default for puppet 4, at least from puppetlabs' perspective, not sure what f23 does01:02
ianwwell the "facter" package installed in f23 is 2.4.301:03
nibalizeri think thats right^^01:04
pabelangercan confirm 2.4.301:04
openstackgerritThanh Ha proposed openstack-infra/jenkins-job-builder: Add view management functionality
nibalizerso then the fedora packagers screwed up?01:04
nibalizerdidnt want to build the c++ toolchaij?01:05
crinklewe should maybe start testing system-config changes on f23 if we want to avoid this in the future01:06
*** bpokorny_ has joined #openstack-infra01:06
crinkleby only testing on ubuntu and centos 7 we're sort of implying that's all we support01:06
crinklei mean beaker testing01:07
jeblairit is01:07
*** hichihara has joined #openstack-infra01:07
nibalizerit doesnt sound like we can build f23 nodes because of this01:07
clarkbdo we have beaker tests for the slaves?01:07
openstackgerritMerged openstack/diskimage-builder: Fix grub-efi-amd64-signed install failure
jeblairthe problem is that we're using puppet to configure slaves01:07
clarkbright chicken and egg01:07
jeblairwe need to stop doing that01:07
nibalizerpabelanger: ya that was harsher than it needed to be01:07
jeblairthat can happen soon01:07
nibalizeri apologize01:07
jeblairso let's just get to there, rather than wasting effort on testing system-config on platforms it won't be used on01:08
ianwnibalizer: yeah, i can find out from maintainers, like i said before, searching for info lead to articles about sarah jessica-parker's head-wear, so no luck there :)01:09
nibalizerok im out01:09
*** Swami_ has quit IRC01:09
*** bpokorny has quit IRC01:09
nibalizeri reccomend we just revert the cfacter thing01:10
*** bpokorny_ has quit IRC01:10
ianwnibalizer: cool, i'll loop you in on some queries so we can understand the way forward01:11
pabelangeropen bugs to bump to 3.x01:13
pabelangerjust lacking help moving it along01:13
jheskethclarkb: I'm travelling at the moment but will reply to your comments on the swift stuff later. The point of the workers though is not to do with race conditions uploading but being able to create metadata higher up the tree than where the temp url has granted01:13
clarkbjhesketh right so thats why one idea I had was to use hashed paths01:14
*** ParsectiX has joined #openstack-infra01:14
clarkbthen allow writes to the metadata container with those objects named after hashed paths01:14
*** Sukhdev has quit IRC01:15
clarkbso logs/foo/bar has 3 subpaths01:15
clarkbeach gets hashed and that object gets updated01:16
clarkbthen to check timestamps pull those objects back out01:16
*** ParsectiX has quit IRC01:16
jheskethRight so we keep a master index elsewhere out of the log tree01:16
jheskethOf sorts01:17
*** ParsectiX has joined #openstack-infra01:17
jheskethThat makes a lot of sesne01:17
jheskethI'll redo the spec with that, thanks!01:17
fungithat got put forth as a solution a while back actually01:17
jheskethclarkb: do you think we should pass a second hmac for the metadata?01:17
fungifeels like we're rewriting the missing parts of a posix filesystem... i guess because we are!01:18
jheskethYeah I likely forgot or missed it01:18
openstackgerritSandro Mathys proposed openstack-infra/system-config: Add statusbot/eavesdrop to #midonet channel
clarkbjhesketh ya I dont think it ahould share the same paths as the logs01:18
clarkbto avoid confusion01:18
clarkbfungi yes we are :(01:18
jheskethAnd yes, we totally are01:18
jheskethWhich is why we revisited our path forward briefly at the summit01:18
fungiwriting a filesystem table bow01:18
fungier, now01:18
jheskethclarkb: agree they shouldn't be on the log path. But is a second hmac the best way forward (I think it's fine, just thinking out loud)01:19
*** tqtran has quit IRC01:19
pleia2^^ that #midonet change has been waiting a bit too (got snagged by the pep8 bug over the weekend), no meetings right now01:19
openstackgerritMerged openstack-infra/shade: Improve test coverage: hypervisor list
clarkbya I think second path requires second hmac?01:20
jheskethpleia2: I'll try to put together an announcement in an etherpad this afternoon for review01:20
jheskethclarkb: unless another service does the metadata stuff, yez01:20
pleia2jhesketh: great, thanks, want to update the wiki too?
jheskethYep, will do after travel01:21
*** sridhar_ram has quit IRC01:22
*** krtaylor has quit IRC01:22
pabelangerHow do I get support for I've tried using the lost password field and entered my email address but I never receive an email01:23
*** sridhar_ram1 has joined #openstack-infra01:24
*** zhenguo has joined #openstack-infra01:24
fungipabelanger: it's trying to e-mail whatever e-mail address(es) you have in your openstack foundation profile01:24
jheskethclarkb: so the downside of the hashes is that we have to update all the metadata for each path. Whereas in a normal posix filesystem only the folders which have new (or moved) items would be updated01:24
jheskethThis is therefore a difference to on disk, but it's possibly not one that will matter01:24
*** sridhar_ram1 has quit IRC01:25
jheskethI'll try to explain it in the spec so we can get those who use the logs heavily to take a look01:25
fungijhesketh: yep, that's a simplicity/performance tradeoff01:26
jheskethAgreed. Just want to make sure it's acceptable first01:27
fungidefinitely pros and cons to each method01:27
pabelangerfungi: should be my address, but never see anything. Will check again in the morning01:27
*** rhallisey has joined #openstack-infra01:30
*** gongysh_ has quit IRC01:30
openstackgerritMerged openstack/diskimage-builder: Load the 8021q kernel module in simple-init
*** _nadya_ has joined #openstack-infra01:32
openstackgerritLi Yingjun proposed openstack/requirements: Add searchlight to projects.txt
*** krtaylor has joined #openstack-infra01:36
*** _nadya_ has quit IRC01:37
*** sridhar_ram has joined #openstack-infra01:37
*** jamesmcarthur has joined #openstack-infra01:38
*** alex_xu has quit IRC01:41
*** alex_xu has joined #openstack-infra01:43
openstackgerritArmando Migliaccio proposed openstack-infra/project-config: Add release note files to the skip-if list for neutron jobs
*** sridhar_ram has quit IRC01:45
*** jamielennox is now known as jamielennox|away01:47
*** dewsday has joined #openstack-infra01:47
*** Liuqing has joined #openstack-infra01:48
*** NobodyCam has joined #openstack-infra01:49
*** cp16net has joined #openstack-infra01:49
*** jesusaurus has joined #openstack-infra01:50
*** gokrokve has quit IRC01:50
openstackgerritMichael Micucci proposed openstack-infra/system-config: Added eavesdrop and statusbot for openstack-zephyr
*** vilobhmm11 has quit IRC01:52
*** vilobhmm11 has joined #openstack-infra01:53
*** vilobhmm11 has quit IRC01:53
*** vilobhmm11 has joined #openstack-infra01:53
vilobhmm11fungi : ping01:55
*** EricGonczer_ has quit IRC01:56
*** Sukhdev has joined #openstack-infra01:56
openstackgerritlifeless proposed openstack-dev/cookiecutter: Update min tox version to match current practice
vilobhmm11lifeless: ping01:58
*** Liuqing has quit IRC01:58
*** Sukhdev has quit IRC01:59
*** sridhar_ram has joined #openstack-infra01:59
*** dewsday has quit IRC02:00
*** sridhar_ram has quit IRC02:02
*** Swami has joined #openstack-infra02:02
*** armax has quit IRC02:02
*** chlong has quit IRC02:03
vilobhmm11lifeless : I am trying to add non-voting job for tooz consul driver
vilobhmm11lifeless : looks like the tag voting: flase can only be applied at project level
vilobhmm11won't it apply voting: false to all the check and gate02:04
vilobhmm11lifeless : ^602:05
lifelessvilobhmm11: there's a huge number of voting: false entries up the top of layout.zzuul02:06
lifelessvilobhmm11: following that pattern should work02:07
*** yamamoto has joined #openstack-infra02:09
*** Swami has quit IRC02:09
vilobhmm11lifeless : I get that but that is at the jobs level nothing at the project "check" "gate" …does the parameter apply there as well02:10
*** yamamoto has quit IRC02:10
openstackgerritVilobh Meshram proposed openstack-infra/project-config: Add Consul Gate Job
*** bharathm has quit IRC02:13
pabelangervilobhmm11: you are missing the job in JJB. There is not such thing as gate-{name}-{envlist}-nv so you need to add it02:13
pabelangerother wise, just add something into the jobs section for zuul making it non-voting02:14
*** gokrokve has joined #openstack-infra02:14
pabelangervilobhmm11: ^ that review will fail, not valid syntax02:14
*** Apoorva has quit IRC02:15
*** Apoorva has joined #openstack-infra02:16
*** eil397 has quit IRC02:17
*** mtanino has quit IRC02:17
*** zhurong has joined #openstack-infra02:18
*** Daisy has joined #openstack-infra02:23
*** zhurong has quit IRC02:25
*** ashleighfarnham has joined #openstack-infra02:25
lifelessvilobhmm11: it does02:26
*** zaro has joined #openstack-infra02:27
*** samueldmq has joined #openstack-infra02:27
*** devananda has joined #openstack-infra02:27
*** gokrokve has quit IRC02:30
*** ashleighfarnham has quit IRC02:30
*** gokrokve has joined #openstack-infra02:31
openstackgerritClaudiu Belu proposed openstack/requirements: Bumps os-win version to 0.0.7
*** zhurong has joined #openstack-infra02:33
ianwif anyone could approve the cfacter revert we agreed on, well that would be great :) ->
*** gokrokve has quit IRC02:38
*** thorst has joined #openstack-infra02:40
*** aeng has quit IRC02:40
openstackgerritKhai Do proposed openstack-infra/puppet-gerrit: Allow puppet to enabling or disabling Gerrit drafts feature
ianwfungi: thanks!  round round nodepool goes ... will it work ... nobody knows!02:41
openstackgerritMerged openstack/requirements: Updated from generate-constraints
openstackgerritMerged openstack-infra/system-config: Revert "Add cfacter to puppet3 nodes"
openstackgerritlifeless proposed openstack-dev/cookiecutter: New projects should come constraint enabled.
*** nelsnelson has quit IRC02:49
*** nelsnels_ has joined #openstack-infra02:49
*** paulbernard has joined #openstack-infra02:50
*** jamielennox|away is now known as jamielennox02:50
openstackgerritKhai Do proposed openstack-infra/system-config: Disable Gerrit drafts feature
*** baoli has joined #openstack-infra02:52
openstackgerritKhai Do proposed openstack-infra/system-config: Disable Gerrit drafts feature
lifelesssamueldmq: you took over right ?02:53
*** aeng has joined #openstack-infra02:53
openstackgerritKhai Do proposed openstack-infra/system-config: Disable Gerrit drafts feature
*** Qiming has joined #openstack-infra02:56
openstackgerritlifeless proposed openstack-dev/cookiecutter: New projects should come constraint enabled.
ianw2015-12-09 02:57:20,327 INFO + output '20-prepare-node completed'02:58
*** gokrokve has joined #openstack-infra02:59
*** yamamoto has joined #openstack-infra02:59
greghaynesianw: w00t03:01
*** yamahata has quit IRC03:01
*** greghayn1 has joined #openstack-infra03:02
*** greghayn1 has quit IRC03:02
*** gokrokve has quit IRC03:04
*** binbincong has quit IRC03:05
*** binbincong has joined #openstack-infra03:05
*** vilobhmm11 has quit IRC03:07
*** baoli has quit IRC03:09
*** Apoorva has quit IRC03:14
*** chlong has joined #openstack-infra03:16
*** Daisy has quit IRC03:18
*** yuanying has quit IRC03:23
*** Piet has joined #openstack-infra03:23
*** links has joined #openstack-infra03:24
*** thorst has quit IRC03:25
*** Piet has quit IRC03:26
*** mtanino has joined #openstack-infra03:31
*** unicell has quit IRC03:32
*** ParsectiX has quit IRC03:32
*** alex_xu_ has joined #openstack-infra03:40
*** alex_xu has quit IRC03:42
*** MarkAtwood has quit IRC03:42
*** MarkAtwood has joined #openstack-infra03:43
*** thorst has joined #openstack-infra03:43
*** aeng has quit IRC03:44
*** zhurong has quit IRC03:44
*** thorst has quit IRC03:44
*** thorst has joined #openstack-infra03:45
*** zhurong has joined #openstack-infra03:45
*** rhallisey has quit IRC03:46
*** chlong has quit IRC03:49
*** zhurong has quit IRC03:50
*** zhurong has joined #openstack-infra03:52
*** fawadkhaliq has joined #openstack-infra03:53
*** chlong has joined #openstack-infra03:53
*** thorst has quit IRC03:54
*** flwang1 has quit IRC03:55
*** MarkAtwood has quit IRC03:57
*** changbl has joined #openstack-infra04:00
*** aeng has joined #openstack-infra04:00
*** fawadkhaliq has quit IRC04:00
*** Hink has quit IRC04:01
*** links has quit IRC04:03
*** gyee has quit IRC04:03
*** mtanino has quit IRC04:03
*** tlian2 has quit IRC04:04
*** SergK has quit IRC04:06
*** rguillebert has quit IRC04:07
*** SergK has joined #openstack-infra04:10
*** dims has joined #openstack-infra04:13
*** dims has quit IRC04:15
*** rlandy has quit IRC04:17
*** dims has joined #openstack-infra04:18
openstackgerritThanh Ha proposed openstack-infra/jenkins-job-builder: Add view management functionality
*** paulbernard has quit IRC04:20
*** changbl has quit IRC04:21
*** gokrokve has joined #openstack-infra04:22
*** ashleighfarnham has joined #openstack-infra04:26
*** akshai has joined #openstack-infra04:28
*** yuanying has joined #openstack-infra04:28
*** dims has quit IRC04:29
*** gokrokve has quit IRC04:30
*** _denisra has quit IRC04:30
*** ashleighfarnham has quit IRC04:31
*** changbl has joined #openstack-infra04:35
*** links has joined #openstack-infra04:38
*** vilobhmm11 has joined #openstack-infra04:38
*** aeng has quit IRC04:43
*** bpokorny has joined #openstack-infra04:44
*** ramineni1 has joined #openstack-infra04:45
*** fawadkhaliq has joined #openstack-infra04:45
*** yuanying has quit IRC04:46
*** fawadkhaliq has quit IRC04:47
openstackgerritAnusha Ramineni proposed openstack-infra/project-config: Congress: Add different gate job for stable branch
*** thorst has joined #openstack-infra04:52
*** Ramanjaneya has joined #openstack-infra04:54
*** bpokorny has quit IRC04:55
*** aeng has joined #openstack-infra04:55
openstackgerritKhai Do proposed openstack-infra/puppet-gerrit: Allow puppet to enabling or disabling Gerrit drafts feature
*** thorst has quit IRC04:58
*** yuanying has joined #openstack-infra05:00
openstackgerritKhai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure the Gerrit drafts feature
*** gokrokve has joined #openstack-infra05:01
openstackgerritKhai Do proposed openstack-infra/system-config: Disable Gerrit drafts feature
*** boris-42_ has quit IRC05:03
*** kushal has joined #openstack-infra05:04
*** persia has quit IRC05:06
*** persia has joined #openstack-infra05:09
openstackgerritSachi King proposed openstack-dev/pbr: WIP: Handle markers to support sdist on pip < 6
*** Sukhdev has joined #openstack-infra05:15
*** sdake has joined #openstack-infra05:15
*** harlowja_at_home has joined #openstack-infra05:19
*** dims has joined #openstack-infra05:20
*** ParsectiX has joined #openstack-infra05:22
*** Qiming has quit IRC05:22
*** ParsectiX has quit IRC05:22
*** Qiming has joined #openstack-infra05:27
*** ParsectiX has joined #openstack-infra05:28
*** gokrokve_ has joined #openstack-infra05:28
*** gokrokve has quit IRC05:28
openstackgerritgreghaynes proposed openstack-infra/nodepool: Decouple nodepool db and config from builders
openstackgerritgreghaynes proposed openstack-infra/nodepool: Builders distinguish between failure and exception
openstackgerritgreghaynes proposed openstack-infra/nodepool: Decouple builders from nodepool instance
*** mohankumar has joined #openstack-infra05:29
*** fawadkhaliq has joined #openstack-infra05:33
*** Daisy has joined #openstack-infra05:33
openstackgerritgreghaynes proposed openstack-infra/nodepool: Builders can be run on their own
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update mistral to correct repo
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update murano to correct repo
openstackgerritLuo Gangyi proposed openstack-dev/pbr: Enable pep8 H405 tests
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update sahara projects to correct repo
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update solum to correct repo and add missing
*** armax has joined #openstack-infra05:38
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Remove projects not in OpenStack repo
ianwgreghaynes: exiting the shell is what you want, right?05:39
greghaynesianw: If you exit in a source it exits the calling shell though05:41
*** ramineni has joined #openstack-infra05:41
greghaynesianw: basically, it'll close your terminal05:42
openstackgerritLuo Gangyi proposed openstack-dev/pbr: Enable pep8 H405 tests
*** abregman has joined #openstack-infra05:43
*** ramineni1 has quit IRC05:45
ianwgreghaynes: but you source it into ./bin/disk-image-create ... that's what will exit?  it's rather academic anyway05:46
ianwif you have no python there's issues05:46
greghaynesianw: ah, yea, you might be right in that it isnt an issue... i'll test05:47
greghaynesianw: I am going to hold off on that one until implementing the thing I mentioned on the ML05:47
greghaynessince that will remove most of the patch05:48
*** _nadya_ has joined #openstack-infra05:48
*** gokrokve_ has quit IRC05:51
openstackgerritMerged openstack-infra/reviewstats: Update mistral to correct repo
*** gokrokve has joined #openstack-infra05:52
openstackgerritMerged openstack-infra/reviewstats: Update murano to correct repo
openstackgerritMerged openstack-infra/reviewstats: Update sahara projects to correct repo
*** _nadya_ has quit IRC05:53
openstackgerritMerged openstack-infra/reviewstats: Update solum to correct repo and add missing
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Add cue to projects
*** thorst has joined #openstack-infra05:56
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: networking-midonet: Enable check-requirements
openstackgerritYAMAMOTO Takashi proposed openstack/requirements: Add networking-midonet to projects.txt
*** salv-orl_ has quit IRC06:00
*** salv-orlando has joined #openstack-infra06:00
*** ramineni has quit IRC06:02
*** thorst has quit IRC06:04
*** MarkAtwood has joined #openstack-infra06:05
*** vilobhmm11 has quit IRC06:06
*** watanabe_isao has joined #openstack-infra06:17
*** salv-orl_ has joined #openstack-infra06:17
*** unicell has joined #openstack-infra06:18
*** hdd has joined #openstack-infra06:19
*** salv-orlando has quit IRC06:20
*** alex_xu has joined #openstack-infra06:25
*** pcrews has quit IRC06:25
*** jerryz has quit IRC06:25
*** ramineni has joined #openstack-infra06:26
*** pcrews has joined #openstack-infra06:27
*** alex_xu_ has quit IRC06:28
*** dewsday has joined #openstack-infra06:29
*** jaypipes has joined #openstack-infra06:31
*** dewsday has quit IRC06:31
*** gokrokve has quit IRC06:32
*** hdd has quit IRC06:32
*** pcrews has quit IRC06:34
openstackgerritLuo Gangyi proposed openstack-dev/pbr: Enable pep8 H405 tests
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints
*** pcrews has joined #openstack-infra06:38
*** chlong has quit IRC06:40
*** _nadya_ has joined #openstack-infra06:42
*** pcrews has quit IRC06:45
*** jamesmcarthur has quit IRC06:47
*** _nadya_ has quit IRC06:47
*** armax has quit IRC06:48
*** gildub has quit IRC06:53
*** _denisra has joined #openstack-infra06:56
*** claudiub has quit IRC07:00
*** thorst has joined #openstack-infra07:01
*** unicell has quit IRC07:01
*** ParsectiX has quit IRC07:02
*** gokrokve has joined #openstack-infra07:02
*** jtomasek has joined #openstack-infra07:03
openstackgerritMerged openstack/requirements: Cap elasticsearch client <2.0
*** unicell has joined #openstack-infra07:04
*** mpavone has joined #openstack-infra07:04
*** gildub has joined #openstack-infra07:06
*** ParsectiX has joined #openstack-infra07:06
*** ParsectiX has quit IRC07:06
*** jtomasek has quit IRC07:06
*** harlowja_at_home has quit IRC07:06
*** ParsectiX has joined #openstack-infra07:06
*** shardy has joined #openstack-infra07:06
openstackgerritMerged openstack-dev/cookiecutter: Update min tox version to match current practice
*** gokrokve has quit IRC07:07
*** ParsectiX has quit IRC07:07
*** ParsectiX has joined #openstack-infra07:07
*** thorst has quit IRC07:09
*** kgiusti has quit IRC07:12
*** kgiusti has joined #openstack-infra07:12
*** kozhukalov_ has joined #openstack-infra07:14
*** mpavone has quit IRC07:15
*** jaosorior has joined #openstack-infra07:19
*** jaosorior has quit IRC07:19
*** MarkAtwood has quit IRC07:20
*** MarkAtwood has joined #openstack-infra07:20
*** jaosorior has joined #openstack-infra07:20
*** rcernin has joined #openstack-infra07:20
*** _nadya_ has joined #openstack-infra07:21
*** jtomasek has joined #openstack-infra07:21
*** AJaeger has joined #openstack-infra07:23
*** flwang1 has joined #openstack-infra07:25
*** yonglihe has quit IRC07:31
*** boris-42_ has joined #openstack-infra07:31
*** scheuran has joined #openstack-infra07:32
openstackgerritHaomeng,Wang proposed openstack-infra/project-config: skip ironic client dsvm and pypy gates for doc patches
openstackgerritMerged openstack-dev/cookiecutter: New projects should come constraint enabled.
*** ihrachys has joined #openstack-infra07:35
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Publish the Python OpenStack SDK docs
openstackgerritMerged openstack-dev/cookiecutter: Do not make project depends on Babel by default
*** kozhukalov_ has quit IRC07:38
*** dizquierdo has joined #openstack-infra07:39
openstackgerritMerged openstack-infra/shade: Improve test coverage: private extension API
*** eandersson has joined #openstack-infra07:43
*** sridhar_ram has joined #openstack-infra07:43
*** cdent has joined #openstack-infra07:50
*** e0ne has joined #openstack-infra07:52
*** ParsectiX has quit IRC07:53
*** ParsectiX has joined #openstack-infra07:53
*** ihrachys has quit IRC07:55
*** jerryz has joined #openstack-infra07:56
*** flwang1 has quit IRC08:00
*** zeih has joined #openstack-infra08:01
*** matrohon has joined #openstack-infra08:03
*** gokrokve has joined #openstack-infra08:03
*** egonzalez has joined #openstack-infra08:06
*** thorst has joined #openstack-infra08:06
*** dizquierdo has quit IRC08:06
*** isaacb has joined #openstack-infra08:06
*** dizquierdo has joined #openstack-infra08:07
*** ramineni has quit IRC08:07
*** MarkAtwood has quit IRC08:07
*** matrohon has quit IRC08:07
*** gokrokve has quit IRC08:08
*** thorst has quit IRC08:14
*** flwang1 has joined #openstack-infra08:16
*** dizquierdo has quit IRC08:20
*** ramineni has joined #openstack-infra08:22
*** amotoki has joined #openstack-infra08:22
*** kushal has quit IRC08:22
*** markus_z has joined #openstack-infra08:23
*** atcitlionok has joined #openstack-infra08:24
raminenihi, could someone please review
*** sputnik13 has joined #openstack-infra08:26
*** ashleighfarnham has joined #openstack-infra08:28
*** _nadya_ has quit IRC08:28
*** _nadya_ has joined #openstack-infra08:29
*** _nadya_ has quit IRC08:29
*** fawadkhaliq has quit IRC08:32
*** ashleighfarnham has quit IRC08:33
*** ifarkas has joined #openstack-infra08:37
openstackgerrityolanda.robla proposed openstack-infra/system-config: Add yolanda to statusbot nicks
*** dtantsur|afk is now known as dtantsur08:39
*** shardy has quit IRC08:42
*** shardy has joined #openstack-infra08:43
*** mrmartin has joined #openstack-infra08:45
*** jlanoux has joined #openstack-infra08:46
openstackgerritMerged openstack-infra/project-config: Publish the Python OpenStack SDK docs
*** matrohon has joined #openstack-infra08:46
*** Sukhdev has quit IRC08:47
*** arxcruz has joined #openstack-infra08:47
openstackgerritMerged openstack-infra/project-config: Add release note files to the skip-if list for neutron jobs
*** fawadkhaliq has joined #openstack-infra08:48
*** shardy has quit IRC08:49
openstackgerritMerged openstack-infra/project-config: networking-midonet: Enable check-requirements
*** fawadkhaliq has quit IRC08:49
*** flwang1 has quit IRC08:49
*** flwang1 has joined #openstack-infra08:49
*** shardy has joined #openstack-infra08:51
openstackgerritMerged openstack-infra/infra-manual: Add previewing a change
*** jaosorior has quit IRC08:55
openstackgerritMerged openstack-infra/system-config: Add statusbot/eavesdrop to #midonet channel
*** salv-orl_ has quit IRC08:58
*** hashar has joined #openstack-infra08:58
*** zeih has quit IRC08:58
*** skolekonov has quit IRC08:58
*** jordanP has joined #openstack-infra08:58
bogdandohi. How can I add myself to the fuel-noop-fixtures-core,members ?09:00
*** yamahata has joined #openstack-infra09:01
*** fhubik has joined #openstack-infra09:02
yolandahi bogdando i can add you. Do you have the link for the project creation so i can check your ownership?09:03
bogdandoyolanda, this ?09:03
yolandabogdando, no, the change you sent to create the project in openstack09:04
*** hashar has quit IRC09:04
*** gokrokve has joined #openstack-infra09:04
bogdandoyolanda, ah this then
yolandathe gerrit change09:04
*** zhurong has quit IRC09:04
bogdandothank you09:04
*** lezbar has joined #openstack-infra09:04
yolandayou may need to relogin and/or wipe browser cache09:05
*** openstackstatus has quit IRC09:05
*** openstack has joined #openstack-infra09:09
*** gokrokve has quit IRC09:09
openstackgerritMerged openstack-infra/ansible-puppet: Add whitespace between tasks for better readability
*** hashar has joined #openstack-infra09:10
*** thorst has joined #openstack-infra09:12
*** ociuhandu has quit IRC09:14
*** yamahata has quit IRC09:15
*** skolekonov has joined #openstack-infra09:15
*** e0ne has quit IRC09:15
*** daemontool has joined #openstack-infra09:16
openstackgerritMerged openstack-infra/puppet-openstack_health: Add a longer timeout to the pip install phase
*** jaosorior has joined #openstack-infra09:16
openstackgerritMerged openstack-infra/puppet-infracloud: Disable novav3, ec2 and metadata services
openstackgerritMerged openstack-infra/project-config: Added new repo for fuel-plugin-bgpvpn in Openstack NS
*** thorst has quit IRC09:18
*** zeih has joined #openstack-infra09:18
*** skraynev has quit IRC09:20
*** Daisy has quit IRC09:20
openstackgerritIgor Belikov proposed openstack-infra/project-config: Add 'delay-release' option to fuel projects
*** nmagnezi has joined #openstack-infra09:24
*** jerryz has quit IRC09:25
BobBallanteaya, pleia2, notmorgan: Regarding the XenProject CI - it did have a period of brokenness on 7th December, but it's resolved now.  As you saw a recheck got a good result on that change.09:27
BobBallanteaya, pleia2, notmorgan: And for the avoidance of doubt... I am Citrix and my primary focus is the XenServer CI, however I do also have admin on the xenproject CI so feel free to ask me questions :)09:28
*** skraynev has joined #openstack-infra09:28
*** sridhar_ram has quit IRC09:29
*** fhubik is now known as fhubik_brb09:30
openstackgerritMerged openstack-infra/project-config: Create bareon project
*** openstackgerrit has quit IRC09:32
*** openstackgerrit has joined #openstack-infra09:32
*** fawadkhaliq has joined #openstack-infra09:34
silehthi, it looks like the dsvm centos7 for kilo branch doesn't setup correclty:
*** fhubik_brb is now known as fhubik09:37
openstackgerritEvgeny Antyshev proposed openstack-infra/puppet-openstackci: Allow logserver to run on CI server (part 1)
openstackgerritMerged openstack-infra/project-config: Add new puppet-vitrage project
openstackgerritMerged openstack-infra/project-config: Add delay-release for Gnocchi
openstackgerritMerged openstack-infra/project-config: Added openstack-zephyr to accessbot
openstackgerritEvgeny Antyshev proposed openstack-infra/puppet-openstackci: Allow logserver to run on CI server (part 2)
openstackgerritValeriy Ponomaryov proposed openstack-infra/project-config: Add new experimental Tempest job for Manila
*** matrohon has quit IRC09:41
*** ildikov has quit IRC09:42
*** skraynev has quit IRC09:48
*** skraynev has joined #openstack-infra09:49
*** ramineni has quit IRC09:52
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: networking-midonet: Move ML2 job out of experimental
*** HeOS has joined #openstack-infra09:53
*** oomichi is now known as oomichi_away09:55
*** ramineni has joined #openstack-infra09:55
*** dewsday has joined #openstack-infra09:57
igorbelikovhi everyone, after this change was merged launchpad bot closed the bug mentioned in commit message AND created a bug
openstackLaunchpad bug 1523989 in Fuel for OpenStack " Remove fuel_package_updates module" [Undecided,New]09:58
*** binbincong has quit IRC09:58
*** salv-orlando has joined #openstack-infra09:59
igorbelikovwas the bug creation some kind of glitch or is it expected (for some reason)?09:59
AJaegerigorbelikov: you have "DocImpact" in that change and a new bug gets created when you add "DocImpact" to it10:00
AJaegerSo, works as designed ;)10:01
igorbelikovAJaeger: thanks! yeah, this totally makes sense:)10:01
*** nijaba has quit IRC10:02
*** ashleighfarnham has joined #openstack-infra10:02
*** rguillebert has joined #openstack-infra10:02
*** electrofelix has joined #openstack-infra10:02
*** openstackgerrit has quit IRC10:02
*** zhenguo has quit IRC10:02
*** openstackgerrit has joined #openstack-infra10:03
*** salv-orlando has quit IRC10:03
*** hichihara has quit IRC10:04
*** dewsday has quit IRC10:04
*** dewsday has joined #openstack-infra10:05
*** dewsday has quit IRC10:05
*** dewsday has joined #openstack-infra10:05
*** sfinucan has joined #openstack-infra10:05
*** dizquierdo has joined #openstack-infra10:06
*** ildikov has joined #openstack-infra10:09
*** e0ne has joined #openstack-infra10:10
*** jamielennox is now known as jamielennox|away10:13
AJaegeryolanda: see - was  that a result of the midonet merge (#252172)?10:14
*** ParsectiX has quit IRC10:15
yolandadoh, i approved a change but at 08:00 UTC10:15
yolandait took long to merge10:15
yolandai double checked the schedule before doing10:15
*** sileht has quit IRC10:16
yolandareally doing that is tricky. i wonder if we shall limit to weekends10:16
*** _nadya_ has joined #openstack-infra10:16
AJaegeryolanda: better ask the other infras on how to do this best.10:17
AJaegerThis was a very late merge ;(10:17
openstackgerritMerged openstack-dev/pbr: Enable pep8 H405 tests
*** thorst has joined #openstack-infra10:18
yolandayes, the fact that we cannot control the time that a change takes to land, makes that risky10:19
*** sileht has joined #openstack-infra10:19
*** binbincong has joined #openstack-infra10:19
*** dtantsur is now known as dtantsur|brb10:19
AJaegerAnd the long list in a single line will mean that changes will create merge conflicts, so you cannot approve 5 at a time ;(10:19
*** sileht has quit IRC10:19
AJaegernibalizer was working on a change to put the data separate which reduces merge conflicts at least10:20
*** sileht has joined #openstack-infra10:20
yolandathat will make it easier10:20
*** ashishb has joined #openstack-infra10:20
AJaeger is the change10:20
*** kozhukalov_ has joined #openstack-infra10:20
openstackgerritNikola Dipanov proposed openstack-infra/irc-meetings: Add the agenda link for the SR-IOV meeting
yolandaan alternative can be to stop puppet deployments on that node while there is a change pending to be merged, and start once we see it landed, and there is no meeting10:22
*** thorst has quit IRC10:24
*** jcoufal has joined #openstack-infra10:25
*** dims has quit IRC10:25
openstackgerritAndrea Frittoli proposed openstack-infra/log_processor: IPC-49 Create auto forwardable parameters
openstackgerritAndrea Frittoli proposed openstack-infra/log_processor: Create auto forwardable parameters
kozhukalov_guys, please add me as a member of this group,members I am an author of the patch
openstackgerritMerged openstack-infra/storyboard-webclient: Fix setting preferences when logged out
yolandaAJaeger, saw it. It seems to be failing on tests, but that's a good idea,these conflicts happen quite often10:29
*** cbader has quit IRC10:32
*** cbader has joined #openstack-infra10:33
openstackgerritJulien Danjou proposed openstack-infra/project-config: Increase Python jobs timeout for Gnocchi
*** olaph has quit IRC10:36
*** olaph has joined #openstack-infra10:36
openstackgerritBeth Elwell proposed openstack-infra/project-config: New project for Ironic UI plugin
*** watanabe_isao has quit IRC10:37
*** jistr has joined #openstack-infra10:39
*** mohankumar has quit IRC10:41
*** rossella_s has joined #openstack-infra10:42
evgenylHi, I'm struggling to find an instruction on how to enable a logging for a new irc-channel, is there some documentation for that?10:44
silehtAJaeger, yolanda any idea for this gate issue on centos7 and kilo branch:
openstackgerritSzymon Wróblewski proposed openstack/requirements: Bump tooz version to 1.28.0
jokke_evgenyl: really sorry, just had to :P10:50
AJaegersileht: sorry, can't help here10:52
yolandasileht, i suggest you spin up a centos 7 vm somehow , and try to reproduce the steps of the script. It seems to fail on yum install, but i cannot tell the reason10:52
silehtAJaeger, the rdo-release-kilo package installation works well locally10:53
*** jyuso1 has joined #openstack-infra10:53
*** ldnunes has joined #openstack-infra10:54
*** fawadkhaliq has quit IRC10:57
*** fawadkhaliq has joined #openstack-infra10:57
*** ParsectiX has joined #openstack-infra10:59
*** ParsectiX has quit IRC11:00
*** ParsectiX has joined #openstack-infra11:00
evgenyljokke_: thanks, I've found this doc, but wasn't sure which of these bots (meetbot/statusbot/gerritbot) perform a regular logging, like this channel has logs here (
*** ParsectiX has quit IRC11:00
*** ParsectiX has joined #openstack-infra11:01
jokke_evgenyl: that would be the logging part of that document11:01
AJaegerevgenyl: you need accessbot to be setup in any case11:02
AJaegerevgenyl: no logging setup without it11:03
*** fawadkhaliq has quit IRC11:03
openstackgerritSergey Nikitin proposed openstack/requirements: Updated oslo.db to 4.1.0
*** kmartin has quit IRC11:05
evgenyljokke_: I've read it thanks11:05
*** kmartin has joined #openstack-infra11:05
evgenylAJaeger: Looks like we have it configured but I still cannot find a directory with logs11:05
AJaegerevgenyl: bareon merged today, so now you have accessbot setup11:06
AJaegerevgenyl: logging is done via change for system-config, see the link that jokke_ gave you11:06
*** yuanying has quit IRC11:07
*** salv-orlando has joined #openstack-infra11:10
*** aysyd has joined #openstack-infra11:14
*** lucas-dinner is now known as lucasagomes11:16
*** aysyd has quit IRC11:16
*** aysyd has joined #openstack-infra11:18
openstackgerritgongysh proposed openstack-infra/project-config: Add python jobs for tacker client
*** hashar has quit IRC11:22
*** thorst has joined #openstack-infra11:23
*** mgrosz_ has joined #openstack-infra11:24
openstackgerritMichael Micucci proposed openstack-infra/system-config: Added eavesdrop and statusbot for openstack-zephyr
*** dtantsur|brb is now known as dtantsur11:25
*** ramineni has left #openstack-infra11:25
*** links has quit IRC11:26
mgrosz_Hello... I am trying to continue with initializing the new puppet-vitrage module, after change was recently merged. I am following the wiki but it seems like the script is not pointing me correctly and I am trying to understand to what repository11:26
mgrosz_ I need to send my changes....11:26
*** thorst has quit IRC11:28
samueldmqlifeless: yes updating right now11:31
mgrosz_Is there somwone oline?11:31
silehtAJaeger, yolanda any idea of who can help me to resolve this gate issue ?11:31
samueldmqlifeless: sorry for the delay11:32
mgrosz_AJaeger, yolanda: I am trying to continue with initializing the new puppet-vitrage module, after change was recently merged. I am following the wiki but it seems like the script is not pointing me correctly and I am trying to understand to what r11:32
mgrosz_epository I need to send my changes....11:32
yolandasileht i'd say you try to debug on a local centos7 vm, to try to see the problem with that install11:32
*** fhubik is now known as fhubik_brb11:32
yolandalater some US people like sdague or clarkb, can know better11:33
silehtyolanda, I have already done this and I can't reproduce11:33
silehtyolanda, thx I will ask them later11:33
yolandamgrosz, what problems do you have with that cookiecutter?11:34
yolandai haven't used but i can try to help11:34
*** ParsectiX has quit IRC11:35
mgrosz_yolanda: The leads me to a directory that doesn't exist... Is is updated?11:36
yolandamm, the modulesyncs?11:37
*** dims has joined #openstack-infra11:37
mgrosz_Yes.... If I ran the script as described here But then I want to execute git review but there is no directory as described...11:38
*** sambetts-afk is now known as sambetts11:38
*** fhubik_brb is now known as fhubik11:39
*** sshnaidm has joined #openstack-infra11:40
mgrosz_as far as I understand, I should have a directory puppet-vitrage created (I have on under /tmp/puppet-vitrage/cookiecutter/puppet-vitrage) but I am not sure this is what I was suppose to have, as the script sends me to /tmp/puppet-vitrage/puppet-modulesync-configs/modules/puppet-vitrage which doesn't exist. Is this update11:40
yolandamgrosz i'm checking myself, as i haven't dealt with that cookiecutter scripts11:41
yolandaok so it fails , it doesn't generate the folder11:42
sdaguesileht: it looks like the rdo release rpm is not installing, and devstack is failing early for that reason11:42
silehtsdague, yes I can't figure why11:42
mgrosz_I have just downloaded the and ran it: ./ vitrage matyg11:42
yolandamgrosz, trying the same11:43
sdaguesileht: possibly because rdo-release-liberty is already installed?11:43
sdagueand it's trying to install rdo-release-kilo11:43
silehtsdague, woot well catch11:43
yolandamgrosz, i get /tmp/puppet-vitrage/puppet-modulesync-configs, but then i don't get modules/puppet-vitrage folder, is that the same there?11:44
silehtsdague, should I fix devstack ? or the gate vm should be cleaned ?11:44
*** ggillies has quit IRC11:44
openstackgerritSamuel de Medeiros Queiroz proposed openstack-infra/infra-manual: Improve docs on setting up development environment
yolandamgrosz, i see "msync not found" error11:45
samueldmqlifeless: ^11:45
yolandado you have the same?11:45
samueldmqbknudson: cc ^11:45
Kennanhi infra-cores11:45
Kennando you know what's the limitation for jenknis jobs run for one VM ?11:46
Kennanwe found that it hit11:46
AJaegermgrosz_: talk with the other puppet folks...#11:46
Kennanfilter_properties)\n', u'  File "/opt/stack/new/nova/nova/compute/", line 2070, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u"RescheduledException: Build of instance 1109482a-bb32-4f90-a67c-8908517c9889 was re-scheduled: internal error: process exited while connecting to monitor: Cannot set up guest memory 'pc.ram': Cannot allocate memory\n\n"]11:46
KennanWe have one VM, and in that vm, we boot two  qemu instances11:47
AJaegermgrosz_: there's puppet-openstack channel...11:47
Kennannot sure what changed in the infra underlay11:47
Kennanbefore it can allocate resource and worked11:47
*** fhubik is now known as fhubik_brb11:47
Kennannow, it seems failed quite oftem11:47
mgrosz_AJaeger: I will. Thanks11:48
Kennanseems we have some resource restriction now in jeknins run job11:48
*** mgrosz_ has quit IRC11:48
Kennananyone core knows about it ? Thanks11:48
yolandamgrosz, yes, better ask there, the script seems to be failing or at least have missing dependencies11:48
*** kmartin has quit IRC11:49
AJaegerKennan: can you compare a successful and a failing run? Are both on machines in the same cloud?11:49
AJaegerWould be good to dig out whether there's a common thing between failures, e.g. all failing in cloud X...11:50
KennanAJaeger how can find the cloud ?11:50
Kennanseems all our jenkins failed about this11:50
*** gokrokve has joined #openstack-infra11:50
AJaegerkenan, see first few lines of
KennanAJaeger: thanks, which line include that message ? I checked not get it11:52
*** dewsday has quit IRC11:52
Kennandid nova changed libvirt these days ? if not infra issue. let me check more11:53
*** thiagop has quit IRC11:54
*** gokrokve has quit IRC11:55
*** gsagie has joined #openstack-infra11:59
gsagieIf anyone can review:   will be great (assuming no meetings are happening right now)11:59
gsagiejust needs workflow11:59
KennanGot this12:00
Kennan internal error: early end of file from monitor: possible problem:12:00
KennanCannot set up guest memory 'pc.ram': Cannot allocate memory12:00
Kennanlibvirt failed12:00
AJaegerKennan: third line "Building remotely on devstack-trusty-ovh-gra1-6437456 (devstack-trusty)"12:01
AJaegerovh is the cloud, gra1 the region12:01
*** mrmartin has quit IRC12:02
KennanOK AJaeger: I remember nova or libvirt support overcommit12:02
openstackgerritZara proposed openstack-infra/storyboard-webclient: Align comment timestamps on the right on dashboard
*** yamamoto has quit IRC12:03
AJaegerKennan: you might want to ask on #openstack-qa or on #openstack-nova...12:03
*** daemontool has quit IRC12:04
*** yamamoto has joined #openstack-infra12:06
*** mohankumar has joined #openstack-infra12:06
silehtsdague, should I fix devstack ? or the gate vm should be cleaned ?12:06
sdaguesileht: maybe a patch to devstack, honestly I haven't seen the centos7 job work in a long time, so I have no idea12:07
silehtsdague, I will tried a devstack patch12:08
silehtsdague, oslo.messaging liberty have gate jobs that vote, so backports are currently stuck12:11
gsagieIf i want to write test for a tempest job that run in the gate that uses the Neutron client, which credentials/auth_url and etc should i create the client with?12:11
gsagieor if anyone can point me to an example, will be great :)12:12
silehtsdague, the latest stable rdo-release package is installed in gate for rhel7 to get openvswitch12:14
*** yamamoto has quit IRC12:15
*** links has joined #openstack-infra12:16
sdaguesileht: ok, so something special probably needs to be sorted there. ianw or afazekas typically look at those12:17
*** salv-orl_ has joined #openstack-infra12:17
*** jyuso1 has quit IRC12:19
*** jyuso1 has joined #openstack-infra12:19
*** salv-orlando has quit IRC12:20
*** hashar has joined #openstack-infra12:23
*** EricGonczer_ has joined #openstack-infra12:24
*** gordc has joined #openstack-infra12:24
*** EricGonczer_ has quit IRC12:25
openstackgerritDmitry Tantsur proposed openstack-infra/os-loganalyze: Whitelist ironic- prefix in
dtantsursdague, please take a look ^^ if you have a minute12:26
*** thorst has joined #openstack-infra12:27
dimssdague : one from me as well :) fix for a grenade upgrade problem -,n,z12:27
sdaguedtantsur: oh, it's not using ir- as it's screen name?12:28
sdagueI think that's what was expected12:28
sdaguebut I'm fine with that as well12:28
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: [sahara] job for sahara-dashboard integration tests
dtantsursdague, yeah, I didn't realize it's important back then :) and I assumed using full name would be more user friendly..12:29
sdaguedims: right, I still don't like or understand this12:29
*** matrohon has joined #openstack-infra12:29
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: [sahara] job for sahara-dashboard integration tests
*** rossella_s has quit IRC12:30
sdaguedims: a change made there is because the project make a non safe upgrade change, and there is a release note about it12:30
sdaguedims: why is this required?12:30
*** rossella_s has joined #openstack-infra12:30
*** ggillies has joined #openstack-infra12:31
dimssdague : kilo nova api-paste.ini has references to oslo.middleware, in Mitaka oslo.middleware we dropped oslo.middleware finally now and that scenario fails12:33
*** thorst has quit IRC12:33
sdaguewhat does nova master currently do?12:33
dimsthat's fine because liberty's nova api-paste.ini was already corrected a while ago12:34
*** gildub has quit IRC12:34
sdagueok, so if liberty api-paste ini is fine, then I don't understand why the master patch is needed12:34
sdagueand I don't think it should be required for kilo12:34
*** ashleighfarnham has quit IRC12:34
*** matrohon has quit IRC12:34
sdaguebecause we can't post date changes like this12:34
sdaguethe oslo.middleware that gets used with liberty needs to never drop this, because it was shipped with something this fundamental12:35
*** matrohon has joined #openstack-infra12:36
*** thorst has joined #openstack-infra12:36
sdagueiirc - liberty is the deprecation cycle, so it can't be removed from liberty using stuff12:36
dimskilo->liberty fails only when oslo.middleware shipped for mitaka fails12:36
sdagueok, but then you just kicked all the ops in the knees if they upgrade from kilo -> liberty 3 months late12:37
sdagueyou broke the deprecation guaruntees12:37
sdagueif that kilo -> liberty upgrade patch is needed to keep the gate working, then we've got a way bigger problem12:38
dimsguess, this is the larger conversation about when we can even get rid of anything or even do any development work in oslo at all12:38
*** fawadkhaliq has joined #openstack-infra12:38
dimsi'll abandon the patches12:38
dimsfor the grenade problem12:38
*** gsagie has quit IRC12:39
sdaguedims: the master patch isn't needed anyway, right?12:39
sdaguebecause it's right in liberty12:39
dimssdague : there's a from-kilo i thought if it is there then it should be fixed there12:39
*** weshay_xchat has joined #openstack-infra12:39
sdagueoh, yeh, that's mostly for bleed over12:39
*** weshay_xchat is now known as weshay12:39
sdaguebecause there is a window of time not all the branches align12:40
sdaguedims: ok, but you do understand that if this is needed for kilo -> liberty we just broke 100s of ops teams, right?12:40
*** Daisy has joined #openstack-infra12:41
*** jaosorior has quit IRC12:42
*** doug-fis_ is now known as doug-fish12:42
*** chlong has joined #openstack-infra12:42
*** jaosorior has joined #openstack-infra12:42
*** rhallisey has joined #openstack-infra12:43
*** boris-42_ has quit IRC12:43
*** _nadya_ has quit IRC12:43
*** salv-orl_ has quit IRC12:44
*** kushal has joined #openstack-infra12:45
*** thorst_ has joined #openstack-infra12:45
*** fhubik_brb is now known as fhubik12:48
*** shardy has quit IRC12:49
*** thorst has quit IRC12:49
*** salv-orlando has joined #openstack-infra12:49
*** shardy has joined #openstack-infra12:51
afazekassileht: which job is failing, can you give me a link ?12:51
*** pahuang has quit IRC12:51
*** gsagie has joined #openstack-infra12:52
markus_zWe have a peak in the failure rates of the grenade gate and check jobs, is that already known?
*** amotoki_ has joined #openstack-infra12:56
markus_zI haven't found a discussion on the ML or in the previous chats in this channel about this peak.12:58
*** amotoki__ has joined #openstack-infra12:59
*** amotoki has quit IRC12:59
*** placko has joined #openstack-infra12:59
*** amotoki__ is now known as amotoki___12:59
*** amuller has joined #openstack-infra13:00
*** dprince has joined #openstack-infra13:01
*** jffische has joined #openstack-infra13:01
*** julim has joined #openstack-infra13:01
*** amotoki___ is now known as amotoki__13:02
*** amotoki_ has quit IRC13:02
*** amotoki__ is now known as amotoki13:02
*** ayoung has quit IRC13:02
*** jed56 has quit IRC13:03
openstackgerritVitaly Gridnev proposed openstack-infra/project-config: [sahara] job for sahara-dashboard integration tests
*** Daisy has quit IRC13:05
*** Liuqing has joined #openstack-infra13:05
*** doug-fish has quit IRC13:06
*** fhubik has quit IRC13:06
*** ildikov has quit IRC13:07
openstackgerritMerged openstack-infra/jenkins-job-builder: Fix pollscm backwards compatiblity
silehtafazekas:,   sdague found the issue, I have proposed this to fix it
*** _nadya_ has joined #openstack-infra13:13
openstackgerritGal Sagie proposed openstack-infra/project-config: Add networking-ovn rally job
afazekassileht, thx13:15
*** _nadya_ has quit IRC13:17
openstackgerritGal Sagie proposed openstack-infra/project-config: Add networking-ovn rally job
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Allow arguments to docs env to be passed
*** asselin has joined #openstack-infra13:19
*** ildikov has joined #openstack-infra13:22
*** Ramanjaneya has quit IRC13:23
*** doug-fish has joined #openstack-infra13:23
*** flepied has joined #openstack-infra13:26
samueldmqI have an issue with my keystoneclient functional tests gate job13:26
samueldmqI can't import openstack_client_config from within it13:26
mordredsamueldmq: that's because that's not the module name13:26
mordredsamueldmq: it's os_client_config13:26
*** fawadkhaliq has quit IRC13:27
* samueldmq facepalm13:27
samueldmqmordred: err; thanks13:27
*** gsagie has quit IRC13:27
samueldmqmordred: I even compared the gate job definition with novaclient's one :/13:28
mordredsamueldmq: it happens13:28
samueldmqthese types of error is sometimes the hardest to find13:28
mordredsamueldmq: btw - notmorgan and I enjoyed the cachaça you brought him finally13:28
notmorganmordred: ++13:29
samueldmqmordred: notmorgan: wow the one from Paris ? :)13:29
mordredsamueldmq: yup. I guess the key to enjoying cachaça is to hang out with me :)13:29
samueldmqnotmorgan: mordred was it good ?13:29
mordredsamueldmq: yup!13:29
*** bryan_att has quit IRC13:29
mordredand all gone now13:29
samueldmqhaha already ? :) yeah now I am sure you enjoyed :)13:30
openstackgerritChangBo Guo(gcb) proposed openstack-dev/cookiecutter: Don't include file openstack-common.conf
asselinyolanda, can you re-review
samueldmqnotmorgan: that is made in my hometown, and mordred knows the 3 varieties they make there13:31
*** mrmartin has joined #openstack-infra13:32
*** regXboi has joined #openstack-infra13:32
mordredsamueldmq: yes I do - I have enjoyed all of them13:33
mordredsamueldmq: in fact, I just had some Ouro last night13:33
samueldmqmordred: which one did you prefer ?13:33
*** EricGonczer_ has joined #openstack-infra13:33
mordredsamueldmq: I like all three- each are different :)13:35
*** edmondsw has joined #openstack-infra13:36
samueldmqmordred: ++ agree13:36
*** gsagie has joined #openstack-infra13:36
*** psanchez has quit IRC13:37
*** moravec has quit IRC13:38
*** EricGonc_ has joined #openstack-infra13:40
*** baoli has joined #openstack-infra13:40
*** jyuso1 has quit IRC13:41
flip214help, please. I keep running into known bugs that already expired some time in the past, because nobody seems to care about them any more.13:41
flip214like right now.13:41
openstackLaunchpad bug 1466485 in OpenStack Identity (keystone) "keystone fails with: ArgsAlreadyParsedError: arguments already parsed: cannot register CLI option" [Critical,Expired]13:41
*** baoli_ has joined #openstack-infra13:41
flip214is there somebody that I could poke about things like this?13:41
flip214obviously, I can't fix them (all) by myself...13:41
*** EricGonc_ has quit IRC13:44
*** eharney has joined #openstack-infra13:44
*** EricGonczer_ has quit IRC13:44
yolandahi crinkle, for the question about inventory you raised.. why do not manage them on hiera? we can have a list of dictionaries, and you can update depending on your situation13:44
*** yamamoto has joined #openstack-infra13:44
yolandai mean, instead of only using hiera to manage the password, use it to manage the full inventory entries13:44
*** baoli has quit IRC13:45
crinkleyolanda: how would we distinguish between which machines are enabled and disabled? i guess just an extra key in the hiera entry?13:46
mordredyolanda, crinkle: what's the question?13:46
mordredand by "manage them in hiera" - do you mean secret hiera, or in-tree hiera?13:47
yolandacrinkle, just some entry in the hiera dict for the server will be fine. And puppet module can compose baremetal.json using that flag13:47
mordredcrinkle: have you eed the enable/disable system we put in place for puppet apply?13:47
yolandamordred, about a question on
* mordred goes to read13:47
crinklemordred: we have a json file with the inventory list, but not every machine in the inventory is working, plus the baremetal host itself shouldn't be enrolled in ironic13:48
crinklemordred: so, how do we distinguish13:48
mordredthis is an interesting question13:48
yolandai'd like to see some yaml in hiera. We do it similar for nodepool secure.conf, to describe jenkins masters for example13:49
*** gongysh has joined #openstack-infra13:50
crinkleyolanda: i think that would work, but it also seems like it's making puppet do extra work to turn yaml data into json data13:51
crinklewe could possibly convince bifrost itself to accept a 'disabled' flag13:52
*** yamahata has joined #openstack-infra13:52
yolandacrinkle, an additional reason... i generally don't like to show that operational details on puppet. We are showing ips, users there13:52
*** jed56 has joined #openstack-infra13:53
*** yamamoto has quit IRC13:53
crinkleyolanda: I agree that separating that data is good practice in puppet, but on the other hand my experience with this group is a preference for keeping the config as simple as possible13:54
crinklewe could maybe retrofit the puppet-apply enable/disable system for this13:55
yolandacrinkle, what do you mean?13:55
*** dkranz has joined #openstack-infra13:55
yolandaalso, is it expensive to conver a yaml to a json in puppet? this shouldn't change very often, only when we add more hosts, or we update some node from enabled to disabled13:57
crinklei don't think it's expensive, just kind of round-a-bout13:58
crinkleyolanda: i think it's a reasonable solution, i'll move forward on that14:00
crinkleyolanda: thank you14:00
*** bcourt has quit IRC14:00
*** dtardivel has joined #openstack-infra14:01
*** sdake has quit IRC14:02
yolandacrinkle great14:02
*** yamahata has quit IRC14:02
*** haleyb has quit IRC14:03
mordredyolanda, crinkle: a few thoughts14:06
mordredTheJulia: how hard would it be to convice you to accept yaml in addition to json directly in bifrost?14:06
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Add separate linkcheck env and allow generic args to docs
mordredyolanda, crinkle: second - I don't think we should put complex data into secret yaml. we _CAN_ however put just the secret data into secret yaml and the non-secret data in in-tree yaml and the hiera merge should do things right14:08
yolandamordred, how will it that work in terms of complexity? are we normally using these two levels of hiera together?14:09
*** gongysh has quit IRC14:11
*** tlian has joined #openstack-infra14:11
crinklemordred: agreed, this would go in the public hiera14:11
*** cdent has quit IRC14:12
anteayaBobBall: thanks for the clarification, glad the issue was already addressed, thank you :)14:13
*** lykinsbd has joined #openstack-infra14:13
mordredyolanda: dunno - I just know that I want most of that data in public git repos14:13
*** bcourt has joined #openstack-infra14:13
yolandai'm normally concerned about exposing operational details publicliy14:13
crinklemordred: i don't think we need to get bifrost to accept yaml (hiera could actually accept json if we wanted), it's more about how we split the data up14:13
*** peristeri has joined #openstack-infra14:13
mordredyolanda: I hear that - but here it's what we do14:14
mordredyolanda: so we just have to make sure that we're careful with how we do it14:14
mordredcrinkle: kk14:14
yolandayes, we need to strengthen the passwords, and make them unique at least14:14
*** mrmartin has quit IRC14:14
TheJuliamordred: already do14:16
TheJuliamordred: if it is json or yaml, regardless it gets passed in14:16
rcarrillocruzi'm happy we are on the same page in regards to hiera/public cloud14:16
*** hashar has quit IRC14:16
rcarrillocruzthat's what i suggested to crinkle yesterday14:16
rcarrillocruzi found horrible to have all the inventory in puppet, given that's data...14:17
*** hashar has joined #openstack-infra14:17
*** cdent has joined #openstack-infra14:18
openstackgerritDarragh Bailey proposed openstack-infra/project-config: Add separate job to check docs links for JJB
*** alivigni has joined #openstack-infra14:20
openstackgerritDarragh Bailey proposed openstack-infra/jenkins-job-builder: Separate linkcheck job allows removal from docs run
yolandaah crinkle, i'm tweaking a bit the bifrost change for static dnsmasq. I found it works with your change, but as we have several other nics hooked to same vlan, dnsmasq was giving ips to them, causing our static allocation to fail14:21
*** denisra_ is now known as denisra14:21
*** AJaeger has quit IRC14:21
*** gokrokve has joined #openstack-infra14:22
*** Piet has joined #openstack-infra14:25
*** mriedem_away is now known as mriedem14:25
*** sputnik13 has quit IRC14:26
openstackgerritDmitry Tantsur proposed openstack/diskimage-builder: Add kmod to package-installs of ironic-agent
*** ayoung has joined #openstack-infra14:28
*** Piet has quit IRC14:28
smcginnisTime for my weekly infra question. ;)14:28
*** ildikov has quit IRC14:28
smcginnisWe have several projects defined in project-config gerritbot channels.14:28
*** bcourt has quit IRC14:29
smcginnisI know we are getting bot notifications for events for the cinder and os-brick, and i'm pretty sure for cinder-soecs14:29
*** amitgandhinz has joined #openstack-infra14:29
smcginnisbut I know for a fact nothing is coming through for python-cinderclient.14:29
smcginnisAny way to see why this is?14:29
smcginnisI know it used to. The only recent change was to add change-merged to the events list.14:30
*** ildikov has joined #openstack-infra14:30
BobBallanteaya: FYI we've got the tracking page ( listed on the third party CIs info page - so it should also be trivial to see if there is an ongoing issue or if the CI currently seems healthy14:31
*** mtanino has joined #openstack-infra14:31
*** spzala has joined #openstack-infra14:32
*** binbincong has quit IRC14:33
*** binbincong has joined #openstack-infra14:33
*** yamamoto has joined #openstack-infra14:34
*** ildikov has quit IRC14:35
crinkleyolanda: the static allocation should only be giving ips to known mac addresses14:35
yolandacrinkle yes, but it missed the static flag so our setup was still offering ips to other nics14:36
anteayaBobBall: what link do you use for the third party CIs info page?14:36
yolandanot sure if that's the same on west, but we have 4 nics attached to each vlan, not only the ones we use14:36
crinkleyolanda: i'm not sure what you mean, 4 nics per host? 4 nics in addition to the ones we're using?14:38
yolandawell, three extra nics in addition to the ones we use14:38
yolandaso dnmasq was giving ips to all those, we need to add an "static" flag to the config:
jd__can I have help wrt ? :)14:39
anteayaBobBall: there are a few options out there and while there is a spec up to bring one under infra service, I don't believe that has happened yet14:39
*** gongysh has joined #openstack-infra14:40
anteayaBobBall: I want to ensure we are talking about the same tool's info page14:40
*** akshai_ has joined #openstack-infra14:40
crinkleyolanda: it shouldn't hurt our infrastructure for dnsmasq to give ips to other nodes, it won't give away the ones where we've assigned a mac address14:41
*** bcourt has joined #openstack-infra14:41
*** Swami has joined #openstack-infra14:41
*** pradk has joined #openstack-infra14:42
crinkleyolanda: i think we might be set up differently, in west there's no vlan on the private ips that the bifrost dnsmasq is managing, and then i have a second dnsmasq that does have dhcp-range:static set for the tagged interfaces14:42
yolandacrinkle, so i found conflicts. dnsmasq was giving an ip that was  statically assigned to some other nic that requested before. So when it arrived to the moment that it had to assign the static ip to the one i wanted, it just complained because of that ip was already allocated14:42
*** moravec has joined #openstack-infra14:43
*** pradk has quit IRC14:43
smcginnisanteaya: Morning! Any thoughts from my comments several lines up ^^?14:43
anteayasmcginnis: yes been looking into it14:43
anteayaI found:
yolandacrinkle yes, that setup is different14:43
crinkleyolanda: you should be able to get dnsmasq to release that lease14:43
*** fawadkhaliq has joined #openstack-infra14:43
smcginnisanteaya: hmmmm14:43
*** akshai has quit IRC14:44
anteayasmcginnis: and
yolandacrinkle yes, manually removing the lease works14:44
*** lykinsbd has quit IRC14:44
smcginnisanteaya: Maybe it fixed itself since I noticed it.14:44
anteayaso you are getting python-cinderclint change merged and change proposed at least14:44
smcginnisanteaya: I'll try pushing up a dummy patch and see if it notifies the channel.14:44
anteayasmcginnis: I see it as working14:44
*** pradk has joined #openstack-infra14:44
anteayasmcginnis: okey dokey14:44
* anteaya moves to -cinder14:44
smcginnisanteaya: Thanks for looking into it. I'll let you know how my test goes.14:44
anteayasure, welcome14:45
asselinBobBall, anteaya, we're getting closer to having an infra hosted ci-watch.14:45
anteayaasselin: figured as much, you are no slouch when it comes to making progress :)14:45
openstackgerritZara proposed openstack-infra/storyboard-webclient: Make it clear that all fields are required
anteayaasselin: I'm just not sure which tool BobBall is referencing in his comment to me in backscroll14:46
*** lucasagomes is now known as lucas-hungry14:46
*** lykinsbd has joined #openstack-infra14:47
asselinanteaya, that's the scoreboard was one of the options considered, but we went with ci-watch instead.14:48
anteayaasselin: have you a link to the scoreboard tool?14:49
* anteaya doesn't have the link14:49
openstackgerritZara proposed openstack-infra/storyboard-webclient: Make it clear that all fields are required
*** gokrokve has quit IRC14:49
asselinI assume you mean the source code.14:50
anteayaI mean the url so I can see the shiny dashboard14:50
anteayathanks though14:50
anteayaI can't keep track of all the things14:50
anteayawhich is why your work is so helpful, asselin, thanks14:51
asselinthis is the version we're working on ^^14:51
anteayanice work14:52
*** bardia has joined #openstack-infra14:53
asselinI can't take any really. mmedvede and apoorva are the active ones. I just review.14:53
anteayathat is very important14:54
anteayaas you know14:54
asselinyes :)14:54
anteayaI'm going to see if I can get a walk in14:55
anteayaback in a bit14:55
*** lv_ has joined #openstack-infra14:57
e0nejgriffith: hi. FYI, I've updated spec to implement cinderclient extension instead of python-brickclient14:58
*** abregman is now known as abregman|mtg14:59
openstackgerritDamien RANNOU proposed openstack-infra/project-config: Change max-servers settings for OVH BHS region
*** ddieterly has joined #openstack-infra15:00
*** kushal has quit IRC15:01
*** eharney has quit IRC15:02
e0neopps, wrong channel:(15:02
*** eharney has joined #openstack-infra15:03
nibalizeryolanda: the in-tree hiera is failing a few tests though15:04
nibalizerill need to investigate and fix15:04
yolandahi nibalizer, great, need a hand?15:05
nibalizerif you want to investigate be my guest15:05
*** bryan_att has joined #openstack-infra15:05
*** sigmavirus24_awa is now known as sigmavirus2415:05
yolandado you have the url handy?15:05
nibalizerya sec15:06
*** xyang1 has joined #openstack-infra15:06
nibalizerstack starting here:
*** EricGonczer_ has joined #openstack-infra15:08
*** AJaeger has joined #openstack-infra15:09
openstackgerritgongysh proposed openstack-infra/project-config: Add python jobs for tacker client
yolandaugh, nibalizer, that's consequence on how we handle defaults in our tests, where there is no hiera default15:10
yolanda$elasticsearch_nodes = 'elasticsearch_nodesNoDefault'15:11
yolandaand fails with15:11
yolanda Detail: undefined method `join' for "elasticsearch_nodesNoDefault":String15:11
*** tsg has joined #openstack-infra15:11
yolandaso easy way to fix.. we add a [] default to hiera15:11
yolandadifficult way, we make defaults a bit more intelligent15:11
*** tsg has quit IRC15:12
*** fredericve has joined #openstack-infra15:13
*** atcitlionok has quit IRC15:16
nibalizeri like easy15:17
BobBallanteaya: / both link to the tracking page15:17
*** mohankumar has quit IRC15:17
BobBallasselin: ci-watch isn't as useful as scoreboard though :/ Very cluttered :)15:18
*** derekh has joined #openstack-infra15:18
nibalizeryolanda: how come it passes on trusty but not the others?15:18
nibalizeroh because of node-type ?15:18
openstackgerritsebastian marcet proposed openstack-infra/openstackid: OIDC - OpenId Connect Implementation
yolandayep, i guess it skips that bits for trusty15:19
*** moravec has quit IRC15:20
*** ildikov has joined #openstack-infra15:20
*** moravec has joined #openstack-infra15:20
samueldmqmordred: just to confirm15:21
*** zz_dimtruck is now known as dimtruck15:21
*** zeih has quit IRC15:22
openstackgerritsebastian marcet proposed openstack-infra/openstackid: OIDC - OpenId Connect Implementation
samueldmqmordred: this is because we need the new release of os-client-config15:22
*** noqa_v_qoovnie has quit IRC15:22
*** bcourt has quit IRC15:22
fredericvehello all, I'm trying to clone the openstack infrastructure for our own projects. I'm reading the "running your own" documentation and I'm a bit confused with how I can override the configuration on top of the defaults in system-config. There is talk about "cloning the entire tree". Does that mean I should just do "git clone" somewhere else on top of the clone I already have in /opt/system-config?15:24
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources: Summit Application API
fredericvemy sincere apologies if that sounds stupid15:24
*** Daisy has joined #openstack-infra15:25
*** mwagone has joined #openstack-infra15:26
*** AJaeger has quit IRC15:26
*** Daisy has quit IRC15:29
*** abregman|mtg is now known as abregman15:30
*** gokrokve has joined #openstack-infra15:30
*** ociuhandu has joined #openstack-infra15:30
eantyshevfredericve: AFAIU, you read an outdated recipe, check out this new version:
openstackgerritRyan Moats proposed openstack-infra/project-config: make gate-tempest-dsvm-neutron-multinode-full job voting
fredericveeantyshev: sweet! thanks!15:33
*** dprince has quit IRC15:33
eantyshevHello, are there anybody capable for merging to puppet projects or system-config? If you do, could you please do so for 253642, 253619, 254128 series15:34
*** gokrokve has quit IRC15:35
*** thorst_ has quit IRC15:35
asselinnibalizer, yolanda and you review eantyshev patches ^^?15:35
asselinBobBall, can you elaborate?15:35
*** mwagone has quit IRC15:36
*** bcourt has joined #openstack-infra15:38
BobBallWell when I want to look at the historical view of a CI, or comparison between it and another non-jenkins CI it's really hard to do.  With scoreboard you could filter based on which CI you're interested in15:40
eantyshevasselin: it would be nice if anybody reviews 239479, too :)15:42
*** lucas-hungry is now known as lucasagomes15:42
*** mwagone has joined #openstack-infra15:42
*** gongysh has quit IRC15:43
sdaguelogstash ui seems wedged15:43
sdaguewhich makes it hard to find our new 50% failure in multinode15:44
*** Sukhdev has joined #openstack-infra15:44
*** thinrichs has joined #openstack-infra15:44
openstackgerritBrian Haley proposed openstack-infra/project-config: Make gate-tempest-dsvm-neutron-dvr job voting
*** gokrokve has joined #openstack-infra15:45
*** kozhukalov_ has quit IRC15:45
*** nmagnezi has quit IRC15:45
*** Qiming has quit IRC15:45
thinrichsDoes anyone know how to convince launchpad to show the most recent release (liberty) as the "Latest version" on the landing page?15:45
thinrichsFor congress it's still showing kilo, even though the liberty release date is after kilo.15:45
thinrichsI expect it's b/c kilo's version is 2015.1.0 and liberty's is 2.0.015:46
*** Liuqing has quit IRC15:47
*** nijaba has joined #openstack-infra15:47
*** nijaba has joined #openstack-infra15:47
*** tsg has joined #openstack-infra15:48
*** mwagone has quit IRC15:48
*** mwagone has joined #openstack-infra15:49
mriedemsdague: it's dead15:49
mriedemDelay in Elastic Search: Indexing behind by 14 hours15:49
fungiyolanda: looks like you approved while there were meetings underway, so you may want to reply to
yolandafungi well, the approval was done an hour ago, but took ages.. yes15:50
yolandahow do you normally deal with that?15:50
yolandawhen i approved i double checked agenda, it was no meeting15:51
sdaguemriedem: fun15:51
fungiif gating is slow/backed up then we usually hold off or wait until some time when there are several hours before any scheduled meeting15:52
fungibecause, as you point out, it can sometimes take more than an hour for a change to merge15:52
fungiif we get someone to rewrite our service bots, ideally we'll have a bot framework that can join new channels without restarting/losing state on existing channels, or worse case delay its restarts until it's idle15:53
fungibut unfortunately that's not the case for now15:54
*** atcitlionok has joined #openstack-infra15:54
cbaderI am looking for someone knowledgeable on zuul ssh to upstream. I am having an issue where I get a zuul-merger error failing on publickey during the git clone. can seem to figure it out.15:54
*** sdake has joined #openstack-infra15:55
yolandaok i'll consider for next time. I'll reply now15:56
*** daemontool has joined #openstack-infra15:56
*** mrmartin has joined #openstack-infra15:56
asselinBobBall, ok, so just need a way to filter out some of the ci. thanks for the feedback.15:56
BobBallasselin: Well - tbh the other fab thing about scoreboard is a summary for how many passed/failed over the period you're interested in for each CI (in this case it'd be per-job)15:57
*** kozhukalov_ has joined #openstack-infra15:57
BobBallTakes the guesswork out of seeing the overall picture15:57
BobBallIf that were added I'd be so very happy indeed :)15:58
*** gokrokve has quit IRC15:58
*** erlon has joined #openstack-infra15:59
*** gokrokve has joined #openstack-infra15:59
asselinBobBall, ok noted as well. If you or someone else is interested to work on it or review patches, let me know.15:59
*** tjones has joined #openstack-infra15:59
*** lezbar has quit IRC16:00
bardiaOur CI's Zuul service just stopped receiving gerrit changeset updates out of nowhere yesterday. It has been running fine for weeks now and constantly receiving updates and running tests. I had to manually reset the Zuul service for it to start receiving changesets and running tests.16:00
bardiaThere are no errors or tracebacks in any of zuul's logs16:00
BobBallMore than happy to review patches (although my knowledge of ci-watch is precisely 0 ATM) but it'd take me a while to get around to implementing the changes if I'm honest.16:00
BobBallbardia: I've seen that too - I think it was caused by a network interruption that zuul didn't spot.16:01
asselinBobBall, sure, it's new for everyone, so a great time to start. :)16:01
bardiaBobBall: so I just have to keep an eye on zuul?16:01
*** ashishb has quit IRC16:02
*** fredericve has quit IRC16:02
asselinbardia, BobBall
*** lezbar has joined #openstack-infra16:04
*** pradk has quit IRC16:05
bardiaasselin: thanks16:05
*** kmartin has joined #openstack-infra16:07
*** rbrndt has joined #openstack-infra16:07
openstackgerritAndrea Frittoli proposed openstack-infra/puppet-log_processor: Pin python-daemon version
*** tonytan4ever has joined #openstack-infra16:09
*** pradk has joined #openstack-infra16:10
*** sdake has quit IRC16:10
openstackgerritMerged openstack-infra/jenkins-job-builder: Correct the entry points of cucumber-testresult
*** mrmartin has quit IRC16:14
anteayaBobBall: thank you16:14
*** mwagone has quit IRC16:14
openstackgerritAndrea Frittoli proposed openstack-infra/puppet-log_processor: Pin python-daemon version
*** dimtruck is now known as zz_dimtruck16:16
anteayaclarkb fungi as a email data point I didn't recieve this post in my thunderbird client inbox, just yolanda's reply:
*** mwagone has joined #openstack-infra16:16
anteayaclarkb fungi I don't know if that is a mailserver thing or a thunderbird client thing16:17
yolandaanteaya, i neither received that, i just quoted in my reply16:17
anteayayolanda: thanks for the confirmation16:18
yolandasaw the comment in the list due to fungi, and prior, due to AJaeger pointing that to me this morning, but i don't have that in my inbox, and i didn't trash it, or not filtered in any folder16:18
yolandai just did a massive search on my thunderbird folders without success16:18
*** ajmiller has joined #openstack-infra16:19
anteayaI don't see it in spam or trash for me either16:19
openstackgerritAntoine Cabot proposed openstack-infra/project-config: Add watcher project to gerritbot
anteayaoh wait there it is, it _was_ in my inbox just way way way down the list16:20
anteayamy fault16:20
*** jcoufal has quit IRC16:20
mriedemjeblair: fungi: clarkb: anything in the logstash job logs about that going down?16:22
openstackgerritMerged openstack/diskimage-builder: Add kmod to package-installs of ironic-agent
EmilienMnibalizer, crinkle: fyi
*** Swami has quit IRC16:23
*** daemontool_ has joined #openstack-infra16:23
yolandai don't have on mine, but i may have a problem, i haven't received infra emails today16:23
openstackgerritFausto Marzi proposed openstack-infra/project-config: Creation of a dedicated repo for API and media storage client
jeblairyolanda, fungi: we should be able to approve now16:24
*** bharathm has joined #openstack-infra16:24
*** placko has quit IRC16:24
*** mrmartin has joined #openstack-infra16:25
*** zz_dimtruck is now known as dimtruck16:25
openstackgerritSean Dague proposed openstack-infra/project-config: test requirements with grenade multinode
*** lv_ has quit IRC16:27
openstackgerrityolanda.robla proposed openstack-infra/system-config: Use In-Tree Public Hiera data
*** harlowja_at_home has joined #openstack-infra16:27
*** daemontool has quit IRC16:28
rcarrillocruzzomg, in-tree public hiera!16:29
* rcarrillocruz dances16:29
anteayattx lifeless would like to hear your thoughts on which adds a job to the requirements repo16:29
*** rossella_s has quit IRC16:30
*** rossella_s has joined #openstack-infra16:30
*** dizquierdo has quit IRC16:31
nibalizerrcarrillocruz: ya, the future is real!16:32
*** amitgandhinz has quit IRC16:32
*** jaosorior has quit IRC16:32
clarkbmriedem: if I had to guess we likely leaked indexes again16:32
*** amitgandhinz has joined #openstack-infra16:33
electrofelixzaro wayne zxiiro: are we happy enough with the parallelization for JJB to review/approve
*** dimtruck is now known as zz_dimtruck16:33
zxiiroelectrofelix: I've been using it for awhile and i'm satisfied with it. I didn't vote since i rebased it16:34
zxiirolooking forward to it getting merged16:34
clarkbjesusaurus: confirmed htat we have far more than 10 indexes, any idea why curator isn't doing its job?16:34
jeblairyou had one job curator16:34
clarkbjeblair: you looked into this last, anything you think we should do before running curator by hand?16:34
jeblairclarkb: ah, check logfiles16:35
jeblairclarkb: we ran out of space last time and there were no logs16:35
clarkbjeblair: the last logfile was written on 11-3016:35
jeblairclarkb: uhoh did we just let the same thing happen?16:35
clarkbwe have 1.3GB of disk16:35
jeblairoh cool16:35
clarkbI think its only running when we run it by hand16:35
zxiiroelectrofelix: I'm looking forward to getting the views feature as well:
clarkbsomething is broken about the cron16:35
jeblairso we're just at 'logstash is real slow'16:35
clarkbjeblair: well no queries are succeeding due to the OOM prevention stuff16:36
clarkbData too large16:36
clarkbnibalizer: yes indexes are not being deleted16:36
jeblairclarkb: where's this run?16:36
electrofelixzxiiro: I've got 13 changes outstanding for JJB excluding the two around entry points :(16:36
clarkbjeblair: on the elasticsearch nodes16:36
*** bpokorny has joined #openstack-infra16:37
jeblairclarkb: it runs on every node?16:37
clarkbjeblair: yes16:37
jeblairclarkb: is that workable?16:37
clarkbthey will all submit deletes for the same indexes and es will only delete once16:37
zxiiroelectrofelix: I hear ya, I'll do another round of reviews soon. I spent a whole night doing reviews a few weeks ago.16:38
jeblairclarkb: makes sense... unless they exit on failure or something16:38
krotscheckmordred: Is kite still a thing?16:38
jeblairclarkb: they'd probably eventually suceed though even if they do that16:38
* krotscheck sees your commit against it.16:38
clarkbI think I know what the problem is16:38
clarkb% is special in cron16:38
clarkbjesusaurus: ^16:38
* krotscheck also notes its the only change since june 25th16:38
nibalizerhaha so 'glance image-create' with no arguments just goes ahead and creates an image with no data and empty string as a name16:38
jeblairclarkb: yeah, doesn't look like its running; i bet if we had syslog back from long enough ago we'd see an error from when it was added16:40
clarkbjeblair: the issue is the %16:40
clarkbit is running but % makes it not do anything useful16:40
openstackgerritMerged openstack-infra/project-config: Change max-servers settings for OVH BHS region
jeblairclarkb: where do you see it running?16:40
clarkbjeblair: I don't, but I know % wil break it16:41
jeblairclarkb: okay, i don't see it running16:41
electrofelixzxiiro: are you aware of,dashboards/important-changes:review-inbox-dashboard16:41
openstackgerritGuido Günther proposed openstack-infra/jenkins-job-builder: delivery-pipeline: support task description template
clarkbjeblair: I think its running and silently succeeding at doing nothing bceause of the %16:41
mordredkrotscheck: aroo?16:41
krotscheckThat's yours.16:42
anteayakrotscheck: barbican feels kite is still a thing I believe16:42
mordredkrotscheck: oh - no clue honestly - I patch-bombed all openstack repos with that patch16:42
jesusaurusclarkb: oh? what does % do in cron?16:42
krotscheckanteaya: Ok, I'll ask over there.16:43
jeblairjesusaurus: it terminates the command and passes the rest of the line as stdin16:43
jeblairneeds to be escaped with \16:43
jesusauruswow TIL16:43
zxiiroelectrofelix: I am not, but that's good to know. I'll bookmark it thanks!16:44
openstackgerritKevin Carter proposed openstack-infra/project-config: Added new projects for the OSA role break out
*** apuimedo is now known as away16:44
*** away is now known as Guest3275216:45
clarkbjesusaurus: you can see the escaped in
clarkbanyways I am going to run by hand16:45
clarkband hopefully we can get escapes in today16:45
openstackgerritFausto Marzi proposed openstack-infra/project-config: Creation of a dedicated repo for API and media storage client
jesusaurusI'll write up a quick change for that if no one else is yet16:46
nibalizercrinkle: i am spawning a vm on omfra cloud!16:47
*** peristeri has quit IRC16:47
mordrednibalizer: zomg16:47
mordreddo we have it in dns yet?16:48
clarkbnibalizer: by hand or with nodepool?16:48
*** HeOS_ has quit IRC16:48
fungimordred: not to my knowledge. i volunteered but need to know what it's to be called16:48
openstackgerritK Jonathan Harker proposed openstack-infra/puppet-logstash: Escape % in cron command
nibalizerclarkb: by hand right now16:49
jesusaurusclarkb: ^16:49
nibalizernodepool after16:49
anteayazomfra cloud16:49
*** HeOS_ has joined #openstack-infra16:49
*** lv_ has joined #openstack-infra16:49
clarkbis that a brand of vanilla?16:49
fungianyway, i assumed there was a bikeshed color scheme discussion coming before i paint dns with it16:50
mordredoh, I thought the bikeshed was about the region name16:50
mriedemclarkb: just read scrollback, thanks for looking into that16:50
*** earlephilhower has joined #openstack-infra16:50
fungiwill the region name be part of the dns entry?16:50
pleia2good morning16:50
mordredthe adminrc file calls the controller node ""16:51
clarkbjesusaurus: I think I only know about it because it bit me when I wrote the first cron16:51
*** gokrokve has quit IRC16:51
nibalizeri think it was suggested that we make a vanilla cloud, a chocolate cloud, and a strawberry cloud16:51
clarkband its obscure enough I didn't remember until just now16:51
mordredand I think we decided that we'd call that region "hpuswest"16:51
mordredbut I could be very misremembering that16:51
*** _nadya_ has joined #openstack-infra16:51
nibalizeranother question I'm wondering is if we do real certs for this16:51
*** rlandy has joined #openstack-infra16:52
nibalizeras it is, it is set up with a self signed cert16:52
nibalizerand I have the cacert file and am setting it with OS_CACERT16:52
anteayamorning pleia216:52
clarkbnibalizer: yes I think so, also what version of pytho nare you on that actually checks the cert? :P16:52
*** HeOS has quit IRC16:52
*** mrmartin has quit IRC16:53
fungiperhaps he did pip install requests[security] (after preinstalling distro packages of lots of c headers)16:53
mordredwell, he should always install requests[security]16:54
mordredif he wants secure connections16:54
clarkbhuh there are insecure platform warnings16:54
mordrednope. he did not16:54
mordredhe's getting the insecure platform warnings16:54
clarkbmaybe python actually does check some things but not others16:54
*** mohankumar has joined #openstack-infra16:54
*** sridhar_ram has joined #openstack-infra16:54
nibalizermordred: the final line16:54
clarkbnot that insecure !16:55
mordredso - install requests[security]16:55
nibalizererror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed16:55
nibalizerthis is something I pip install?16:55
nibalizerwith the brackets16:55
clarkbnibalizer: in theory python isn't supposed to care about that hence the insecure platform warnings16:55
clarkbrequests[security] intsalls bindings for requests that actually check things properly16:56
*** sdake has joined #openstack-infra16:56
nibalizeri have insufficient headers16:56
*** thorst has joined #openstack-infra16:56
nibalizeranyways not important right now16:56
*** gyee has joined #openstack-infra16:56
mordrednibalizer: what's in your venv?16:56
mordredbecause the things are on that node16:57
mordredand keystone catalog works for me without any warnings just raw on the box16:57
*** mrmartin has joined #openstack-infra16:57
*** Swami has joined #openstack-infra16:57
mordredcrinkle: what's the difference between the _member_ role and the user role?16:57
nibalizeri created the user role by following the kilo docs16:57
nibalizeris where that came from16:58
*** dtantsur is now known as dtantsur|afk16:58
mordrednibalizer: ok. so there's already a _member_ role which is, I believe created by the puppet16:58
nibalizeralso bad news, i've booted two nodes, neither ping :(16:58
mordrednibalizer: doesn't seem we need it - the only role listed in nova's policy.json is admin16:59
anteayagood news you booted two nodes16:59
*** kushal has joined #openstack-infra16:59
clarkbnibalizer: where ar eyou pinging from?16:59
*** unicell1 has joined #openstack-infra16:59
mordrednibalizer: they ping for me17:00
clarkbconsidering hte network setup there you may have to ping from not the cloud nodes17:00
mordredfrom not controller node17:00
mordred ping
*** mrmartin has quit IRC17:00
crinklemordred: nibalizer ha yeah _member_ is a real thing and user is not17:00
nibalizeryou can ping  ?17:00
mordredcrinkle: awesome. I'm going to delete it17:00
mordrednibalizer: no17:01
timrcyolanda: Heeey.. ;-)
*** unicell has quit IRC17:01
crinklebtw i'm at a conference so i may be slow to respond17:01
mordrednibalizer: I don't see in the nova list output for the admin user17:01
mordredI see17:01
mordred| 39790724-228d-43b7-a048-75daf37c04ca | test | ACTIVE | -          | Running     | provider-net= |17:01
mordred| 8e138786-bb85-41d2-8894-a4eda9516a4b | test | ACTIVE | -          | Running     | provider-net= |17:01
nibalizeri didn't create them as admin17:01
*** links has quit IRC17:01
nibalizeri made an infra user17:01
mordredawesome - what project?17:01
nibalizerill add _member_ as the mapping between infra and infra17:02
*** zz_dimtruck is now known as dimtruck17:02
openstackgerritLucas Alvares Gomes proposed openstack-infra/project-config: Make the ironic-pxe_ipa-ipxe voting
anteayacrinkle: what conference?17:03
crinkleanteaya: the internal Tech Tour conference for hp17:04
mordrednibalizer: I think I'd actually like to make an infra domain17:04
anteayacrinkle: ah fun17:04
anteayacrinkle: hope you are having a good time17:05
mordrednibalizer: and make the infra project be in the infra domain17:05
*** bharathm has quit IRC17:05
mordrednibalizer: largely because I think a test domain where we can test project creation/deletion would be nice17:05
* mordred should write an ansible playbook to do this17:06
*** mwagone has quit IRC17:06
openstackgerritMatt Riedemann proposed openstack/requirements: Block oslo.messaging 3.1.0 since it breaks Ironic
*** bcourt has quit IRC17:07
mordrednotmorgan: is it possible for users to change their passwords via keystone api in v3?17:07
nibalizermordred: eh? seems like you could do that against devstack17:07
notmorganmordred: fairly certian so17:07
mordrednotmorgan: cool17:07
notmorganmordred: sec looking at to find the API17:08
nibalizercrinkle: my nodes do not network, halp!17:08
mordrednibalizer: where can I find the password for the infra user?17:08
nibalizermordred: ~nibz/userrc17:08
nibalizeron the controller17:08
*** daemontool_ has quit IRC17:08
mordrednibalizer: cool.17:08
*** acabot has quit IRC17:08
crinklenibalizer: is this with the infra user? maybe need to acl some things17:08
nibalizercrinkle: yes17:09
nibalizerif it works for admin but not for user then that totally points to acl something mumble17:09
crinklenibalizer: i can't really look at it right this second17:09
clarkbnova security-group-list or something17:09
nibalizer| d094313b-9f11-4507-be96-fed257885832 | testmachine2 | ACTIVE | -          | Running     | provider-net= |17:09
nibalizerbut then the thing doesn't ping17:09
clarkbnibalizer: then nova security-group-update icmp -1 -1
nibalizernova console-log indicates that the node things it has that ip17:09
clarkbnibalizer: yes likely security groups17:10
*** dprince has joined #openstack-infra17:10
timrcclarkb: I vaguely remember some documentation about current possibilities around multi-node testing with nodepool, but I can't remember where I saw that.  It may be insufficient for what I'd like to demo at Blue Box but I was hoping maybe you could point me in the right direction?17:10
nibalizeroh i dind't thing security gorups were a thing with provider networks17:10
mordrednibalizer: they're orthogonal17:10
clarkbnibalizer: I think they are always a thing17:10
clarkbbecause ya ....17:10
nibalizercrinkle: ok no worries17:11
crinklenibalizer: works?17:11
* mordred wants to learn how to configure the default security group17:11
nibalizerim super excite that there is a cloud17:11
notmorganmordred: so we have UserV3 controller which has change_password17:11
timrcWe have a project called ursula that's responsible for deploying clouds.  Whenever a change is made we provision some nodes and install a cloud using trunk+commit and then run some integration tests.  I'd like to see if we can use nodepool for the provisioning part.17:11
mordrednotmorgan: woot17:11
clarkbtimrc: nodepool can spin up sets of nodes with one being primary and attached ot jenkins. All nodes are told who the others are via files in /etc/nodepool17:11
nibalizercrinkle: im gonna try clarks things17:11
*** psanchez has joined #openstack-infra17:11
notmorganmordred: trying to find the route17:11
crinklei set icmp/ssh for the default security group, i didn't realize there were per-user differences17:11
mordrednibalizer: anywho - I also want to create an infra domain largely because we have a v3 cloud and I'd like to dtrt wrt domains17:11
*** rcernin has quit IRC17:11
clarkbtimrc: details on that info there17:11
timrcclarkb: Hm that might actually be enough.17:11
mordrednotmorgan: it's cool - if it's a thing, that menas I can find it17:12
*** acabot has joined #openstack-infra17:12
nibalizermordred: ya you can change things if you wish, note that this is a kilo cloud17:12
clarkbtimrc: I would be very surprised if this doesn't work for that use case17:12
mordrednibalizer: lemme cook up a quick playbook that describes what I think we should do17:12
notmorganmordred: not sure if we have it in ksc17:12
clarkbtimrc: since its basically identical to how we test upstream17:12
mordrednotmorgan: user password set is in osc17:12
mordrednotmorgan: I just wasn't user if a user could use it for themselves17:12
notmorganmordred: i mean as the lib17:12
mordredand hadn't gotten to creating a user so I could test it17:12
*** jlanoux has quit IRC17:13
clarkbnibalizer: security groups are applied on hypervisor iptables so its always there17:14
notmorganmordred: so ksc has it as part of the lib, so it should all work.17:14
mordrednotmorgan: awesome17:14
notmorganprovided osc has the command17:14
mordredit does17:14
*** sshnaidm has quit IRC17:14
mordrednibalizer, crinkle: I have made an adminrcv3 in /root17:14
mordredwhich has the v3 version of the rc file17:14
timrcclarkb: Sounds exactly like what we need.17:14
crinklemordred: cool17:15
fungii put controller00.hpuswest.ic.o.o in dns for now17:15
crinklefungi: thank you, that is super helpful17:15
clarkbtimrc: the way we use it is have devstack-gate write out an ansible inventory file with that data17:15
openstackgerritKevin Carter proposed openstack-infra/project-config: Added new projects for the OSA role break out
clarkbtimrc: then its ansible primary dosomething and ansible subnodes dosomethingelse and ansible all otherthing17:16
clarkbtimrc: because its ssh based it maps really well17:16
*** BobBall is now known as BobBall_AWOL17:16
* mordred is enjoying playing with our cloud API17:16
*** mrmartin has joined #openstack-infra17:17
fungimordred: it's even better because you know that any bug you find, you can just fix without needing to explain your reasons to teh provider17:17
mordredfungi: yup!17:17
mordredfungi: well, I have to convince YOU PEOPLE17:17
crinkleplease include reasons in commit messages :P17:18
*** ajmiller_ has joined #openstack-infra17:18
* fungi is easily convinced17:18
yolandahi, can i have some eyes on ? that's to start dealiing with apt-get autoremove in our servers17:18
clarkbyolanda: I wouldn't even parameterize it17:19
clarkbyolanda: we know that if you don't set it to true bad things happen17:19
*** jistr has quit IRC17:19
clarkbyolanda: also is there a change for the autoclean of the cache?17:19
*** mwagone_ has joined #openstack-infra17:19
mordredcrinkle: so ...17:19
yolandaclarkb not yet17:19
mordredcrinkle: the patch to disable ec2 and novav317:19
mordredcrinkle: seems to have had an odd effect on the catalog17:20
yolandafirst wanted to show that. About paremeterizing, as it's no system-config, but the puppet module i think we shall consider all uses cases17:20
*** tlian has quit IRC17:20
yolandafor example in downstream we wouldn't want unattended autoremoves17:20
clarkbyolanda: I mean17:20
mordredcrinkle: the services still show in the catalog, but they show with urls like:
clarkbyolanda: its broken when set to false right?17:20
clarkbyolanda: bceause you run out of inodes17:20
mordredcrinkle: which makes me think that there is a default value in puppet somewhere that disabling things is causing to not fully disable17:20
timrcclarkb: I mean... that's pretty much what we do.  We provision hosts, write a hosts file, deploy cloud to them, run some tests.  So I could see nodepool factoring out the provisioning piece and hopefully eliminating some of the expense of setup by having ready nodes.17:21
crinklemordred: i expected the first part, the second part is sorta strange17:21
yolandaclarkb, unless you schedule some periodical cleanups for example17:21
clarkbyolanda: if its not broken lets not bother with it17:21
timrcdeploy cloud to them with the ansible*17:21
*** ajmiller has quit IRC17:21
clarkbyolanda: this is the periodic cleanup17:21
crinklemordred: we'll need to delete the endpoints/services by hand17:21
*** Apoorva has joined #openstack-infra17:21
clarkbyolanda: this is what that is for17:21
crinklemordred: then puppet shouldn't put it back weird17:21
yolandaclarkb, i know. I'm trying to expose some downstream case17:21
openstackgerritKhai Do proposed openstack-infra/system-config: Allow Gerrit to use 4 threads for indexing changes.
*** bcourt has joined #openstack-infra17:21
yolandain our nodes we don't want that to be executed automatically, for x reasons17:21
clarkbyolanda: what is the use case to set that to false if we know your server will break when set to false?17:21
mordredcrinkle: I'm going to disable them by hand now and see what happens next puppet17:22
yolandabut we execute that cleanup manually17:22
*** ashishb has joined #openstack-infra17:22
yolandaclarkb, it won't break if you monitor disk space and do a manual cleanup when needed17:22
yolandai'd like to leave it open to parameterize, if you prefer we can change the defaults to true17:22
clarkbyolanda: and doing it manually when it can be done automatially isn't a bug?17:22
yolandaclarkb well, i can tell you that our use case downstream is to don't do updates or autoremoves automatically, for certain servers17:23
yolandait can be a bug but is some use case we hit17:24
mordredcrinkle: there is another weird - which is not a problem but I mention purely in case is should be an improvement to the puppet modules17:24
clarkbyolanda: for those servers you should not use the unattended updates module then17:24
clarkbyolanda: because this is why the module exists to do updates automatically17:24
mordredcrinkle: we now have 4 services, for them, glance shows internal of and public of
*** dims has quit IRC17:24
sdagueclarkb: I just had another one of those connection failed blips17:24
zaroyolanda, crinkle : help?  don't understand why that's not working.17:25
mordredcrinkle: oh - wait, I was going to give you that backwards17:25
sdaguewas there an es restart or something?17:25
clarkbsdague: no, just deleted indexes again17:25
mordredcrinkle: non-keystone services show in their internal url17:25
clarkbbut let me see if rax restarted a node17:25
yolandaclarkb ok i'll hardcode it and we'll revisit our setup17:25
sdaguein the last 3 minutes?17:25
*** jaypipes has quit IRC17:25
mordredcrinkle: but keystone's internal is
clarkbsdague: status is green17:25
sdagueI seriously want to kick someone at ES in the knees for this17:26
crinklemordred: i think we're setting public and admin and not internal, so it defaults to http/localhost17:26
clarkband there are 6 data nodes so nothing went away17:26
crinklemordred: for no particular reason, i think that was just how SpamapS set it in his original manifests17:26
mordredcrinkle: ok. so something is setting internal for keystone to something else as a default then17:26
crinklemordred: easy to set it17:26
sdaguebecause of their caching semantics all you need is 1 second of network failure, then you are boned until you dump cache and restart your browser17:26
mordredcrinkle: kk. we don't need the internal url for any purpose17:26
mordredcrinkle: I was mostly reporting in case there was a bug for people who do17:26
clarkbyolanda: I think that if you do not want automated updates the proper approach is to not use the module that configures automated updates17:26
clarkbyolanda: the module should however properly setup updates such that they do not break the server17:27
clarkbyolanda: that way people that do want automated updates get them in a safe manner17:27
*** olaph has quit IRC17:27
*** unicell1 has quit IRC17:27
*** olaph has joined #openstack-infra17:27
clarkbsdague: its not a great workaround but maybe use private browsing mode in $browser to work around it for now?17:27
*** esker has joined #openstack-infra17:28
*** isaacb has quit IRC17:28
mordredclarkb: otoh - is wrong for keystone as I'm listing that using v3 - so we should have the v3 endpoints in the v3 catalog17:28
mordredsorry clarkb17:28
mordredcrinkle: ^^17:28
fungioh, wow. we were flush with nodes stuck in delete for a while there according to the graph17:28
mordrednotmorgan: when did the version-less endpoint show up in keystone?17:28
*** Sukhdev has quit IRC17:29
yolandaclarkb, we got some breakages in the past , even using that, specially with kernel upgrades. But we always hit on an specific use case of baremetal servers executing lo-level testing17:29
mordrednotmorgan: was it available in kilo? or not until liberty?17:29
*** jordanP has quit IRC17:29
crinklemordred: ah that's a bug17:29
notmorgani'd need to ask jamielennox|away17:29
clarkbyolanda: then you shouldn't use the automated updates puppet module17:29
crinklemordred: the reason keystone is different from the rest is
crinklemordred: just inconsistency in the module17:29
mordredcrinkle: cool17:29
*** salv-orlando has quit IRC17:30
mordredalso, looks like the versioned endpoint didn't change until liberty17:30
*** eil397 has joined #openstack-infra17:30
*** gokrokve has joined #openstack-infra17:30
notmorganbut yeah liberty is likely where that changed17:30
openstackgerritDong Ma proposed openstack/python-jenkins: Fixed create_node() Error
mordredk. so we can't do that until we're liberty17:31
*** claudiub has joined #openstack-infra17:31
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for grenade bug 1524418
openstackbug 1524418 in oslo.messaging "gate-grenade-dsvm-multinode fails with "AttributeError: 'LocalManager' object has no attribute 'l3driver'"" [Undecided,New]
*** abregman has quit IRC17:34
*** egonzalez has quit IRC17:35
*** loaf has joined #openstack-infra17:35
*** bardia has quit IRC17:36
*** thinrichs has quit IRC17:36
*** ayoung has quit IRC17:37
*** armax_ has joined #openstack-infra17:38
*** matrohon has quit IRC17:39
openstackgerritDong Ma proposed openstack-infra/jenkins-job-builder: Add support for cucumber reports plugin
*** sfinucan has quit IRC17:40
*** ashleighfarnham has joined #openstack-infra17:40
*** mwagone_ has quit IRC17:40
crinklemordred: there's an ugly bug in the keystone module for specifying the version, we should be able to make it unversioned but someone didn't understand undef
crinklemordred: i'll work on it later today17:41
openstackgerritMerged openstack-infra/elastic-recheck: Add query for grenade bug 1524418
openstackbug 1524418 in oslo.messaging "gate-grenade-dsvm-multinode fails with "AttributeError: 'LocalManager' object has no attribute 'l3driver'"" [Undecided,New]
*** amitgandhinz has quit IRC17:41
*** hdd has joined #openstack-infra17:43
mordredcrinkle: neat!17:43
*** changbl has quit IRC17:43
mordredcrinkle: well, we don't have unversioned keystone auth url until we upgrade to liberty anyway17:43
mordredcrinkle: however, we do need to set /v2.0 in the v2 catalog entries and /v3.0 in the v3 catalog entries17:44
mordredbecause that's TRT17:44
mordrednotmorgan: yes? ^^17:44
fungineed to hop over to the next island for a travel vaccine booster. should be back in a couple hours17:44
notmorganopenstack: i ... think so17:45
anteayafungi: enjoy17:45
notmorganmordred: ^17:45
crinklemordred: it doesn't seem like the keystone module wants to let you set both17:45
mordredcrinkle: that might be a bug in the keystone module17:45
mordredcrinkle: HOWEVER17:45
mordredcrinkle: just setting unversioned for v3 for liberty is a thing anyway17:45
mordredcrinkle: so we should not fix that bug17:45
mordredcrinkle: it seems like the kilo perspective in the modules is "let's deal with v3 starting with liberty" anyway17:46
*** bharathm has joined #openstack-infra17:46
crinklemordred: yeah i think so17:46
*** kozhukalov_ has quit IRC17:46
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: update 1524418 with root cause
*** yonglihe has joined #openstack-infra17:49
*** salv-orlando has joined #openstack-infra17:49
*** salv-orlando has quit IRC17:52
*** amuller is now known as amuller_afk17:53
*** sputnik13 has joined #openstack-infra17:53
mordredinfra-root, crinkle: the openstack command line client and client libs on controller00 are super old - I don't recommend spending much time with them17:53
mordredrather, I recommend doing most things related to openstack APIs from your own machine17:54
crinkle"super old" == kilo?17:54
mordredpython-openstackclient from kilo is hella suck17:54
crinkleya upgrading to liberty can be a priority17:55
*** tonytan4ever has quit IRC17:55
stevemarmordred: ja, i think that's just after 1.0.017:55
mordredwe also won't be able to use shade or the openstack ansible modules on that host17:55
mordredbecause the client lib versions will be too old17:55
mordredhowever, we don't have to17:55
mordredas we can run ansible playbooks on puppetmaster pointed at that cloud for anything we might want to playbook17:56
mordredstevemar: I was cursing about things about it - but then normorgan showed me his output and I realized we were just running old version17:57
*** hashar has quit IRC17:57
mordredstevemar: however, openstack project list --long should show domain name either instead of or in addtion to domain id17:58
mordredstevemar: because:17:58
mordred| 228a38069ea74e1d9b3f16ef86a74614 | infra     | fb7965423257443ab31844c20a92faa7 |                                   | True    |17:58
mordred| 66625c58ad43409d867e4d203b60c67d | infra     | default                          | OpenStack Infra                   | True    |17:58
mordredstevemar: is not terribly helpful17:58
dtroyermordred: ++17:58
dtroyerdo you have a preference?17:59
dtroyeralso, -c helps17:59
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: update 1524418, remove job restriction
mordreddtroyer: honestly, I _personally_ would like to just see name17:59
*** arxcruz has quit IRC17:59
mordreddtroyer: but I could totally understanding keeping id and just adding name for compat reasons17:59
*** jerryz has joined #openstack-infra17:59
stevemarmordred: dtroyer we would have to call list_domains() and cache the resultant list, and do a lookup to replace domain id/name17:59
*** markus_z has quit IRC17:59
dtroyercool, me too.  this is one area where client caching would be helpful, otherwise the lcient gets to resolve those in many cases18:00
stevemarwe do this with images and volumes today anyway18:00
*** _nadya_ has quit IRC18:01
dtroyerI don't mind doing the work if it's optional (as in —long) but I wasn't sure what the user preference was other than myself.18:01
mordreddtroyer: remind me to make you some trial ballon patches for being able to consume the cache config stuff we've got in occ18:01
mordreddtroyer: well, I'm currently developing new preferences18:01
mordreddtroyer: becaues I just got a brand new cloud delivered that I'm an admin on18:01
mordreddtroyer: so I'm using these commands in anger now :)18:01
dtroyerre occ…osc is soooo far behind on that18:01
dtroyeroh, nice!18:02
mordredyah - I've got like 50 patches I need to write you -18:02
dtroyerThat drove a set of decisions I made in OSC too after arriving at $FORMER_DAY_JOB18:02
*** changbl has joined #openstack-infra18:02
mordredanger is a great motivator to reduce anger18:02
dtroyernothing like real-world use cases ;)18:02
openstackgerritDawid Malinowski proposed openstack-infra/jenkins-job-builder: Add support for OWASP Dependency-Check Plugin
mordreddtroyer: I was about ot explode about how openstack project list --help just showed me the entire command list18:03
mordreddtroyer: but you've alreayd fixed that :)18:03
dtroyerI have an asbestos suit for help discussions18:03
dtroyernobody likes it, including me18:03
*** ayoung has joined #openstack-infra18:03
mordreddtroyer: :)18:04
mordreddtroyer: what's the osc env var equiv for --insecure?18:04
dtroyerthere isn't one, intentionally18:04
crinklemordred: i can give you the CA that's set up18:04
*** thinrichs has joined #openstack-infra18:04
mordredcrinkle: sweet! please do18:05
mordredcrinkle: or I can fetch it if you give me the patch18:05
stevemardtroyer: mordred hey! help is cleaned up nicely now :P18:05
*** yamamoto has quit IRC18:05
* stevemar ducks18:05
dtroyerstevemar: I'll give you 'better than before', sure18:05
mordreddtroyer, stevemar, notmorgan: BTW... I have OS_IDENTITY_API_VERSION=318:06
mordredand I'm getting18:06
mordredDiscovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL.18:06
notmorgani've gotten that in a lot of cases18:06
mordredI think I would like to describe that as a bug18:06
dtroyerI've slept since I look at this, but is that from auth or from the CRUD call?18:06
*** amitgandhinz has joined #openstack-infra18:06
mordredsince we have explicitly requested a version with OS_IDENTITY_API_VERSION=318:07
notmorgandtroyer: auth call iirc18:07
notmorgandtroyer: because it's in the auth plugin18:07
mordredyeah - this is a cloud that does not have version discovery18:07
dtroyerand I don't think auth pays attention to that18:07
mordredoh - because that's only talking about crud api?18:07
dtroyerso yeah, maybe a bug with some ideas on precedence18:07
lifelessanteaya: will comment on the review18:08
*** sdake has quit IRC18:08
dtroyermordred: right, I don't think auth is hooked up to look at that18:08
mordredperhaps we need to introduce an OS_AUTH_API_VERSION variable that you can set to be explicit about that?18:08
*** thinrichs has left #openstack-infra18:08
lifelesssamueldmq: thanks, np18:08
dtroyernot sure I have a strong opinion on it, but that would be a bit of a change for us18:08
mordredor - lemme try something18:08
samueldmqlifeless: :)18:08
anteayalifeless: thank you18:08
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: update 1524418
mordredif I set OS_AUTH_TYPE=v3password, that goes away18:09
mordredso there _IS_ a way to silence it18:09
openstackgerritMerged openstack/requirements: Remove httpretty from requirements
notmorganmordred: so yeah this is Password plugin being bad :(18:09
*** tlian has joined #openstack-infra18:10
openstackgerritJesse Pretorius proposed openstack-infra/project-config: Added new projects for the OSA role break out
*** amitgandhinz has quit IRC18:11
mordrednotmorgan: well, no, I think it's correct - OS_IDENTITY_API_VERSION is different than auth version or auth plugin18:11
dtroyerpassword plugin implies doing discovery18:11
clarkbthere is a nova change that has passed all three dibtest jobs in expreimental18:11
notmorganmordred: again thi is really a bad error [as we discussed before]18:11
clarkbI think we can maybe go ahead switching to ubuntu trusty diskimage on the devstack-trusty label18:12
*** amitgandhinz has joined #openstack-infra18:12
clarkbinfra-root ^18:12
notmorganso password+discovery needs a little work.18:12
greghaynesclarkb: So was the thing you hit yesterday a red herring?18:12
clarkbgreghaynes: ya18:12
clarkbseems to be18:12
greghaynesThat was working on all experimental for me many months ago, so that is a good sign18:12
stevemarmordred: oh i ran into that18:12
stevemarmordred: i think it's a stale clouds.yaml in ~/.config/openstack ...18:13
mordredclarkb: ++18:13
mordredstevemar: nah - it's the discover thing18:13
stevemarmordred: ah18:13
anteayaclarkb: yay18:13
stevemarmordred: different error then, i was getting this one:
*** bcourt has quit IRC18:14
stevemar"Could not determine a suitable URL for the plugin"18:14
openstackgerritLucas Alvares Gomes proposed openstack-infra/project-config: Make the ironic-pxe_ipa-ipxe voting
*** _nadya_ has joined #openstack-infra18:16
openstackgerritDawid Malinowski proposed openstack-infra/jenkins-job-builder: Add cloudbees folder creation support
lifelesssdague: I presume the multinode job actually tests more codepaths ?18:16
openstackgerritJames E. Blair proposed openstack-infra/zuul: Update the test config with a function
lifelesssdague: (that might impact libraries)18:16
openstackgerritJames E. Blair proposed openstack-infra/zuul: Merge configurations with multiple layout files
*** unicell has joined #openstack-infra18:17
*** mohankumar has quit IRC18:17
sdaguelifeless: I think it does18:17
mordredstevemar: yah - that's a different one18:18
*** markvoelker has quit IRC18:18
sdaguerequirements peeps - we need that blacklist, the grenade jobs are at 50% failure now until it's landed18:18
*** sambetts is now known as sambetts-afk18:18
*** gyee has quit IRC18:19
*** bryan_att has quit IRC18:19
mordredsdague: done18:19
openstackgerritDawid Malinowski proposed openstack-infra/jenkins-job-builder: Add support for OWASP Dependency-Check Plugin
*** tonytan4ever has joined #openstack-infra18:20
sdaguea promote would be in order as well, we're just fail grinding in the gate right now with grenades failure rate18:21
openstackgerritKen Dreyer proposed openstack-infra/jenkins-job-builder: builders: fix mavenName sorting with py35
openstackgerritKen Dreyer proposed openstack-infra/jenkins-job-builder: publishers: iterate over copy of checkstyle data
openstackgerritRyan Moats proposed openstack-infra/project-config: make gate-tempest-dsvm-neutron-multinode-full job voting
*** markvoelker has joined #openstack-infra18:21
*** electrofelix has quit IRC18:22
*** yamamoto has joined #openstack-infra18:22
*** yamamoto has quit IRC18:23
jeblairsdague: i keep trying to read that tinyurl as a word18:25
sdaguehave I snow crashed you yet?18:26
*** shayneburgess has quit IRC18:28
*** vilobhmm11 has joined #openstack-infra18:28
sigmavirus24pleia2: I might ping you about in the future since I know nothing about puppet :)18:29
* sigmavirus24 does know ruby though18:29
pleia2sigmavirus24: we're a good match then :)18:29
*** vilobhmm11 has quit IRC18:29
jeblairsdague: you want an enqueue+promote of 254911 ?18:30
sigmavirus24pleia2: awesome. Idk when I'll have time for it, but I think it would be awesome to work on18:30
pleia2sigmavirus24: taron is the one who did most of the work, so they could be of assistance too18:30
pleia2sigmavirus24: oh, it's already done :\18:30
jeblairsdague: (it did fail the resolver job, i'm assuming that's still generally broken?)18:30
sigmavirus24pleia2: oh well18:30
sigmavirus24I'm so sad I have less work to do ;)18:30
pleia2sigmavirus24:, it was taron's outreachy project over the summer18:30
pleia2sigmavirus24: are you an outreachy student?18:31
openstackgerritMerged openstack-infra/elastic-recheck: update 1524418
clarkbI am about to try logstash2.0ing logstash-worker2018:31
jeblairpleia2: oh should we update the status on that wiki page?18:31
sigmavirus24pleia2: nope. :)18:31
sigmavirus24Oh, that's for outreachy mentorship. Got it. I found that link via lbragstad18:31
pleia2jeblair: apparently :)18:31
pleia2sigmavirus24: what is lbragstad?18:32
pleia2I've updated that wiki page anyway18:32
sigmavirus24pleia2: lbragstad is Lance, Keystone Core18:33
sigmavirus24I was chatting about something like for OpenStack and he found that18:33
*** ashishb has quit IRC18:33
jeblairpleia2: that's so satisfying18:33
pleia2sigmavirus24: oh, a person :) gotcha, I won't update him18:34
lbragstadlbragstad o/18:34
anteayalbragstad: plays guitar18:34
lbragstadi'm actually a bot18:34
stevemarpleia2: i dunno, lbragstad could use an update or two18:34
jeblairlook it works ^ :)18:34
anteayalbragstad: that plays guitar18:34
pleia2I'll remove the "internship idea" category too, so it stops showing up as an idea (since it's a real thing now)18:35
lbragstadstevemar i'm at least 3 releases behind18:35
lbragstadanteaya :) not as well as ayoung plays the sax18:35
*** Swami has quit IRC18:35
anteayanevar as well as ayoung plays sax18:35
anteayabut you still do play guitar18:35
*** Swami has joined #openstack-infra18:35
clarkbinfra-root FYI, if you are going to be puppet applying system-config changes make sure you rebase them on master otherwise the puppet config gets broken18:36
mordredclarkb: ++18:37
sdaguejeblair: I'm assuming that is generally broken18:38
clarkbI may have discovered this the hard way :) to fix scroll back to your puppet conf diff and undo the changes then rebase and reapply18:39
*** e0ne has quit IRC18:41
mtreinishjeblair: heh, searching for me I found something fun:
*** burgerk has joined #openstack-infra18:41
mordredclarkb: excellent18:41
mtreinishI never realized I was so talkative18:41
mordredmtreinish: THERE ARE PEOPLE WHO TALK MORE THAN ME?????18:42
*** sridhar_ram has quit IRC18:42
anteayamtreinish: searching for me, sounds like a play I know18:42
mtreinishmordred: hah, both clarkb and openstack talk more than you :)18:43
mgagneI'm more impressed by those results:
mordredhowever - I'm impressed by the humans who talk more than openstack18:43
*** pvaneck has joined #openstack-infra18:43
*** sdake has joined #openstack-infra18:43
mordredmgagne: I'm pleased that is as smal as it is!18:44
anteayathe activity board stats are old18:44
anteayathey haven't been updated as stackalytics is the new hotness18:44
mtreinishanteaya: it says q3 201518:44
anteayaexcept I don't think stackalytics tracks irc chattiness18:44
mtreinishit doesn't18:45
clarkbok logstash-worker20 is running logstash2.018:45
clarkbneed one small change to the puppet18:45
anteayamtreinish: let's see come jan 1 then if it gives us a q418:45
mordredSergeyLukjanov: I forget, who do we send stackalytics feature requests to?18:45
mtreinishanteaya: it's such a useful metric, you should file a crtitical bug against stackalytics to include irc tracking18:45
anteayaI should?18:46
anteayaI guess I can18:46
mordrednibalizer: you still poking at things?18:46
mordrednibalizer: on the infra cloud?18:46
*** hdd has quit IRC18:47
mordrednibalizer: if you have a down period, I'd like to delete and re-create the infra project18:47
mordrednibalizer: HOWEVER18:47
mordredinfra-root: FYI - deleting a project in a cloud does not delete resources owned by that project. one must first delete the resources owned by the project18:48
jeblairmordred: fascinating!18:48
openstackgerritClark Boylan proposed openstack-infra/system-config: Change logstash multiline stream identity
openstackgerritClark Boylan proposed openstack-infra/system-config: Logstash 2.0 compat ES output rule
mordredjeblair: you know how clouds are a loose confederation of services? this is one of the offshoots - telling keystone to delete the project only deletes it from keystone - it does not tell any of the other projects to do anything18:49
mtreinishmordred: that's been a huge pain point for a long time...18:49
mordredmtreinish: it is clearly a design problem18:49
jeblairmordred: i'm guessing every cloud deployer writes the same "for $service in $services; delete project;" script?18:49
mordredjeblair: I'm guessing so - the fun part is that I am not aware of a "nova delete-project-stuff" ...18:50
jeblairis there something in the ops tools repo, i wonder?18:50
mordredsdague: does nova have such a command?18:50
mtreinishmordred, jeblair: there are things for it:
mtreinishthose are the 2 I know about18:50
sdaguemordred: no18:50
jeblairit's a whole project!18:50
mtreinishthe tempest one is for the tempest created things though18:50
sdagueit came up at summit18:50
*** flwang1 has quit IRC18:50
sdagueand our suggestion is that it becomes an osc command18:50
mtreinishI think rally was working on something similar18:50
mordredI agree18:51
mordredit should be an osc command18:51
*** flwang1 has joined #openstack-infra18:51
mordredit should not be in rally18:51
jeblairmordred: ospurge rather looks like it does what you want18:51
*** ifarkas has quit IRC18:51
clarkb and its depends on are ready for review now18:51
mordredI'm going to try it18:51
stevemarthats a snazzy project18:51
anteayaokay so stackalytics has a launchpad page for bugs: and storyboard has two stories: one to create it:!/story/2000274 and one to add ilya to the stackalytics-core group:!/story/119926618:51
sdagueyeh, we were nudging the ospurge folks to show up to osc and get it in18:51
anteayafungi: so stackalytcs bug, launchpad or storyboard?18:51
mordredas soon as nibalizer acks or nacks that that I can delete his infra project18:52
mordredI will try it18:52
stevemarsdague: i'd be happy to review it :)18:52
stevemarhad no idea it existed !18:52
sdaguestevemar: yeh, I don't remember who those folks were now18:52
sdaguethey showed up in the nova sessions and wanted it part of nova18:52
sdagueor more importantly, have nova auto delete stuff when a project is deleted18:53
*** _nadya_ has quit IRC18:53
sdaguebut people typically want to actually archive stuff for accountability later18:53
sdagueso auto doing that seems terrible18:53
clarkbkeystone rm-user --i-really-mean-it18:54
*** tsg has quit IRC18:54
mordredbut yeah18:54
mordredopenstack project delete --purge perhaps18:54
sdagueyeh, doing it in osc, and having osc go through services in a sane order is fine18:54
mordredor maybe, given how osc is structured - openstack project purge18:55
sdaguebut seriously, people fat finger this often enough that we've got an undelete window in nova for computes18:55
mordredsdague: people ... what?18:55
clarkbI know someone who did that18:55
clarkbbut I don't think we kne wabout an undelete feature18:56
clarkbjesusaurus: transition to logstash2.0 is slightly painful in that there is no forward and backward compatible config but other than that its just move log processors over ot their own thing, remove logstash stuff so that package can apply it on its own, run puppet18:57
*** gokrokve has quit IRC18:57
mordrednibalizer is ignoring me18:57
* mordred pokes nibalizer18:57
* mordred pokes nibalizer again18:57
clarkbjesusaurus: steps documented in comnments on the change18:57
* mordred hands nibalizer an annoying bunny rabbit who pokes nibalizer and then eats nibalizer's salad18:58
pleia2they also ruin your furniture18:58
mordredpleia2: or quite literally any object you place them on18:58
ayoungI don't care how well you guys play./ anteaya lbragstad I want a Jam session in Austin18:59
mordredpleia2: but they're cute and cuddly18:59
clarkboh so they are like children18:59
*** gokrokve has joined #openstack-infra18:59
anteayaayoung: I'll partcipate if lbragstad will18:59
*** gokrokve has quit IRC18:59
clarkbfungi: ^ check it out18:59
sdagueclarkb: yeh, there is a restore action18:59
*** flwang1 has quit IRC19:00
ayounganteaya, participate regardless.19:00
lbragstadayoung anteaya :)19:00
mordredclarkb: yes. they are like beakless chickens that do not lay eggs19:00
mordredclarkb: in all other respects, they are exactly like chickens19:00
*** yamamoto has joined #openstack-infra19:00
sdagueI can't find the soft_delete timeout atm, but it's in there, and I think it was set to 5m by default or something19:00
*** david-ly_ has joined #openstack-infra19:00
*** SorinSen has quit IRC19:01
wendarSince release status page is no longer updated, is there a decent Launchpad query to run for Mitaka series?19:01
wendar(Don't need it to be authoritative, just informational.)19:01
anteayaI make no promises19:01
anteayadhellmann ttx ^^19:02
*** yamamoto has quit IRC19:02
clarkbwendar: I think the idea is to use that new tool, reno19:02
*** bpokorny_ has joined #openstack-infra19:02
*** SorinSen has joined #openstack-infra19:03
*** dprince has quit IRC19:03
*** david-ly_ is now known as david-lyle_19:03
mordredwendar: you may also want to go poking in #openstack-release19:03
wendarmordred: kk, I can join there too19:03
*** david-lyle has quit IRC19:03
*** bpokorny_ has quit IRC19:03
*** fawadkhaliq has quit IRC19:04
*** bpokorny_ has joined #openstack-infra19:04
*** gokrokve has joined #openstack-infra19:04
wendarclarkb: I just need a list of blueprints for now19:04
sdagueclarkb: there it is - - I guess it's set to 0 by default, so deletes are permanent19:04
*** david-lyle_ is now known as david-lyle19:04
dhellmannwendar: part of the shift we're making this cycle is to stop using launchpad at all for tracking contents of new releases, at least as a whole. It's up to project teams to decide if they want to keep targeting blueprints & bugs.19:05
dhellmannwendar : #openstack-release may be a quieter place to get into more detail19:05
*** salv-orlando has joined #openstack-infra19:05
*** gokrokve has quit IRC19:05
*** Swami has quit IRC19:06
*** gokrokve has joined #openstack-infra19:06
*** Swami has joined #openstack-infra19:06
*** bpokorny has quit IRC19:06
wendardhellmann: aye, I know we're moving away from it, just have a short-term need for the info19:06
wendarOne note for -infra before I relocate, it's probably worth removing the "Release" link from header, since it's still pointing at Liberty info.19:07
*** doug-fish has quit IRC19:08
pleia2or updating it to point to mitaka if it still makes sense19:08
anteayawendar: ttx has a patch up for that19:08
pleia2anteaya: ah, good19:08
*** doug-fish has joined #openstack-infra19:09
nibalizermordred: hi19:09
wendaranteaya: for removal, or updating to mitaka? (I'm guessing removal, just a conversation sequence thing)19:09
*** Apoorva has quit IRC19:09
*** dkehn has quit IRC19:09
wendaranteaya: got it, makes sense19:09
* anteaya puts phone down and looks19:09
*** derekh has quit IRC19:10
nibalizermordred: yes you can delete all my stuff19:10
*** gokrokve has quit IRC19:10
clarkbnibalizer: did you determine if security groups were at fault?19:10
clarkbjust note if things get deleted you will ahve to do it again19:10
nibalizerclarkb: didn't get very far in investigating19:10
nibalizerbut i looked at the rules for the default secgroup and it looked like there weren't any19:11
nibalizerso possible19:11
sdagueinfra-root: can we get a promote of
pleia2anteaya: ah, yolanda makes a good point in there19:11
openstackgerritRyan Moats proposed openstack-infra/project-config: make gate-tempest-dsvm-neutron-multinode-full job voting
sdagueit should dramatically decrease failure rate, so stop burning so many nodes19:11
*** bpokorny_ has quit IRC19:11
*** dkehn has joined #openstack-infra19:11
anteayapleia2: oh good, a good -1 is valuable19:12
*** sridhar_ram has joined #openstack-infra19:12
*** bpokorny has joined #openstack-infra19:12
jeblairsdague, infra-root: i will promote 25491119:12
jeblairIOError: [Errno 2] No such file or directory: '/usr/lib/python2.7/dist-packages/PyYAML-3.10.egg-info'19:13
jeblairactually, i will start by figuring out what that's about ^19:13
*** doug-fish has quit IRC19:14
*** bpokorny_ has joined #openstack-infra19:14
*** gyee has joined #openstack-infra19:14
*** yukod_ has joined #openstack-infra19:14
jeblairthere are a bunch of dangling symlinks in dist-packages for zuul deps19:15
*** e0ne has joined #openstack-infra19:15
*** lykinsbd has quit IRC19:15
jeblairso, erm, i deleted them and ran pip install -U19:15
*** vilobhmm11 has joined #openstack-infra19:16
*** bpokorny has quit IRC19:17
*** atcitlionok has quit IRC19:17
jeblairsdague: done19:18
mordrednibalizer: :)19:18
mordrednibalizer: I will recreate and update your creds file when I'm done19:18
pabelangerSo, I've managed to get diskimage-builder 1.4.0 into epel7 :) For the people wanting to run centos7!19:20
pabelangerwell, it is in epel-testing ATM19:20
*** devananda has quit IRC19:22
sdaguejeblair: thanks!19:22
*** doug-fish has joined #openstack-infra19:23
openstackgerritgreghaynes proposed openstack-infra/nodepool: Builders can be run on their own
*** doug-fish has quit IRC19:23
greghaynesclarkb: ^ does the builder run separately in devstack, let me know if that is what you had in mind19:23
*** doug-fish has joined #openstack-infra19:23
*** Apoorva has joined #openstack-infra19:24
*** amuller_afk is now known as amuller19:24
*** kozhukalov_ has joined #openstack-infra19:26
*** lv_ has quit IRC19:26
clarkbgreghaynes: I think you may have to enable_service nodepool-builder in devstack/settings too19:27
openstackgerritgreghaynes proposed openstack-infra/nodepool: Builders can be run on their own
greghaynesclarkb: ^ cool19:28
* mordred stabs ospurge19:28
*** kozhukalov_ has quit IRC19:31
*** scheuran has quit IRC19:31
*** unicell1 has joined #openstack-infra19:32
*** armax_ is now known as armax19:32
openstackgerritJay_Clark proposed openstack-infra/irc-meetings: Adding the meeting_id value to the meeting yaml file.
*** unicell has quit IRC19:34
*** devananda has joined #openstack-infra19:34
*** salv-orlando has quit IRC19:36
openstackgerrityolanda.robla proposed openstack-infra/puppet-unattended_upgrades: Enable automatic cleanup
yolandaclarkb, i hardcoded the setting as we agreed ^19:37
*** rguillebert has quit IRC19:37
fungion my way home, i saw a car with the virginia license plate "ZUUL"19:38
fungiit's a sign19:38
fungi(obligatory ghostbusters quote)19:39
anteayaha ha ha19:39
anteayafungi: hope you had an enjoyable vacination19:40
fungiclarkb: i didn't know about the restore method, but i was aware that resources aren't automatically cleaned up. it's come up in security-relevant contexts before19:40
fungianteaya: yes, having needles jabbed in me is always enjoyable19:40
pleia2immunity \o/19:41
fungiat least now i've completed the hep-a series, so my mom can stop worrying about me eating "local" food when i travel19:41
*** thomasem has joined #openstack-infra19:41
pleia2welcome to a lifetime of blood results telling you you have hep-a antibodies ("yes, I got the vaccine")19:42
ianwhey infra-root, there was a good f23 build yesterday, but jobs are coming up as NOT_REGISTERED.  if someone could look in the nodepool logs to tell me where it's at, that would be great19:43
fungiianw: image successfully built but no confirmation that it was able to boot, i assume?19:44
nibalizermordred: thank you19:44
anteayathe things we do to keep Mom happy19:45
ianwfungi:  yeah / upload etc19:45
ianwSpamapS: i'm seeing failures : File "/opt/git/subunit2sql-env/lib/python2.7/site-packages/subunit2sql/", line 86, in sql2subunit from ... know about this?19:46
*** kozhukalov_ has joined #openstack-infra19:46
mordredfyi - for anyone reading along - I'm making a rather large patch to ospurge to get it to work19:46
*** briancurtin has quit IRC19:46
SpamapSianw: That error doesn't give me a lot of context.19:46
mtreinishianw: do you have the full stack trace?19:46
mordredbecause it turns out using python client libraries correctly is ... hard19:46
*** briancurtin has joined #openstack-infra19:46
mordredand the code is doing is wrong - and it doesn't work :)19:47
SpamapSmordred: ospurge + shade?19:47
mordredSpamapS: not even19:47
mordredSpamapS: I'm just replacing the incorrect calls to legacy client constructors rightnow19:47
mordredI don't want ot mess with changing semantics19:47
openstackgerritJames E. Blair proposed openstack-infra/zuul: Update the test config with a function
openstackgerritJames E. Blair proposed openstack-infra/zuul: Merge configurations with multiple layout files
fungiianw: i can confirm that nodepool image-list shows fedora-23 images less than a day old ready in all hpcloud and rax regions/networks19:48
fungibut nodepool list shows no nodes, even in building state19:49
fungidigging in the logs next19:49
*** flwang1 has joined #openstack-infra19:49
fungiianw: no, wait, one just started building now19:49
*** tqtran has joined #openstack-infra19:49
fungipresumably it will fail, and then i'll see what the logs say about it19:50
*** Sukhdev has joined #openstack-infra19:50
ianwfungi: thanks19:50
clarkbfungi: you might want to nova show on it too since nova sometimes has more info than nodepool19:50
*** bpokorny_ has quit IRC19:51
clarkb(I wonder if nova would be open to fixing that)19:51
*** gokrokve has joined #openstack-infra19:51
ianwmtreinish: from my monitoring, seems like only fedora 23 hit this at build time today19:51
mtreinishianw: looks like you found a legit bug in the write_subunit module. It was assuming there was always going to be run metadata for the runs it's pulling19:51
*** lucasagomes is now known as lucas-dinner19:51
mtreinishbut whatever one it was trying to create a stream for didn't have any19:51
*** gokrokve has quit IRC19:52
fungiclarkb: ianw: nova still lists it in status BUILD19:52
*** gokrokve has joined #openstack-infra19:52
fungiprogress 1019:52
mtreinishianw: and the metadata key isn't set unless it's present:
mtreinishianw: although if that's running against the infra subunit2sql db I'm not sure how you hit that condition19:53
mtreinishbecause everything should be populated with metadata19:53
*** gokrokve has quit IRC19:53
fungiclarkb: ianw: now nova says it's status ACTIVE19:53
ianwmtreinish: well, that's extracted from
fungiclarkb: ianw: though nodepool still lists it as building19:53
mtreinishyeah, I figured that was the case, I'm just at a loss from how that happened19:54
*** kevinbenton has quit IRC19:54
*** harlowja_at_home has quit IRC19:54
fungiclarkb: ianw: nova has ip addresses listed for the instance but nodepool does not19:54
*** baoli_ has quit IRC19:54
clarkbso that doesn't get forgotten I am going to approve shortly unless anyone else wants to review19:55
clarkbmriedem: ^ I responded to your comments19:55
mtreinishianw: my only guess is that the prepare_testrepository script was racing against an insert19:55
openstackLaunchpad bug 1524176 in PBR "H405 multi line docstring summary not separated with an empty line" [Undecided,Fix released] - Assigned to Luo Gangyi (luogangyi)19:55
mtreinishso it queried the db after the run was created but before the metadata was added19:55
lifelessmordred: how/why is easy-install happening in that gate job?19:55
mtreinishbut that should be a really small window19:55
lifelessmtreinish: races usually are :)19:55
mtreinishianw: anyway I can push a patch to make the write_subunit module more defensive. Unless you wanted to? :)19:56
mtreinishlifeless: heh, fair19:56
clarkblifeless: ^F easy no results, where are you seeing easy install?19:56
ianwmtreinish: maybe you should, and i'll review it :)19:57
*** spzala has quit IRC19:57
*** dtardivel has quit IRC19:57
*** david-lyle has quit IRC19:58
lifelessclarkb: .eggs19:58
*** ociuhandu has quit IRC19:58
lifelessclarkb: thats a gnocchi job, failing flake8 on pbr's code19:58
*** david-lyle has joined #openstack-infra19:58
lifelessclarkb: in an egg in .eggs, which is where easy-install puts its stuff for setup_requires19:58
fungiclarkb: ianw: attempts to ssh into the instance are timing out (tcpdump shows no syn+ack back from my syn to 22/tcp both for v4 and v6 addresses)19:58
*** dprince has joined #openstack-infra19:58
*** tonytan4ever has quit IRC19:59
clarkblifeless: for setup_requires does anything but easy install service that?19:59
lifelessclarkb: which means that a compatible pbr version wasn't installed in the host environment19:59
fungiclarkb: ianw: also nodepool concurs... "Exception: Timeout waiting for ssh access"19:59
fungiclarkb: ianw: network configuration issue? iptables issue?19:59
*** tjones has quit IRC19:59
clarkblifeless: we don't install pbr like that19:59
clarkblifeless: never have aiui19:59
lifelessclarkb: I thought we always did, we certainly do in devstack to avoid easy-install triggering19:59
clarkbbut the reason this changed is mordred added an explicit sdist step to pep8 to make sure we can make sdists19:59
clarkblifeless: nope we don20:00
*** bpokorny has joined #openstack-infra20:00
lifelessclarkb: but tox does sdist for us ...20:00
clarkblifeless: no it doesn't :)20:00
clarkbat least not in many/most/all projects20:00
lifelessclarkb: did we disable it?20:00
fungitox calls pip which builds (and caches) wheels for us20:00
clarkblifeless: most projects use develop equivalent20:00
clarkbfungi: I am going t oguess glean isn't working right on f23 systemd20:01
fungipretty sure our tox invocations aren't creating an sdist indirectly though20:01
ianwfungi: all very possible ... can we get serial out of it?  trying to debug no network on a system you can't see and don't have the image for is, well, kind of interesting :)20:01
lifelessclarkb: usedevelop = True? I didn't think that skipped tox's sdisting20:01
clarkbfungi: lifeless by default tox always makes an sdist then installs it then runs tests. This is apparently too slow for our users so we do the use develop thing20:01
ianwclarkb: my suspicion too...20:01
*** peristeri has joined #openstack-infra20:01
clarkblifeless: it does20:01
clarkblifeless: well that plus the install command override iirc20:01
fungiianw: someone will need time to manually attempt booting that image so that it won't get short in the head by nodepool so quickly20:01
clarkbso anyways I think the real fix here is to .gitignore .eggs20:02
lifelessclarkb: ack - covers it20:03
lifelessanyhow, that explains it.20:03
lifelesswe need to have pbr in the deps then for pep820:03
lifelessif its going to run sdist20:03
clarkblifeless: except that doesn't fix it either right?20:03
lifelessthe gnocchi pep8 doesn't have an sdist command that I can see20:03
clarkbsince setup_requires happens before any other deps20:03
clarkblifeless: right its in the job not the tox20:03
clarkblifeless: so every pep8 job runs `python sdist ; tox`20:04
lifelessclarkb: depends on how its wired in20:04
*** sridhar_ram has quit IRC20:04
lifelessclarkb: ugh ok so yeah20:04
clarkblike I said the fix here is to ignore .eggs20:04
lifelessclarkb: we don't actually want to trigger easy install at all20:04
openstackgerritKevin Carter proposed openstack-infra/project-config: Added new projects for the OSA role break out
clarkblifeless: there is no way of avoiding that?20:05
clarkblifeless: we have never done the thing you are descirbing20:05
lifelessclarkb: this should be virtualenv d; pip install -U pip wheel setuptools pbr; d/bin/python sdist; rm -r d20:05
clarkband it hasn't broken on us as far as I can tell20:05
mordredwe could make an sdist virtualenv20:05
clarkblifeless: right but we have never ever ever ever done that20:05
mordredwhat lifeless just said20:05
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Handle test_runs without metadata in write_subunit
clarkbso this isn't a regression20:05
mtreinishianw: ^^^20:05
*** mrmartin has quit IRC20:05
clarkbthe regression is gitignore .eggs and move on20:05
mordredwell, we only added the sdist thing like a weeks ago20:05
clarkbif you want to reengineer how tox et al install stuff thats a separate problem20:05
lifelessclarkb: I've been told by mordred that infra have assiduously avoided easy-install since forever20:05
*** ZZelle_ has joined #openstack-infra20:05
clarkbmordred: yes but nothing else does what lifeless wants either20:05
lifelessclarkb: but, I really don't care20:05
mordredwe only need to do this20:06
lifelessclarkb: I've reported, I've other fires to fight, if you want to argue against mordred, do so.20:06
mordredfor the sdist fail trigger check20:06
mordredwe don't need to do it for the pep8 install itself20:06
clarkbmordred: I get that20:06
clarkbmordred: my point is more general in that we don't preinstall pbr anywhere else20:06
clarkbso doing it in this one case is silly20:06
mordredand becuase we use tox elsewhere20:06
mordredthis thing isn't using tox20:06
lifelessclarkb: but we *do* in devstack20:06
clarkblifeless: devstack is unrelated20:07
*** kushal has quit IRC20:07
ianwmtreinish: L62 ... if metadata"s" ?20:07
mtreinishianw: oops, good catch :)20:08
mtreinishwhy did I add an 's' on that param? :)20:08
clarkbthis only happens to work if people declare a runtime dep on pbr20:08
clarkbbecause tox preinstalls runtime deps20:08
*** cdent has quit IRC20:08
clarkbbut it is "broken" everywhere that no runtime dep on pbr exists20:08
openstackgerritMonty Taylor proposed openstack-infra/project-config: Do the sdist check in a virtualenv
mordredno - it's broken because we added something20:09
mordredthat triggeres easy_install20:09
clarkbok yes we added athing20:09
clarkbbut back up20:09
clarkbrunning is currently broken everywhere but devstack20:09
clarkbrunning tox is broken everywhere that doesn't have pbr in *requirements.txt20:10
*** salv-orlando has joined #openstack-infra20:10
clarkbit seems odd we owuld focus on this one incidence and freak out when its been this way for eternity20:10
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Handle test_runs without metadata in write_subunit
clarkbeither we fix it properly, or it isn't actually an issue and we gitignore .eggs20:10
clarkband aiui there is no proper fix because of how setuptools works20:10
mordredclarkb: you just said two different things that do not interrelate that do not make sense to me20:11
mordredtox isn't going to be broken/fixed by gitignore20:11
clarkbmordred: yes it will be20:11
mordredtox cares about gitignore?20:11
clarkbbecause flake8 won't flake8 gitignored files20:11
clarkbso you can have whatver you want in .eggs and flake8 will not touch it20:11
clarkbthis is the actual issue20:12
clarkbnot that easy install ran20:12
lifelessso I've just been poking at things, and most of our jobs do this:20:12
mordredok. so, that's one side of the thing ... but we do not support people running blah20:12
lifelesstox -e$venv python sdist20:12
clarkblifeless: yes so that will only work in the case that pbr is an explicit runtime dep20:12 is the *exception*20:12
clarkbotherwise easy install will be used20:12
clarkbwhich is my point20:12
mordredlifeless: yes - I did not do in because I did not want to install nova's deps TWICE20:12
clarkbfixing this one case doesn't fix the actual problem20:12
lifelessmordred: remember that that only takes a few seconds20:13
lifelessmordred: because caching20:13
*** kevinbenton_ has joined #openstack-infra20:13
mordredlifeless: no, we do not have a wheel mirror yet in the gate20:13
lifelessmordred: its local20:13
clarkbmordred: what we can do is run tox to just install stuff, then run the command out of that venv then run tox -epep820:13
mordredoh - the second one20:13
*** rguillebert has joined #openstack-infra20:13
mordredok - we can do that20:13
clarkbBUT this doesn't actually fix easy install...20:13
lifelessclarkb: I agree that we should git ignore .eggs.20:14
mordredyes. we should ignore eggs. but we should ALSO not easy install in run-pep820:14
lifelessclarkb: I think we should also be doing the sdist in a venv (e.g. via tox)20:14
mordredit's two different things20:14
mordredand both should fix20:14
*** kevinbenton_ has quit IRC20:14
clarkbmordred: you need to make pbr a runtime dep for every project then20:14
*** kevinbenton has joined #openstack-infra20:14
mordredright. which is why we can also do my patch above20:14
lifelessclarkb: and I understand that that needs pbr listed either in tox.ini deps= or as a runtime dep of the project (which it is for most, and should be for all since exporting your version number is a good citizen thing to do)20:14
mordredto fix the gate-triggers-easy-install path20:14
*** ldnunes has quit IRC20:15
*** moravec has quit IRC20:15
*** kozhukalov_ has quit IRC20:15
clarkb(I am not opposed to using a venv to sdist I just want ot point out it doens't ctually fix the issue)20:15
mordredclarkb: it fixes part of the issue20:15
mordredclarkb: there are two issues20:15
mordredin two different places20:15
mordredthat overlap20:15
mordredvenv sdist in pep8 job prevents easy_install from being executed in the gate20:16
mordredthat is one thing that is worth fixing20:16
*** rfolco has joined #openstack-infra20:16
clarkbanyways I would tox --notest -epep8, .tox/pep8/bin/python sdist, then tox -epep8 in that job20:16
clarkband don't do anything specific with pbr20:16
mordredgitignore eggs fixes the problem of pep8 running on in-tree eggs20:16
clarkbmordred: no the pbr install prevents easy_install from running20:16
clarkbvirtualenvs are actually unrelated20:16
mordredclarkb: your thing does not fix the gate20:17
*** gildub has joined #openstack-infra20:17
clarkbmordred: ?20:17
*** egonzalez has joined #openstack-infra20:17
mordredclarkb: if there are projects that lack pbr in their requirements20:17
*** xyang1 has quit IRC20:17
clarkbyes but thats true of every single tox run20:17
mordredthen gate job will still trigger easy_install for those as you have pointed out20:17
armaxhi folks, how can I make changes to this team?,members?20:17
clarkbso that isn't a regression which is my point20:17
clarkband you aren't fixing that regression here anyways20:17
clarkb(if we were to treat it as one)20:17
*** xyang1 has joined #openstack-infra20:17
mordredclarkb: I'm not sure what you are arguing against20:17
clarkbmordred: I am aguing against explicitly installing pbr in one place20:18
mordredclarkb: why. that will fix the intent of the line that it is replacing20:18
clarkbbecause that doesn't actually fix anything it masks the problem20:18
mordredclarkb: it may not fix the other things that it does not attempt to fix20:18
mordredbut it WILL fix easy_install being triggered by EVERY sdist test20:18
clarkbmordred: because p27 pep8 etc etc won't install pbr first20:18
jeblairarmax:,members this group owns it and can make changes20:18
clarkbmordred: so its still broken20:18
mordredI don't care20:18
mordredas you pointed out, that's not a regression20:18
*** tonytan4ever has joined #openstack-infra20:18
mordredso I'm not trying to fix that20:19
mordredI'm trying to fix20:19
mordredthe oversight20:19
clarkbright so why treat the other thing as a regression?20:19
clarkbinstead just do the other thing with tox as I described20:19
mordredbecuase we're talking about two different things20:19
clarkbthen you are either still broken or not broken already and sdist just works20:19
mordredthere are more than one issue20:19
nibalizermordred: do my credentials work again?20:19
armaxjeblair: ok, so I should talk to any member of that group then20:19
armaxjeblair: thanks20:19
clarkbbut in both cases the fix is simple, add pbr to deps20:19
mordrednibalizer: no, ospurge sucks20:19
mordredclarkb: add pbr to deps is not simple20:19
clarkbit is in tox20:19
mordredclarkb: there are literally hundreds of problems20:20
mordredclarkb: there are literally hundreds of prjects20:20
clarkbmordred: right so why does bandaiding one inconsequential location matter?20:20
clarkbthats my point20:20
clarkbwe don't fix anything with that change20:20
mordredbecause it makes the sdist command at least as good as the other tox commands20:20
clarkbno it doesn't20:20
mordredright now it is not as good20:20
mordredit does20:20
clarkbit explicitly adds a dep that may not actually be a dep20:20
clarkbif instead you use tox as  Idescribe you avoid that20:21
mordredright. to teh sdist command20:21
clarkbbut it will be fixed for anyone with an actual pbr dep20:21
clarkbso it works for both pbr and not pbr20:21
mordredwow. omg20:21
mordredwe're intractable20:21
mordredand saying opposite things20:21
mordredand I'm not going to contine talking about this20:21
clarkbmordred: what is the issue iwht using tox as I describe?20:21
*** rfolco has quit IRC20:21
clarkbtox --notest -epep8 && .tox/pep8/bin/python sdist && tox -epep820:22
clarkbthat won't artificially inject any deps to the sdist process20:22
mordredbut I don't CARE about deps in the sdist process20:22
clarkbwhich is a problem because if we say you must do random things that are undocumented to actually make an sdist what are we testing?20:22
mordredthat's not the point of the sdist test20:23
mordreddeps or not deps in it literally do not matter AT ALL20:23
*** lykinsbd has joined #openstack-infra20:23
mordredthere are zero need for deps in to check whether or not the project can build an sdist20:23
clarkbthey do if the point is can we make an sdist20:23
mordreddeps aren't related20:23
mordredthe only reason fo rhte venv20:23
mordredis to have a place that pbr is20:23
mordredso that easy install will not install it20:23
mordred_before_ we run something else20:24
clarkband my suggestion does that for projects that want to20:24
clarkband does not for projects that dont20:24
mordredit's an artificial sequence20:24
clarkbyour change does it for everyone20:24
clarkbregardless of want/intent/need20:24
*** gokrokve has joined #openstack-infra20:24
mordredI don't care about their intent/need/want20:24
mordredthat's not what we're testing20:24
*** spzala has joined #openstack-infra20:24
mordredwe're not testing their tox environment's ability to not leave cruft in the source dir after an sdist20:24
clarkbwe are testing their ability to make an sdist20:24
mordredwe're testing being able to run sdist THEN a clean pep820:24
mordredwe are testing20:25
mordredbut we're also ACcIDENTALLY20:25
*** armax has quit IRC20:25
mordredttesting being able to run sdist THEN a clean pep820:25
mordredwhich is not a thing we care about20:25
clarkbyour change is testing our ability to install pbr and sdist20:25
mordredwhat I'm doing is disconnecting the testing of being able to run a tarball from being able to run an sdist20:25
clarkbwhich is different20:25
mordredsince that sequence does not matter20:25
mordredit's not different20:26
mordredit doesn't matter20:26
clarkbit is for any project without an explicit dep on pbr20:26
clarkbbecause it will mkae them artificially pass when they should fail20:26
clarkbsince you need pbr first but they won't do that20:26
mordredwhy should they fail?20:26
clarkbbceause they don't depend on the thing they depend on20:26
mordredthey setup_requires on them20:27
mordredwhich will make it work20:27
mordredit will just make it work in a way that will leave cruft in the source dir20:27
clarkbbut we are asserting for things to work they must also runtime dep on pbr20:27
clarkbthats what this assertion is20:27
*** gokrokve has quit IRC20:27
mordredthat's not what the assertion is20:28
*** gokrokve has joined #openstack-infra20:28
clarkbthen py27 et al should also fail20:28
mordrednot at all20:28
*** kozhukalov_ has joined #openstack-infra20:28
clarkbbut they wil leasy install20:28
clarkband you have said thats bad20:28
clarkbso? thats the whol ething you are avoiding20:28
clarkbif thats not a problem then no change needed20:28
mordredI'm avoiding an easy install that WE ADDED20:28
mordredfixing the other easy_installs is fixed by added pbr to requirements20:28
clarkbyes and I am saying the way to fix that is what I described20:29
clarkbbecause it will remove additional easy install but preserve existing easy install20:29
clarkbrather than an indisciminant removal20:29
*** rossella_s has quit IRC20:30
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add ansible-role-(diskimage-builder|jenkins-job-builder)
*** rossella_s has joined #openstack-infra20:30
clarkbif you run tox to install first without running tests, and sdist there the easy_install count will not change20:30
openstackgerritPaul Belanger proposed openstack-infra/project-config: Add ansible-role-(diskimage-builder|jenkins-job-builder)
openstackgerritDan Prince proposed openstack/requirements: Bump inspectorclient to 1.3.0 to support get_data
jeblairmordred, fungi: i updated git restack to work on more branches by default: (it tries the argument if given, or if that fails, what's in .gitreview, or if that fails, master)20:31
mordredjeblair: awesome!20:31
jeblairmordred, fungi: restack = "!f() { B=$(grep -oP 'defaultbranch=\\K.*' .gitreview); B=${1:-$B}; B=${B:-master}; git rebase -i $(git merge-base HEAD origin/$B); }; f"20:31
lifelessclarkb: why don't you want indiscriminate removal ?20:32
mordredclarkb: yes. I understand your point. but what I'm saying is that the problem you are solving is an increase in scope to what that sdist was there to check. the patch I put up cleans up an unclean thing. it can be imrpoved on, so perhaps a follow up patch that does even more awesome things would be great20:32
mordredbut instead what we've done20:32
mordredis argue for 30 miuntes20:32
clarkblifeless: because I want projects that have had the control to continue to have control20:32
clarkblifeless: I want to minimally change things20:32
clarkbrather than just change things because I can20:32
jeblairmordred, fungi: that's pretty close to the line of being too complicated for a one-liner... possibly over the line :)  should maybe consider making a dedicated script or adding it to git-review...20:33
fungijeblair: that's wicked! and also starting to rattle the alias cage door and risk breaking loose into being a separate tool20:33
fungiyeah. that20:33
mordredjeblair: I'm thinking perhaps adding to git-review might be good :)20:33
*** esker has quit IRC20:33
mordredjeblair: or making a new project called git-restack20:33
jeblairmordred, fungi: anyway, i have to run get lunch, think about it / try it out, and let's talk later20:33
*** hashar has joined #openstack-infra20:34
fungii'm on the fence about whether adding more git subcommand entrypoints to git-review is sane, but yeah that's an awesome improvement20:34
openstackgerritMerged openstack-infra/project-config: test requirements with grenade multinode
*** sridhar_ram has joined #openstack-infra20:34
openstackgerritMatthew Treinish proposed openstack-infra/subunit2sql: Handle test_runs without metadata in write_subunit
*** gokrokve has quit IRC20:43
*** derekh has joined #openstack-infra20:45
*** derekh has quit IRC20:45
*** egonzalez has quit IRC20:45
*** dims has joined #openstack-infra20:49
greghaynesianw: You make any progress on glean + f23?20:51
greghaynesianw: something that occured to me - we could make glean func tests using dib20:51
clarkbif I see another f23 node booting I will try to grab the console log from nova20:53
greghaynesI can make an image and boot it real quick20:53
*** baoli has joined #openstack-infra20:54
*** kozhukalov_ has quit IRC20:55
*** jffische has quit IRC20:55
TheJuliawin 620:55
*** krtaylor has quit IRC20:56
*** masber has joined #openstack-infra20:56
*** Apoorva has quit IRC20:57
lifelessmordred: oh btw
lifelessmordred: if it becomes an actual thing thing, I think we'll want that in oslo20:57
fungiinfra-root: yet another offer for nodepool resources... the intel/rackspace "oic" joint effort has reached out to invite us to fill out with details for what we need/could use20:57
fungii'm considering telling them we'll get back to them since we already have several new providers in the active pipeline20:58
anteayafungi: woooo20:58
*** bpokorny has quit IRC20:58
mtreinishfungi: oic?20:58
*** dims has quit IRC20:58
fungimtreinish: that thing where intel donated some thousands of machines to rackspace to use for "developing openstack"20:58
anteayawell considering the speed at which they move, I'm guessing it might be 6 months before we actually have nodes to test20:58
*** bcourt has joined #openstack-infra20:59
* fungi is almost certainly misrepresenting the relationship there20:59
*** hdd has joined #openstack-infra20:59
anteayafungi: can't hurt to ask for 600 or so and see what happens20:59
mordredfungi: I dunno - we're losing the 600 nodes from hp sooner than later20:59
mordredfungi: would be nice to request an account with 600 as anteaya says20:59
mtreinishfungi: heh, figured that from the link. but what's the acronym mean?20:59
greghaynesfungi: agree with anteaya, maybe mention that we have other stuff in the pipeline but doesnt hurt to also give them info so they can start20:59
*** Daisy has joined #openstack-infra21:00
fungiyep, so what we're really lacking is a volunteer to handle that new addition21:00
*** dims has joined #openstack-infra21:00
clarkbI can do21:00
clarkb*do it21:00
anteayaI mean I'm all for omfra cloud being the great way forward but since they are knocking on the door and all21:00
fungiclarkb: awesome! i'll cc you on the thread21:00
mtreinishfungi: they say the've got 2x1000 nodes on that link, so I say ask for 100021:00
anteayaclarkb: I can help if there are any non-root duties to be done21:01
*** dims has quit IRC21:01
openstackgerritMerged openstack-infra/project-config: add 'tempest' in puppet-openstack-integration jobs
fungimtreinish: i planned to ask for all 2000...? ;)21:01
*** yukod_ has quit IRC21:02
sdagueso, I know ianw has a -2 on - but I'm confused why, because f21 isn't working21:02
ianwgreghaynes: well the first thing i'm seeing is the systemd unit file looking for glean in /usr/local/bin21:02
fungianteaya: i expect there are some things to be done that don't require a root admin. clarkb can cut you in on what he needs21:02
clarkbsdague: tehre is no f23 yet21:02
fungiif so21:02
ianwsdague: well i just think it's more confusing to get NOT_REGISTERED21:02
anteayafungi: clarkb works for me, thanks21:02
greghaynesianw: is that not where it gets installed?21:02
*** masber has quit IRC21:02
sdaguewait, there is no f23 at all?21:03
clarkbsdague: not yet21:03
sdagueI thought it was in experimental queues21:03
ianwgreghaynes: no, pip puts it in /usr/bin21:03
clarkbwe have images but no VMs21:03
ianwsdague: it is, working on getting it up21:03
mtreinishfungi: heh, I figured leave the other half alone just so there is leftover capacity for when 1 half goes down21:03
fungisdague: he only just got the images successfully building for f23 last night, now they're not booting for some reason (looks like networking configuration, possibly glean-related?)21:03
sdagueah, ok, I didn't realize it was in a NOT_REGISTERED state21:03
fungimtreinish: touché!21:04
ianwgreghaynes: i bet debian puts it there though ...21:04
greghaynesianw: it does. that is wierd21:04
*** Daisy has quit IRC21:04
fungiyeah, debian considers /usr/bin the domain of distro packages, and /usr/local/bin for tools installed outside of distro packaging21:05
clarkbyes fedora/redhat do things the "right" way which is actually the "wrong" way21:05
ianwgreghaynes: use of "local" and pip has been a pita difference with fedora/ubuntu for ages21:05
fungiin theory so that you can back up /usr/local and just rebuild the rest of /usr from scratch21:05
*** baoli has quit IRC21:05
clarkbquotes because no one seems to agree on what right and wrong are here but there is a pep21:05
greghaynesclarkb: glad that it's clear21:05
*** baoli has joined #openstack-infra21:06
clarkband red hat packaging follows the pep, debian doesn't but typically debian works when fedora explodes21:06
clarkbso its a case of pragmatically the pep maybe isn't so good21:06
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources: Summit Application API
*** annegentle has joined #openstack-infra21:07
fungiwell, it's a case of the python community trying to tell distros one way to do things, when different distros are already pretty set in their ways and don't see eye-to-eye as it is21:07
*** timothyb89 has quit IRC21:08
*** timothyb89 has joined #openstack-infra21:08
anteayaat least with cars and highways there are only two choices, right and left21:08
fungisort of unrealistic to think that a third party would be able to convince the others that their disagreements can be solved by just not disagreeing and doing things the same way21:09
mordrednibalizer: ok. your userrc is up to date21:10
mordrednibalizer: and your user and project are re-created21:10
greghaynesianw: This is annoying to fix. We kind of dont want to just assume if you are using systemd then you must be on RH21:10
clarkbgreghaynes: ianw do unit files not have a path of some sort?21:11
*** kozhukalov_ has joined #openstack-infra21:11
greghaynesgood question21:11
clarkbcan just set path to include both then refer by name not path?21:11
*** aysyd has quit IRC21:11
ianwclarkb: refuses to start anything that doesn't have a fully qualified path name21:11
ianw"executable path is not absolute"21:12
clarkb$(which glean)21:12
fungiportability, thy name is systemd21:12
*** annegentle has quit IRC21:12
greghaynesI have a bad idea - two services with ConditionPathExists=21:12
clarkbexcept which isn't installed21:12
mordredinfra-root: on infra-cloud west I have created an infra domain and put the infra user and infra project that nibalizer created in it. if you create more projects or users, please create them in a domain21:12
mordredclarkb: I was just about to say that :)21:12
*** thorst has quit IRC21:12
clarkbmordred: :)21:12
fungii have an even badder idea: ubuntu/debian unit files separate from fedora/rhel unit files21:12
greghaynesfungi: Yea, that is the better form of my bad idea ;)21:13
*** thorst has joined #openstack-infra21:13
mordredinfra-root: we do not strictly need the domain - but I want to make sure that our stuff works with domains where we've got control before we get a cloud that requires them and we're under duress or something (it all should Just Work)21:13
* ianw is not saying anything about distro packages21:13
fungithe promise of systemd uniting the distros with "one true init config" was at best fleeting21:13
mordredcrinkle: ^^21:13
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: add reno notes to announce script output
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: format reno section of release notes more nicely
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: add URLs from links to the text output
clarkbnibalizer: now that mordred has that domained you should setup the security groups. We open them up wide open. So allow all icmp, udp, and tcp from all addresses21:14
greghaynesfungi: hopefully it can at least unite them against a common enemy21:14
openstackgerritMerged openstack-infra/puppet-graphite: Add python-tz package
fungigreghaynes: down with the system v unbelievers?21:14
mordredclarkb: oh - I was going to look for a second into configuring the default security group that everybody gets21:14
greghaynesianw: best option I have heard so far is debian/rh dirs of init files21:15
clarkbmordred: oh that works too but I thin kyou have to recreate the users to have it take effect21:15
clarkbmordred: this was part of the problem with our initial use of hpcloud 1.1, we got in before al lthe defaults were sane21:15
mordredclarkb: awesome. well, if I figure it out before nibz gets back, I can delete and re-create again21:15
mordredI have a working ospurge now21:15
mordredoh - if anybody needs ospurge - you'll want my patches that I have put up for review21:16
fungii have a feeling systemd wants to be smart and figure out whether it should mount filesystem /foo before it starts a service /foo/bar/baz21:16
fungiand so relative executable names and search paths are sort of at odds with that21:16
*** erlon has quit IRC21:16
greghaynesah, ya, makes sense21:16
*** baoli has quit IRC21:17
ianwgreghaynes: but would this actually break networking at start?  if it didn't run?  should still have address via dhcp21:17
*** baoli has joined #openstack-infra21:17
greghaynesianw: depends on where the node booted. In rax it would break networking21:18
clarkbfungi: ^ do you remember which cloud it was in?21:18
fungigreghaynes: ianw: yeah the one i was watching booted in rax-iad and was unreachable21:18
fungiclarkb: ^21:18
*** dprince has quit IRC21:19
ianwgreghaynes: it really sucks for glean to be in the "is this a redhat platform" business.  it's notoriously tricky to figure that out in a sane way (we'll be fine for our limited platforms, but writing something generically portable)21:19
fungii too tried to ssh into it as soon as it went active according to nova show. repeatedly. never got a syn+ack back21:19
*** eharney has quit IRC21:19
clarkbwhat if21:19
clarkbianw: greghaynes: the dib build itself could which glean then write the full path to the unit files21:19
fungiso it seemed like it never correctly configured eth021:19
greghaynesianw: Yep, that is kind of it's job though (it has to do that for all the networking bits). For picking the correct init scripts that is dib's job though21:19
clarkbianw: greghaynes then you don't care what distro it is you just always write out the path correct for the current chroot21:20
greghaynesclarkb: Yep, that would be trivial to do if there were multiple distro specific dirs21:20
fungithat seems like a sane enough solution21:20
*** Sukhdev has quit IRC21:20
greghaynesright now dib does the init script installing already21:20
fungibasically template the unit file21:20
*** timothyb89 has quit IRC21:20
greghaynesya, that would work too21:20
mordredclarkb: oh - wow. the default security group rules are just totally blank21:21
*** timothyb89 has joined #openstack-infra21:21
clarkbmordred: yup21:21
fungimordred: does that make it pass everything, or drop everything?21:21
clarkbmordred: so by default no traffic flows21:21
clarkbneed to open the spice trade21:21
clarkbfungi: drop everything21:21
fungithe spice must flow21:21
*** moravec has joined #openstack-infra21:21
*** thorst has quit IRC21:21
greghaynesI am a fan of just having two dirs (debian / redhat) in glean with init configs - then glean is a lot more useful on it's own without some extra processing logic which lives in dib21:22
greghaynesdib can just dtrt for copying the correct file21:22
clarkbmordred: for an ipv4 only cloud you need 3 rules, imcp/udp/tcp -1 -1
*** timothyb89 has quit IRC21:22
*** bdemers has quit IRC21:22
ianwgreghaynes: i think it should see where "" is, and template that in, rather than doing platform detection21:23
*** timothyb89 has joined #openstack-infra21:23
clarkbI think if you set the ports to -1 -1 on udp and tcp it means all ports21:23
clarkbalternative is 1 6553521:23
ianwgreghaynes: just got to run kids to school ... i'll work on that in a bit21:23
mordredclarkb: nm - openstack client doesn't show them right21:23
mordredso yeah - it only has rules for traffic from the same group21:24
clarkboh ya delete any group rules21:24
clarkbsince they apparently cause perofrmance problems21:24
greghaynesianw: ok. Also check out this old tripleo element for a laugh
greghaynessince it does the general case of what we want here21:24
mordredneutron security-group-rule-create --direction ingress --ethertype IPv4 default21:26
mordredneutron security-group-rule-create --direction ingress --ethertype IPv6 default21:26
mordredSam-I-Am: you know all the things ... where do I look to find what security groups an openstack project gets by default?21:26
Sam-I-Ammordred: howdy21:27
clarkbmordred: I think every project gets a default security group of default by default21:27
mordredclarkb: yah21:27
greghaynesyes, there is a secgroup named default21:27
mordredSam-I-Am: is there any way to configure the rules that get added to a project's default security group by default?21:27
greghaynesthere is also secgroup-list-default-rules21:27
greghaynessecgroup-add-default-rule ?21:28
mordredoh! neat21:28
greghaynesthat worked in my cloud, at least21:28
fungisounds like what we want21:29
mordredgreghaynes: oh - was that with nova ?21:29
greghaynesmordred: yes21:29
*** bdemers has joined #openstack-infra21:29
fungiand what most openstack users actually want, yet public cloud deployers seem to not realize21:29
mordredI do not see the same thing in neutron21:29
greghaynesin neutron land I think you just use the default security group21:29
Sam-I-Amby default instances will get the default secgroup21:29
Sam-I-Amwhich i think is the same for all projects by default21:29
*** baoli has quit IRC21:29
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Improve test coverage: volume attach/detach API
mordredso ...21:30
mordredthe user account I just used21:30
mordredwhich is not an admin account21:30
*** baoli has joined #openstack-infra21:30
mordredwas able to modify the security group rules for the default security group21:30
greghaynesI think that is your default security group21:30
fungithat seems odd. i wonder if that's a per-tenant default group?21:30
fungiand not a cloud global default group21:30
Sam-I-Amfungi: if you modify the default secgroup rules for your project, it only modifies them for the project21:31
greghaynesI notice a whole list of security group's named default in my cloud with different id's21:31
fungiSam-I-Am: makes sense, so it's context sensitive whether you use an admin account or a non-admin tenant-scoped account?21:31
Sam-I-Amgreghaynes: there's one per project21:31
openstackgerritsebastian marcet proposed openstack-infra/openstackid: OIDC - OpenId Connect Implementation
greghaynesSam-I-Am: makes sense21:32
mordredyes. I confirm21:32
mordredthere is a different default security group per project21:32
mordredSam-I-Am: so, that impllies to me that something is creating that default security group for the project, yeah?21:32
*** e0ne has quit IRC21:32
Sam-I-Ammordred: yeah, if you're using neutron it lives in neutron21:32
fungibut there's one global template default security group that's used to initially create the per-project default security groups (so that you can decide what they should start out with)?21:32
Sam-I-Amalthough you can modify it using nova commands21:33
fungier, what mordred said21:33
mordredSam-I-Am: ok. so I have to use nova command to modify it21:33
openstackgerritgordon chung proposed openstack/requirements: add aodh to projects.txt
mordredand I cannot modify it with neutron commands21:33
*** bryan_att has joined #openstack-infra21:33
openstackgerritMerged openstack/requirements: Block oslo.messaging 3.1.0 since it breaks Ironic
greghaynesYou should be able to use neutron, just refer to the group default21:34
mordredgreghaynes: no - that gets me the project's security group named default21:34
jesusaurusclarkb: thanks for pointing out those logstash upgrade steps in the comment21:34
greghaynesmordred: oh, you want to modify the global default?21:34
mordredI want to modify the thing that gets created per project21:35
*** armax has joined #openstack-infra21:35
jeblairi have not read all of the scrollback, but i see something about security groups...21:35
jeblairdon't specify protocol21:35
jeblairuse -1 which means all protocols21:35
*** trown|BOS is now known as trown|outttypeww21:35
jeblairhalves the number of rules21:35
mordredthat is what I did21:35
Sam-I-Ammordred: you can use nova or neutron commands21:35
jeblairthis is not exactly what clarkb said21:35
mordredI'm trying to learn how to modify the cloud's idea of what a new project's default security group looks like21:35
jeblairso just wanted to make sure that got corrected21:35
mordredjeblair: it did21:35
jeblaircool carry on, sorry, i'll get back to trying to catch up21:36
clarkbjeblair: you can wildcard icmp/tcp/udp too?21:36
mordredSam-I-Am: ok. bear with me here - because I cannot find the command in neutron21:36
mordredclarkb: yes21:36
mordredneutron security-group-rule-create --direction ingress --ethertype IPv4 default21:36
jeblairclarkb: yep, that's what we actually have in place in most of our providers21:36
Shrewsi don't think -1 works in newer versions, fwiw21:36
mordredthat is how to make a wildcard rule using neutron21:36
fungiip protocol -1? interesting21:36
mordredyou don't need it21:36
clarkbmordred: but that applies the default rules right?21:36
mordredjust omit it21:36
clarkbmordred: what does that do?21:36
jeblairwell, i'm speaking from gui experience, never used the command line21:36
Shrewsmordred: that was a pita in shade (the -1 thing)21:37
jeblairit might be -1 in the gui and omit on the command line21:37
mordredthat is how to create a rule that applies to any protocol from any source21:37
greghaynesmordred: I have a default security group that belongs to the admin tenant. I wonder if that is the global default?21:37
Sam-I-Amits -1 for the cli21:37
mordredgreghaynes: no - that is the default security group for the admin's project21:37
mordredSam-I-Am: it's not21:37
mordredit's omit in the CLI21:37
mordredyou do not need to pass -121:37
fungii wonder why it uses -1 and not 0 (which is basically documented as the "pseudo protocol number" for all ip protocol)21:37
clarkbmordred: so saying default there does not refer to the default group but rather ot default open?21:37
Shrews-1 is nova only. doesn't apply in neutron21:37
Sam-I-AmShrews: ahh thats probably why21:38
Sam-I-AmShrews: we use the nova command in a lot of the docs21:38
mordredclarkb: default on the command line is "which security group would you like to add this rule to"21:38
fungioh, maybe because v6 uses 0 for hop-by-hop option (rfc 1883)21:38
mordredeach project gets a security group named "default"21:38
clarkbmordred: right so where in that command are you saying ports -1 to -1 and source ?21:38
mordredthat security group is the security group that is used by nova by default21:38
Sam-I-Ammordred: correct21:38
mordredclarkb: by omitting those arguments, I am saying that21:38
*** Apoorva has joined #openstack-infra21:39
clarkbsomething something UX21:39
mordredI like it21:39
mordredbecause it's what we want :)21:39
*** gordc has quit IRC21:39
mordredbut sure21:39
clarkbmordred: well ya but generally with ACL type things you don't do that21:39
clarkbmordred: bceause the rule to allow everything should be explicit21:39
jeblairclarkb: doesn't iptables work like that?21:39
clarkband implicit should be bloc keverything21:39
mordredbut, let's come back to my question for a sec plz21:39
fungioh, i bet they did protocol -1 as the layer 3 wildcard to be consistent with port -1 for the layer 4 wildcard (since there is also a port 0)21:39
mordredSam-I-Am: if I use the neutron command line program21:40
greghaynesmordred: so I have no idea how to modify the default security group that is initially made... this is wierd21:40
greghaynesbut I do want to know how21:40
jeblairgreghaynes, mordred: ++21:40
clarkbjeblair: I am not sure what a iptables -Iwould do21:40
greghaynesI wonder if everyone just has some logic to make a sane default security group when they make a new tenant21:40
fungiheh... wwiptd21:40
Sam-I-Amlook at the 'security group' bits21:40
mordredSam-I-Am: and I want to do the equiv of nova secgroup-create-default-rule21:41
notmorgani should review some ospurge shouldn't i...21:41
notmorganme is just thinking21:41
Sam-I-Amin this case, nova just talks to neutron and the rules are implemented in neutron21:41
mordredSam-I-Am: what do I do21:41
greghaynesmordred: Youre sure that doesn't just modify your project's default security group too?21:41
*** baoli has quit IRC21:41
mordredgreghaynes: yes. it is an admin command that requires an aPI extension21:41
mordredERROR (Forbidden): Policy doesn't allow compute_extension:security_group_default_rules to be performed. (HTTP 403) (Request-ID: req-4f79a454-bc34-4c2f-b253-63e436fc9214)21:41
mordredthat is what I get when I try that nova command21:42
*** baoli has joined #openstack-infra21:42
greghaynessweet, so sounds like a winner21:42
clarkbjeblair: without a -j flag the manpage tells me that rule will not have an effect on the packets fate21:42
mordredI don't want to use nova21:42
mordredthis is important21:42
mordrednova does not undersatnd IPv621:42
mordredso I _cannot_ use nova to modify the default neutron IPv6 rule21:42
clarkbjeblair: so iptables noops in that case21:42
mordredit also does not understand the difference between egress and ingress rules21:42
*** alivigni has quit IRC21:42
Sam-I-Ammordred: i dont think there's an equivalent of that secgroup-add-default-rule in neutron21:43
mordredwithout using nova proxy commands I'm trying to figure out how to tell the thing how to modify the cloud's default rules, not the rules in a projects default grou21:43
mordredthat's awesome21:43
mordredSam-I-Am: thank you21:43
openstackgerritJames E. Blair proposed openstack-infra/zuul: Merge configurations with multiple layout files
Sam-I-Amprobably a good idea to verify this with the neutron folks21:44
jeblairclarkb: sure, i think i'm saying this is like "iptables -j ACCEPT" because you're leaving off "-p 80"21:44
anteayaSam-I-Am: I thought you are the neutron folks21:44
anteayamordred: I asked sc68cal about this earlier today when it first came up21:45
jeblairclarkb: so at least, for me, it makes some sense.  i can see you might make a different mental equivalency and it wouldn't make sense to you though.21:45
Sam-I-Amanteaya: yeah... but its hard to know everything21:45
mordredanteaya: neat! did sc68cal have any answers?21:45
anteayahe tried to find an opening in the conversation but was unsuccessful21:45
anteayaSam-I-Am: true21:45
anteayasc68cal: he had to go offline this aft and said he would come and find you tomorrow21:45
Sam-I-Ami need for my brain21:45
anteayaSam-I-Am: no, no we don't21:45
mordredanteaya: thank you21:45
anteayaSam-I-Am: we need Matt needs some time offline, and perhaps flying21:46
anteayamordred: welcome21:46
clarkbjeblair: to me that maps to just saying iptables -I INPUT (add a rule to this table without any content) but I guess neutron doesn't allow deny rules just allows?21:46
Sam-I-Amanteaya: its been over 2 weeks since i've flown, but i am just 20 miles shy of a 200 mile cycling week21:46
anteayano idea what that means21:46
anteayabut numbers21:46
Sam-I-Am200 miles of cycling in a week21:46
Sam-I-Amthats a lot for any week, and a record for december21:47
anteayaoh bicycle21:47
Sam-I-Amyeah. pedal thing.21:47
anteayaI walked 8k yesterday and 4k so far today21:47
*** gordc has joined #openstack-infra21:47
anteayahoping to walk another 4 to 8 tongith21:47
*** baoli has quit IRC21:47
Sam-I-Ammordred: i gotta split for a bit, but i'll get your question answered21:47
*** baoli has joined #openstack-infra21:48
mordredSam-I-Am: thanks!21:48
*** annegentle has joined #openstack-infra21:49
*** Guest32752 is now known as apuimedo21:49
clarkblooking at the neutron api doc the valid values for protocol are null, tcp, udp, icmp, so clients must be translating -1/all/whatever to none?21:50
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Don't swallow interrupts
*** flepied1 has joined #openstack-infra21:51
*** flepied has quit IRC21:51
sdaguewell, the f21 jobs are hard fail now, so if we don't get something up soon, maybe we should stop running them anyway21:51
fungiclarkb: when i cc you on this oic thread, should i use your, or address?21:52
* fungi can;t remember what you prefer21:52
clarkbfungi: not gmail, either of the other two is fine21:52
clarkbI am trying to not use gmail for work because its unreliable21:52
clarkbnot to mention all the other problems iwht it21:53
fungimakes sense. just saw you were using it for git authorship21:53
*** tdurakov_ has joined #openstack-infra21:53
tdurakov_hi folks21:53
clarkbfungi: ya I mostly haven't changed git authorship for historical reasons21:53
tdurakov_what is the password for stack user in upstream ci?21:53
clarkbtdurakov_: whatver is in the openrc21:53
*** annegentle has quit IRC21:55
*** annegentle has joined #openstack-infra21:56
tdurakov_clarkb, openrc, right!  ADMIN_PASSWORD=secretadmin - this one? Need to provide it for: sudo $ANSIBLE subnodes -u stack --sudo -f 5 -i "$WORKSPACE/inventory" -m copy -a "src=/etc/ceph/ceph.conf dest=/etc/ceph/ceph.conf"21:56
*** sridhar_ram has quit IRC21:56
*** tonytan4ever has quit IRC21:56
tdurakov_got FAILED => Missing become password21:56
openstackgerritRamy Asselin proposed openstack-infra/puppet-openstackci: Initial puppet setup should be done as root
clarkbtdurakov_: devstack-gate should not configure ceph21:57
clarkbtdurakov_: devstack needs to do that21:57
tdurakov_it's not d-g21:57
Sam-I-Amclarkb: if you do the neutron cli equivalent for a protocol without a port (icmp), neutron simply shows no port. if you use nova to read the rules, it plug -1 in when there's no port or port range21:57
tdurakov_it's my custom hook21:57
clarkbtdurakov_: ok I would recommend having a devstack plugin do it21:57
tdurakov_clarkb, is it possible to apply plugin after env deployed?21:58
clarkbSam-I-Am: and no port is match all?21:58
clarkbtdurakov_: you would enable the plugin as part of the env deployment21:58
clarkbtdurakov_: then it knows what the passwords and all the other config is21:58
Sam-I-Amclarkb: in the neutron cli, for a proto that uses ports, not specifying any ports is match all (1-65535)21:58
tdurakov_clarkb, well, it's not my case(21:58
clarkbSam-I-Am: gotcha21:58
*** hashar has quit IRC21:58
tdurakov_need to setup ceph on live env21:59
tdurakov_live= already deplloyed21:59
Sam-I-Amclarkb: the difference is the nova client equivalent command has <from-port> and <to-port> as required options21:59
clarkbtdurakov_: I guess my confusion is you asked about the upstream ci21:59
*** baoli has quit IRC21:59
clarkbtdurakov_: for upstream ci I would deploy ceph using a plugin (which I htink already exists)21:59
clarkbSam-I-Am: I see21:59
zarofungi: almost forgot that i should remind you to get me a db dump so i can take a look at cleaning up that project typo.21:59
*** baoli has joined #openstack-infra22:00
tdurakov_clarkb, it's custom job in upstream ci22:00
mordredclarkb, Sam-I-Am: I'm going to have to disagree again22:00
fungizaro: yep--i need to step out to dinner in about 15 minutes but that should be plenty of time. just a sec22:00
mordredclarkb, Sam-I-Am: is the output of mordred@camelot:~$ neutron security-group-rule-list22:00
notmorgancrinkle: let me know when richm is around and you want to revisit that convo22:00
clarkbtdurakov_: what are you trying to test?22:00
clarkbtdurakov_: that ceph works or that a specific way of deploying ceph works?22:00
clarkbmordred: thats using your command from above? I think that matches up with wat Sam-I-Am says22:01
clarkbmordred: if you don't provide values "any" equivalents are used?22:01
Sam-I-Amclarkb: with the neutron cli, the fewer options you use, the more open the rule becomes22:02
Sam-I-Ammissing options are read as "whatever"22:02
mordredclarkb: he said "neutron simply shows no port."22:02
mordredwhich is what I was arguing with22:02
mordredit shows the word "any"22:02
clarkbya it says "any" a bunch22:02
Sam-I-Ammordred: oh, sorry... my bad. the nova cli translates 'any' as 1-6553522:03
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Allow queue regex specification
clarkbtdurakov_: ya deploy ceph as part of devstack for the live migration job22:03
mordredSam-I-Am: yah. I have been focused on using osc where it has support for a thing, and the non-proxy versionof the clients where that is not possible22:03
*** edmondsw has quit IRC22:04
clarkbtdurakov_: since I htink thats already a thing22:04
Sam-I-Ami hope we can use openstackclient soon22:04
Sam-I-Amfor a while neutron was an afterthought in it22:04
tdurakov_clarkb, idea was to test several configs in one run22:04
clarkbtdurakov_: and do the same for nfs etc22:04
clarkbtdurakov_: yup22:04
clarkbtdurakov_: then you only have to change the nova config and restart it to select which one you are using22:05
clarkbtdurakov_: you don't need to do a new deploy each time22:05
Sam-I-Amaight, i really need to afk now... back in a bit22:05
mordredjeblair, greghaynes, clarkb: ok. answer from neutron channel - it is not possible to change the globel default rules that get created in each project22:05
clarkbtdurakov_: you just have to change what nova will use each time22:05
clarkbmordred: time for an RFE?22:06
mordredjeblair, greghaynes, clarkb: the reason for this is that allowing that would cause divergent user experience22:06
tdurakov_i also want to test env with ceph for glance+nova, and then add  cinder for ceph too22:06
clarkbmordred: uh22:06
tdurakov_clarkb, ^22:06
*** sridhar_ram has joined #openstack-infra22:06
mordredso, honestly, I'm inclined to be in support of that as a thing and just know that first step is running a playbook that makes rules like I want them22:06
clarkbtdurakov_: so similar thing then22:06
mordredif I can always count on that being the case, then taht playboot is simple22:06
greghaynesmordred: fun22:06
tdurakov_clarkb, yup, do you want me to set up ceph during deployment?22:07
clarkbmordred: at least for hpcloud you can't couldn't count on it22:07
clarkbtdurakov_: yes, then when you use ceph just switch the config to use it and restart services22:07
greghaynesmordred: I dont agree with that point (because everyone will and does work around it by doing what youre saying) but w/e22:07
clarkbtdurakov_: I don't think you should reinvent devstack's ceph deployment22:07
greghaynesnot worth solving that problem right now22:07
mordredI know the answer now22:07
mordredand am happy that I have knowledge that is knowledge22:07
clarkb(hpcloud neutron has rules for ssh)22:07
tdurakov_clarkb, how to separate glance+nova with ceph and cinder?22:07
*** baoli has quit IRC22:08
clarkbtdurakov_: that I do not know, I imagine you can just remove the config in cinder for ceph and restart it22:08
clarkbtdurakov_: then remove it from glance and add it to cinder and restart22:08
*** baoli has joined #openstack-infra22:08
*** dewsday has joined #openstack-infra22:08
tdurakov_clarkb, need to check that, thanks for advice:)22:09
*** Piet has joined #openstack-infra22:09
*** tdurakov_ has quit IRC22:10
clarkbgreghaynes: mordred I wonder if consistent user experience is th egoal if we shouldn't have a better default then22:11
clarkbbut maybe thats too much bikeshedding22:11
clarkb(being unable to ssh to VMs or ping VMs seems like a bad default)22:11
greghaynesclarkb: I worry that the only one everyone will agree on is no ingress22:11
fungiyeah, i think the _default_ for a router is to route my packets. firewalls are a non-default router experience, so blocking packets you would have routed is, also by extension, not a default router experience22:11
greghayneswhich is not what I want22:11
clarkb(as does using group rules which are known to be bad)22:12
*** baoli has quit IRC22:12
mordredthere's lots of people apparently doing lots of no-ingress stuff22:12
mordredclarkb: it's possible the group rules were simply broken at HP and not generally22:12
mordredbut adding 2 allow rules is _REALLY_ easy to do22:12
fungimordred: no, group rules were a terrible, terrible implementation22:12
fungithere was a security vulnerability around the ability to easily dos a provider that way22:13
greghaynesclarkb: In the devstack plugin, if I name the nodepool logger "nodepool.builder" will the logger config for qualname=nodepool which we use also pick that up?22:13
greghaynesclarkb: er, if I name the nodepool builder logger that22:13
fungireported to us by a provider, who saw it happening in the wild... one of the providers we use...22:13
mordredconsidering that "for ether in IPv4 IPv6 ; do neutron security-group-rule-create --direction ingress --ethertype $ether default ; done" fixes it22:13
clarkbgreghaynes: yes it should be hierarchical iirc22:13
fungiwe very well may have been the ones to bring the behavior to their attention22:13
openstackgerritgreghaynes proposed openstack-infra/nodepool: Add dib deletion test
openstackgerritgreghaynes proposed openstack-infra/nodepool: Trigger image delete using gearman
openstackgerritgreghaynes proposed openstack-infra/nodepool: Trigger image uploads using gearman
openstackgerritgreghaynes proposed openstack-infra/nodepool: Builders can be run on their own
openstackgerritgreghaynes proposed openstack-infra/nodepool: Decouple builders from nodepool instance
openstackgerritgreghaynes proposed openstack-infra/nodepool: Builders distinguish between failure and exception
openstackgerritgreghaynes proposed openstack-infra/nodepool: Make secure.conf optional and backwards-compat
openstackgerritgreghaynes proposed openstack-infra/nodepool: Move nodepool config loading into module
openstackgerritgreghaynes proposed openstack-infra/nodepool: Trigger image builds using gearman
openstackgerritgreghaynes proposed openstack-infra/nodepool: Decouple nodepool db and config from builders
greghaynesclarkb: ok, wanted to ask before doing ^22:14
* nibalizer hands greghaynes the patchbomb hat22:14
mordredhrm. actually, I should figure out a playbook that removes the group rules to and make sure we have them22:14
fungibut basically you could cripple a cloud by creating a security host group (or whatever it's called) and then rapidly booting and deleting instances to add and remove them from the group22:14
nibalizermordred: so am i good with creds now (or can I keep using the ones I had?)22:15
fungizaro: do you need a tarball of the git tree too?22:15
mordrednibalizer: yup. your userrc file is updated22:15
*** baoli has joined #openstack-infra22:16
greghaynesclarkb: We had a wierd failure on the last patch in the series - one of the dsvm's passed and the other didnt, we didnt have any logging for builders though so :(22:16
*** annegentle has quit IRC22:17
zarofungi: no, i think just the db dump is all i need.22:17
anteayafungi: oh I forgot, I had a question earlier, stackalytics bugs, continue on launchpad or file one on storyboard?22:17
fungizaro: cool, in that case almost done22:17
*** _Alexandra_ has joined #openstack-infra22:18
*** dewsday has quit IRC22:18
fungianteaya: stackalytics isn't an infra project is it?22:18
fungianteaya: so stackalytics bugs should get filed wherever the stackalytics team expects them22:19
*** thorst has joined #openstack-infra22:19
anteayaoh I guess we are just puppet-stackalytics22:20
anteayathought we were getting the whole thing22:20
mordrednibalizer: I re-uploaded your ubuntu image, btw22:20
fungizaro: there's a sanitized database dump in your homedir on review-dev, called reviewdb_2015-12-09.sql.gz22:21
funginow i need to run out to dinner. back later22:21
mordrednibalizer: and I have fixed the security groups in that account, and it's mapped to _member_ instead of user22:21
mordrednibalizer: so if you upload a key and then boot a vm it SHOULD work22:21
*** thorst has quit IRC22:22
*** thorst has joined #openstack-infra22:22
*** EricGonczer_ has quit IRC22:22
openstackLaunchpad bug 1524519 in Stackalytics "irc activity is not tracked" [Undecided,New]22:23
*** _Alexandra_ has quit IRC22:23
openstackgerritSachi King proposed openstack-dev/pbr: Handle markers to support sdist on pip < 6
nibalizermordred: many thabks22:23
anteayayup infra is just the puppet module22:23
*** baoli has quit IRC22:23
*** tmcpeak has joined #openstack-infra22:23
*** dkranz has quit IRC22:24
*** baoli has joined #openstack-infra22:24
tmcpeakhey guys — I've got a unit test that passes on my machine and fails in gate.  I suspect that it isn't letting me make a specific named file in /tmp, does that sound right?22:24
*** thorst has quit IRC22:24
*** thorst has joined #openstack-infra22:24
nibalizermordred: /me tests22:25
clarkbyou should use a tmpfile fixture instead22:25
tmcpeakclarkb: the problem is that the program itself uses a tmp file, so it will never run correctly in the gate22:25
clarkbyou should have perms but using files like that is racy22:26
lifelesstmcpeak: you can do temp files, but *specific names* is a problem22:26
mtreinishanteaya: cool22:26
tmcpeakproblem for the usual reasons or something special about the gate?22:26
lifelesstmcpeak: you can however export a variable to make your tempdir be a random subdir of /tmp, avoiding the race/security implications of specifically named tempfiles22:26
lifelesstmcpeak: we may have misinterpreted your meaning though - link me to the code?22:27
openstackgerritDerek Higgins proposed openstack-infra/project-config: Switch TripleO jobs to F22 jenkins nodes
anteayamtreinish: you are so easy to please :)22:27
*** lykinsbd has quit IRC22:27
tmcpeaklifeless: ok the thing I'm trying to test uses that specific named tmp file, but the reason it does it (rather than using the normal tempfile.mkstemp) is because it itself calls a program22:28
tmcpeakhere's what's failing in the gate:
tmcpeakso this is confusing but basically22:28
tmcpeakthe unit test calls the script I'm trying to test, the script calls another program22:29
*** thorst has quit IRC22:29
tmcpeakthe program needs to write (and then read) output from a specific named file, so tempfile.mkstemp isn't what I want because that actually creates a file already for me, and the thing I'm trying to run (Bandit) won't execute if the file I specify exists already22:29
mtreinishanteaya: tbh, I was kinda joking about that. :) I think stackalytics is toxic for the community so I'd rather just see it disappear then expand in scope22:30
mtreinishalthough, on the other hand maybe people will concentrate on gaming irc message stats instead of commits and reviews. So there could be a benefit to it :)22:30
*** kozhukalov_ has quit IRC22:30
clarkbtmcpeak: the test user can read and write to /tmp22:31
*** regXboi has quit IRC22:31
clarkbso no there shouldn't be anything special about the gate there22:31
tmcpeakok, so that's not the problem then22:31
tmcpeakclarkb: thank you22:31
anteayamtreinish: more channel noise, wooooo22:31
tmcpeakit passes on my local machine but doesn't in the gate22:31
anteayamtreinish: and no argument with any of your above points22:31
zarofungi: thanks, enjoy dinner22:31
anteayamtreinish: perhaps we could set up a channel and place a premium on activity in there?22:32
clarkbtmcpeak: keep in mind its running your tests in 8 partitions at the same time22:32
tmcpeakclarkb: partitions?22:33
clarkbtmcpeak: if you have >1 test that depend on this single tmpfile they may race each other. This is one very good reason to not do it this way22:33
tmcpeakahh ok, no there's only one test22:33
clarkbtmcpeak: yes, it lists all of your tests, divides them into 8 sets, hands a different set to each runner22:33
*** rhallisey has quit IRC22:34
*** david-lyle has quit IRC22:34
lifelessclarkb: looking at the code it is random22:34
*** flepied has joined #openstack-infra22:35
tmcpeakclarkb: how about - does that have any problem in the gate?22:35
*** baoli has quit IRC22:35
*** Sukhdev has joined #openstack-infra22:36
clarkbtmcpeak: it shouldn't22:36
lifelesstmcpeak: I'm just reviewing it22:36
*** baoli has joined #openstack-infra22:36
tmcpeaklifeless: cool, thank you22:36
tmcpeakI'm confused ;)22:36
clarkbtmcpeak: well thats not entirely true, if you fork bomb we shold have ulimits in place to limit that22:36
*** betherly has quit IRC22:36
clarkbbut the limit is well above one fork22:36
openstackgerritMerged openstack-infra/elastic-recheck: Allow queue regex specification
tmcpeakI do my best to only fork bomb my own systems22:37
*** larainema has quit IRC22:37
*** kgiusti has quit IRC22:37
*** flepied1 has quit IRC22:37
*** briancurtin has quit IRC22:38
*** david-ly_ has joined #openstack-infra22:39
tmcpeaklifeless: oh sweet, it's happening on my local system now :) I can debug22:39
*** patrickeast has quit IRC22:40
*** bryan_att has quit IRC22:41
openstackgerritDoug Wiegley proposed openstack-infra/project-config: Add python3 and requirements jobs to Kosmos
*** ashtokolov has quit IRC22:41
*** bryan_att has joined #openstack-infra22:41
asselin_nibalizer, when you get a chance, your review of would be appreciated22:42
*** pcrews has joined #openstack-infra22:43
*** shardy has quit IRC22:45
clarkbinfra-root: so I should be asking for ~600 8GB/8vcpu/80GB disk VMs from oic?22:45
*** piet1 has joined #openstack-infra22:45
*** david-ly_ is now known as david-lyle22:45
*** baoli has quit IRC22:47
clarkbanteaya: putting a thing toegether at
*** baoli has joined #openstack-infra22:48
*** mriedem is now known as mriedem_away22:51
nibalizerasselin_: ack22:51
jeblairclarkb: yeah, don't forget the mirror22:51
*** harlowja has quit IRC22:51
clarkbjeblair: oh right22:51
*** harlowja has joined #openstack-infra22:51
tmcpeaklifeless: thanks for the comments, checking them out now22:51
jeblairclarkb: i think is up to date22:51
clarkbgreat will go read that22:52
jeblairclarkb: aside from the 80g disk22:52
lifelesstmcpeak: you're welcome22:52
*** sdake has quit IRC22:52
jeblairclarkb: (we really only need 40, but we sort of defacto have 80 everywhere so....)22:52
jeblairclarkb: oh, also should add a link to the infra contributors page on o.o22:52
clarkbjeblair: want to make those hcanges then I can link the doc when I submit22:53
jeblairclarkb: on it22:53
tmcpeaklifeless: fwiw I did consider the security implications, specific temp files suck but at the time it didn't seem to be worth fixing.  I agree with you though, may as well just create a directory and stick the file in there22:53
*** bcourt has quit IRC22:53
*** peristeri has quit IRC22:53
*** yuanying has joined #openstack-infra22:56
*** esker has joined #openstack-infra22:56
*** esker has quit IRC22:57
*** esker has joined #openstack-infra22:58
*** boris-42_ has joined #openstack-infra22:59
*** baoli has quit IRC22:59
*** akshai_ has quit IRC22:59
*** baoli has joined #openstack-infra22:59
*** tonytan4ever has joined #openstack-infra23:00
*** burgerk has quit IRC23:00
openstackgerritJames E. Blair proposed openstack-infra/system-config: Update cloud contributors doc
jeblairclarkb: ^23:00
*** akshai has joined #openstack-infra23:01
anteayaI didn't know donors were being acknowledged on this page:
anteayapersia: ^^23:03
lifelesswhats the name of the new doc for projects23:04
clarkbjeblair: tyop on the 80GB of disk line23:04
persiaanteaya: Nice find!23:04
*** gokrokve has joined #openstack-infra23:04
anteayajeblair's patch:
anteayajeblair: I am seeing a capital G where I think you might mean 823:04
persiaAny idea how that page is administered?  I thought we also had bluebox as a donor currently.23:05
clarkbpersia: bluebox isn't in "production" yet23:05
anteayanot in production23:05
anteayawe are also testing internap, also not yet in production23:05
*** doug-fish has quit IRC23:06
*** piet1 has quit IRC23:06
clarkbok hows look?23:06
anteayagobs of storage23:06
persiaAh, so it is up to date.  Excellent :)23:06
*** amitgandhinz has quit IRC23:07
anteayaclarkb: looks good to me23:07
anteayapersia: :)23:07
jeblairclarkb, anteaya: i ran out of 8s so i substituted a G; i guess if that doesn't work, i'll pull an 8 from a different font23:08
anteayajeblair: I can see you running out of 8's in that font23:08
openstackgerritJames E. Blair proposed openstack-infra/system-config: Update cloud contributors doc
PietWhere should I go to add blueprints for the infra team?23:08
*** gokrokve has quit IRC23:08
clarkbjeblair: I have never run out of 8s using vim >_>23:08
anteayaPiet: we use specs23:09
anteayaPiet: infra-specs repo23:09
anteayaPiet: however that uses gerrit and you don't use gerrit23:09
jeblairclarkb: nicely done :)23:09
anteayaPiet: so, how about you draw up what you want to say in an etherpad23:09
*** baoli has quit IRC23:09
anteayathen I can put it into a gerrit patch and we can go from there23:09
anteayaPiet: fair?23:09
clarkbanteaya: Piet is there a problem with using gerrit?23:10
clarkbI would rather we avoid a carousel for addressing comments23:10
*** megm has quit IRC23:10
anteayaclarkb: Piet doesn't use gerrit23:10
anteayahe doesn't command line at all23:10
*** baoli_ has joined #openstack-infra23:10
clarkbyou don't need to command line to use gerrit... but its the tool we use23:11
anteayahe has two gerrit patches to his credit, I wrote one on his laptop and travisT wrote the other23:11
clarkb(pretty sure people have $IDE hooked to gerrit)23:11
anteayayes but since Piet is the ux person I made an exception23:11
nibalizerPiet: we use a spec process23:11
nibalizerso propose a spec to openstack-infra/infra-specs23:11
nibalizerthrough gerrit23:11
anteayaPiet doesn't gerrit23:12
*** megm has joined #openstack-infra23:12
anteayahe never has and asking him to will honestly probably close the door on him23:12
anteayamy opinion23:12
clarkbso I don't think we should set up a roundabout one off process for one contributor23:12
anteayaPiet: does ux23:12
anteayahe doesn't command line23:12
anteayahe is the ux ptl23:13
clarkbyou don't need to command line23:13
clarkbif there are problems with the current tool we should address them "but I don't use the command line" isn't really something we can fix23:13
anteayaPiet: this is clarkb23:13
anteayaclarkb: Piet23:13
anteayaPiet: clark would like to understand your situation better23:14
anteayaPiet: please explain if you could23:14
clarkbfor one command line alternative23:15
nibalizerclarkb: thats looks good to me23:16
openstackgerritMerged openstack-infra/jenkins-job-builder: Add flatten and encryption support to s3 publisher
*** flepied has quit IRC23:16
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Limit elastic-search graph to additional query
*** claudiub has quit IRC23:17
openstackgerritMerged openstack/diskimage-builder: Use ironic-agent for source-repositories
*** baoli_ has quit IRC23:19
*** Piet has quit IRC23:20
*** hichihara has joined #openstack-infra23:21
*** dingyichen has joined #openstack-infra23:21
anteayathe etherpad looks good to me clarkb23:21
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: format reno section of release notes more nicely
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: add URLs from links to the text output
*** gmmaha has joined #openstack-infra23:22
nibalizeranteaya: if we're really at a place where we need a transcriber then we need to figure out the problem causing that23:23
anteayapiet is a visual person23:23
anteayaI've seen him and talked to him23:23
anteayahanding him a list of instructions that would make sense to command line folks doesn't make any sense to him23:24
openstackgerritSachi King proposed openstack-dev/pbr: Handle markers to support sdist on pip < 6
anteayabrillant person in his element23:24
anteayawhich is ux23:24
clarkbanteaya: yes that is why I have offered an alternative23:24
gmmahahi, might i bother someone to help me check this log ? the error it seems to fail with indicates some file missing on the server & my patch didnt touch any of that.
clarkbyou can interact iwth gerrit without using the command line23:24
anteayaclarkb: I understand23:24
anteayaI hope Piet comes back and shares his thoughts23:25
*** baoli has joined #openstack-infra23:25
*** xyang1 has quit IRC23:25
nibalizercrinkle: mordred i've been able to boot a node and ping/ssh it !23:25
nibalizerunfortunately, it isn't taking my ssh key but thats likely my fault or cloud-inits fault or the images fault23:25
nibalizerbut booting/networking is working in infra-cloud which is sweet23:26
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Add support for extra elastic-search graph filter
anteayanibalizer: yay23:26
nibalizeranteaya: thats not true23:26
anteayawhat is not true?23:26
*** thorst has joined #openstack-infra23:27
clarkbgmmaha: I believe its supposed to cop that file but I am readin the script now23:28
gmmahaclarkb: thanks a lot23:28
*** briancurtin has joined #openstack-infra23:28
nibalizeranteaya: ?23:28
anteayanibalizer: I wrote that23:29
anteayaon his laptop23:29
anteayaread the comments23:29
clarkbgmmaha: it either copies the file for converts it, but according to your job log neither was present23:29
anteayahe is the owner of one other patch in gerrit, travisT wrote it also on piet's workstation23:29
anteayahe can however comment on a patch23:30
anteayaas you can see, he did write the comment, not the patch23:30
jeblairi can not see that he did not write the patch23:30
anteayaAnita built the patch for me23:31
clarkbgmmaha: is swift using testr yet?23:31
gmmahaclarkb: ohh.. :)23:31
gmmahaclarkb: this patch is to set it up to use ostestr23:31
anteayaI was on his computer23:31
gmmahawell its a dependent patch for that to help suppress some logging from the swift servers23:31
clarkbgmmaha: ok then that patch is probably not complete because the subunit file is not getting written properly23:31
*** pahuang has joined #openstack-infra23:31
clarkbgmmaha: there should be a .testrepository/0 or a .testrepository/0.223:31
clarkbdepending on the version of the output23:31
gmmahalet me check on my machine23:32
anteayait was the evening of an hp internal opensource event, where I told him about extra-atcs23:32
anteayait was the thursday after the qa mid-cycle23:32
gmmahaclarkb: i have a .testrepository/023:32
anteayamtreinish: remember when I kept you waiting in the cafeteria because I was helping Piet?23:32
*** MarkAtwood has joined #openstack-infra23:32
gmmahaclarkb: is there a way we can check the job's repo ? I am guessing it gets blown away right after the build23:33
mtreinishanteaya: who could forget. While I was sitting there I saw some mice run across the cafeteria23:33
anteayamtreinish: I didn't know about the mice, or did you tell me and I forgot23:33
anteayaand I met Piet's labrador retriever23:34
anteayathen mtreinish and travisT and myself went out for subs23:34
mtreinishI don't remember if I mentioned it or not23:34
clarkbgmmaha: yes they are all single use VMs23:34
*** thorst has quit IRC23:34
*** larainema has joined #openstack-infra23:34
anteayamtreinish: I don't remember mice23:34
nibalizerso Piet can't or won't gerrit going forward?23:34
anteayamtreinish: I do remember feeling bad because it took so long, and you were so patient waiting in the cafeteria, thank you23:34
nibalizerI don't understand the issue23:35
*** baoli has quit IRC23:35
gmmahaclarkb: ohh.23:35
anteayanibalizer: he doesn't know how23:35
gmmahanot sure how i can test it out23:35
clarkbgmmaha: I wonder if its actually failing to run properly and not writing those files23:35
anteayanibalizer: if you want to take the time to teach him, he isn't closed to the idea23:35
*** baoli has joined #openstack-infra23:36
clarkbgmmaha: can you rm -rf .testrepository and try run it again locally?23:36
gmmahaclarkb: okie. thats what i was trying and its happening now.23:36
anteayanibalizer: but he hasn't before and doesn't work with tools in which this workflow would give him any vocabulary to build on23:36
gmmahawill confirm in a bit what the status is23:36
anteayanibalizer: the issue is this is not how his mind works23:36
clarkbgmmaha: oh I know what the issue is I think23:36
clarkbhrm except the parent patch passed23:37
gmmahaclarkb: is that the descriptor leak. i have seen that in my old jenkkins setups23:37
anteayanibalizer: so getting him to use gerrit is more than just here are the instructions23:37
clarkbgmmaha: descriptor leak?23:37
clarkboh the file desriptor leak? no thats benign and just noise23:37
*** rcernin has joined #openstack-infra23:38
gmmahaProcess leaked file descriptors. See for more information23:38
gmmahaBuild step 'Execute shell' marked build as failure23:38
gmmahaaah ok :)23:38
anteayanibalizer: as an example I have spent time working with family members with dyslexia23:38
anteayanibalizer: when working wtih someone with dyslexia you can't just hand them things and say do this23:38
clarkbgmmaha: its just saying you made files outside the workspace which is normal if you do anything like write to /tmp23:39
anteayanibalizer: you have to listen and understand how they process information23:39
gmmahaclarkb: aah ok. thanks23:39
anteayanibalizer: then work to give them information in a way they can process it23:39
anteayanot saying this is dyslexia, saying different people's brains process information different ways23:39
gmmahaclarkb: okie. i have one failure when i run it locally on my setup23:39
*** pradk has quit IRC23:40
clarkbanteaya: right, I think we can appreciate that not everyone works the same. The issue is that "I don't use the command line so I don't use gerrit" doesn't lead to actionable positive change23:40
anteayaclarkb: what change do you want to see here/23:40
anteayabecause for me my time helping piet is doing as I offered23:40
nibalizerme too23:40
clarkbanteaya: I would like to see Piet (and everyone else) be able to use the tools we use.23:40
anteayaunderstanding what he wants to say and putting it in a patch for him23:40
anteayaclarkb: great23:40
clarkbthat does not mean we should have gerrit proxy humans23:41
anteayaclarkb: if you want to teach him I support that23:41
openstackgerritMerged openstack-infra/system-config: Update cloud contributors doc
anteayathat is my offer and I stand by it23:41
clarkbif there are deficiencies in Gerrit they need to be explained, etc23:41
anteayaif you want to take the time to do that with Piet I have no argument23:41
clarkbI am always happy to help people work thorugh problems interacting with Gerrit23:42
anteayawell he disappeared at the mention of it23:42
anteayahopefully he will return and share his thoughts23:42
*** weshay has quit IRC23:42
anteayaso we can see how he feels23:42
clarkbmordred has pointed out that bluebox is having some trouble with nodepool23:43
clarkbI am currently tailing the debug log to see if I can see wat is going on23:44
*** gordc has quit IRC23:44
gmmahaclarkb: i am getting a single test failure also on my side but it seems erratic. a test taht passed last time is now showing as failing (under .testrepository/failing)23:45
gmmahalet me dig into it more.. thanks for the help23:45
*** spzala has quit IRC23:45
*** ashtokolov has joined #openstack-infra23:47
*** sigmavirus24 is now known as sigmavirus24_awa23:48
*** betherly has joined #openstack-infra23:48
clarkbOpenStackCloudException: Could not determine capabilities (Inner Exception: No tenant specified) appears to be the blue box issue23:49
clarkbno tenant specified to swift auth?23:49
clarkbduring upload image23:50
clarkbhrm maybe that unrelated, (but also an issue_23:50
*** baoli has quit IRC23:50
clarkbah yup Exception: Timeout waiting for ssh access is the actual problem looks like23:50
clarkbjlk jeblair mordred ^ seems to be ssh timeout issues post boot with the bluebox cloud and nodepool23:51
*** patrickeast has joined #openstack-infra23:51
*** Qiming has joined #openstack-infra23:51
clarkbnow to see what the image upload fail was23:52
* jlk cries23:52
*** rbrndt has quit IRC23:54
clarkbI think the image upload was against rax-ord which makes sense since its the swift task thing23:54
*** rlandy has quit IRC23:54
clarkbso lets ignore that for now23:54
clarkbjlk: anything I cna do to help debug that more? I can get uuids and ip addrs from the log23:54
clarkb169.45.71.79 and f69cf053-606d-40d9-acb7-731a991b918423:55
*** dimtruck is now known as zz_dimtruck23:55
*** zz_dimtruck is now known as dimtruck23:56
*** Sukhdev has quit IRC23:56
jlkis this reliably happening?23:58
openstackgerritIan Wienand proposed openstack-infra/glean: Basic logging
openstackgerritIan Wienand proposed openstack-infra/glean: Handle different paths to
openstackgerritMichael Krotscheck proposed openstack/requirements: Updated gabbi to 1.11.0
clarkbjlk: 623 times since the log was rotated (haven't confirmed all are bluebox yet)23:59
ianwgreghaynes / clarkb / fungi : ^ glean path detection ... urgh i know but should work23:59
jlkclarkb: ugh, you'll probably best off getting a support ticket opened. I'm not going to be a reliable POC on this23:59
jlkMight be able to convince Jason Kennedy to open one though23:59
*** ZZelle_ has quit IRC23:59

Generated by 2.14.0 by Marius Gedminas - find it at!