Tuesday, 2015-10-20

clarkbbut to avoid breaking devstack by not installing those packages in devstack we force it to do th eowrk itself00:00
*** heyongli has joined #openstack-infra00:00
*** lawrancejing has quit IRC00:00
tonybclarkb: Thanks.  I expect that hard part will be working out what the righ package names for suse are :)00:02
*** aysyd has joined #openstack-infra00:02
openstackgerritMerged openstack-infra/puppet-grafyaml: Make config_dir a required variable  https://review.openstack.org/23710400:02
openstackgerritMerged openstack-infra/project-config: Add check-requirements job for python-scciclient  https://review.openstack.org/23421100:03
pabelangerany dogpile.cache experts around?  First time I seen this exception:  http://logs.openstack.org/34/237134/5/check/gate-grafyaml-python34/0dc42d7/console.html#_2015-10-19_23_34_26_59300:03
openstackgerritIan Wienand proposed openstack-infra/project-config: puppet: retire beaker-upgrade jobs  https://review.openstack.org/23557200:03
*** ybathia has quit IRC00:04
*** jaypipes has quit IRC00:04
*** achanda has quit IRC00:05
*** dims_ has left #openstack-infra00:05
*** vilobhmm11 has joined #openstack-infra00:05
openstackgerritIan Wienand proposed openstack-infra/project-config: puppet: retire beaker-upgrade jobs  https://review.openstack.org/23557200:05
*** ybathia has joined #openstack-infra00:05
*** ybathia has quit IRC00:06
pabelangerfungi: I think this is the last piece for puppet-grafyaml today: https://review.openstack.org/#/c/237179/00:08
vilobhmm11have some question about devstack…is it the right channel to ask for one00:08
vilobhmm11fungi : ^^00:08
nibalizervilobhmm11: best to just ask your question00:09
*** heyongli has quit IRC00:09
pabelangerfungi: once that lands, the update command in puppet will also invoke when the vcsrepo for grafyaml changes00:09
nibalizerif someone knows the answer they'll help you out, or redirect you00:09
*** heyongli has joined #openstack-infra00:10
vilobhmm11nibalizer : have 1 16GB virtual box created to setup devstack…trying to boot a small flavor vm (4G RAM) it fails with no valid host found…no valid host error  is mostly found when nova-compute is down or doesn't have enough resource i my case none is true00:11
openstackgerritMerged openstack/diskimage-builder: Add flake8 to requirements  https://review.openstack.org/22978000:11
openstackgerritgreghaynes proposed openstack-infra/nodepool: Trigger image delete using gearman  https://review.openstack.org/19811400:12
openstackgerritgreghaynes proposed openstack-infra/nodepool: Move nodepool config loading into module  https://review.openstack.org/23159200:12
openstackgerritgreghaynes proposed openstack-infra/nodepool: Make secure.conf optional and backwards-compat  https://review.openstack.org/23280000:12
*** DericHorn-HP has quit IRC00:12
*** amotoki has joined #openstack-infra00:14
vilobhmm11nibalizer : ^^00:14
greghaynesianw: you dib changed merged, so releasing now00:14
*** asettle is now known as asettle-afk00:15
*** angdraug has quit IRC00:15
jeblairrm_work: sorry i missed the convo with clarkb and mordred earlier; would be happy to join you all in chatting about this at the summit :)00:16
jrollclarkb: rm_work: mordred: just kicked off a thread internally, let's see where this goes00:16
jrolljeblair: ^ ditto for you apparently :)00:17
*** salv-orlando has quit IRC00:17
jeblairjroll: thanks! :)00:18
*** amotoki has quit IRC00:18
*** rockyg has quit IRC00:19
*** heyongli has quit IRC00:19
clarkbvilobhmm11: check the nova compute logs it should tell you what resources it thnks it has00:19
*** heyongli has joined #openstack-infra00:20
*** amit213 has quit IRC00:20
*** amit213 has joined #openstack-infra00:21
vilobhmm11clarkb : it has enough resources i can boot a tiny vm but having problem booting small flavor00:23
*** otter768 has quit IRC00:24
*** laughterwym has joined #openstack-infra00:24
*** otter768 has joined #openstack-infra00:24
clarkbvilobhmm11: right, I think the nova compute log should tell you exactly how many resources it thinks it has00:25
openstackgerritTony Breeds proposed openstack/requirements: Remove upper cap on Pillow  https://review.openstack.org/23685000:25
clarkbvilobhmm11: if virtualbox isn't claiming all of the memory upfront n-cpu may think it has far less memory than it does in reality for example00:25
vilobhmm11clarkb : I think thats happening is there a way to not let virtualbox claim all of memory upfront00:26
*** heyongli has quit IRC00:29
*** heyongli has joined #openstack-infra00:30
openstackgerritMerged openstack-infra/project-config: puppet: retire beaker-upgrade jobs  https://review.openstack.org/23557200:30
clarkbjenkins07 is finally gtting the shutdown script run again00:31
*** lawrancejing has joined #openstack-infra00:32
openstackgerritAlexey Shtokolov proposed openstack-infra/project-config: Enabled python27 and pep8 gates and fixed checks for Fuel repos  https://review.openstack.org/23606800:36
*** gyee has quit IRC00:36
*** glauco___ has quit IRC00:36
patrickeasthey, so i'm in the process of updating my nodepool configs to use the most recent stuff and am having a problem with the authentication now like so http://paste.openstack.org/show/476784/00:37
*** armax has quit IRC00:37
patrickeastany ideas what i might be missing? the error isn't super specific :(00:37
*** dims_ has joined #openstack-infra00:38
patrickeastmy old nodepool.yaml -> cloud.yaml looks like this http://paste.openstack.org/show/476785/ username and password are the same, urls match up, i think i got the format right...00:38
*** rguillebert has quit IRC00:38
*** heyongli has quit IRC00:40
asselinpatrickeast, I think it's an issue with shade that mordred fixed...not sure it was released yet00:40
patrickeastasselin: oh sweet00:40
*** Sukhdev has quit IRC00:40
*** heyongli has joined #openstack-infra00:40
*** vilobhmm11 has quit IRC00:40
*** armax has joined #openstack-infra00:41
asselinwell..maybe it's a different issue: http://lists.openstack.org/pipermail/openstack-dev/2015-September/075368.html00:41
openstackgerritIan Wienand proposed openstack-infra/project-config: Remove devstack install job for networking-ovn.  https://review.openstack.org/23711200:42
openstackgerritIan Wienand proposed openstack-infra/project-config: Add zaqar integration builder  https://review.openstack.org/23695800:42
patrickeastyea it looks like the latest shade was released very recently anyway00:42
patrickeastwhich i've got (shade==0.16.0)00:42
openstackgerritIan Wienand proposed openstack-infra/project-config: Refactoring GlusterFS jobs  https://review.openstack.org/23672700:42
openstackgerritMerged openstack/requirements: Updated oslo.db to 3.0.0  https://review.openstack.org/23401500:44
asselinpatrickeast, from my tests, using clouds.yaml is still optional. You need to use secure.conf howerver.00:45
*** asettle-afk has quit IRC00:45
*** pc_m has joined #openstack-infra00:45
patrickeastasselin: hehe yea i got burned trying to take a shortcut and not use the secure.conf stuff00:45
*** annegentle has joined #openstack-infra00:45
patrickeastasselin: ok so i guess for now i'll go back to not using the cloud.yaml00:46
patrickeastand see how that works00:46
asselinyeah...if it still fails, seems to be an issue with keystone client update or something related00:46
*** annegentle has quit IRC00:47
*** pvaneck has quit IRC00:48
openstackgerritOpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements  https://review.openstack.org/23735100:49
openstackgerritAdam Spiers proposed openstack-infra/project-config: Add openstack-resource-agents project  https://review.openstack.org/23622700:50
*** heyongli has quit IRC00:50
*** heyongli has joined #openstack-infra00:51
*** signed8bit_ZZZzz is now known as signed8bit00:52
*** Swami has quit IRC00:52
*** jjohnson2 has joined #openstack-infra00:53
*** ddieterl_ has quit IRC00:53
*** ashleighfarnham has quit IRC00:57
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: (Re)add midonet tempest jobs  https://review.openstack.org/23093200:59
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: Add experimental Neutron Fwaas api tests  https://review.openstack.org/23676200:59
*** signed8bit has quit IRC01:00
*** heyongli has quit IRC01:00
mordredpatrickeast: your clouds.yaml looks fine - I expect that to work01:01
*** heyongli has joined #openstack-infra01:01
*** xyang1 has joined #openstack-infra01:01
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: Add experimental Neutron Fwaas api tests  https://review.openstack.org/23676201:01
openstackgerritIan Wienand proposed openstack-infra/project-config: Fix oslo.messaging+amqp1+dsvm to run from master  https://review.openstack.org/23616801:01
patrickeastmordred: ok cool, thats good to know01:02
mordredpatrickeast: try this:01:03
mordredfrom shade import *01:03
*** aeng has quit IRC01:03
mordredcloud = openstack_cloud(cloud='cinder-init2')01:04
mordredprint cloud.list_flavors()01:04
mordredpatrickeast: just from python01:04
mordredin that account, so that it can see the clouds.yaml01:04
mordredthat'll test whether there is a problem with shade and libraries and stuff talking to your cloud01:04
* patrickeast testing it01:04
openstackgerritIan Wienand proposed openstack-infra/project-config: Disable tacker dsvm functional test on kilo branch  https://review.openstack.org/23558601:07
openstackgerritMartin André proposed openstack/diskimage-builder: Fix uniqueness check of initrd in fedora-minimal  https://review.openstack.org/23570501:07
openstackgerritIan Wienand proposed openstack-infra/project-config: Update gerrit javamelody plugin builds  https://review.openstack.org/23603501:08
patrickeastmordred: ok so that seems to work fine http://paste.openstack.org/show/476793/01:08
openstackgerritIan Wienand proposed openstack-infra/project-config: Make openstack-health npm lint job voting  https://review.openstack.org/23613801:09
mordredpatrickeast: yes! that's fantastic. That means it contacted keystone, got a token, found the compute url and then contacted nova01:10
openstackgerritAlex Schultz proposed openstack-infra/project-config: Add puppet gating jobs to fuel-library  https://review.openstack.org/23555001:10
mordredpatrickeast: which is GREAT01:10
openstackgerritIan Wienand proposed openstack-infra/project-config: Add a ceilometer-functional job for mysql  https://review.openstack.org/23578401:10
mordredpatrickeast: in that it means that today's shade release did not lose the ability to talk to your cloud01:10
*** heyongli has quit IRC01:10
mordredpatrickeast: otoh - why is nodepool not working :)01:10
*** FallenPegasus has joined #openstack-infra01:10
*** asettle has joined #openstack-infra01:11
mordredpatrickeast: you have 'cinder-init2' listed as cloud: now in your clouds.yaml yah?01:11
patrickeastmordred: yep01:11
*** heyongli has joined #openstack-infra01:11
patrickeastand over in nodepool a provider with "- name: cinder-init2"01:12
openstackgerritIan Wienand proposed openstack-infra/project-config: Don't run dsvm tests on rst/unit test changes  https://review.openstack.org/23455001:12
*** yamamoto has quit IRC01:12
openstackgerritIan Wienand proposed openstack-infra/project-config: Add ambari images for sahara-image-elements  https://review.openstack.org/23198301:13
mordredpatrickeast: yah. you've got "providers: - name: cinder-init2\n cloud: cinder-init2"01:14
openstackgerritIan Wienand proposed openstack-infra/project-config: Experimental fedora-22 node for puppet-apply tests  https://review.openstack.org/23167901:14
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Tag dashboards with grafyaml label  https://review.openstack.org/23713401:16
*** zhenguo has joined #openstack-infra01:16
*** DericHorn-HP has joined #openstack-infra01:18
patrickeastmordred: ok yea so with name *and* cloud set in nodepool.yaml it works better..01:18
patrickeasti was missing the 'cloud' param01:19
*** heyongli has quit IRC01:21
patrickeastmordred: sure enough everything is great now, nodes are happily building01:22
*** heyongli has joined #openstack-infra01:22
patrickeastmordred: thanks for the help!01:22
openstackgerritMerged openstack-infra/project-config: Disable tacker dsvm functional test on kilo branch  https://review.openstack.org/23558601:22
mordredpatrickeast: woot!01:29
*** bpokorny_ has joined #openstack-infra01:29
mordredpatrickeast: yah - name is the name of the provider for nodepool. cloud is the refernce to the clouds.yaml name01:29
openstackgerritDave McCowan proposed openstack-infra/project-config: Add Project Marshal  https://review.openstack.org/23453001:29
patrickeastmordred: haha yea, it makes a lot of sense... i guess i figured nodepool made the assumption that the names were the same01:30
*** heyongli has quit IRC01:31
*** heyongli has joined #openstack-infra01:32
*** ddieterly has joined #openstack-infra01:32
*** bpokorny has quit IRC01:32
*** DericHorn-HP has quit IRC01:33
*** bpokorny_ has quit IRC01:33
openstackgerritPatrick East proposed openstack-infra/project-config-example: Add some documentation for using os-client-config with nodepool.yaml  https://review.openstack.org/23736501:34
patrickeastmordred: asselin: ^ hopefully that will help avoid more folks doing what i did01:35
openstackgerritMerged openstack-infra/project-config: Don't run dsvm tests on rst/unit test changes  https://review.openstack.org/23455001:35
openstackgerritSachi King proposed openstack-infra/project-config: Add non-voting constraints jobs to nova  https://review.openstack.org/21958201:36
*** markvoelker has joined #openstack-infra01:40
openstackgerritMerged openstack/diskimage-builder: Fix devuser pubkey defaults  https://review.openstack.org/23449001:41
*** sridhar_ram has quit IRC01:41
*** dims_ has quit IRC01:41
*** heyongli has quit IRC01:41
*** sridhar_ram1 has joined #openstack-infra01:41
*** aysyd has quit IRC01:42
*** heyongli has joined #openstack-infra01:42
*** DericHorn-HP has joined #openstack-infra01:46
*** Daisy has joined #openstack-infra01:46
*** sridhar_ram1 has quit IRC01:46
openstackgerritPatrick East proposed openstack-infra/project-config-example: Add some documentation for using os-client-config with nodepool.yaml  https://review.openstack.org/23736501:46
*** tqtran has quit IRC01:46
*** Daisy has quit IRC01:46
mordredpatrickeast: awesome - thanks!!!01:47
*** rfolco_ has joined #openstack-infra01:47
openstackgerritfumihiko kakuma proposed openstack-infra/project-config: Add job for the ovs mechanism driver with native mode  https://review.openstack.org/22114301:47
*** aeng has joined #openstack-infra01:47
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Tag dashboards with grafyaml label  https://review.openstack.org/23713401:47
*** burgerk has joined #openstack-infra01:49
*** yamamoto has joined #openstack-infra01:50
*** yamahata has quit IRC01:51
*** heyongli has quit IRC01:51
*** tiswanso has joined #openstack-infra01:51
*** heyongli has joined #openstack-infra01:52
*** pc_m has quit IRC01:53
*** ivar-laz_ has joined #openstack-infra01:54
*** ivar-lazzaro has quit IRC01:58
*** ivar-laz_ has quit IRC01:58
*** laughterwym_ has joined #openstack-infra01:59
*** heyongli has quit IRC02:02
*** unicell has quit IRC02:02
*** laughterwym has quit IRC02:02
*** heyongli has joined #openstack-infra02:03
*** BharatK has joined #openstack-infra02:06
*** cody-somerville has quit IRC02:08
clarkbthe jenkins restarting completed02:09
jeblairmordred, clarkb, fungi, sdague: ovh improved their custom filters and nodepool is capable of launching instances at normal rate now.02:09
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Disable cache for cmd testcases  https://review.openstack.org/23737302:10
jeblairinfra continues to help production clouds :)02:11
clarkbso I think the ansible cron is ready for friday and shouldn't blow up too badly while we are all away02:11
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Tag dashboards with grafyaml label  https://review.openstack.org/23713402:11
*** FallenPegasus has quit IRC02:12
*** heyongli has quit IRC02:12
*** FallenPegasus has joined #openstack-infra02:12
*** otter768 has quit IRC02:13
*** heyongli has joined #openstack-infra02:13
jeblair(i saw that on a 'gravestone' haloween decoration and chuckled)02:13
*** otter768 has joined #openstack-infra02:13
*** amotoki has joined #openstack-infra02:14
*** Daisy has joined #openstack-infra02:15
*** devlaps has quit IRC02:15
mordredjeblair: I chuckle every time I drive past the town of Yolo, CA on the way to Redding02:15
jeblairmordred: it's a whole county too :)02:16
*** tiswanso_ has joined #openstack-infra02:16
openstackgerritAugustina Ragwitz proposed openstack/diskimage-builder: Sanitize distro elements for minimal vs cloud  https://review.openstack.org/21185902:18
clarkbI bet they make great tshirts just like weed, CA02:18
jeblairmordred: is that a random internet find, or do you know the photographer?02:18
mordredjeblair: I do know the photographer - I'm sitting in the same chair as he02:18
jeblairmordred: i like his work02:19
mordredjeblair: thanks!02:19
mordredhttps://www.smugmug.com/gallery/n-4DhTvc/i-wMdkg7m/A is from the same series02:19
*** tiswanso has quit IRC02:19
*** amotoki has quit IRC02:19
mordredjeblair: (I may have moved the photo collection to smugmug, which may have involved learning how rest apis _actually_ work one weekend)02:19
mordredteaser: it's not how they work in openstack02:20
*** aeng has quit IRC02:20
clarkbmordred: do they actually work anywhere?02:20
clarkbI have yet to find a good REST api02:20
jeblairmordred: those urls end up showing me the same thing02:20
mordredjeblair: awesome02:20
mordredjeblair: oh! those are for the album02:21
jeblairmordred: (the pic of 2 trees, which seems to be the start of the 11 pic series)02:21
mordredthe first time I thought I was sending you the far away yolo sign02:21
*** inteq has joined #openstack-infra02:21
mordredthe second time the close up yolo sign02:21
mordredclarkb: I actually think the smugmug one is good02:21
mordredclarkb: the docs were a little shoddy one one point when I was doing uploads02:21
mordredclarkb: BUT - they actually include api descriptions with readable docs in the actual resource responses02:22
*** heyongli has quit IRC02:22
mordredclarkb: so the link type things work - AND - you know what they're for02:22
*** ashleighfarnham has joined #openstack-infra02:22
clarkbmordred: I find that most apis anymore are "mass of undocumented parameters at undocumented path and none of them make sense"02:23
*** heyongli has joined #openstack-infra02:23
mordredthat was not the case with smugmug02:23
mordredthe problem there02:23
mordredis that I'm so used to "mass of undocumented parameters at undocumented path and none of them make sense"02:23
mordredthat I had a hard time following the clearly self-documenting api02:23
*** achanda has joined #openstack-infra02:24
mordredclarkb: that said - when I was doing it - the image payload portion had docs problems02:25
mordredclarkb: as they have noticed the glance image upload concern and addressed it02:25
mordredso the actual uploading of image content is to a differernt url (that iirc is returned from the image resource)02:25
clarkbthat the api gets slow if you shove content through it?02:25
mordredbut how you were supposed to send data to it was unclear02:25
*** armax has quit IRC02:25
mordredI sent bug reports02:25
mordredand they were like "yeah, it's a new api rev and we're working on those docs"02:26
*** jjohnson2 has quit IRC02:27
*** devlaps has joined #openstack-infra02:29
mordredmtreinish: if you're awake ... https://review.openstack.org/#/c/237148/02:29
mordredmtreinish: if you're not, that's more sensible02:29
*** FallenPegasus has quit IRC02:30
lifelessStevenK: ping02:31
*** heyongli has quit IRC02:32
*** heyongli has joined #openstack-infra02:33
mordredlook! it's lifeless!02:34
*** salv-orlando has joined #openstack-infra02:34
openstackgerritSachi King proposed openstack-infra/project-config: Add tarball-constraints jobs  https://review.openstack.org/22834602:35
lifelessmordred: o/02:36
StevenKlifeless: Now that I've had lunch, pong02:36
lifelessStevenK: hi! following up on yesterdays start02:37
lifelesssince I've only an hour left before heading into town for a meetup, and thats not enough time to get into a new patch of my own02:38
*** devlaps has quit IRC02:40
*** dims_ has joined #openstack-infra02:41
StevenKlifeless: I looked at trello, PRs and issues, and sighed about probably needing to sign up for github02:42
*** heyongli has quit IRC02:43
lifelessStevenK: I am amazed you've survived this long w/o that02:43
*** heyongli has joined #openstack-infra02:43
*** DericHorn-HP has quit IRC02:44
StevenKHandy, stevenk already exists, but has done nothing ever02:47
*** dims_ has quit IRC02:47
*** rfolco_ has quit IRC02:49
*** aeng has joined #openstack-infra02:51
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: Add a job for Neutron FWaaS scenario tests  https://review.openstack.org/23737602:53
*** lawrancejing has quit IRC02:53
*** heyongli has quit IRC02:53
*** lawrancejing has joined #openstack-infra02:54
*** heyongli has joined #openstack-infra02:54
*** achanda has quit IRC02:54
StevenKlifeless: So while I ponder signing up, is there something you want to help guide me through?02:54
*** ashleighfarnham has quit IRC02:55
*** salv-orlando has quit IRC02:55
*** zz_dimtruck is now known as dimtruck02:57
*** kozhukalov_ has quit IRC02:59
*** heyongli has quit IRC03:03
lifelessStevenK: looking, dstufft has merged some bits03:03
*** heyongli has joined #openstack-infra03:04
lifelessStevenK: I suspect cloning https://github.com/pypa/pip/pull/310503:04
lifelessStevenK: and adding the additional tests I've asked for in the review would be a good warm-up thing03:04
lifelessNakato: ^ do you concur?03:05
Nakatolifeless: Yes03:08
*** heyongli has quit IRC03:13
*** nadya has joined #openstack-infra03:14
*** heyongli has joined #openstack-infra03:14
*** burgerk has quit IRC03:17
*** julim has joined #openstack-infra03:17
*** armax has joined #openstack-infra03:21
*** salv-orlando has joined #openstack-infra03:22
*** amotoki has joined #openstack-infra03:23
*** heyongli has quit IRC03:24
*** heyongli has joined #openstack-infra03:24
lifelessStevenK: ok, -> town, will be online again in a while03:27
*** salv-orlando has quit IRC03:29
*** bpokorny has joined #openstack-infra03:30
*** thorst has quit IRC03:31
*** heyongli has quit IRC03:34
*** heyongli has joined #openstack-infra03:35
*** asselin__ has joined #openstack-infra03:36
*** asselin_ has quit IRC03:37
*** ddieterly has quit IRC03:38
*** links has joined #openstack-infra03:39
*** tlian2 has quit IRC03:41
*** dims_ has joined #openstack-infra03:43
*** heyongli has quit IRC03:44
*** heyongli has joined #openstack-infra03:45
*** Sukhdev_ has joined #openstack-infra03:47
*** vilobhmm11 has joined #openstack-infra03:47
*** laughterwym_ has quit IRC03:48
*** dims_ has quit IRC03:48
*** dougshelley66 has quit IRC03:49
*** schang has quit IRC03:50
*** dougshelley66 has joined #openstack-infra03:50
*** pahuang has quit IRC03:50
*** Sukhdev_ has quit IRC03:51
*** schang has joined #openstack-infra03:52
*** unicell has joined #openstack-infra03:52
*** links has quit IRC03:52
*** cody-somerville has joined #openstack-infra03:54
*** Ramanjaneya has joined #openstack-infra03:54
*** heyongli has quit IRC03:54
*** pahuang has joined #openstack-infra03:55
*** heyongli has joined #openstack-infra03:55
*** yfried has quit IRC03:57
*** amotoki has quit IRC03:58
*** xyang1 has quit IRC04:00
*** dimtruck is now known as zz_dimtruck04:01
*** lawrancejing has quit IRC04:01
*** Daisy has quit IRC04:02
openstackgerritfumihiko kakuma proposed openstack-infra/project-config: Add job for the ovs mechanism driver with native mode  https://review.openstack.org/22114304:02
*** lawrancejing has joined #openstack-infra04:02
*** heyongli has quit IRC04:05
*** heyongli has joined #openstack-infra04:05
*** FallenPegasus has joined #openstack-infra04:06
openstackgerritMikhail S Medvedev proposed openstack-infra/project-config: Add puppet-ciwatch module to openstack-infra  https://review.openstack.org/23169904:07
*** sridhar_ram has joined #openstack-infra04:09
SpamapSfungi: wow, I'm guessing that dump file is still growing...04:09
SpamapS| 1066062 | query | | subunit2sql | Query   | 27258 | Writing to net | SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_run_metadata` |04:09
*** amotoki has joined #openstack-infra04:11
*** cody-somerville has quit IRC04:11
*** Sukhdev_ has joined #openstack-infra04:12
*** gongysh has quit IRC04:12
*** igorbelikov has quit IRC04:13
*** heyongli has quit IRC04:15
*** BharatK has quit IRC04:15
*** links has joined #openstack-infra04:15
openstackgerritIan Wienand proposed openstack-infra/system-config: Switch Fedora 22 nodes to fedora-minimal build  https://review.openstack.org/23738904:15
*** amotoki has quit IRC04:15
ianwgreghaynes / mordred : ^ with dib release today, i think that's good to go...04:16
*** heyongli has joined #openstack-infra04:16
*** Sukhdev_ has quit IRC04:18
*** reed_ has joined #openstack-infra04:18
*** sridhar_ram has quit IRC04:18
*** sridhar_ram has joined #openstack-infra04:18
*** amotoki has joined #openstack-infra04:19
*** Sukhdev_ has joined #openstack-infra04:19
*** Sukhdev__ has joined #openstack-infra04:20
*** sridhar_ram1 has joined #openstack-infra04:23
*** Sukhdev__ has quit IRC04:23
*** Sukhdev_ has quit IRC04:23
*** Sukhdev_ has joined #openstack-infra04:24
*** sridhar_ram has quit IRC04:24
*** amotoki_ has joined #openstack-infra04:25
*** heyongli has quit IRC04:25
*** heyongli has joined #openstack-infra04:26
*** vilobhmm11 has quit IRC04:26
*** ashleighfarnham has joined #openstack-infra04:26
*** amotoki has quit IRC04:28
*** BharatK has joined #openstack-infra04:28
*** sridhar_ram1 has quit IRC04:28
*** sridhar_ram has joined #openstack-infra04:29
*** olaph has quit IRC04:30
*** mwagone_ has joined #openstack-infra04:30
*** mwagone has quit IRC04:30
*** olaph has joined #openstack-infra04:31
*** nadya has quit IRC04:32
*** denisra has quit IRC04:33
*** armax has quit IRC04:33
*** stevemar_ has quit IRC04:34
openstackgerritAngus Lees proposed openstack-infra/project-config: Add new oslo.privsep project  https://review.openstack.org/20447704:34
*** alex_xu has quit IRC04:34
*** bpokorny has quit IRC04:34
*** asettle has quit IRC04:35
*** alex_xu has joined #openstack-infra04:35
*** heyongli has quit IRC04:35
*** heyongli has joined #openstack-infra04:36
*** rlandy has quit IRC04:38
*** vilobhmm11 has joined #openstack-infra04:39
*** ddieterly has joined #openstack-infra04:39
*** amotoki_ has quit IRC04:40
*** BharatK has quit IRC04:41
*** Daisy has joined #openstack-infra04:41
*** ashleighfarnham has quit IRC04:41
*** BharatK has joined #openstack-infra04:44
*** salv-orlando has joined #openstack-infra04:44
*** ddieterly has quit IRC04:44
*** dims_ has joined #openstack-infra04:45
*** heyongli has quit IRC04:46
*** yfried has joined #openstack-infra04:46
*** amotoki has joined #openstack-infra04:46
*** heyongli has joined #openstack-infra04:46
*** Ramanjaneya has quit IRC04:48
*** Ramanjaneya has joined #openstack-infra04:49
*** dims_ has quit IRC04:50
*** tiswanso_ has quit IRC04:50
*** mohankumar has joined #openstack-infra04:52
*** oomichi has joined #openstack-infra04:52
*** heyongli has quit IRC04:56
*** salv-orlando has quit IRC04:56
*** heyongli has joined #openstack-infra04:57
*** e0ne has joined #openstack-infra05:01
*** stevemar_ has joined #openstack-infra05:02
*** sergmelikyan has joined #openstack-infra05:03
*** achanda has joined #openstack-infra05:04
*** heyongli has quit IRC05:06
*** heyongli has joined #openstack-infra05:07
StevenKlifeless: I have a testcase added that runs 'pip install bar bar[simple] bar[singlemodule]' and it fails with Double requirement given: bar[simple] (already in bar, name='bar')05:08
*** boris-42 has joined #openstack-infra05:09
*** spredzy_ has joined #openstack-infra05:12
*** yfried has quit IRC05:14
*** heyongli has quit IRC05:16
*** heyongli has joined #openstack-infra05:17
openstackgerritMerged openstack-infra/project-config: Make openstack-health npm lint job voting  https://review.openstack.org/23613805:17
*** mohankumar has quit IRC05:18
*** aukhan has joined #openstack-infra05:18
*** mohankumar has joined #openstack-infra05:19
openstackgerritMerged openstack-infra/project-config: Fix oslo.messaging+amqp1+dsvm to run from master  https://review.openstack.org/23616805:20
lifelessStevenK: \o/ means its not fixed :)05:23
*** e0ne has quit IRC05:23
*** salv-orlando has joined #openstack-infra05:23
lifelessStevenK: or rather the narrow special case we'd initially identified was fixed, but not the broader set of (still simply-addressable) cases05:24
lifelessStevenK: btw, #pypa-dev is the channel we discuss upstream packaging work on05:24
*** abregman has joined #openstack-infra05:25
*** reed_ has quit IRC05:25
*** reed_ has joined #openstack-infra05:26
*** e0ne has joined #openstack-infra05:26
*** heyongli has quit IRC05:27
*** heyongli has joined #openstack-infra05:27
*** stevemar_ has quit IRC05:28
*** stevemar_ has joined #openstack-infra05:29
*** amotoki_ has joined #openstack-infra05:30
*** salv-orl_ has joined #openstack-infra05:31
*** skylerberg has quit IRC05:33
*** salv-orlando has quit IRC05:34
*** skylerberg has joined #openstack-infra05:34
*** reed_ has quit IRC05:36
*** heyongli has quit IRC05:37
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Add a repository for I18n team  https://review.openstack.org/23528005:37
*** heyongli has joined #openstack-infra05:38
*** jamielennox is now known as jamielennox|away05:39
*** ddieterly has joined #openstack-infra05:40
*** edmondsw has quit IRC05:41
*** cody-somerville has joined #openstack-infra05:44
*** ddieterly has quit IRC05:45
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: (Re)add midonet tempest jobs  https://review.openstack.org/23093205:46
*** dims_ has joined #openstack-infra05:47
*** heyongli has quit IRC05:47
*** heyongli has joined #openstack-infra05:48
*** stevemar_ has quit IRC05:51
*** dims_ has quit IRC05:52
*** amotoki_ has quit IRC05:54
*** aeng has quit IRC05:55
*** heyongli has quit IRC05:57
*** heyongli has joined #openstack-infra05:58
*** amotoki_ has joined #openstack-infra05:59
*** amotoki_ has quit IRC06:00
*** adreznec has quit IRC06:00
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/23740906:02
*** adreznec has joined #openstack-infra06:03
*** e0ne has quit IRC06:05
*** heyongli has quit IRC06:08
*** heyongli has joined #openstack-infra06:08
*** asettle has joined #openstack-infra06:09
*** sergmelikyan has quit IRC06:09
*** sergmelikyan has joined #openstack-infra06:10
*** manas has joined #openstack-infra06:10
*** yfried has joined #openstack-infra06:12
*** unicell1 has joined #openstack-infra06:12
*** achanda has quit IRC06:12
*** shardy_a1k has joined #openstack-infra06:13
*** unicell has quit IRC06:13
*** shardy_afk has quit IRC06:14
*** stevemar_ has joined #openstack-infra06:14
*** ParsectiX has joined #openstack-infra06:15
*** yfried_ has joined #openstack-infra06:16
*** yfried has quit IRC06:16
*** sergmelikyan has quit IRC06:16
*** sergmelikyan has joined #openstack-infra06:17
*** sergmelikyan has quit IRC06:17
*** heyongli has quit IRC06:18
*** shardy_a1k has quit IRC06:18
*** shardy_afk has joined #openstack-infra06:19
*** heyongli has joined #openstack-infra06:19
openstackgerritOpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints  https://review.openstack.org/23742006:23
*** markvoelker_ has joined #openstack-infra06:26
*** heyongli has quit IRC06:28
*** markvoelker has quit IRC06:29
*** heyongli has joined #openstack-infra06:29
*** e0ne has joined #openstack-infra06:30
*** aukhan has quit IRC06:30
*** ildikov has quit IRC06:31
*** shashank_hegde has quit IRC06:32
*** aukhan has joined #openstack-infra06:33
*** asmigala has joined #openstack-infra06:35
*** yfried_ is now known as yfried|afk06:35
*** sergmelikyan has joined #openstack-infra06:37
*** yfried|afk is now known as yfried_06:37
*** heyongli has quit IRC06:38
*** heyongli has joined #openstack-infra06:39
*** amotoki_ has joined #openstack-infra06:40
*** ddieterly has joined #openstack-infra06:41
*** sergmelikyan has quit IRC06:42
*** fifieldt has joined #openstack-infra06:43
*** stevemar_ has quit IRC06:44
*** ddieterly has quit IRC06:45
*** kragniz has quit IRC06:46
*** dims_ has joined #openstack-infra06:48
*** heyongli has quit IRC06:49
*** heyongli has joined #openstack-infra06:49
*** peristeri has joined #openstack-infra06:50
*** AJaeger has joined #openstack-infra06:51
*** flepied1 has joined #openstack-infra06:51
AJaegeryolanda, ianw: Here're two new repos to create - could you review these, please? https://review.openstack.org/#/c/235280/ https://review.openstack.org/#/c/236227/06:52
*** dims_ has quit IRC06:53
*** flepied has quit IRC06:54
*** flepied1 has quit IRC06:56
*** ParsectiX has quit IRC06:56
*** heyongli has quit IRC06:59
*** k4n0 has joined #openstack-infra07:00
*** heyongli has joined #openstack-infra07:00
*** amotoki_ has quit IRC07:01
*** nmagnezi has joined #openstack-infra07:02
*** e0ne has quit IRC07:05
*** ifarkas has joined #openstack-infra07:06
*** jcoufal has joined #openstack-infra07:07
*** skylerberg has quit IRC07:07
*** e0ne has joined #openstack-infra07:09
*** heyongli has quit IRC07:09
*** woodster_ has quit IRC07:09
*** heyongli has joined #openstack-infra07:10
*** e0ne has quit IRC07:11
*** nadya has joined #openstack-infra07:12
*** arxcruz has joined #openstack-infra07:12
openstackgerritMerged openstack-infra/project-config: Normalize projects.yaml  https://review.openstack.org/23740907:12
*** asettle has quit IRC07:13
*** scheuran has joined #openstack-infra07:14
*** ildikov has joined #openstack-infra07:16
*** e0ne has joined #openstack-infra07:16
*** markus_z has joined #openstack-infra07:16
*** HeOS has quit IRC07:17
*** camunoz is now known as camunoz_gone07:18
*** e0ne has quit IRC07:19
*** asettle has joined #openstack-infra07:19
*** heyongli has quit IRC07:19
*** asettle has quit IRC07:20
*** heyongli has joined #openstack-infra07:20
*** dizquierdo has joined #openstack-infra07:25
*** shardy_afk is now known as shardy07:27
openstackgerritJean-Emile DARTOIS proposed openstack-infra/project-config: Add watcher-specs  https://review.openstack.org/23367907:27
*** flepied has joined #openstack-infra07:28
*** flepied1 has joined #openstack-infra07:29
*** mpavone has joined #openstack-infra07:29
*** heyongli has quit IRC07:30
*** heyongli has joined #openstack-infra07:30
yolandamorning AJaeger07:31
AJaegermorning, yolanda ! Thanks07:31
*** flepied has quit IRC07:32
StevenKAJaeger: Maybe we need Zanata to throw errors when translators attempt to save a string with invalid Unicode :-(07:34
StevenKI can't see a project setting WRT that07:34
AJaegerStevenK: Feature request?07:34
StevenKAJaeger: I'm hoping aeng will pop up on the -i18n thread07:35
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Revert "Temporarily remove zuul-cloner before tarball"  https://review.openstack.org/23744207:36
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Revert "Temporarily remove zuul-cloner before tarball"  https://review.openstack.org/20720107:38
*** vilobhmm11 has quit IRC07:39
*** heyongli has quit IRC07:40
AJaegerStevenK: If not, please ask aeng directly07:40
yolandaok approved07:40
*** Sukhdev_ has quit IRC07:40
*** heyongli has joined #openstack-infra07:41
*** ddieterly has joined #openstack-infra07:42
*** Ala has joined #openstack-infra07:42
*** ParsectiX has joined #openstack-infra07:43
*** asettle has joined #openstack-infra07:44
openstackgerritMerged openstack-infra/project-config: Add a repository for I18n team  https://review.openstack.org/23528007:44
*** ParsectiX has quit IRC07:44
openstackgerritMerged openstack-infra/project-config: Add openstack-resource-agents project  https://review.openstack.org/23622707:45
*** asettle has quit IRC07:45
*** jlanoux has joined #openstack-infra07:46
*** openstackgerrit has quit IRC07:46
*** asettle has joined #openstack-infra07:46
*** ddieterly has quit IRC07:46
*** openstackgerrit has joined #openstack-infra07:46
*** dtantsur|afk is now known as dtantsur07:47
*** asettle has quit IRC07:48
*** jordanP has joined #openstack-infra07:48
*** asettle has joined #openstack-infra07:48
openstackgerritStéphane Albert proposed openstack-infra/project-config: Updated CloudKitty information due to big tent integration  https://review.openstack.org/23708407:49
*** dims_ has joined #openstack-infra07:49
*** asettle has quit IRC07:49
*** heyongli has quit IRC07:50
*** asettle has joined #openstack-infra07:50
*** sshnaidm has joined #openstack-infra07:50
*** heyongli has joined #openstack-infra07:51
*** asettle has quit IRC07:51
*** yfried_ is now known as yfried|afk07:51
DaisyHello. The patch "Add a repository for I18n team" is merged. Who could help to give me the right to edit ACL of i18n repo ?07:52
*** bexelbie has joined #openstack-infra07:52
openstackgerritMerged openstack-infra/project-config: open stable/liberty for ceilometer  https://review.openstack.org/23709807:52
*** asettle has joined #openstack-infra07:52
*** asettle has quit IRC07:53
AJaegeryolanda: https://review.openstack.org/#/admin/groups/1132,members07:53
yolandashall i add Daisy?07:53
openstackgerritMerged openstack-infra/project-config: Remove devstack install job for networking-ovn.  https://review.openstack.org/23711207:53
*** yfried|afk is now known as yfried_07:54
AJaegeryolanda: yes, please07:54
yolandaAJaeger, daisy-ycguo right?07:54
DaisyIt's me.07:54
yolandahi :)07:54
DaisyI see. Thank you, yolanda07:55
AJaegerDaisy: you're setup now ;)07:55
*** dims_ has quit IRC07:55
*** salv-orl_ has quit IRC07:56
DaisyThank you, AJaeger.07:56
openstackgerritDmitry Tantsur proposed openstack-infra/project-config: Add a non-voting Ironic job to Neutron  https://review.openstack.org/23696407:58
*** heyongli has quit IRC08:00
*** fhubik has joined #openstack-infra08:01
*** salv-orlando has joined #openstack-infra08:01
*** heyongli has joined #openstack-infra08:01
*** nmagnezi has quit IRC08:02
*** ParsectiX has joined #openstack-infra08:03
*** salv-orlando has quit IRC08:05
*** salv-orlando has joined #openstack-infra08:05
*** sshnaidm has quit IRC08:05
openstackgerritMerged openstack-infra/project-config: Refactoring GlusterFS jobs  https://review.openstack.org/23672708:06
*** jhesketh has quit IRC08:07
*** luqas has quit IRC08:09
*** jhesketh has joined #openstack-infra08:09
openstackgerritMarian Horban proposed openstack-infra/project-config: [WIP]Make job tempest-dsvm-nova-wsgi-full voteable  https://review.openstack.org/23696908:09
*** luqas has joined #openstack-infra08:09
*** heyongli has quit IRC08:11
*** fhubik is now known as fhubik_brb08:11
*** heyongli has joined #openstack-infra08:11
*** kiran-r has joined #openstack-infra08:13
*** hashar has joined #openstack-infra08:15
*** luqas has quit IRC08:16
openstackgerritMerged openstack-infra/project-config: Add ambari images for sahara-image-elements  https://review.openstack.org/23198308:16
openstackgerritMerged openstack-infra/project-config: Experimental fedora-22 node for puppet-apply tests  https://review.openstack.org/23167908:17
openstackgerritMerged openstack-infra/project-config: Add zaqar integration builder  https://review.openstack.org/23695808:17
*** rossella_s has joined #openstack-infra08:17
*** kozhukalov_ has joined #openstack-infra08:18
*** aeng has joined #openstack-infra08:18
*** boris-42 has quit IRC08:18
*** rguillebert has joined #openstack-infra08:19
*** luqas has joined #openstack-infra08:20
*** heyongli has quit IRC08:21
*** derekh has joined #openstack-infra08:21
*** aukhan has quit IRC08:21
*** cylopez has joined #openstack-infra08:21
*** heyongli has joined #openstack-infra08:22
*** BobBall_AWOL is now known as BobBall08:23
cylopezI tried to sign Contributor Agreement in gerrit without success... I got "Cannot store contact information". Someone could help me please ?08:23
openstackgerritMerged openstack-infra/puppet-grafyaml: Run grafana_dashboard_update when install_grafyaml changes  https://review.openstack.org/23717908:24
*** jistr has joined #openstack-infra08:27
*** fhubik_brb is now known as fhubik08:30
*** heyongli has quit IRC08:31
*** heyongli has joined #openstack-infra08:32
openstackgerritMerged openstack-infra/puppet-openstackci: Add pass through for project-config revision  https://review.openstack.org/19854608:32
*** _skolekonov is now known as skolekonov08:33
openstackgerritMerged openstack-infra/puppet-phabricator: Update SQL Migration Script  https://review.openstack.org/23580208:33
openstackgerritMerged openstack-infra/puppet-httpd: Fix proxy and redirect configuration file names.  https://review.openstack.org/22533108:36
*** aukhan has joined #openstack-infra08:37
*** cylopez has left #openstack-infra08:37
openstackgerritMerged openstack-infra/puppet-httpd: Grant access for vhosts on Apache >= 2.4  https://review.openstack.org/22860408:39
*** jed56 has joined #openstack-infra08:41
*** kragniz_ has joined #openstack-infra08:41
*** alexpilotti has joined #openstack-infra08:41
*** kragniz_ is now known as kragniz08:41
*** heyongli has quit IRC08:41
openstackgerritStéphane Albert proposed openstack-infra/project-config: Updated CloudKitty information due to big tent integration  https://review.openstack.org/23708408:42
*** ddieterly has joined #openstack-infra08:42
*** heyongli has joined #openstack-infra08:42
jed56yolanda :  I have one question for you regarding python jobs for watcher-specs08:43
jed56I was not sure. But I saw the same thing in several projects . For example in ceilometer-specs. We want to have tests.08:43
yolandai want to know... if that's an spec, why do you need to run python jobs?08:43
jed56it's seems that we can have tests08:44
jed56for example : https://github.com/openstack/ceilometer-specs/tree/master/tests08:44
jed56But maybe is not a good practice08:44
yolandalet me check ,if that's a general rule i am ok08:45
yolandabut it raised my attention08:45
jed56thanks a lot08:46
*** luqas has quit IRC08:46
*** ddieterly has quit IRC08:47
yolandajed56, i see some projects with python tests, some others without. If your plan is to have python code in the specs, and tests it is good to add them. But if you just plan to add some text specs, these jobs are not needed, and better skip them08:47
*** sergmelikyan has joined #openstack-infra08:47
jed56yolanda : yes we want to add tests like in the ceilometer project08:48
yolandawhat was the link for your change?08:48
jed56One more question : I'm not familiar with gerrit i tried to response to your comment did you see it ?08:49
yolandalast comment in the review it's mine08:50
yolandawhat did you try?08:50
*** dingyichen has quit IRC08:50
jed56my comment is marked as a draft08:51
*** dims_ has joined #openstack-infra08:51
*** amotoki has quit IRC08:51
jed56I can edit it08:51
lennybyolanda: Hi, could you review the following commit #link https://review.openstack.org/#/c/206513/ . It's a small documentation issue and missing one core only.08:51
yolandaah, then you have to click on review, and publish comments08:51
*** HeOS has joined #openstack-infra08:52
*** heyongli has quit IRC08:52
*** sergmelikyan has quit IRC08:52
*** FallenPegasus has quit IRC08:52
*** heyongli has joined #openstack-infra08:52
jed56yolanda : thanks i got it !08:53
openstackgerritDimitri Mazmanov proposed openstack-infra/project-config: Add Kingbird to the project list  https://review.openstack.org/23525308:54
yolandalennyb, .looking08:54
silehthum is http://logs.openstack.org/ dead ?08:56
*** sergmelikyan has joined #openstack-infra08:56
*** lawranc__ has joined #openstack-infra08:56
yolandasileht, wow yes08:56
silehtyolanda, thx08:57
*** dims_ has quit IRC08:57
silehtyolanda, the serves look alive, but not the webserver08:57
yolandaok, there is a bug in some apache config, checking08:57
lennybyolanda: thanks.08:58
yolandasileht, fixed08:59
yolandathanks for the alert08:59
*** shardy_ has joined #openstack-infra08:59
*** aopincar has joined #openstack-infra08:59
silehtyolanda, thx for the quick fix08:59
*** lawrancejing has quit IRC08:59
yolandai'll check other endpoints as well08:59
*** stevemar_ has joined #openstack-infra09:00
*** shardy has quit IRC09:00
*** nmagnezi has joined #openstack-infra09:01
openstackgerritMerged openstack-infra/system-config: Send email on merge-failure instead of commenting  https://review.openstack.org/20651309:01
*** tdasilva has quit IRC09:02
*** heyongli has quit IRC09:02
*** heyongli has joined #openstack-infra09:03
*** stevemar_ has quit IRC09:03
*** sergmelikyan has quit IRC09:04
*** sergmelikyan has joined #openstack-infra09:04
*** shardy_ has quit IRC09:04
*** nadya has quit IRC09:04
*** frickler has quit IRC09:05
*** luqas has joined #openstack-infra09:05
*** Mmike has quit IRC09:05
*** shardy has joined #openstack-infra09:05
*** jcoufal has quit IRC09:06
*** marzif has joined #openstack-infra09:07
*** e0ne has joined #openstack-infra09:07
*** manas has quit IRC09:08
AJaegeryolanda: specs often have tests - to ensure the RST is valid09:08
yolandaAJaeger, new thing learnt :)09:08
*** jcoufal has joined #openstack-infra09:09
*** Daisy has quit IRC09:09
AJaegeryolanda: specs publish HTML documents built from RST - so testing that the RST can be published is nice ;)09:09
*** amotoki has joined #openstack-infra09:10
*** Mmike has joined #openstack-infra09:11
*** luqas has quit IRC09:11
*** heyongli has quit IRC09:12
*** yamahata has joined #openstack-infra09:13
*** heyongli has joined #openstack-infra09:13
*** dkehn has quit IRC09:14
AJaegeryolanda: something else to mention: publishing to specs.openstack.org and docs.openstack.org is only available for official projects - the repo needs to be in governance/reference/projects.yaml. Watcher is not there, so is not allowed to publish to specs.openstack.org.09:14
*** lawranc__ has quit IRC09:15
yolandaah, thanks for the info. So in that case we need to ask for the owner to create the change to governance09:15
AJaegeryolanda: yes - first to become part of the big tent.09:15
*** lawrancejing has joined #openstack-infra09:15
AJaegerBut for watcher-specs, they can just setup the specs and not publish for now.09:15
*** dkehn has joined #openstack-infra09:15
*** notnownikki has joined #openstack-infra09:16
*** kiran-r has quit IRC09:16
openstackgerritEmma Foley proposed openstack-infra/project-config: Opening a repository for a ceilometer plugin for collectd.  https://review.openstack.org/22276309:17
openstackgerritGael Chamoulaud proposed openstack-infra/project-config: zuul/layout: Enable non-voting puppet check jobs on tripleo-heat-templates.  https://review.openstack.org/23412909:18
AJaegeryolanda: didn't you see my comment on https://review.openstack.org/237112 ?09:18
*** frickler has joined #openstack-infra09:18
AJaegeryolanda: for certain repos - most notably neutron family but also glance - we have liasons that should review any change, see https://wiki.openstack.org/wiki/CrossProjectLiaisons#Infra09:19
*** sergmelikyan has quit IRC09:19
*** luqas has joined #openstack-infra09:19
*** sergmelikyan has joined #openstack-infra09:20
*** ihrachys has joined #openstack-infra09:20
*** tdasilva has joined #openstack-infra09:21
AJaegeryolanda: I don't think harm done and we can revert if necessary - but please check comments next time.09:22
*** heyongli has quit IRC09:22
*** heyongli has joined #openstack-infra09:23
yolandaAJaeger so i missed that sorry09:24
openstackgerritGael Chamoulaud proposed openstack-infra/project-config: zuul/layout: Enable non-voting puppet check jobs on tripleo-heat-templates.  https://review.openstack.org/23412909:25
*** gchamoul has joined #openstack-infra09:25
*** jamielennox|away is now known as jamielennox09:26
*** nadya has joined #openstack-infra09:27
*** sergmelikyan has quit IRC09:28
openstackgerritJohn Garbutt proposed openstack-infra/devstack-gate: DO NOT MERGE - testing tempest-dsvm-lxc  https://review.openstack.org/23748009:29
openstackgerritEmma Foley proposed openstack-infra/project-config: Opening a repository for a ceilometer plugin for collectd.  https://review.openstack.org/22276309:31
*** heyongli has quit IRC09:33
*** heyongli has joined #openstack-infra09:34
*** alex_xu has quit IRC09:34
*** lawrancejing has quit IRC09:34
*** sergmelikyan has joined #openstack-infra09:35
*** alex_xu has joined #openstack-infra09:36
*** sergmelikyan has quit IRC09:36
*** manas has joined #openstack-infra09:36
*** amotoki has quit IRC09:37
*** sergmelikyan has joined #openstack-infra09:37
*** dtardivel has joined #openstack-infra09:38
openstackgerritJohn Garbutt proposed openstack-infra/project-config: Add tempest-dsvm-lxc as experimental  https://review.openstack.org/22656509:39
*** dtantsur is now known as dtantsur|bbl09:39
*** amotoki has joined #openstack-infra09:43
*** heyongli has quit IRC09:43
*** ddieterly has joined #openstack-infra09:43
*** heyongli has joined #openstack-infra09:44
*** rlu has joined #openstack-infra09:44
*** fawadkhaliq has joined #openstack-infra09:46
openstackgerritJean-Emile DARTOIS proposed openstack-infra/project-config: Add watcher-specs  https://review.openstack.org/23367909:47
*** ddieterly has quit IRC09:48
*** markus_z is now known as markus_z_lunch09:49
*** dims_ has joined #openstack-infra09:49
*** sergmelikyan has quit IRC09:49
jed56AJaeger : Hi09:50
AJaegerhi jed5609:51
*** aeng has quit IRC09:52
*** ihrachys has quit IRC09:52
*** amotoki has quit IRC09:53
*** heyongli has quit IRC09:53
jed56I'm trying to add watcher-specs and I thought that there is not more difference between stackforge project and openstack project09:54
*** heyongli has joined #openstack-infra09:54
*** rlu has quit IRC09:54
AJaegerjed56: we only merged the namespaces, the idea of stackforge still exists09:54
*** rlu has joined #openstack-infra09:55
*** dtardivel has quit IRC09:56
jed56but not we have http://github.com/openstack/watcher09:57
jed56so your code will be on openstack and the specs will be on stackforge ?09:57
jed56I'm not sure to understand09:57
*** dtardivel has joined #openstack-infra09:58
openstackgerritJohn Garbutt proposed openstack-infra/devstack-gate: DO NOT MERGE - testing tempest-dsvm-lxc  https://review.openstack.org/23748009:58
openstackgerritJohn Garbutt proposed openstack-infra/devstack-gate: DO NOT MERGE - testing tempest-dsvm-lxc  https://review.openstack.org/23748009:59
jed56One collegue of my asked this question : Question (Oct 9) : Hi Infra team. I want to create a new "stackforge" project. Do I need to wait for Stackforge/OpenStack projects migration, or can I create it now and update your wiki page for migration? Thanks        09:59
jed56=> mordred           dtardivel: you can create a new project now, just do it in the openstack namespace10:00
jed56the response was given by mordred the 9 oct10:00
jed56mordred : hi10:01
Clintjed56: you can be in openstack/ and not be an official openstack project10:02
*** amotoki has joined #openstack-infra10:02
AJaegerjed56: and that answer is correct - as Clint says10:02
AJaegerOfficial openstack projects are those that are recorded in governance repository.10:03
AJaegerProjects can use the openstack namespace, even those that are not official10:03
*** zhenguo has quit IRC10:03
*** heyongli has quit IRC10:03
AJaegerjed56: the specs will not be on stackforge!10:03
AJaegerjed56: watcher-specs will be part of openstack namespace.10:04
AJaegerBUT you cannot publish to specs.openstack.org unless you are an official project10:04
*** heyongli has joined #openstack-infra10:04
AJaegerSo, don't use the template for publishing to specs.openstack.org but use the template for projects that do not publish - which is called stackforge-specs-job10:04
AJaegerjed56: is that clearer?10:04
jed56yes,  this more clear now. So i will change openstack-specs-jobs by stackforge-specs-jobs.10:06
jed56in the new patchset10:06
AJaegerjed56: exactly10:08
lennybbtw, http://stackalytics.com is down. Is it 'ours' ?10:09
AJaegerlennyb: no10:09
lennybAjaeger: thanks.10:09
openstackgerritYAMAMOTO Takashi proposed openstack-infra/project-config: (Re)add midonet tempest jobs  https://review.openstack.org/23093210:10
*** marzif has quit IRC10:10
*** bdemers has quit IRC10:11
*** tlbr has joined #openstack-infra10:11
openstackgerritJean-Emile DARTOIS proposed openstack-infra/project-config: Add watcher-specs  https://review.openstack.org/23367910:11
jed56AJaeger:  thanks10:12
*** markvoelker_ has quit IRC10:13
*** heyongli has quit IRC10:14
*** fhubik is now known as fhubik_brb10:14
*** heyongli has joined #openstack-infra10:15
*** jed56 is now known as identity10:15
*** lawrancejing has joined #openstack-infra10:16
*** identity is now known as jed5610:16
*** SamYaple has joined #openstack-infra10:18
*** Kennan2 has quit IRC10:19
*** amotoki_ has joined #openstack-infra10:21
*** amotoki has quit IRC10:23
*** Kennan has joined #openstack-infra10:24
*** heyongli has quit IRC10:24
*** heyongli has joined #openstack-infra10:25
openstackgerritDmitriy Ukhlov proposed openstack-infra/project-config: Oslo to adopt dukhlov's devstack pika plugin  https://review.openstack.org/23700110:25
*** pc_m has joined #openstack-infra10:28
*** sergmelikyan has joined #openstack-infra10:31
*** claudiub has joined #openstack-infra10:32
*** heyongli has quit IRC10:34
*** aukhan has quit IRC10:35
*** sergmelikyan has quit IRC10:35
*** yfried_ is now known as yfried|afk10:35
*** heyongli has joined #openstack-infra10:35
*** aukhan has joined #openstack-infra10:35
*** markus_z_lunch is now known as markus_z10:36
*** sergmelikyan has joined #openstack-infra10:38
*** yfried|afk is now known as yfried_10:38
aspiersyolanda / anyone: please could I be added to openstack-resource-agents-core? my gerrit username is adam.spiers.  https://review.openstack.org/#/admin/groups/1133,members10:39
*** sergmelikyan has quit IRC10:39
*** sergmelikyan has joined #openstack-infra10:39
*** sergmelikyan has quit IRC10:42
openstackgerritDerek Higgins proposed openstack-infra/project-config: Add tripleo F22 periodic jobs  https://review.openstack.org/23532110:43
openstackgerritDerek Higgins proposed openstack-infra/project-config: Remove redundant sections of tripleo job names  https://review.openstack.org/23539510:43
*** nadya has quit IRC10:44
*** ddieterly has joined #openstack-infra10:44
*** heyongli has quit IRC10:44
*** heyongli has joined #openstack-infra10:45
AJaegeraspiers: you should get added to -release as well - https://review.openstack.org/#/admin/groups/1134,members10:45
*** bdemers has joined #openstack-infra10:48
*** ddieterly has quit IRC10:48
openstackgerritEmma Foley proposed openstack-infra/project-config: Opening a repository for a ceilometer plugin for collectd.  https://review.openstack.org/22276310:49
*** ParsectiX has quit IRC10:50
*** aysyd has joined #openstack-infra10:50
*** ldnunes has joined #openstack-infra10:52
*** ZZelle has joined #openstack-infra10:52
*** ihrachys has joined #openstack-infra10:52
*** nadya has joined #openstack-infra10:54
*** aysyd has quit IRC10:54
*** heyongli has quit IRC10:55
*** paboldin has joined #openstack-infra10:55
*** heyongli has joined #openstack-infra10:56
*** manas has quit IRC10:56
paboldinHello channel. I'm here to complain about logs.openstack.org giving out css files with a 'x-asm' content type. This breaks things like this: http://logs.openstack.org/23/231423/6/check/rally-coverage/8eb0ab1/cover/10:56
*** rvba has quit IRC10:57
*** dtardivel has quit IRC10:58
*** ociuhandu has quit IRC11:00
*** jamielennox is now known as jamielennox|away11:00
*** aysyd has joined #openstack-infra11:01
aspiersAJaeger: true11:02
*** fhubik_brb is now known as fhubik11:03
*** EricGonczer_ has joined #openstack-infra11:03
*** weihan has joined #openstack-infra11:04
*** heyongli has quit IRC11:05
*** rlu has quit IRC11:05
*** heyongli has joined #openstack-infra11:06
*** dtantsur|bbl is now known as dtantsur11:08
*** ParsectiX has joined #openstack-infra11:09
*** nadya has quit IRC11:11
*** EricGonc_ has joined #openstack-infra11:12
*** markvoelker has joined #openstack-infra11:14
*** heyongli has quit IRC11:15
*** amotoki_ has quit IRC11:15
*** manas has joined #openstack-infra11:15
*** EricGonczer_ has quit IRC11:16
*** heyongli has joined #openstack-infra11:16
*** sergmelikyan has joined #openstack-infra11:17
*** EricGonc_ has quit IRC11:19
*** e0ne has quit IRC11:21
openstackgerritEmilien Macchi proposed openstack-infra/project-config: Create openstack/puppet-openstack-release-tools  https://review.openstack.org/23551311:23
*** yamamoto has quit IRC11:24
*** jyuso1 has joined #openstack-infra11:24
*** heyongli has quit IRC11:25
*** ociuhandu has joined #openstack-infra11:26
*** heyongli has joined #openstack-infra11:26
*** ildikov has quit IRC11:26
*** flip214 has joined #openstack-infra11:27
flip214Hi. I've got a link to graphite.openstack.org, but it doesn't show any new data since 2 days. Have the CI jobs been renamed or something like that?11:27
flip214It's about gate-tempest-dsvm-full-drbd-devstack-nv in cinder.11:28
flip214URL: http://graphite.openstack.org/render/?width=600&height=344&_salt=1434709688.361&from=-3days&xFormat=%25b%25d-%25T&title=DRBD%20Cinder%2FDevstack%20stats&colorList=red%2Cgreen%2Cblue&target=stats_counts.zuul.pipeline.check.job.gate-tempest-dsvm-full-drbd-devstack-nv.FAILURE&target=stats_counts.zuul.pipeline.check.job.gate-tempest-dsvm-full-drbd-devstack-nv.SUCCESS11:28
*** sergmelikyan has quit IRC11:30
*** aysyd has quit IRC11:30
*** aysyd has joined #openstack-infra11:31
openstackgerritBruno Tavares proposed openstack-infra/zuul: Adds information about metric types.  https://review.openstack.org/23752311:31
*** salv-orl_ has joined #openstack-infra11:31
*** sergmelikyan has joined #openstack-infra11:32
*** markvoelker has quit IRC11:32
sdagueyeh, I think graphite stopped taking data over the weekend11:32
sdaguefungi / jhesketh / SergeyLukjanov anyone able to look?11:33
sdaguesorry - http://tinyurl.com/nd5blp711:33
SergeyLukjanovsdague, here11:33
sdagueis another instance demonstrating that we stopped having job data in graphite sometime over the weekend11:33
SergeyLukjanovsdague, any ideas?11:34
sdagueSergeyLukjanov: no idea, I just see the problem11:34
*** marcusvrn_ has joined #openstack-infra11:34
sdagueI saw it yesterday, but assumed it was something wrong with my tools11:34
*** salv-orlando has quit IRC11:34
SergeyLukjanovsdague, but we still have info on status.o.o/zuul for example11:34
sdagueyep, so it's specific to the job data11:35
*** heyongli has quit IRC11:36
*** maiteb has joined #openstack-infra11:36
*** ParsectiX has quit IRC11:36
SergeyLukjanovwe have gate-grenade-dsvm jobs running even right now - https://jenkins04.openstack.org/job/gate-grenade-dsvm/11:37
*** heyongli has joined #openstack-infra11:37
*** manas has quit IRC11:38
*** yamahata has quit IRC11:38
*** markvoelker has joined #openstack-infra11:39
*** markvoelker has quit IRC11:39
sdagueright, that's why I'm sure it's wrong11:40
*** gordc has joined #openstack-infra11:40
flip214so, it's nothing to do with that specific job, but some general graphite problem? okay, thanks - then I can stop looking.11:43
*** salv-orl_ has quit IRC11:44
*** dprince has joined #openstack-infra11:44
*** ddieterly has joined #openstack-infra11:45
*** salv-orlando has joined #openstack-infra11:45
*** stevemar_ has joined #openstack-infra11:45
*** heyongli has quit IRC11:46
*** heyongli has joined #openstack-infra11:47
openstackgerritJean-Emile DARTOIS proposed openstack-infra/project-config: Add watcher-specs  https://review.openstack.org/23367911:47
*** stevemar_ has quit IRC11:48
*** lawrancejing has quit IRC11:49
*** ddieterly has quit IRC11:49
*** glauco has joined #openstack-infra11:50
*** dtardivel has joined #openstack-infra11:53
openstackgerritDerek Higgins proposed openstack/diskimage-builder: Nothing to see here  https://review.openstack.org/13962711:53
*** ParsectiX has joined #openstack-infra11:55
*** heyongli has quit IRC11:56
*** fhubik is now known as fhubik_brb11:56
*** heyongli has joined #openstack-infra11:57
*** fawadkhaliq has quit IRC11:58
*** ParsectiX has quit IRC11:58
*** jordanP has quit IRC11:58
*** jyuso has joined #openstack-infra12:00
*** ildikov has joined #openstack-infra12:01
*** amuller has joined #openstack-infra12:02
*** bdemers has quit IRC12:02
*** jyuso1 has quit IRC12:03
*** rfolco_ has joined #openstack-infra12:06
*** heyongli has quit IRC12:06
*** weihan has quit IRC12:07
*** heyongli has joined #openstack-infra12:07
*** otter768 has quit IRC12:07
*** otter768 has joined #openstack-infra12:08
*** zeih has joined #openstack-infra12:08
*** dtardivel has quit IRC12:09
*** yfried__ has joined #openstack-infra12:09
*** thorst1 has joined #openstack-infra12:09
*** mrmartin has joined #openstack-infra12:10
*** yfried_ has quit IRC12:10
*** trown|outttypeww is now known as trown12:13
*** Daisy has joined #openstack-infra12:13
*** ParsectiX has joined #openstack-infra12:14
*** heyongli has quit IRC12:17
*** heyongli has joined #openstack-infra12:18
openstackgerritBruno Tavares proposed openstack-infra/zuul: Adds information about metric types.  https://review.openstack.org/23752312:18
*** doug-fish has quit IRC12:19
*** doug-fish has joined #openstack-infra12:19
*** yamamoto has joined #openstack-infra12:20
*** doug-fish has quit IRC12:21
*** fhubik_brb is now known as fhubik12:21
*** doug-fish has joined #openstack-infra12:21
shardyHi all, can anyone please direct me to the process for creating new stable branches on repos which don't currently have them?12:21
shardyI've found various stable branch content in the docs/wiki but not spotted how that's supposed to work12:21
*** aopincar has quit IRC12:24
*** jaypipes has joined #openstack-infra12:24
*** yfried__ has quit IRC12:24
AJaegershardy: does this work for you: http://docs.openstack.org/infra/manual/drivers.html#create-proposed-branch12:24
*** pradk has joined #openstack-infra12:24
*** burgerk has joined #openstack-infra12:25
AJaegerdhellman, fungi, I think we need to update the Infra Manual a bit, it still contains proposed branches... http://docs.openstack.org/infra/manual/drivers.html12:25
*** dtardivel has joined #openstack-infra12:25
*** otter768 has quit IRC12:25
AJaegershardy: as mentioned in that document, you can do this in webui - can't you?12:25
shardyAJaeger: Thanks! I think I looked at that, but I lack the superpowers to create the branch in gerrit I think12:26
* shardy checks again12:26
AJaegershardy: which repo?12:26
*** otter768 has joined #openstack-infra12:26
*** dguitarbite has quit IRC12:26
shardyAJaeger: Several TripleO repos, ref https://review.openstack.org/#/c/221811/12:26
*** hashar has quit IRC12:26
*** nadya has joined #openstack-infra12:27
shardyAJaeger: I'm tripleo core, but I was assuming this required interaction with the release manager, or other infra process12:27
*** heyongli has quit IRC12:27
AJaegershardy: correct, looking at https://review.openstack.org/#/admin/projects/openstack/tripleo-common,access there's no branch creation setup12:27
*** hashar has joined #openstack-infra12:27
shardyYeah, same there, it's just a read-only view of the current master12:27
*** heyongli has joined #openstack-infra12:28
AJaegershardy: http://docs.openstack.org/infra/manual/creators.html#creating-of-branches explains the needed permissions12:28
*** jordanP has joined #openstack-infra12:28
AJaegeryou miss a "create" line12:28
*** yamamoto has quit IRC12:29
*** amotoki has joined #openstack-infra12:29
AJaegershardy: do you want to tag releases from those branches as well?12:29
shardyAJaeger: aha, so it's a project-config thing, I see, thanks!12:29
*** rossella_s has quit IRC12:30
shardyAJaeger: not sure, that's not part of the current proposal, we're initially just planning to cut a new branch per openstack release12:30
shardyso we can support deploying e.g stable/liberty of all other projects12:30
AJaegershardy: yes, send a patch to project-config for these files, we review, approve, you will get added manually to the release group and then have the superpowers to use and delegate ;)12:30
AJaegershardy: so, just add the create line for the repos...12:30
shardyAJaeger: Great, thanks for the explanation, will do! :)12:31
openstackgerritDavanum Srinivas (dims) proposed openstack/requirements: Expose extras information in requirements  https://review.openstack.org/23754412:31
SamYaplecan someone point me to some documentation about how i get the proposal bot to start proposing things for a project?12:31
AJaegerSamYaple: all documented in the Infra Manual...12:32
AJaegerhttp://docs.openstack.org/infra/manual/creators.html explains the setup12:32
AJaegerSamYaple: wait, sorry.12:32
*** edmondsw has joined #openstack-infra12:32
AJaegerI misparsed your line. What do you want to propose?12:32
*** e0ne has joined #openstack-infra12:33
SamYapleAJaeger: currently the openstack proposal bot does not propose changes to the Kolla project, I want to make it do that thing12:34
AJaegerSamYaple: Proposal bots are documented basically only in source ;)12:34
AJaegerSamYaple: REquirements changes?12:34
*** markvoelker_ has joined #openstack-infra12:35
AJaegerSamYaple: That's done via the requirements repo, add your project to projects.txt and run the check-requirements changes as well on kolla. See also http://docs.openstack.org/developer/requirements/12:35
AJaegerproject-config needs updating for the check-requirements change12:35
SamYapleAJaeger: awesome! thats what i needed to know12:35
*** heyongli has quit IRC12:37
SamYapleAJaeger: yup that looks like everything. thanks will get patches submitted12:37
*** heyongli has joined #openstack-infra12:38
*** sshnaidm has joined #openstack-infra12:39
*** jyuso has quit IRC12:39
SamYapleAJaeger: do you know the order for these patches? requirements first or project-config?12:39
*** amotoki has quit IRC12:43
*** dguitarbite has joined #openstack-infra12:43
openstackgerritSam Yaple proposed openstack-infra/project-config: Add check-requirements for Kolla  https://review.openstack.org/23755012:44
openstackgerritSam Yaple proposed openstack/requirements: Add kolla to projects.txt  https://review.openstack.org/23755112:44
AJaegerSamYaple: project-config first12:45
SamYaplecool thanks12:45
*** ddieterly has joined #openstack-infra12:45
AJaegerSamYaple: Let me just comment, no need to change...12:46
SamYapleok i was just going to drop in a depends-on12:46
SamYapleworks for me though12:46
*** heyongli has quit IRC12:47
AJaegerSamYaple: depends-on would be nicer indeed ;)12:48
*** annegentle has joined #openstack-infra12:48
*** heyongli has joined #openstack-infra12:48
SamYapleyea its going to bug me if i dont add it12:48
openstackgerritSam Yaple proposed openstack/requirements: Add kolla to projects.txt  https://review.openstack.org/23755112:49
*** ddieterly has quit IRC12:50
*** esker has joined #openstack-infra12:51
*** weshay_xchat has joined #openstack-infra12:51
*** esker has quit IRC12:52
*** ddieterly has joined #openstack-infra12:52
*** esker has joined #openstack-infra12:52
*** markvoelker_ has quit IRC12:54
*** doug-fish has quit IRC12:54
SamYapleAJaeger: I dont see the 'gate-{name}-requirements' added to any of the other projects in projects.yaml12:55
*** doug-fish has joined #openstack-infra12:55
*** dramalho has joined #openstack-infra12:55
SamYapleoh i do see a few12:56
AJaegerSamYaple: Most projects use python-jobs job-group and that includes requirments. You can do that as well or do it as openstack-doc-tools does where it's included12:56
SamYapleah that makes more sense12:56
openstackgerritSam Yaple proposed openstack-infra/project-config: Add check-requirements for Kolla  https://review.openstack.org/23755012:58
*** heyongli has quit IRC12:58
SamYaplethanks for your help12:58
*** fhubik is now known as fhubik_brb12:58
*** yamamoto has joined #openstack-infra12:58
*** fawadkhaliq has joined #openstack-infra12:59
*** heyongli has joined #openstack-infra12:59
*** doug-fish has quit IRC13:00
*** ddieterly has quit IRC13:01
*** sshnaidm has quit IRC13:01
*** derekh is now known as derekh_afk13:01
*** regXboi has joined #openstack-infra13:03
*** fawadkhaliq has quit IRC13:04
*** mjb has quit IRC13:04
*** Daisy has quit IRC13:04
*** jjohnson2 has joined #openstack-infra13:04
*** heyongli has quit IRC13:08
*** heyongli has joined #openstack-infra13:09
*** jjohnson2 has quit IRC13:09
*** rlandy has joined #openstack-infra13:10
*** yfried__ has joined #openstack-infra13:10
fungisdague: i'm looking into it. odd that just some data isn't ending up in graphite but other data is (for example, zuul's job queue and pipeline counts are used to create the graphs on the status page successfully)13:13
*** jed56 has quit IRC13:13
*** ParsectiX has quit IRC13:13
*** sshnaidm has joined #openstack-infra13:14
sdaguefungi: I agree that it is odd13:14
*** maiteb has quit IRC13:14
fungifilesystem isn't full, oom killer hasn't fired for a week... i wonder if there was a recent code change in zuul which regressed job stats13:15
*** changbl has quit IRC13:16
*** Zhongjun has joined #openstack-infra13:16
*** thorst1 has quit IRC13:16
fungithat timing could coincide with the zuul restart for our maintenance13:17
*** AzherKhan has joined #openstack-infra13:17
*** aukhan has quit IRC13:17
*** sergmelikyan has quit IRC13:17
fungimost recent patch to zuul was "Adds missing statsd counter increment." which merged on friday13:17
*** mjb has joined #openstack-infra13:17
fungithough it's possible we hadn't restarted zuul since https://review.openstack.org/202867 merged on october 613:18
*** heyongli has quit IRC13:18
fungiwhich brings much more intrusive alterations to job statistics13:18
fungichecking now to see if i can narrow down the window between when restarts happened13:19
*** heyongli has joined #openstack-infra13:19
*** alivigni has joined #openstack-infra13:19
*** doug-fish has joined #openstack-infra13:20
*** tlian has joined #openstack-infra13:20
*** BharatK has quit IRC13:21
*** baoli has joined #openstack-infra13:21
*** baoli has quit IRC13:21
*** tiswanso has joined #openstack-infra13:23
fungiyep, saturday was the first time in a month we've restarted zuul13:23
*** nelsnelson has quit IRC13:23
*** BharatK has joined #openstack-infra13:24
fungiso i suspect 202867 is likely the cause13:24
*** baoli has joined #openstack-infra13:25
*** yamamoto has quit IRC13:27
*** baoli_ has joined #openstack-infra13:27
*** ddieterly has joined #openstack-infra13:28
*** heyongli has quit IRC13:28
*** Daisy has joined #openstack-infra13:29
*** heyongli has joined #openstack-infra13:29
*** zeih has quit IRC13:30
*** dims_ has quit IRC13:30
*** baoli has quit IRC13:30
*** dims_ has joined #openstack-infra13:30
*** yfried__ has quit IRC13:31
*** burgerk has quit IRC13:31
*** jsavak has joined #openstack-infra13:32
warpcHi, guys. I need help. We use gevent eggs in fuel-ostf. According to the log http://logs.openstack.org/91/237491/1/check/gate-fuel-ostf-python26/612c605/console.html this eggs is failed to install. What can i do to fix it and pass tests?13:32
fungioh, wait, my math is off by a day13:32
*** ParsectiX has joined #openstack-infra13:32
fungiit looks like this actually started around 05:00 utc sunday, not saturday13:32
dims_fungi: has anyone poked at "E: Unable to correct problems, you have held broken packages." problems?13:33
dims_fungi: http://logs.openstack.org/44/237544/1/check/gate-tempest-dsvm-full/140ecc3/logs/devstacklog.txt.gz#_2015-10-20_12_43_44_65113:33
*** mpavone has quit IRC13:33
*** pradk has quit IRC13:34
*** pradk has joined #openstack-infra13:34
*** derekh_afk is now known as derekh13:35
fungidims_: http://logs.openstack.org/44/237544/1/check/gate-tempest-dsvm-full/140ecc3/logs/devstacklog.txt.gz#_2015-10-20_12_43_43_28313:35
fungidims_: there's the actual error13:35
fungidims_: bad distro package repo13:35
fungius.archive.ubuntu.com seems to have a bad sources file for trusty-updates13:36
*** zeih has joined #openstack-infra13:36
*** amotoki has joined #openstack-infra13:36
dims_fungi: ugh! thanks13:37
*** jcoufal has quit IRC13:37
dims_fungi: what should we do? guessing recheck will not help13:37
*** Piet has quit IRC13:38
*** heyongli has quit IRC13:39
sdaguedims_: it will probably, the mirrors aren't atomic13:39
dims_sdague: ack thanks13:39
sdagueso they are in corrupt states for windows every dya13:39
*** jcoufal has joined #openstack-infra13:40
mordredfungi: wow: http://logs.openstack.org/63/236763/2/gate/gate-shade-dsvm-functional-keystone2/96e5acf/console.html#_2015-10-20_13_05_59_11713:40
*** heyongli has joined #openstack-infra13:40
mordredfungi: read only filesystem FTW13:40
mordredfungi: clouds \0/13:40
*** yamamoto has joined #openstack-infra13:40
*** abregman has quit IRC13:41
*** amotoki has quit IRC13:41
fungiread-only /tmp no less!13:42
fungithat's rather unhelpful13:42
sdaguemordred: that's the second weird filesystem issue on rax in a few days13:42
*** wenlock has joined #openstack-infra13:42
sdagueI wonder if they are blowing physical disks13:42
mtreinishfungi: heh, I like that13:43
fungiif that job also collected syslog i might be able to see why, but i suspect i/o errors caused the kernel to remount the rootfs read-only at some point during the job13:43
sdaguefungi: yeh, there were no other logs for that reason13:44
*** eharney has quit IRC13:44
fungioh... right-oh. jenkins agent barfed before it collected any13:44
*** weihan has joined #openstack-infra13:47
sdaguefungi: also, something is wrong with the indexer, where it's merging 2 jobs13:48
*** heyongli has quit IRC13:49
*** armax has joined #openstack-infra13:49
fungithe index generator for os-loganalyze?13:50
*** heyongli has joined #openstack-infra13:50
mrmartinfungi, krotscheck: I've restored the old stackmeat.org from backup, you can check the project catalog here: http://dev.stackmeat.org/catalog13:50
fungisdague: as for the missing graphite data, i'm noticing that the wait_time value is the only one still reporting data in the per-job pipeline stats13:51
*** oomichi has quit IRC13:51
*** thorst1 has joined #openstack-infra13:52
*** AzherKhan has quit IRC13:52
*** rossella_s has joined #openstack-infra13:52
sdaguefungi: no, sorry, the es indexer13:52
sdaguelet me find you an anomoly13:52
fungistill not convinced it's the wait_time patch that has caused this regression, since i can't explain the ~12-hour delay13:52
*** peristeri has quit IRC13:53
mtreinishfungi: did we ever create the server for health.o.o? I'm not seeing it in puppetboard or anything13:53
sdaguefungi: @timestamp:"2015-10-20T12:31:49.956+00:00"13:53
fungimtreinish: not to my knowledge13:53
sdaguefungi: so that entry is accounted against 2 jobs13:54
fungisdague: seems possible there were two jobs with log lines with the same timestamp, but i'm unsure what behavior you're expecting13:55
sdaguefungi: look at the *single* line returned by that13:55
sdaguethere is one line returned13:55
sdaguewith 2 masters13:55
fungilemme see if this netbook will load kibana without falling over13:55
*** fawadkhaliq has joined #openstack-infra13:56
fungigraphite just about oom'd the browser a moment ago13:57
mtreinishsdague: http://logstash.openstack.org/#dashboard/file/logstash.json?query=_id:YA4MnFMoT5OaKLZIKJHAdA is a query to show exactly what you're looking at13:57
*** dramalho has quit IRC13:57
sdaguemtreinish: nope13:58
mtreinishit has 2 sets of metadata on one document13:58
sdagueI think _id relies on the writable indexes13:58
sdaguewhich we don't have13:58
mtreinishyou might have to bump the date range, it looks13:58
mtreinishlike clarkb's share patch doesnt copy that bit to the url13:59
*** mestery has joined #openstack-infra13:59
sdagueoh, yeh, have to bump that13:59
*** heyongli has quit IRC13:59
sdagueyeh, it's kind of unfortunate that we can't come up with a better share model here, because timerange keeps getting reset13:59
sdagueand all the perspectives that you can build can't be shared14:00
*** Daisy has quit IRC14:00
*** heyongli has joined #openstack-infra14:00
*** Daisy has joined #openstack-infra14:00
fungisounds like you figured it out while i'm still waiting for this browser to stop being unresponsive14:01
mtreinishsdague: it should be doable, I think we just overlooked it on the hotpatch to fix the share button14:01
*** trown is now known as trown|mtg14:02
*** aukhan has joined #openstack-infra14:02
mtreinishsdague: https://review.openstack.org/#/c/236070/14:02
sdaguefungi: well, we know what's wrong14:02
*** doug-fish has quit IRC14:02
sdaguebut why is really unclear14:02
sdaguewhy do we smash in someone else's metadata to the document14:02
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: make intermediary.sh support stable branches  https://review.openstack.org/23759514:03
*** Piet has joined #openstack-infra14:03
*** alexpilotti has quit IRC14:04
*** dramalho has joined #openstack-infra14:04
*** Daisy has quit IRC14:04
*** hdd has joined #openstack-infra14:05
warpcHi! If some of eggs won't  install during testing in case of gate-X-py27, what can i do to fix it?14:06
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: make intermediary.sh support stable branches  https://review.openstack.org/23759514:06
*** hashar has quit IRC14:07
*** fawadkhaliq has quit IRC14:07
*** ParsectiX has quit IRC14:08
fungisdague: hrm, got that query loaded and i see four log lines with that timestamp. doesn't seem unusual to me but i probably have no clue what i'm looking for14:08
*** aukhan has quit IRC14:08
*** annegentle has quit IRC14:08
*** burgerk has joined #openstack-infra14:08
mtreinishfungi: look at the document with the id from the link I posted14:08
*** nelsnelson has joined #openstack-infra14:08
*** aukhan has joined #openstack-infra14:08
mtreinishthere are 2 sets of metadata associated with it14:08
openstackgerritSteven Hardy proposed openstack-infra/project-config: Enable stable branches for tripleo repositories  https://review.openstack.org/23759714:09
sdaguefungi: the line that starts with 'set +o xtrace'14:09
*** burgerk_ has joined #openstack-infra14:09
sdagueit's 2nd in my list of the 414:09
*** hashar has joined #openstack-infra14:09
*** heyongli has quit IRC14:09
fungilooks like i have to change the default query range when following links like that14:10
sdaguefungi: ye14:10
*** heyongli has joined #openstack-infra14:10
fungiyeah, mtreinish's link gets me just one log line with a set +o xtrace interleaved with an authentication failure14:11
mtreinishhmm, adding a &from= to the query links is hanging firefox for me14:11
mtreinishthat might be a problem...14:11
sdaguefungi: right, so look at the metadata on that doc14:11
sdagueanother way to make these things pop is:14:11
sdaguebuild_status:SUCCESS AND build_status:FAILURE14:11
*** tonytan4ever has joined #openstack-infra14:12
fungioh, and it has two different log urls?14:12
sdagueit has 2 different everything14:12
sdague2 different build_names14:12
*** glauco___ has joined #openstack-infra14:12
sdagueeven 2 different build_status14:12
*** burgerk has quit IRC14:13
fungigot it. yep this looks strange. i wonder if it's happening on indexing or retrieval14:13
sdaguebuild_status:SUCCESS AND build_status:FAILURE is honestly the best way to see how big the problem is14:13
fungicould this be a hash collision?14:14
sdague build_status:SUCCESS AND build_status:FAILURE (26822)  count per 1h | (26822 hits)14:14
sdaguefungi: don't know14:14
*** glauco has quit IRC14:15
fungiyeah, i'm not familiar enough with es internals to know how the correlation for the index is maintained14:15
sdagueif I had to guess, I'd say it's a race on indexing14:15
*** eharney has joined #openstack-infra14:15
sdaguethat doc id is allocated some how14:15
sdagueand there is time between that and add for metadata14:16
*** zhenguo has joined #openstack-infra14:16
sdagueI tripped over this a couple of weeks ago when trying to build the "what will fail in M-1" devstack list14:16
sdagueand put up a link in channel for that oddity14:16
*** ajmiller has joined #openstack-infra14:16
sdaguebut just hit it again randomly searching for things14:16
fungithe the filename and timestamp are common between those two lines, but so is the id14:16
*** stevemar_ has joined #openstack-infra14:16
*** aukhan has quit IRC14:17
*** ayoung has quit IRC14:17
sdaguethey do seem to always have the "multiline" tag14:17
sdagueso it may be a grok parser issue14:18
*** aukhan has joined #openstack-infra14:18
fungiso wondering whether two entries happened to get identical ids though sheer chance, or as you suggest some race caused one id to get assigned to two lines14:18
sdagueanyway, we probably need clarkb involved to figure it out further14:18
fungiyep, but agreed it's strange14:18
sdagueanyway, back to graphite :)14:19
sdaguebecause zuul not reporting to graphite is kind of hard to figure out what's up with jobs14:19
*** heyongli has quit IRC14:20
*** stevemar_ has quit IRC14:20
*** sigmavirus24_awa is now known as sigmavirus2414:20
fungiwarpc: no idea. looks like gevent is not successfully compiling. there's a lot of gcc spew in there you can hopefully search on14:20
*** heyongli has joined #openstack-infra14:21
fungisdague: agreed. i'm still at a loss to explain why the wait_time value hasn't stopped being reported even though the others have, but the reporting seems to have stopped 12 hours after we restarted zuul, not when we restarted zuul14:21
sdaguefungi: oh, are you looking at my graphs?14:22
sdaguethose have a 12 hr rolling average14:22
fungisdague: ohhh14:22
sdagueso 12 hr delay would be expected14:22
fungii'll see if i can narrow it down with direct graphite queries in that case14:23
warpcfungi: thank you for answer. As i know, such error can be fixed if libevent-dev will present in the system. Can i somehow install it on test env?14:23
sdaguefungi: that would point directly to the restart time14:23
sdaguebecause that's when we'd loose data14:23
fungisdague: in that case we likely do have a regression introduced by the wait_time stats implementation14:24
sdagueso 12 hrs later we'd finally have nothing in the window and it would go blank14:24
fungitimrc: ^14:24
fungii need to step away for just a few minutes, but will continue looking at this shortly14:24
*** hdd has quit IRC14:24
openstackgerritMerged openstack-infra/tripleo-ci: Destroy previous test env VM's earlier  https://review.openstack.org/23012914:26
*** weshay_xchat is now known as weshay14:28
*** aukhan has quit IRC14:30
*** heyongli has quit IRC14:30
*** heyongli has joined #openstack-infra14:31
*** mohankumar has quit IRC14:31
*** mohankumar has joined #openstack-infra14:33
*** jsavak has quit IRC14:33
*** zeih has quit IRC14:33
openstackgerritMarios Andreou proposed openstack-infra/irc-meetings: Update tripleo meeting to weekly  https://review.openstack.org/23760914:33
*** jsavak has joined #openstack-infra14:34
openstackgerritMerged openstack-infra/grafyaml: Update pypi metadata to reference -infra  https://review.openstack.org/23158914:36
*** Daisy has joined #openstack-infra14:36
*** MarkAtwood has joined #openstack-infra14:37
*** aukhan has joined #openstack-infra14:37
*** AJaeger has quit IRC14:38
*** heyongli has quit IRC14:40
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create fuel-menu project  https://review.openstack.org/23517714:40
*** aukhan has quit IRC14:40
*** aukhan has joined #openstack-infra14:41
*** heyongli has joined #openstack-infra14:41
*** angdraug has joined #openstack-infra14:42
*** bradjones has joined #openstack-infra14:42
*** bradjones has quit IRC14:42
*** bradjones has joined #openstack-infra14:42
*** EricGonczer_ has joined #openstack-infra14:44
paboldinHello channel. I'm here to complain about logs.openstack.org giving out css files with a 'x-asm' content type. This breaks things like this: http://logs.openstack.org/23/231423/6/check/rally-coverage/8eb0ab1/cover/14:45
*** EricGonc_ has joined #openstack-infra14:46
*** BharatK has quit IRC14:47
*** aukhan has quit IRC14:48
*** aukhan has joined #openstack-infra14:49
pabelangerfungi: have some time to help troubleshoot grafana.o.o again?14:49
*** EricGonczer_ has quit IRC14:49
*** heyongli has quit IRC14:50
pabelangerfungi: trying to see if syslog reported a puppet error14:51
*** heyongli has joined #openstack-infra14:51
*** pauloewerton has joined #openstack-infra14:52
*** sabeen1 has joined #openstack-infra14:54
*** dtardivel_ has joined #openstack-infra14:54
fungiwarpc: would need to add it to http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/jenkins_params.pp14:55
fungiwarpc: and http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/thick_slave.pp14:55
fungiwarpc: and also http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/data/bindep-fallback.txt14:55
fungiwarpc: and then the next time worker images are generated, new nodes will boot with it preinstalled14:55
*** esker has quit IRC14:55
openstackgerritCyril Roelandt proposed openstack-infra/project-config: Add a gate-dsvm-functional-py34 job to Neutron  https://review.openstack.org/23317714:56
*** boris-42 has joined #openstack-infra14:56
fungipabelanger: in a bit. about to get on a conference call and still looking into zuul's recent stats reporting regression14:56
*** jsavak has quit IRC14:58
*** tiswanso has quit IRC14:58
openstackgerritDavanum Srinivas (dims) proposed openstack/requirements: Expose extras information in requirements  https://review.openstack.org/23754414:59
*** sergmelikyan has joined #openstack-infra14:59
*** jsavak has joined #openstack-infra14:59
*** sabeen3 has joined #openstack-infra14:59
*** tiswanso has joined #openstack-infra14:59
*** e0ne has quit IRC15:00
*** trown|mtg is now known as trown15:00
*** heyongli has quit IRC15:01
*** sabeen1 has quit IRC15:01
*** hdd has joined #openstack-infra15:01
*** mrc3 has joined #openstack-infra15:01
*** fhubik_brb is now known as fhubik15:02
*** heyongli has joined #openstack-infra15:02
kozhukalov_clarkb, fungi, anteaya, SergeyLukjanov, other, could you guys please review? https://review.openstack.org/#/c/23517715:02
*** e0ne has joined #openstack-infra15:02
*** links has quit IRC15:02
*** yamamoto has quit IRC15:04
openstackgerritMerged openstack-infra/shade: Rely on devstack for clouds.yaml  https://review.openstack.org/23676315:04
*** annegentle has joined #openstack-infra15:06
*** ujuc has joined #openstack-infra15:06
*** acabot has joined #openstack-infra15:06
*** doug-fish has joined #openstack-infra15:06
pauloewertonhi all. is there any way to recheck a patch against a specific gate bug? I keep hitting the same bug but elastic-recheck does not seem to detect it no matter how many recheck comments I leave in my patch =\15:07
*** fhubik is now known as fhubik_brb15:08
mwhahahahey openstack-infra folks, where are the rules for gate jobs & their voting settings documented?15:08
*** mahatic has joined #openstack-infra15:09
*** ashleighfarnham has joined #openstack-infra15:10
*** jlvillal has quit IRC15:11
*** heyongli has quit IRC15:11
*** ashleighfarnham has quit IRC15:11
*** Ala has quit IRC15:11
*** dramalho has quit IRC15:12
*** heyongli has joined #openstack-infra15:12
fungipauloewerton: if the bug isn't listed at http://status.openstack.org/elastic-recheck/ then follow the instructions on that page to add it to the queries list15:12
fungimwhahaha: can you be more specific as to what you mean by "rules"? (procedural? social?)15:13
*** abregman has joined #openstack-infra15:13
pauloewertonfungi, it is listed there already http://status.openstack.org/elastic-recheck/#149164615:13
*** k4n0 has quit IRC15:14
mwhahahawell, people keep referring to "please don't add non-voting jobs to the gate" (https://review.openstack.org/#/c/235550/) trying to figure out what best practice is15:14
*** nmagnezi has quit IRC15:14
*** abregman has quit IRC15:14
*** yfried__ has joined #openstack-infra15:15
*** changbl has joined #openstack-infra15:15
*** yamamoto has joined #openstack-infra15:16
mwhahahaand if it's documented somewhere so i could understand what is ok and what isn't. if you look at what is in the code, there's a whole bunch of non-voting items so i'm not sure what is right15:16
*** jlvillal has joined #openstack-infra15:17
fungimwhahaha: that convention is probably not currently documented, but rather mostly tribal knowledge: just about nobody looks at gate pipeline jobs which are non-voting because they don't report until the change merges, and some qa data analysis relies on assuming failures in the gate pipeline are impactful15:17
*** salv-orlando has quit IRC15:18
*** rossella_s has quit IRC15:18
*** xyang1 has joined #openstack-infra15:20
pabelangerfungi: Ya, cannot figure out why the Exec is not logging in puppetdb for grafana.o.o.  This works locally for what ever reason15:21
mwhahahafungi: thanks15:21
fungimwhahaha: also don't combine other check or gate pipeline jobs with the noop-jobs template (that's another one we likely should document somewhere)15:21
*** heyongli has quit IRC15:21
mwhahahaok i'll try and figure something out15:21
*** sergmelikyan has quit IRC15:22
*** heyongli has joined #openstack-infra15:22
*** denisra has joined #openstack-infra15:23
warpcfungi: thank you!15:24
fungipauloewerton: it looks like at least some hits for that bug have been recorded recently, and the logstash job queue doesn't seem to be high... which change did you hit it on recently? maybe the query is too strict15:25
*** mrmartin has quit IRC15:26
*** bpokorny has joined #openstack-infra15:26
*** rlandy has quit IRC15:27
*** Zhongjun has quit IRC15:27
*** rlandy has joined #openstack-infra15:28
*** mrmartin has joined #openstack-infra15:28
*** juggler has joined #openstack-infra15:29
*** bpokorny has quit IRC15:29
*** Ramanjaneya has quit IRC15:29
glauco___Hey, did anybody else noticed that some pages just vanished from the project15:29
*** bpokorny has joined #openstack-infra15:29
glauco___I can't find job detail page15:29
*** mohankumar has quit IRC15:29
glauco___sorry, wrong window! :)15:30
clarkbfungi: pauloewerton: usually the way I like to debug that is find the specific run you expected to fail in logstash using the build_short_uuid:"someuuidhere" and go from there15:30
fungiclarkb: did you see earlier discussion on the merged log line in ls/es from two separate jobs?15:31
clarkbfungi: no, but it does that when we confuse the line merging15:31
*** heyongli has quit IRC15:31
clarkbfungi: its one of those things that happens in .01% or less of events so tracking down is hard15:31
fungiclarkb: recent example is id:YA4MnFMoT5OaKLZIKJHAdA15:32
*** heyongli has joined #openstack-infra15:32
clarkbits a known issue and has been there for years15:33
*** dims__ has joined #openstack-infra15:33
clarkbis there a specific reason it has come up today?15:33
fungiseems that the filename and timestamp are identical but everthing else (aside from id) is not15:33
fungii think it must be rare enough it keeps taking people by surprise15:33
sdagueclarkb: mostly, I've started hitting it more failure analysis stuff15:33
*** aukhan has quit IRC15:34
sdaguewhich kind of makes the results suspect15:34
sdaguewhen you say "this job will break" except, it won't, because it's randomly munged into an unrelated job15:34
*** wenlock has quit IRC15:34
*** rhe00 has joined #openstack-infra15:34
clarkbsdague: the filters are published and you know where the two pieces of data originated from. Anyone can debug that15:35
sdagueanyway, more of a back burner concern, but it would be good to get to the bottom of if we could, because it does mud up the data15:35
*** sridhar_ram has joined #openstack-infra15:35
clarkbit has been a low enough incidence and the multiline munging is naive enough that it has not been a priority for me15:35
clarkbif we want to properly fix it we would switch to json or some other form of structured logging everywhere15:35
*** dims_ has quit IRC15:36
fungii do find it interesting that the filename is the same for both. makes me wonder if that matches in other occurrences as well15:36
fungior if it was just random chance15:36
*** wenlock has joined #openstack-infra15:36
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create network-checker project  https://review.openstack.org/23582215:36
*** hichihara has quit IRC15:37
clarkbfungi: it shouldn't be random. logstash looks at one line at a time and checks if the event is complete or if it needs to inspect the next line15:37
*** weshay is now known as weshay_mtg15:37
sdagueclarkb: so why does it roll across jobs? I guess that's the big that's confusing to me15:37
*** aukhan has joined #openstack-infra15:37
clarkbfungi: then when it decides the line is complete it sends it through. so in these cases something is making logstash decide to hold the current event as incomplete and look at the next one15:38
*** amotoki has joined #openstack-infra15:38
clarkbsdague: because we shove multiple streams of data through at a time15:38
fungii see, and events are interleaved from multiple jobs15:38
clarkbyes identified by what should be a unique string (the source location)15:38
clarkbI thinkt he source is also recordred in the event data15:38
sdagueoh, I think this is because of ansible15:38
sdaguewhy it started cropping up a lot more15:39
sdaguebecause these logs - http://logs.openstack.org/07/237607/1/check/gate-aodh-dsvm-functional-mysql/6c6af22/logs/devstack-gate-cleanup-host.txt15:39
sdaguewhich get scanned for "are we done" start with a line with no datestamp15:39
clarkbya "host" is the source key, its an ip:port combo identifying the tcp connection the file was streamed from15:40
openstackgerritAlex Schultz proposed openstack-infra/project-config: Add puppet jobs to fuel-library  https://review.openstack.org/23555015:40
*** shardy_ has joined #openstack-infra15:41
fungiclarkb: in unrelated news, the bug 1348954 fix is getting a do-over http://packages.ubuntu.com/trusty-updates/python3.4 (appearing there a few hours ago now)15:41
openstackbug 1348954 in python3-defaults (Ubuntu) "update Python3 for trusty" [Undecided,Confirmed] https://launchpad.net/bugs/134895415:41
*** fhubik_brb is now known as fhubik15:41
fungimaybe it'll stick this time15:41
clarkbwe can hope15:41
clarkbthough at this rate upgrading to 16.04 might be quicker15:41
fungiit's taken them until almost 16.04 lts to fix this regression in 14.04 lts15:41
fungiheh, yep, what i was just thinking15:42
*** heyongli has quit IRC15:42
*** shardy has quit IRC15:42
*** MarkAtwood has quit IRC15:42
*** heyongli has joined #openstack-infra15:43
*** bexelbie has quit IRC15:43
*** stevemar_ has joined #openstack-infra15:43
pabelangeryolanda: timrc: mind the review? https://review.openstack.org/#/c/237373/15:44
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: add support for ZUUL_CACHE_DIR  https://review.openstack.org/23764515:44
*** ldnunes has quit IRC15:44
pabelangerthat fixes 2 issue.  Bouncing test cases, and rotating md5hash15:44
pabelangerwhich is causing an issue with grafana.o.o15:44
*** ldnunes has joined #openstack-infra15:46
*** shardy_ has quit IRC15:46
*** ddieterl_ has joined #openstack-infra15:46
*** ZZelle has quit IRC15:46
*** shardy has joined #openstack-infra15:47
*** dtantsur is now known as dtantsur|afk15:48
*** Piet has quit IRC15:48
*** ddieterly has quit IRC15:49
*** annegentle has quit IRC15:50
*** amuller is now known as amuller_afk15:51
krotscheckmrmartin: stackmeat. Nice.15:51
*** amotoki has quit IRC15:51
krotscheckmrmartin: Huhn.15:52
*** heyongli has quit IRC15:52
krotscheckmrmartin: That's actually rather nice15:52
*** rlandy has quit IRC15:52
sdagueclarkb: yeh, so this is clearly mostly related to ansible. Any idea if we can change that ansible line 1 to have a timestamp?15:52
*** jcoufal_ has joined #openstack-infra15:52
*** fawadkhaliq has joined #openstack-infra15:53
*** heyongli has joined #openstack-infra15:53
fungitimrc: if you're around yet, trying to figure out what about the wait_time implementation caused job result stats to stop getting sent15:53
*** flepied has joined #openstack-infra15:53
fungitimrc: http://graphite.openstack.org/render/?width=586&height=308&_salt=1445356331.002&from=00%3A00_20151017&until=23%3A59_20151020&target=stats.zuul.pipeline.check.job.gate-tempest-dsvm-full.FAILURE&target=stats.zuul.pipeline.check.job.gate-tempest-dsvm-full.SUCCESS&target=stats.zuul.pipeline.check.job.gate-tempest-dsvm-full.wait_time15:53
*** nadya has quit IRC15:54
fungitimrc: it's a pretty clear delineation between getting success/failure stats and the zuul restart for saturday's maintenance at which point we started getting only wait_time stats15:54
*** rkukura has quit IRC15:54
anteaya<-- still trying to get home after watching the election tonight, people keep dropping in, hoping to get home in time for the meeting15:54
anteayas/tonight/last night15:55
*** jcoufal has quit IRC15:55
*** gyee has joined #openstack-infra15:56
*** flepied1 has quit IRC15:56
*** yamamoto has quit IRC15:56
*** mahatic_ has joined #openstack-infra15:57
mrmartinkrotscheck: so it is using an apache solr as a backend, and if we like to make the projects searchable, we need to refactor the actual project content to represent the project's yaml file with groups and delivarables.15:57
*** mahatic has quit IRC15:57
*** hashar is now known as hasharMeeting15:57
krotscheckmrmartin: Honestly, I feel like the majority of that can just be done in the browser. Openstack's not _that_ big that fancy search engines are needed.15:58
*** fawadkhaliq has quit IRC15:58
mrmartinand find out how to sync the projects yaml with the solr backend.15:58
mrmartinok, it was just an idea, i have this project archived somewhere15:58
*** ifarkas has quit IRC15:59
mrmartina nice angularjs project with a bootstrap theme can solve that problem.15:59
*** pradk has quit IRC15:59
clarkbsdague: I am not sure but can look into it15:59
sdaguenothing is popping out to me in the ansible docs16:00
pauloewertonfungi, clarkb, sorry, I was afk. this is the query: https://github.com/openstack-infra/elastic-recheck/blob/master/queries/1491646.yaml16:00
fungipauloewerton: yep, i know where to find the queries... was wanting an example where that should have matched and didn't16:01
pauloewertonfungi, oh I see16:01
*** BharatK has joined #openstack-infra16:02
*** scheuran has quit IRC16:02
*** asmigala has quit IRC16:02
pauloewertonnot sure how to do that though16:02
*** jsavak has quit IRC16:02
*** jsavak has joined #openstack-infra16:02
*** heyongli has quit IRC16:02
fungipauloewerton: you said you had a change which failed a job because of that bug and elastic-recheck never commented on it. can you please link the change?16:02
*** vilobhmm11 has joined #openstack-infra16:02
pauloewertonfirst time trying to debug gate failures ;)16:02
*** dizquierdo has quit IRC16:03
fungino problem. just trying to get sufficient information so i can look at the same failure you are16:03
pauloewertonfungi, sure, here's the patch https://review.openstack.org/#/c/192438/16:03
openstackgerritAlex Schultz proposed openstack-infra/project-config: Add puppet jobs to fuel-library  https://review.openstack.org/23555016:03
*** heyongli has joined #openstack-infra16:03
clarkbsdague: jenkins is adding the timestamps not ansible16:04
openstackgerritVladimir Sharshov proposed openstack-infra/project-config: Support egg Gevent (http://www.gevent.org/)  https://review.openstack.org/23765716:05
*** amotoki has joined #openstack-infra16:05
clarkbsdague: or are we looking at different parts of the lgo?16:06
*** jlanoux has quit IRC16:06
*** yamamoto has joined #openstack-infra16:06
sdagueclarkb: at least one of the wrap issues is the following16:07
sdaguedevstack-gate-cleanup-host.txt is being parsed as potentially multiline16:07
sdagueso it gets to the end of the file16:07
sdaguewhich is16:07
sdague{datetimestam} set +o xtrace16:07
sdaguethe first line of a follow on file is a thing without a {datetimestamp}16:08
sdagueso it appends that to this document16:08
sdagueeven though it's from an entirely different run16:08
sdagueand then you get merged metadata16:09
fungipauloewerton: interesting. so if i search the last 6 hours at http://logstash.openstack.org/ for message:".tox/functional/bin/python: can't open file '/usr/local/jenkins/slave_scripts/subunit2html.py': [Errno 2] No such file or directory" AND tags:"console"16:09
fungipauloewerton: i do see your run as the only match16:09
*** woodster_ has joined #openstack-infra16:09
fungipauloewerton: it's possible something happened to elastic-recheck that caused it to not find this or timeout waiting for the resulty16:10
*** rlandy has joined #openstack-infra16:10
fungii'm checking its logs next16:10
sdagueclarkb: http://logstash.openstack.org/#/dashboard/file/logstash.json?query=_id:AH13fgoeQnqMh-7rzSxsfQ as an instance16:10
*** zz_dimtruck is now known as dimtruck16:10
*** flepied has quit IRC16:11
pauloewertonfungi, ok, thanks16:12
openstackgerritVladimir Sharshov proposed openstack-infra/system-config: Support egg Gevent (http://www.gevent.org/)  https://review.openstack.org/23766016:12
clarkbI see its the localhost | success | rc = 0 leading line iwthout a timestamp causing problems16:12
*** heyongli has quit IRC16:12
*** BharatK has quit IRC16:13
sdagueclarkb: right16:13
*** unicell has joined #openstack-infra16:13
clarkband thats a layer above the timestamp wrapping16:13
*** heyongli has joined #openstack-infra16:13
*** unicell1 has quit IRC16:14
sdagueso, there might be other collisions as well, but this is definitely one of the bigger ones, and might be why it popped to my radar more recently16:14
anteayamorning zaro16:14
*** flepied has joined #openstack-infra16:14
sdagueso, I don't know if there is anyway to say "build_uuid is different, stop"16:14
sdaguebut conceptually that's what we'd want16:14
clarkbsdague: no thats the problem16:15
zarowow! gerrit test migration took all night and still didn't finish.16:15
clarkblogstash is naive there is no way to inject a stop16:15
anteayazaro: :(16:15
warpcfungi: Please have a look if you have time: https://review.openstack.org/#/c/237657/ and https://review.openstack.org/#/c/237660/ Thank you!16:15
clarkbsdague: one alternative would be to always write an event that would force a flush16:15
*** aukhan has quit IRC16:15
anteayazaro: where do you feel we are to start discussing timing an upgrade?16:15
clarkbbut then we are modifying the logs to work around a logstash derp16:15
clarkbwhich isn't really better imo16:15
anteayazaro: do you feel we need to continue to test 2.11 before we can set a time?16:15
*** aukhan has joined #openstack-infra16:15
sdagueclarkb: yeh16:15
sdagueso, honestly, I suspect we won't end up doing structured logs until we're on 16.04 with journalctl16:16
zaroanteaya: yes, i was testing on my laptop which was  probably cpu limited16:16
anteayazaro: ah okay16:16
* anteaya references https://etherpad.openstack.org/p/test-gerrit-2.1116:16
zaroplanning to test on beefy VM next16:17
anteayazaro: sounds good16:17
anteayazaro: so you want to ensure that works first before we start talking about timeing for an upgrade?16:17
*** krtaylor has quit IRC16:17
aspiersis anyone around who could add me to these two -core and -release groups on gerrit? https://review.openstack.org/#/admin/projects/openstack/openstack-resource-agents,access16:17
sdagueclarkb: I suppose we could build an output filter which dumps the ansible reporting line, we did that for devstack16:17
*** shashank_hegde has joined #openstack-infra16:17
zaroyes, just testing that it works.  but would be nice to get an estimate on the migration as well.16:17
clarkbsdague: https://github.com/ansible/ansible/issues/2454 looks like ansible isn't interested in fixing this16:18
anteayazaro: yes, well I think the estimate would include your thoughts16:18
*** david-lyle has quit IRC16:18
clarkbsdague: ya that was going to be my next suggestion, but the return code info is useful16:18
anteayazaro: do you think we are ready to upgrade?16:18
*** flepied has quit IRC16:18
clarkbsdague: maybe we can get it to not multiline and get an injected timestamp to at least preserve that info16:18
anteayazaro: are you confident in 2.11?16:18
*** david-lyle has joined #openstack-infra16:18
clarkbsdague: something like word | word | rc=digit is special16:19
*** dizquierdo has joined #openstack-infra16:19
*** dstanek has quit IRC16:19
sdagueclarkb: yeh, it sort of sucks to have to build such a filter, but it could be a thing16:20
*** salv-orlando has joined #openstack-infra16:20
*** ashleighfarnham has joined #openstack-infra16:21
zaroanteaya: not done testing yet, i think a few more things to test: gerritlib, zuul pipelines, manage projects and migration.16:22
*** dstanek has joined #openstack-infra16:22
anteayazaro: and that is all on the etherpad? https://etherpad.openstack.org/p/test-gerrit-2.1116:22
zaroalso we might need some sort of rollback script.  might need mordred help to do that.16:22
*** aukhan has quit IRC16:22
zaroyes, it's on there.16:23
*** heyongli has quit IRC16:23
anteayazaro: awesome16:23
anteayazaro: do you want a gerrit upgrade item on today's agenda?16:23
*** dramalho has joined #openstack-infra16:23
zaroanteaya: do you think there's anything special we need to test for thirdy party as it pertains to ACLs?16:23
zaroi've already added it.16:24
anteayazaro: hmmmmm, nothing immediate comes to mind16:24
anteayazaro: thank you16:24
*** heyongli has joined #openstack-infra16:24
anteayazaro: can you add the ehterpad link to the agenda under gerrit upgrade?16:24
anteayathat way we signal we have something to discuss today16:24
zaroi can't seem to find the priority effort gerrit search anymore, do you know where it's at?16:24
anteayalet me see what the topic16:25
anteayagive me a minute and I will look for it16:25
openstackgerritSebastian Schuberth proposed openstack-infra/jenkins-job-builder: quick-start: Fix the configuration file argument to read "--conf"  https://review.openstack.org/23766416:25
clarkbsdague: so I see why the stream_identity isn't good enough, the filenames can match16:25
*** salv-orlando has quit IRC16:25
SpamapSfungi: so, whats the word on that dump?16:25
clarkbsdague: so I think we want to add build uuid to that then rely on the periodic flushing of multilines to dump the last one through after a while16:25
clarkbdouble checking on the docs to make sure that will work properly16:26
*** bdemers has joined #openstack-infra16:26
clarkbbut logstash seems to have killed their doc hosting16:26
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create fuel-menu project  https://review.openstack.org/23517716:26
clarkbso I need to read the source16:26
openstackgerritDerek Higgins proposed openstack/diskimage-builder: Use DIB_EPEL_MIRROR when finding the epel-release package  https://review.openstack.org/23767616:27
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create fuel-menu project  https://review.openstack.org/23517716:27
clarkbhrm the version we have may not flush16:27
*** jistr has quit IRC16:27
*** sridhar_ram has quit IRC16:28
*** shashank_hegde has quit IRC16:28
*** Daisy has quit IRC16:29
*** Daisy has joined #openstack-infra16:30
*** krtaylor has joined #openstack-infra16:30
fungiSpamapS: it completed while i was sleeping. compressed size (with xz at default compression level) is 14gb, uncompressed looks like just shy of 50gb16:30
mtreinishfungi: that's not too bad16:31
*** shashank_hegde has joined #openstack-infra16:31
*** sridhar_ram has joined #openstack-infra16:31
funginow i just need to figure out how to serve it up16:31
clarkbsdague: ya so the trick is getting the pending event to be flushed16:32
mtreinishclarkb: ^^^ that answers our questions about the subunit2sql backups too16:32
*** weihan has quit IRC16:32
mtreinishI think we've got 14gb free on logstash.o.o16:32
clarkbmtreinish: we do16:32
*** weihan has joined #openstack-infra16:33
clarkbsdague: it uses the stream_identity as an array index into a list of pending events. Which means if we never get an event with the same key as something pending we never flush16:33
*** dimtruck is now known as zz_dimtruck16:33
SpamapSfungi: I realized after saying 40:1 that there are 118 million uuids in there16:33
*** heyongli has quit IRC16:33
SpamapSfungi: those aren't likely to compress at all ;)16:33
*** e0ne has quit IRC16:33
fungipauloewerton: based on a review of the elastic-recheck bot logs and some additional discussion in #openstack-qa, our best guess is that the gerrit event stream reader in it is buggy and has blind spots16:33
*** aukhan has joined #openstack-infra16:33
*** jordanP has quit IRC16:33
fungineeds deeper analysis when someone has time16:33
zxiirozaro: Ok I'm back to work for a week before heading to OpenStack Summit, any JJB Gerrits I should prioritize review for this week?16:34
*** heyongli has joined #openstack-infra16:34
fungipauloewerton: but it seems to have not noticed job results posted for that change in gerrit, basically16:34
openstackgerritAlex Schultz proposed openstack-infra/project-config: Add puppet jobs to fuel-library  https://review.openstack.org/23555016:34
*** Daisy has quit IRC16:34
openstackgerritDavanum Srinivas (dims) proposed openstack/requirements: Expose extras information in requirements  https://review.openstack.org/23754416:34
anteayazaro: looks like it might be topic:gerrit-upgrade https://review.openstack.org/#/c/172534/16:34
*** kozhukalov_ has quit IRC16:34
*** arxcruz has quit IRC16:35
anteayazaro: I am offline for lunch and more celebration of the election, hoping to be home in time for the meeting16:35
SpamapSgreghaynes: crinkle pabelanger I just remembered that we have this https://storyboard.openstack.org/#!/story/200017516:36
*** Swami has joined #openstack-infra16:36
SpamapSand it is probably woefully out of date16:36
crinkleSpamapS: yes, and it's sort of hard to figure out what from that is still relevant16:37
pauloewertonfungi, I see. I thought that maybe it had something to do with the change being submitted to a feature branch16:37
SpamapScrinkle: indeed16:37
*** mrmartin has quit IRC16:37
pauloewertonfungi, thank you for your help o/16:37
SpamapSWell, "Write manifest for one node deployment cloud" is still relevant16:37
*** weihan has quit IRC16:37
crinkleSpamapS: I think we skipped over that into writing a multinode deployment cloud16:38
crinkleSpamapS: https://review.openstack.org/231631 and https://review.openstack.org/23163216:38
SpamapScrinkle: ok, I'd count your bifrost work as that. Would you say it is done?16:38
zaroanteaya: yes, topic will be gerrit-upgrade but i'm looking for the Gerrit search for all priority efforts.  not sure what happened to that but i can't seem to find it anymore16:39
crinkleSpamapS: I need to test it again but it is basically done16:39
zaromaybe fungi knows?^^16:39
crinkleSpamapS: I rolled some bits from https://review.openstack.org/231632 into https://review.openstack.org/23175716:39
*** shashank_hegde has quit IRC16:40
SpamapScrinkle: ok, I redid that taks to be pointed at you and status of Review16:40
SpamapScrinkle: so now it is just "write manifest for deployment cloud"16:40
crinkleSpamapS: cool16:40
SpamapSYou know..16:40
SpamapSI really don't like not having direct 1:1 review<->task16:40
*** amuller_afk is now known as amuller16:40
*** achanda has joined #openstack-infra16:41
openstackgerritMehdi Abaakouk (sileht) proposed openstack-infra/project-config: heat: ceilometer-alarm is deprecated  https://review.openstack.org/23768316:41
openstackgerritMehdi Abaakouk (sileht) proposed openstack-infra/project-config: rally: ceilometer-alarm is deprecated  https://review.openstack.org/23768416:41
openstackgerritMehdi Abaakouk (sileht) proposed openstack-infra/project-config: heat: remove legacy functional jobs  https://review.openstack.org/23768516:41
*** hdd has quit IRC16:41
*** AJaeger has joined #openstack-infra16:41
*** markus_z has quit IRC16:41
zarozxiiro: welcome back for a short bit, i think pelix is keen to land this change.  https://review.openstack.org/18046316:42
*** rkukura has joined #openstack-infra16:42
SpamapScrinkle: are you working on the script to move vlan config onto bridge/veths, or is greghaynes ?16:42
crinkleSpamapS: I am not working on it16:43
*** heyongli has quit IRC16:43
*** pradk has joined #openstack-infra16:43
zarozxiiro: i've reviewed it before the last few rebases. looks good, just need someone to test it if you are interested to tackle that :)16:43
*** heyongli has joined #openstack-infra16:44
fungizaro: the priority efforts search is linked at http://specs.openstack.org/openstack-infra/infra-specs/16:44
*** blogan is now known as testudp16:44
openstackgerritClark Boylan proposed openstack-infra/system-config: Special case ansible logs  https://review.openstack.org/23768616:44
*** testudp is now known as blogan16:44
clarkbsdague: ^ something like that maybe, not tested with logstash but tested with http://grokconstructor.appspot.com/do/match#result16:44
crinkleSpamapS: though actually i added half of it in puppet, which should be removed when the script exists https://review.openstack.org/#/c/231631/3/manifests/veth.pp16:45
crinkleSpamapS: that doesn't include creating the bridge or the vlan interface though16:45
zxiirozaro: cool. I'll add it to my todo list for this week. I'll see if I can run it at least once.16:45
*** derekh has quit IRC16:46
*** kozhukalov_ has joined #openstack-infra16:47
*** unicell has quit IRC16:50
*** trown is now known as trown|lunch16:50
openstackgerritIhar Hrachyshka proposed openstack-infra/project-config: Gate neutron-lbaas on -minimal job  https://review.openstack.org/23464316:50
*** stevemar_ has quit IRC16:52
*** flepied has joined #openstack-infra16:52
*** heyongli has quit IRC16:53
*** fawadkhaliq has joined #openstack-infra16:54
*** heyongli has joined #openstack-infra16:54
openstackgerritMonty Taylor proposed openstack-infra/shade: Make designate record methods private for now  https://review.openstack.org/23769216:55
mordredShrews: ^^16:56
*** jsavak has quit IRC16:57
*** zz_dimtruck is now known as dimtruck16:57
sdagueclarkb: cool16:57
*** bswartz has joined #openstack-infra16:57
*** jsavak has joined #openstack-infra16:58
*** aukhan has quit IRC16:58
*** aukhan has joined #openstack-infra16:58
*** fawadkhaliq has quit IRC16:58
*** sergmelikyan has joined #openstack-infra16:58
*** ivar-lazzaro has joined #openstack-infra16:58
*** stevemar_ has joined #openstack-infra16:59
*** dramalho has quit IRC16:59
*** achanda has quit IRC17:00
openstackgerritSebastian Schuberth proposed openstack-infra/jenkins-job-builder: cmd: Add a fallback for jenkins_jobs.ini to the user's home directory  https://review.openstack.org/23769417:00
*** claudiub has quit IRC17:01
*** ZZelle has joined #openstack-infra17:01
*** shashank_hegde has joined #openstack-infra17:01
*** vilobhmm11 has quit IRC17:02
fungitimrc: sdague: i should have dug into zuul's logs sooner... 2015-10-20 06:48:34,900 ERROR zuul.Scheduler: Exception reporting runtime stats17:02
SpamapSgreghaynes: ok, so you're working on the veth/vlan init script, yes?17:02
*** yamamoto has quit IRC17:02
*** dizquierdo has quit IRC17:02
fungitimrc: sdague: https://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/scheduler.py#n562 is trying to iterate over build.node_labels but it's coming up as a NoneType17:03
clarkbsdague: I have also confirmed that my test of ansible 2.0 doesn't have a different log output format17:03
clarkbsdague: so we should be fine with that rule after the upcoming ansible release17:03
*** dtardivel_ has quit IRC17:03
*** heyongli has quit IRC17:04
*** heyongli has joined #openstack-infra17:04
*** aukhan has quit IRC17:05
*** aukhan has joined #openstack-infra17:05
sdagueclarkb: sounds good to me17:05
sdagueclarkb: thanks for looking at it17:05
*** dzamboni has joined #openstack-infra17:06
fungitimrc: sdague: oh... i wonder if https://review.openstack.org/205097 is missing on the jenkins masters17:08
openstackgerritSebastian Schuberth proposed openstack-infra/jenkins-job-builder: builder: Fix hashing arbitrary job configurations  https://review.openstack.org/23769917:08
*** hasharMeeting has quit IRC17:09
* timrc reads up17:10
timrcfungi: I do not think the new gearman-plugin is deployed.17:11
fungitimrc: it is not. i just confirmed17:11
timrcfungi: I had a review up to add it to jenkins_dev for testing as per zaro's recommendation.17:11
*** kozhukalov_ has quit IRC17:11
fungitimrc: so anyway, the fallback to set that to [] in zuul seems to not be happening17:11
timrcfungi: So I should create a patch for zuul that defaults to [] and not NoneType if the labels are missing.17:11
timrcfungi: (hivemind)17:12
fungitimrc: yep17:12
fungitimrc: once we have that i can do a zuul restart with the fix in place17:12
timrcfungi: fwiw: https://review.openstack.org/#/c/236019/17:12
fungiand hopefully we'll start getting job stats again after that17:12
timrcfungi: Cool.  Let me work on that patch then.17:12
*** achanda has joined #openstack-infra17:13
fungithanks timrc!!!17:13
*** heyongli has quit IRC17:14
openstackgerritKhai Do proposed openstack-infra/infra-specs: add gerrit-upgrade to priority efforts query  https://review.openstack.org/23770117:14
nibalizerfungi: what is our procedure for a zuul restart? just the zuul-changes/zuul-enqueue stuff from here? http://docs.openstack.org/infra/zuul/zuul.html#starting-zuul17:14
*** mahatic_ has quit IRC17:14
*** nmagnezi has joined #openstack-infra17:15
*** jsavak has quit IRC17:15
*** heyongli has joined #openstack-infra17:15
*** mahatic has joined #openstack-infra17:15
funginibalizer: yup17:15
* nibalizer nods17:15
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create fuel-menu project  https://review.openstack.org/23517717:15
nibalizerI can do it if you want to do more interesting things17:15
clarkbyou can do a safe shutdown but it requires all queues to quiesce which takes forever17:16
funginibalizer: or wait until a really dead time and so a graceful restart (and cross your fingers it works)17:16
nibalizerafter we get tims patch17:16
fungiyeah, that17:16
*** dramalho has joined #openstack-infra17:16
*** dkranz has joined #openstack-infra17:16
funginibalizer: you're awesome. i'll leave it to you while i get back to prepping for meetings and adding details to our summit schedule items17:16
nibalizerdo you do the #notice thing when you bounce zuul like this? or just hope no one noticees?17:17
*** nadya has joined #openstack-infra17:17
jeblairit's close to functionally equivalent to a gate reset, so we don't usually #notice it unless it will impact people17:18
*** boris-42 has quit IRC17:18
*** annegentle has joined #openstack-infra17:19
*** trown|lunch is now known as trown17:19
*** fhubik has quit IRC17:19
jeblairclarkb, sdague, mordred: apparently in ovh we have dedicated compute hosts (i guess our flavor is tied to them?).  the perf characteristics we observed are related to that -- we oertaxed our oversubscribed hosts; and i think the 2-tier performnace was probably different hosts (at different levels of usage)17:20
jeblairwe have a less-oversubscribed flavor now17:20
*** unicell has joined #openstack-infra17:20
sdaguejeblair: the probably put us in a host aggregate17:21
jeblairi have to fly now, but we have next steps at least17:22
clarkbwe probably doubled up the use of the hypervisors when we doubled the size of the flavor17:22
mordredjeblair: ah17:23
*** salv-orlando has joined #openstack-infra17:24
*** salv-orlando has quit IRC17:24
*** heyongli has quit IRC17:24
*** mahatic has quit IRC17:24
*** salv-orlando has joined #openstack-infra17:25
*** austin81 has joined #openstack-infra17:25
*** heyongli has joined #openstack-infra17:25
openstackgerritMerged openstack-infra/shade: Rename designate domains to zones  https://review.openstack.org/23717217:25
openstackgerritMerged openstack-infra/shade: Rename identity_domain to domain  https://review.openstack.org/23717717:25
openstackgerritTimothy R. Chavez proposed openstack-infra/zuul: Use default if gearman-plugin None for node_labels  https://review.openstack.org/23770317:25
timrcfungi: https://review.openstack.org/#/c/237703/17:26
*** kozhukalov_ has joined #openstack-infra17:26
*** dims__ has quit IRC17:27
fungiclarkb: nibalizer: a second review ^?17:28
*** dims_ has joined #openstack-infra17:28
*** Sukhdev has joined #openstack-infra17:29
*** tonytan4ever has quit IRC17:30
*** ujuc has quit IRC17:30
clarkbfungi: aroo? Sorry I am reviewing the glance image upload spec now17:30
*** dkranz has quit IRC17:30
*** harlowja has quit IRC17:30
*** ociuhandu has quit IRC17:31
*** dkranz has joined #openstack-infra17:31
fungiclarkb: no worries. this was just addressing the problem where we haven't been sending job stats to graphite since saturday's maintenance17:31
*** aukhan has quit IRC17:34
*** aukhan has joined #openstack-infra17:34
*** heyongli has quit IRC17:34
greghaynesclarkb: image upload spec?17:35
clarkbtimrc: fungi nibalizer why not []17:35
greghaynesclarkb: that sounds relavent to my interests17:35
clarkbtimrc: otherwise we are relying on a default that may change?17:35
timrcclarkb: It is that []... the Build initializes it to []17:35
clarkbgreghaynes: https://review.openstack.org/#/c/232371/417:35
clarkbtimrc: until that changes17:35
*** heyongli has joined #openstack-infra17:35
fungiclarkb: would also be fine. i don't have an opinion on explicit vs implicit there17:35
timrcclarkb: How many onCompletes would there be?17:36
timrcThe only time that value changes is when the job finishes.17:36
*** harlowja has joined #openstack-infra17:36
clarkbtimrc: just one, but if the model changes you may run into problems again17:36
* nibalizer has no opinion17:36
clarkbmy point is we know explicitly what the value should be17:36
timrcI'm happy to do []17:36
clarkbso we should set the value explicitly17:36
timrcGive me a sec.17:36
fungithough i guess a test with the default value would be useful in either case (since it's possible the default or the expected type might change in the future)17:36
greghaynesSpamapS: Not so much - The plan was to get terrible cloud up just doing eni with hard coded interfaces via element. As of yesterday there were some hardware issues being sorted out17:36
clarkbfungi: that would be good too17:36
openstackgerritTimothy R. Chavez proposed openstack-infra/zuul: Use default if gearman-plugin None for node_labels  https://review.openstack.org/23770317:37
clarkbgreghaynes: if you refresh I just posted comments to it17:37
greghaynesSpamapS: I think they replaced a few mobos, but the ILOs were no longer pingable so \O/17:37
nibalizertimrc: now the commit message is wrong17:37
*** jsavak has joined #openstack-infra17:37
fungithird time's a charm17:37
timrcIf this is not hugely urgent I could be persuaded to write a test as part of this change.17:37
*** nadya has quit IRC17:38
*** yamahata has joined #openstack-infra17:38
clarkbtimrc: a followup would be fine too17:38
fungitimrc: it would be nice to get it merged and zuul restarted sometime todayish17:38
*** Sukhdev has quit IRC17:38
fungisince the qa team and others are starved for health stats on jobs at the moment17:38
*** tonytan4ever has joined #openstack-infra17:38
*** nadya has joined #openstack-infra17:38
*** kragniz has quit IRC17:39
kevinbentonhi, stackforge networking-bigswitch code was replaced with a commit message indicating that it's no longer maintained because someone didn't update the wiki17:39
*** achanda has quit IRC17:39
*** kragniz has joined #openstack-infra17:39
fungikevinbenton: sounds like it's abandoned? we sent multiple e-mails to the listed maintainers for it17:39
fungiover a span of months17:40
openstackgerritTimothy R. Chavez proposed openstack-infra/zuul: Use default if gearman-plugin None for node_labels  https://review.openstack.org/23770317:40
kevinbentonfungi: it's not abandoned. the force into the openstack namespace where they would be under the neutron stadium put them into an untenable situation WRT stable policies17:41
kevinbentonfungi: so they were waiting to discuss this at the summit17:41
*** jpeeler has quit IRC17:41
AJaegerkevinbenton: openstack namespace != under neutron17:41
fungikevinbenton: the namespace move is unrelated to governance. it was just a cosmetic change so that repos switching to official status later didn't need separate renames17:41
kevinbentonfungi: ah, that was the confusion then17:42
*** sabeen3 has quit IRC17:42
openstackgerritMerged openstack-infra/project-config: Add a ceilometer-functional job for mysql  https://review.openstack.org/23578417:42
kevinbentonfungi: can it be moved to openstack then?17:42
fungikevinbenton: the announcements and e-mail notifications explained that, and also mentioned the deadline (which was before the summit obviously)17:42
*** nmagnezi has quit IRC17:42
*** sshnaidm has quit IRC17:42
* AJaeger wonders why nobody spoke up before the deadline...17:42
*** baoli_ has quit IRC17:43
fungikevinbenton: we can, though we didn't discuss how we're going to handle getting the stragglers moved later if there are more like this one17:43
*** jpeeler has joined #openstack-infra17:43
*** jpeeler has quit IRC17:43
*** jpeeler has joined #openstack-infra17:43
AJaegerdhellmann, dims_: Is it time to merge the trove liberty switch to post-versioning? It passes now: https://review.openstack.org/#/c/235149/17:44
kevinbentonthe email went to a gerrit email address which i think most were filtering because of CI noise17:44
*** sridhar_ram has quit IRC17:44
dims_AJaeger: i don't know the intricacies :) so would wait on dhellmann17:44
fungikevinbenton: it's a good thing we also announced it broadly on mailing lists, and more than once17:45
*** heyongli has quit IRC17:45
fungi(in case people were filtering their personal e-mail addresses)17:45
kevinbentonfungi: yeah, it's a good thing there isn't much email traffic on the mailing lists :)17:45
*** heyongli has joined #openstack-infra17:45
*** amotoki has quit IRC17:46
fungikevinbenton: participating in the openstack community is hard, i get it. not sure what we could have done that we didn't already do17:46
fungimultiple prominent announcements (including mailing lists, cross-project meetings, a mention in the weekly newsletter at least once) and also direct e-mails just in case all the announcements went unnoticed17:47
*** achanda has joined #openstack-infra17:47
*** shashank_hegde has quit IRC17:48
*** aukhan has quit IRC17:48
kevinbentonfungi: thank you for explaining all of the things i did wrong17:48
kevinbentonis there any way that this can be fixed?17:48
*** nadya has quit IRC17:48
*** bnemec has quit IRC17:48
fungikevinbenton: yes, i did say we can fix it, we just haven't discussed how we're going to handle people who missed all the notices (frankly we didn't expect any, and you're the first to my knowledge)17:49
AJaegerclarkb: do you want to merge https://review.openstack.org/#/c/234510/ now? It was approved before the depends-on was approved. I waited with another +A to give you time to monitor any fallout. Do you want to +A or shall I? title is "Add grenade to PROJECTS list in grenade jobs"17:50
kevinbentonto clarify, not unnoticed. the issue was that it was carefully read enough. we didn't realize the project would be replaced with the 'no longer maintained' graffiti and we didn't realize that 'openstack/' networking projects didn't have to be under neutron17:50
kevinbentonnot carefully*17:50
*** vilobhmm11 has joined #openstack-infra17:51
kevinbentonthe commit freeze was not an issue because we had the summit coming up to figure out if it should live under neutron17:51
AJaegerfungi: What about adding this as topic for the infra meeting - and have kevinbenton add a section in the wiki with the project mentioned.17:51
clarkbAJaeger: I can approve it17:51
fungikevinbenton: understood. the unmaintained/retirement commit on them was to make sure people didn't continue to rely on software which had been abandoned by its authors17:51
*** weshay_mtg is now known as weshay17:51
fungiAJaeger: yeah, i expect we'll treat it like a normal rename, but in the meantime we need a commit to restore the previos acl so the core reviewers on it can approve a change reverting the last commit17:52
fungier, previous acl17:52
fungii'm working on the acl change right now17:53
fungiacl and jobs that is17:53
nibalizertimrc: failed in the gate?17:54
aspiersinfra-root: please could someone add me to these two -core and -release groups on gerrit? https://review.openstack.org/#/admin/projects/openstack/openstack-resource-agents,access17:54
nibalizeraspiers: i got ya'17:54
*** pradk has quit IRC17:54
*** heyongli has quit IRC17:55
nibalizeraspiers: what is the email you use with gerrit?17:55
timrcnibalizer: Looking17:55
aspiersaspiers@suse.com, username adam.spiers17:55
AJaegerwow, we have around 100 open reviews for project-config - and I guess more than 70 have a -1 by Jenkins, probably merge conflicts. More than 50 older than 6 days...17:55
aspiersAJaeger: I'm guessing the resort caused a lot of conflicts17:56
fungiyep, my expectation as well17:56
aspiersAJaeger: it did for my review anyway17:56
*** heyongli has joined #openstack-infra17:56
nibalizeraspiers: done!17:56
aspiersnibalizer: awesome, many thanks!17:56
fungiAJaeger: did you have a change proposed to delete all the stackforge repos from zuul/layout.yaml? i notice most are still in there17:57
AJaegeraspiers: it did - but the -1 is older than 6 days, so before the resort17:57
AJaegerfungi: I didn't sign up for that task and didn't notice yet.17:57
* timrc scratches head and zuul test17:58
AJaegerfungi: shall I do it?17:58
fungiAJaeger: not trying to assign work! just thought that seemed like something you probably would have done while working on the resort17:58
openstackgerritMerged openstack-infra/project-config: Add grenade to PROJECTS list in grenade jobs  https://review.openstack.org/23451017:58
fungii was starting to add back the networking-bigswitch jobs and noticed they were still there17:58
AJaegerfungi: I probably should but was looking elsewhere. I can take it...17:58
AJaegerfungi: I'll base my change on top of yours17:58
fungiAJaeger: it's not urgent. just want to make sure we don't delete any for this repo since it's going to get renamed in some weeks17:59
AJaegerI guess we need to cleanup jenkins/projects as well...17:59
timrcOh I think I know.17:59
fungiAJaeger: and if you already had one proposed i wanted to make sure we amended it to avoid blowing these away18:00
*** sergmelikyan has quit IRC18:00
openstackgerritMerged openstack-infra/project-config: Be explicit when installing clients from src  https://review.openstack.org/23451118:01
*** jamielennox|away is now known as jamielennox18:02
*** tiswanso has quit IRC18:02
*** yamamoto has joined #openstack-infra18:03
*** tiswanso has joined #openstack-infra18:03
*** ybathia has joined #openstack-infra18:03
*** ihrachys has quit IRC18:04
*** heyongli has quit IRC18:05
openstackgerritMerged openstack-infra/project-config: Use Barbican gate_hook in preparation for removal of extras.d  https://review.openstack.org/23453818:06
*** heyongli has joined #openstack-infra18:06
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove stackforge IRC notification  https://review.openstack.org/23771818:07
AJaegerfungi: I'll do some cleanup now like the above ^18:07
kevinbentonfungi, AJaeger: i added an item to open discussion for the meeting. is it still needed?18:07
openstackgerritJeremy Stanley proposed openstack-infra/project-config: Unretire the stackforge/networking-bigswitch repo  https://review.openstack.org/23772018:08
fungikevinbenton: AJaeger: ^18:08
*** yamamoto has quit IRC18:08
fungii have a revert of the head commit for that repo prepared to send as soon as manage-projects applies the acl there18:08
timrcfungi: nibalizer So I dunno why zuul failed.. tox locally ran successfully18:09
*** notnownikki has quit IRC18:09
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Enable configurable uris in uncategorized_fails.py  https://review.openstack.org/23504418:09
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Enable configurable uris in graph.py  https://review.openstack.org/23617118:09
timrcShould I just attempt a recheck and light a candle and say prayer?18:09
AJaegerfungi: LGTM.18:09
kevinbentonfungi: thanks. want me to pull that off the meeting agenda then?18:09
fungikevinbenton: i think what AJaeger was suggesting was adding it to the project renames list at the bottom of our agenda18:10
nibalizertimrc: sgtm18:11
*** achanda has quit IRC18:11
kevinbentonfungi: ack18:11
*** dprince has quit IRC18:11
fungitimrc: i'm trying to spot why/what failed there18:11
*** ociuhandu has joined #openstack-infra18:12
fungiprocess-returncode ended up with a 14218:12
fungiand also test_client_enqueue_ref went sideways oddly18:13
fungieven if it's not related to the proposed change, it would be good to get a handle on what's possibly nondeterministic in those tests18:13
fungior whether they're being caused by some sort of setup error which didn't get caught and short-circuit the job18:14
*** jpeeler has quit IRC18:15
*** jpeeler has joined #openstack-infra18:15
*** jpeeler has joined #openstack-infra18:15
*** heyongli has quit IRC18:15
*** heyongli has joined #openstack-infra18:16
*** marzif has joined #openstack-infra18:16
*** shashank_hegde has joined #openstack-infra18:17
stevemar_fungi: who should i bug about removing a feature branch in keystoneclient?18:17
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: WIP: Stackforge retirement: Job cleanup  https://review.openstack.org/23772718:17
*** sabeen has joined #openstack-infra18:18
fungistevemar_: infra or release managers. which branch? i can delete it now18:18
*** kozhukalov_ has quit IRC18:18
*** Sukhdev has joined #openstack-infra18:18
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Enable configurable uris in uncategorized_fails.py  https://review.openstack.org/23504418:19
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Enable configurable uris in graph.py  https://review.openstack.org/23617118:19
stevemar_fungi: this one https://review.openstack.org/#/q/status:open+project:openstack/python-keystoneclient+branch:feature/keystoneauth_integration,n,z18:19
*** Sukhdev has quit IRC18:19
*** Sukhdev has joined #openstack-infra18:20
*** Mmike has quit IRC18:20
*** Mmike has joined #openstack-infra18:20
fungistevemar_: you have a few open changes on that branch still which will need to be abandoned first18:21
*** ayoung has joined #openstack-infra18:21
stevemar_fungi: i'll abandon them18:21
fungistevemar_: let me know once that's done and i'll delete the branch for you18:21
stevemar_fungi: should be good now18:22
*** vilobhmm11 has left #openstack-infra18:22
openstackgerritMerged openstack-infra/zuul: Use default if gearman-plugin None for node_labels  https://review.openstack.org/23770318:22
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Create hidden row for md5 hash  https://review.openstack.org/23772918:23
*** marzif has quit IRC18:23
pabelangertimrc: ^ came up with that idea to better compare dashboards with grafyaml18:23
*** Sukhdev has quit IRC18:24
openstackgerritMerged openstack-infra/project-config: Unretire the stackforge/networking-bigswitch repo  https://review.openstack.org/23772018:24
*** heyongli has quit IRC18:26
timrcpabelanger: I'll look soon.  Will add to review queue18:26
fungistevemar_: done. final state of feature/keystoneauth_integration was commit 4e498a54d0034b2ce5c87130f080ff580d241600 prior to deletion18:26
*** heyongli has joined #openstack-infra18:26
timrcfungi: So.. a little scary :) https://review.openstack.org/#/c/237703/ but that merged after the recheck18:26
*** nadya has joined #openstack-infra18:28
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Create hidden row for md5 hash  https://review.openstack.org/23772918:28
*** nadya has quit IRC18:28
*** dprince has joined #openstack-infra18:28
*** marzif has joined #openstack-infra18:29
*** kmartin has quit IRC18:31
*** pvaneck has joined #openstack-infra18:31
pabelangeryolanda: do you mind adding https://review.openstack.org/#/c/225228/ to your list too18:32
AJaegerfungi, anteaya: Why do we have gerrit/acls/openstack/swiftsync.config in project-config? That looks wrong, isn't it?18:33
*** achanda has joined #openstack-infra18:33
AJaegeranteaya: argh, that's wrong for ages...18:34
AJaegerfungi, anteaya : Found my answer, sorry for the noise18:34
*** heyongli has quit IRC18:36
*** heyongli has joined #openstack-infra18:37
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Stackforge retirement: Job cleanup  https://review.openstack.org/23772718:37
AJaegerfungi, here's the big cleanup ^18:37
AJaeger"3 files changed, 875 deletions(-)"18:37
fungiAJaeger: wow--nice!18:38
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove unused gerrit ACLs  https://review.openstack.org/23773218:39
*** achanda has quit IRC18:39
AJaegerfungi: and one more cleanup ^18:39
* AJaeger run "git grep" over all removed repositories, so I hope I got all18:40
AJaegerOnly noise was puppet-openstack ;)18:40
*** xyz has joined #openstack-infra18:40
*** xyz is now known as Guest8965218:41
*** sergmelikyan has joined #openstack-infra18:41
*** sridhar_ram has joined #openstack-infra18:44
*** fawadkhaliq has joined #openstack-infra18:44
Guest89652I just signed up to be an openstack contributor.  When I ran "git review -v", it ran happily and then asked me to "review your contact information" at  https://review.openstack.org/#/settings/contact.  I did just that and hit the "save changes" button.  Then the browser shows "server error.  Cannot save contact information".18:45
*** dramalho has quit IRC18:45
pabelangernibalizer: you got time spare cycles to help with grafana.o.o puppet issue?18:45
fungiGuest89652: were you following the instructions at http://docs.openstack.org/infra/manual/developers.html#account-setup or something else?18:45
fungiGuest89652: that error is usually a sign that the foundation membership step was skipped18:46
Guest89652I think I have signed up.  But I'll check.  Thx.18:46
*** heyongli has quit IRC18:46
fungiGuest89652: some additional troubleshooting suggestions for that error are at https://ask.openstack.org/question/5672018:46
AJaegerinfra-root, I would appreciate some reviews for 237718 and 237727 - part of stackforge retirement. These are merge-conflict potential changes, so let's keep the window small before everybody rebased, ;) please18:46
AJaegerWith those changes Jenkins jobs go down from 7419 to 705218:47
*** heyongli has joined #openstack-infra18:47
mordredfungi: I have slow derpy l=plane wifi - could you add a topic about hound onto the meeting today?18:47
fungimordred: you bet18:48
*** dramalho has joined #openstack-infra18:48
*** achanda has joined #openstack-infra18:48
mordredfungi: thanks!18:49
nibalizerpabelanger: after lunch sure18:51
*** bhuvan has joined #openstack-infra18:51
anteayaAJaeger: thank you18:51
openstackgerritAlexey Shtokolov proposed openstack-infra/project-config: Enabled python27 and pep8 gates and fixed checks for Fuel repos  https://review.openstack.org/23606818:51
anteayazaro: oh I think I understand what you are looking for now18:52
anteayazaro: I'm going to have to think where to find that or compose a new one18:52
*** pradk has joined #openstack-infra18:52
openstackgerritAlexey Shtokolov proposed openstack-infra/project-config: Enabled python27 and pep8 gates and fixed checks for Fuel repos  https://review.openstack.org/23606818:52
AJaegeranteaya, fungi: If there's further cleanup to do, feel free to point it out.18:53
*** harlowja has quit IRC18:53
zaroanteaya: fungi pointed me to it, it's in infra-specs18:53
anteayaAJaeger: I'm sure you will spot it before I will, but will do if I beat you to one18:53
anteayazaro: awesome18:53
AJaegeranteaya: since I was travelling last weekend I missed all the fun and fungi needed to point out the obvious to me18:55
fungikevinbenton: https://review.openstack.org/237741 is the retirement revert for your repo18:56
*** heyongli has quit IRC18:56
fungiinfra-root: protip... you can't apply an acl to a project in gerrit without manually switching it from read-only to active first. manage-projects was failing until i did that18:57
*** heyongli has joined #openstack-infra18:57
AJaegermorning, jhesketh !18:57
*** HeOS has quit IRC18:58
mordredfungi: nice18:59
anteayaAJaeger: you pick up the obvious quickly :)18:59
anteayamorning jhesketh18:59
*** rkukura has quit IRC19:00
anteayafungi: good to know19:00
fungimeeting time!19:00
anteayakevinbenton: I asked about bigswitch19:00
Guest89652fungi: I'm a community member of openstack.  Do I have to be a foundation member?19:00
anteayakevinbenton: you know there is no way I wanted to retire a project you wanted, right?19:00
fungiGuest89652: yes19:00
Guest89652alright, thanks.19:01
*** jsavak has quit IRC19:04
*** jsavak has joined #openstack-infra19:04
*** shardy has quit IRC19:04
openstackgerritSam Yaple proposed openstack-infra/project-config: Add gates for deploying Kolla  https://review.openstack.org/23774519:04
*** jcoufal_ has quit IRC19:05
AJaegerkevinbenton: can you propose a proper rename switch to move your repo to openstack? Check the project-config repo for examples on how this was done in the past.19:05
*** gyee has quit IRC19:05
*** shardy has joined #openstack-infra19:06
*** heyongli has quit IRC19:07
*** heyongli has joined #openstack-infra19:07
*** rguillebert has quit IRC19:08
*** armax has quit IRC19:10
openstackgerritSam Yaple proposed openstack-infra/project-config: Add gates for deploying Kolla  https://review.openstack.org/23774519:10
*** harlowja has joined #openstack-infra19:11
*** jsavak has quit IRC19:13
*** jsavak has joined #openstack-infra19:13
Guest89652fungi: I can't apply for foundation membership (email in use).  So how do I switch from community membership to foundation?19:13
openstackgerritIan Wienand proposed openstack-infra/project-config: Remove dib-lint flake8 requirement  https://review.openstack.org/23502019:14
fungiGuest89652: there's a button in your profile, as pictured in the screenshot at https://ask.openstack.org/question/5672019:14
*** megm_ has quit IRC19:15
*** megm has joined #openstack-infra19:15
kevinbentonanteaya: sorry. i dropped the ball by failing to read the warning closely. i thought it would just go read-only which they were fine with while they figured out their Neutron stadium strategy19:16
kevinbentonanteaya: but then the change to show that it was an unmaintained project forced things to pick up the pace19:16
kevinbentonAJaeger: yep, i'll do that19:16
*** heyongli has quit IRC19:17
anteayakevinbenton: okay I don't know when our next rename window is scheduled19:17
anteayakevinbenton: :(19:17
anteayakevinbenton: next time can you ping just to be sure, saves a ton of work19:17
*** heyongli has joined #openstack-infra19:18
*** abetterlie_ has joined #openstack-infra19:19
*** bdemers has quit IRC19:19
*** bdemers has joined #openstack-infra19:20
Guest89652fungi: found the way.  Also saw your reply.  Thanks!19:20
*** jsavak has quit IRC19:22
*** salv-orlando has quit IRC19:23
openstackgerritKevin Benton proposed openstack-infra/project-config: Move networking-bigswitch from stackforge to openstack  https://review.openstack.org/23775319:23
*** achanda has quit IRC19:23
kevinbentonAJaeger: ^^ does that look right?19:23
*** bdemers has quit IRC19:23
kevinbentonanteaya: definitely will watch the infra notifications more closely next time :)19:24
AJaegeranteaya: that's for you ^ ;)19:24
anteayaAJaeger: we both can help :) but sure can take this one19:25
openstackgerritKevin Benton proposed openstack-infra/project-config: Move networking-bigswitch from stackforge to openstack  https://review.openstack.org/23775319:25
anteayakevinbenton: I'm just a ping away19:25
kevinbentonanteaya: you'll be in tokyo, right?19:26
anteayakevinbenton: yes19:26
*** qba73 has joined #openstack-infra19:26
anteayakevinbenton: let's have a drink19:26
*** bdemers has joined #openstack-infra19:26
kevinbentonanteaya: sounds like a plan!19:26
*** tiswanso has quit IRC19:26
anteayakevinbenton: thank you, sorry this got missed, it will be a while before we have another window to change it back19:26
anteayakevinbenton: just so you know there will be a wait19:27
kevinbentonanteaya: will the stackforge project be unblocked so they can revert the change that deleted everything?19:27
*** heyongli has quit IRC19:27
*** marzif has quit IRC19:27
anteayawe can fix it19:27
anteayabut it won't be fixed for a while19:27
anteayait will be after summit19:28
*** bdemers has quit IRC19:28
*** qba73 has quit IRC19:28
*** marzif has joined #openstack-infra19:28
anteayabefore the maintenance is scheduled19:28
*** heyongli has joined #openstack-infra19:28
*** jsavak has joined #openstack-infra19:28
kevinbentonis it possible for someone with commit rights to at least revert that change so it doesn't look like an abandoned project?19:28
*** tiswanso has joined #openstack-infra19:29
kevinbentonor is it frozen for everyone?19:29
anteayait is frozen19:29
anteayathat is the point19:29
anteayaand why we worked on this for 3 months19:29
anteayaand sent posts to the mailing list19:29
anteayaand individually emailed maintainters twice19:29
anteayaand why I asked in -neutron the wednesday prior to the rename19:30
*** dangers_away is now known as dangers19:30
*** bdemers has joined #openstack-infra19:30
AJaegeranteaya: I understand fungi that he unfroze the repo, let's wait for him to chime in after the infra meeting19:32
anteayaAJaeger: oh19:32
anteayawell then that is news to me19:32
anteayaand I haven't read all the backscroll19:32
anteayakevinbenton: well I may be wrong here then19:33
*** hdd has joined #openstack-infra19:33
AJaegerkevinbenton: please add a link to the review to https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Upcoming_Project_Renames19:34
kevinbentonanteaya: ok. i'll stay tuned19:34
*** tonytan4ever has quit IRC19:34
anteayakevinbenton: okay thanks19:34
angdraugAJaeger: can you have a look again: https://review.openstack.org/235550 -- mwhahaha has fixed it up19:34
kevinbentonAJaeger: done19:34
openstackgerritJamie Lennox proposed openstack/requirements: Update upper keystoneauth to latest version  https://review.openstack.org/23775619:35
*** dtardivel has quit IRC19:35
AJaegerangdraug: please get https://review.openstack.org/#/c/235582/ reviewed and merged first19:36
AJaegerangdraug: I'll look at it again later or tomorrow19:36
*** bpokorny has quit IRC19:36
openstackgerritAlex Schultz proposed openstack-infra/project-config: Add puppet jobs to fuel-library  https://review.openstack.org/23555019:37
*** heyongli has quit IRC19:37
AJaegerangdraug: is the dependency wrong?19:38
angdraugremoved the Depends-On for https://review.openstack.org/235582 -- no longer depends since it's non-voting now19:38
AJaegerangdraug: ah...19:38
AJaegerangdraug: that's cheating ;)19:38
*** rkukura has joined #openstack-infra19:38
angdraugno, that just makes it easier for us to test https://review.openstack.org/235582 :)19:38
*** heyongli has joined #openstack-infra19:38
*** annegentle has quit IRC19:39
*** rguillebert has joined #openstack-infra19:40
*** annegentle has joined #openstack-infra19:40
*** nadya has joined #openstack-infra19:40
*** dprince has quit IRC19:42
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Create hidden row for md5 hash  https://review.openstack.org/23772919:43
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources: Summit Application API  https://review.openstack.org/22196419:44
*** pradk has quit IRC19:46
angdraugpretty please? :)19:46
mwhahahagotta wait for CI19:46
mwhahahawhich just popped through :o19:47
*** claudiub has joined #openstack-infra19:47
*** heyongli has quit IRC19:48
AJaegerianw: you have some old changes like 175625, 179329, 175626 in the review queue for project-config. Could you either rework or abandon them, please?19:48
*** heyongli has joined #openstack-infra19:48
*** armax has joined #openstack-infra19:49
*** EricGonc_ has quit IRC19:50
*** mrmartin has joined #openstack-infra19:51
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources: Summit Application API  https://review.openstack.org/22196419:53
*** erikwilson has joined #openstack-infra19:55
*** rkukura has quit IRC19:56
*** shardy has quit IRC19:56
openstackgerritMerged openstack-infra/shade: Make designate record methods private for now  https://review.openstack.org/23769219:57
*** rkukura has joined #openstack-infra19:57
*** shardy has joined #openstack-infra19:58
*** heyongli has quit IRC19:58
*** Piet has joined #openstack-infra19:58
*** fawadkhaliq has quit IRC19:58
*** heyongli has joined #openstack-infra19:59
anteayanibalizer: can you review these for the gerrit work? https://review.openstack.org/#/q/status:open+project:openstack-infra/gerrit+branch:openstack/2.11.4+topic:gerrit-upgrade,n,z19:59
openstackgerritDoug Hellmann proposed openstack-infra/release-tools: make intermediary.sh support stable branches  https://review.openstack.org/23759520:00
openstackgerritKevin Benton proposed openstack-infra/project-config: Move networking-bigswitch from stackforge to openstack  https://review.openstack.org/23775320:00
*** notnownikki has joined #openstack-infra20:00
*** HeOS has joined #openstack-infra20:01
*** mnencia has quit IRC20:01
*** sridhar_ram1 has joined #openstack-infra20:01
taronah, I'd forgotten the power of dumping your thoughts in a text file while debugging20:02
* taron finds a whole bunch of at least questionable, if not faulty logic while poking around hound again20:02
anteayafungi: can you share with kevinbenton the current status of networking-bigswitch?20:02
*** boris-42 has joined #openstack-infra20:02
anteayafungi: sorry I haven't read all the backscroll20:02
anteayataron: yay!20:02
fungitaron: aka rubber duck debugging20:03
*** bnemec has joined #openstack-infra20:03
*** sridhar_ram has quit IRC20:03
taronI knew there was a term for it I was forgetting20:03
fungiit's surprisingly useful20:03
fungiwell, technically it's explaining your problems to a fictional or inanimate conversant20:03
fungibut a text file is a close proxy20:03
nibalizeranteaya: when i get back to my main pc, yes20:03
anteayakevinbenton: did you ever have -bigswitch report patches submitted and merged to any of the channels?20:03
anteayanibalizer: thanks20:03
nibalizerin transit at the moment20:04
anteayanibalizer: /me nods20:04
anteayanibalizer: from somewhere where clarkb was I'm guessing20:04
kevinbentonanteaya: no20:04
anteayakevinbenton: okay thank you20:04
nibalizeryep. infratacolunch20:04
openstackgerritMonty Taylor proposed openstack-infra/shade: Set cache information from clouds.yaml  https://review.openstack.org/23705020:04
fungikevinbenton: so i see from my proposed https://review.openstack.org/237741 change that your unit tests have bit-rotted20:04
anteayanibalizer: nice20:04
nibalizera pdx openstack tradition20:04
anteayanibalizer: I'm going to have to time a visit20:05
taronlos gorditos?20:05
fungikevinbenton: we should trim those down so gate-networking-bigswitch-python27 only runs in the check pipeline and is non-voting20:05
fungikevinbenton: that'll allow the revert to merge and whatever else until you can fix your unit tests20:05
*** salv-orlando has joined #openstack-infra20:05
*** jsavak has quit IRC20:06
*** jsavak has joined #openstack-infra20:06
fungianteaya: did you want to help with that? ^20:06
*** bpokorny has joined #openstack-infra20:06
mordredShrews: ^ I figured out what was breaking in that nodepool-shade-src job20:06
kevinbentonfungi: ah, the wonders of depending on unstable neutron internals20:06
mordredshardy: it was that the upper-constraints file hadn't been updated20:06
kevinbentonfungi: either that or i could ammend your commit with a fix for those20:06
fungikevinbenton: if it's easy, then go for it20:07
kevinbentonfungi: sure, i think it's just an import statement. one sec20:07
zaroyolanda: i'm interested about your kubertnetes solution for running ci.20:07
anteayafungi: I can try, sure20:07
AJaegerfungi, before changing layout.yaml for kevinbenton , let's merge https://review.openstack.org/237727, please ;)20:07
anteayafungi: so right now the repo is unfrozen?20:08
taronnibalizer: https://review.openstack.org/#/c/216549/7/modules/openstack_project/manifests/codesearch.pp would I be correct in assuming that since this exec is set to refreshonly and has nothing to notify it it'll never run?20:08
fungianteaya: well, sounds like kevinbenton may have a solution20:08
anteayafungi: kevinbenton is so smart20:08
fungianteaya: yeah, it's been unfrozen since before the infra meeting20:08
anteayafungi: ah okay thanks, was confused20:08
*** heyongli has quit IRC20:08
taronnibalizer: because if so, well, no wonder it's breaking, there's been no config.json the whole time :P20:09
openstackgerritPaul Belanger proposed openstack-infra/grafyaml: Check for existing md5 hash in dashboard  https://review.openstack.org/23776420:09
openstackgerritMerged openstack-infra/shade: Create neutron floating ips with server info  https://review.openstack.org/23540220:09
*** heyongli has joined #openstack-infra20:09
openstackgerritMerged openstack-infra/shade: Make a few IP methods private  https://review.openstack.org/23702020:09
AJaegerkevinbenton: you need to sort *all* files - https://review.openstack.org/#/c/237753/3/zuul/layout.yaml is not sorted properly...20:10
*** adam_g has quit IRC20:10
AJaegerthanks for the reviews, anteaya !20:11
anteayakevinbenton: you'll never get past AJaeger without things in alphabetical order20:11
anteayaAJaeger: :)20:11
fungii'm going to step out and get a bite to eat momentarily, since there's no tc meeting to skulk around in and no cross-project meeting i need to be around for. let me know if there's anything else urgent, otherwise i can help with things in a couple hours20:11
anteayakevinbenton: he let us turn it off for Saturday so I don't think we can ask again for about 2 years20:11
anteayafungi: enjoy!20:11
AJaegeranteaya: we have it automated, just wanted to help him debugging ;)20:13
*** mrmartin has quit IRC20:13
AJaegerfungi: enjoy!20:13
anteayaAJaeger: yup20:13
openstackgerritBruno Tavares proposed openstack-infra/puppet-httpd: Fixes vhost and vhost::proxy on Precise.  https://review.openstack.org/23776520:14
*** adam_g has joined #openstack-infra20:14
*** adam_g has quit IRC20:14
*** adam_g has joined #openstack-infra20:14
openstackgerritSam Yaple proposed openstack/requirements: Block requests 2.8.1  https://review.openstack.org/23776620:15
*** ldnunes has quit IRC20:15
*** rfolco_ has quit IRC20:15
mtreinishinfra-root: if someone gets a second creating a server for: http://git.openstack.org/cgit/openstack-infra/system-config/commit/?id=05c16e2e35077260b21e286a8eb0948c085750c7 would unblock the deployment side of that20:16
*** rfolco_ has joined #openstack-infra20:16
mtreinishit'd be awesome if we could get that running before the summit session next week20:16
clarkbanteaya: yes we decided it was a taco day but meeting from taco grocery store didn't work so great20:16
anteayaclarkb: ah20:17
anteayaclarkb: and you have no phone currently20:17
anteayaclarkb: nibalizer was a good proxy20:17
anteayaclarkb: were tacos good/20:17
* AJaeger waves good night...20:17
clarkbyes they were excellent20:17
clarkbAJaeger: good night20:17
anteayaAJaeger: night20:17
anteayaAJaeger: thank you20:17
anteayaclarkb: wonderful20:17
*** AJaeger has quit IRC20:17
mtreinishclarkb: wait, there's a taco grocery store? do they only sell taco ingredients?20:18
* mtreinish remebers it is portland20:18
*** kgiusti has left #openstack-infra20:18
anteayaa grocery store that also serves tacos?20:18
mordredit's also probably a food truck20:18
*** heyongli has quit IRC20:18
clarkbanteaya: ya you go into a grocery store, walk through the back to a place with a counter and bunch of plastic chairs and tables20:19
mtreinishmordred: haha, that's very likely20:19
anteayamtreinish: I think you might have to find someone to take that on20:19
anteayaclarkb: nice20:19
zaroseattle has em.  usually pretty good too20:19
anteayamtreinish: who most needs a favour from you?20:19
anteayazaro: what is em?20:19
*** heyongli has joined #openstack-infra20:19
zarocanadian don't use em?20:20
anteayamtreinish: hey I would ask nibalizer as practice for him20:20
anteayamtreinish: try that angle20:20
nibalizersaw that comming20:20
mtreinishanteaya: that works for me20:20
mtreinishnibalizer: want to create the health.o.o server?20:21
anteayanibalizer: ha ha ha, new guy gets all the tasks20:21
clarkbits good learning20:21
anteayazaro: yes, often I see it as 'em20:21
clarkbnibalizer: I can walk through it and the zuul thing if you want20:21
clarkbonce your laundry is started20:21
anteayaha ha ha20:21
anteayalaundry first20:21
zaromy bad, i can see how you would be confused20:22
nibalizertacos and laundry20:22
anteayazaro: not at all, glad I understand now :)20:22
anteayanibalizer: nice20:22
*** marzif has quit IRC20:22
nibalizeri have no secrets from infra20:22
zaronibalizer: i thought i heard you moved? but still in portland?20:22
anteayaI'm so glad I work with a team that has its priorities figured out20:22
anteayanibalizer: good approach20:22
mtreinishanteaya: tacos being the priority right?20:23
*** smyers has left #openstack-infra20:23
anteayamtreinish: of course20:24
*** achanda has joined #openstack-infra20:24
anteayayou always have to eat first20:24
anteayayou can order and load landry in the washer20:24
anteayathen get it started and eat20:24
anteayathen into the dryer as you digest20:24
anteayadepends on the line up for food and for washers20:25
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Enable configurable uris in uncategorized_fails.py  https://review.openstack.org/23504420:25
openstackgerritRamy Asselin proposed openstack-infra/elastic-recheck: Enable configurable uris in graph.py  https://review.openstack.org/23617120:25
*** marzif has joined #openstack-infra20:25
openstackgerritKevin Benton proposed openstack-infra/project-config: Move networking-bigswitch from stackforge to openstack  https://review.openstack.org/23775320:25
*** achanda has quit IRC20:25
*** pc_m has quit IRC20:26
clarkbianw: +2 on https://review.openstack.org/#/c/237389/20:27
clarkbis an easy one and nothing uses it yet so best to just try20:27
*** kozhukalov_ has joined #openstack-infra20:27
*** tonytan4ever has joined #openstack-infra20:28
*** hdd has quit IRC20:28
ianwclarkb: thanks, yep as usual i'm sure there's something i missed, but am around to debug it20:28
*** heyongli has quit IRC20:29
*** skylerberg has joined #openstack-infra20:29
anteayakevinbenton: I have to grab something to eat, let me know if you can get your unit tests passing20:29
dims_anteaya: talking about food. found any good vegetarian places for tokyo?20:29
*** heyongli has joined #openstack-infra20:29
anteayadims_: not so far20:30
anteayadims_: I understand the benito flakes might be an issue20:30
dims_ah i see20:30
anteayamy plan thus far is to stay close to mtreinish20:30
dims_++ :)20:30
anteayaand have him assign me someone who lives in japan as a guide my first night20:30
kevinbentonanteaya: ack. i think they're happy now20:30
anteayaand then if I have to eat at the same restaurant all week, I have no problem with that20:30
anteayakevinbenton: awesome thank you20:31
anteayadims_: I'm under obligation to go to the fish market in tokyo and I don't want to go20:31
anteayadims_: I'm going to have to find a proxy for taht task20:31
anteayadims_: I'm to see the fish market20:31
mtreinishdims_: I know a decent indian restaurant so I've got that meal covered20:31
anteayaI don't want to see it20:31
anteayamtreinish: yay20:31
dims_mtreinish: nice, what's the name?20:31
anteayadims_: when are you there?20:31
dims_anteaya: sunday evening20:32
mtreinishdims_: dipmahal20:32
anteayadims_: cool I am going to be at the tc/board meeting20:32
anteayadims_: let's see what develops for dinner that night20:32
dims_mtreinish: cool. will look up20:32
dims_anteaya: ++20:32
nibalizerok im actually back now20:32
anteayadims_: great, I just have to check if I have you on text, I thinnk I do20:33
nibalizerzaro: i moved inside portland20:33
mtreinishdims_: google isn't pointing it to the right place, I'll dig up a link20:33
*** marzif has quit IRC20:33
mtreinishdims_: https://goo.gl/maps/gLJjw1tmECk20:34
*** baoli has joined #openstack-infra20:34
dims_mtreinish:  thanks20:34
mtreinishdims_: there might be good ones closer to the summit. I just know that one because it's not far from where I've stayed before20:35
anteayameh I'd go there20:36
*** moravec1 has quit IRC20:36
*** moravec has joined #openstack-infra20:36
*** bpokorny has quit IRC20:37
openstackgerritPaul Belanger proposed openstack-infra/project-config: Match on dsvm-fedora-22 and dsvm-f21 nodes  https://review.openstack.org/23777120:38
*** sdake has joined #openstack-infra20:38
sdakehey, is it possible that someone in the infra core team could attend a kolla gating session20:38
sdakeat summit20:39
sdakewe have a big ask of openstack infra team20:39
*** heyongli has quit IRC20:39
pabelangerianw: nibalizer ^ do you mind?  That will fix our gate-infra-puppet-apply-dsvm-fedora-22 NOT_REGISTERED issue20:39
clarkbsdake: we do tend to spend lots of our time in other folk ssessions helping with things like that. Can you link the session and maybe give us a heads up on what youexpect to happen (link to etherpad or agenda is good for that)20:39
timrcfungi: What is the curses slide deck software you used for your OSCON 2015 talk called again?20:39
pabelangerianw: nibalizer https://review.openstack.org/#/c/231681/20:39
sdakethe etherpad isn't filled out yet20:39
clarkbtimrc: presentty20:39
sdakebut here is the session - sec20:40
timrcclarkb: Thanks20:40
nibalizerzaro: anteaya gerrit upgrade revviews done20:40
*** heyongli has joined #openstack-infra20:40
sdakethursday at 2:4020:41
zaronibalizer: thanks!20:41
nibalizertimrc: presentty?20:41
sdakebasically clarkb we want mirrors of all of the external rpm repos we use20:41
sdakethat are internal to openstack so we can vote our gates20:41
*** sergmelikyan has quit IRC20:41
timrcnibalizer: Yeah, that's it.20:41
sdakeits probably 200-300gb of storage20:41
*** sergmelikyan has joined #openstack-infra20:41
timrcfungi: If you just wait long enough, questions directed to you may just get answered without you doing any of the work :)20:42
asselinHi, I'm running into an issue where elastic recheck keeps spawning new processes every time there's an update. I noticed that there's no pid file being created. would an infra root be able to see if upstream has the same issue? /var/run/elastic-recheck/elastic-recheck.pid20:42
sdakenot concerned with fast updates, once eveyr couple of days sync is good enough20:42
nibalizeri found that the current pip version of presentty is unable to pip install because of pillow issues20:42
ianwpabelanger: cool, yeah i think as we retire f21 we should move things to "fedora"20:42
nibalizeror maybe it can't install from git20:42
nibalizeror maybe jeblair fixed it20:42
pabelangerianw: Ya, fedora-22, fedora-23, etc20:42
timrcnibalizer: Requires a little Pillo talk.  That's all.20:42
pabelangerianw: that ubuntu-trusty, centos-7, fedora-2320:42
*** gyee has joined #openstack-infra20:43
nibalizerpabelanger: +220:43
clarkbsdake: distro mirrors are a thing  that various groups hvae wanted. I think its mostly a matter of somoene doing the work20:43
anteayanibalizer: thank you!20:43
sdakeclarkb the problem is the usptream repos fail and then our gates fail, which means we can never make our gates voting, which means our gates are always busted20:43
clarkbsdake: write the puppet to run the mirror software, figure out how much cinder volume is required for it, then babysit it when the indexes break because distro mirroring seems to do that20:43
clarkbsdake: sure I am saying we just need someone to do the work20:43
sdakeclarkb if I can find someone to do the work, infra can sponsor the effort financially?20:44
clarkbits been a thing that we have said ya lets do that its just lo wenough priority because the current setup sort of mostly works (I think fedora is actually the only one that fails consistently)20:44
sdakesorry wrong wording20:44
sdakei mean resource wise20:44
sdakenot people resource hardware resource20:44
*** notnownikki has quit IRC20:44
clarkbsdake: yes we already run mirror hosts, we just need to have puppet configure the distro mirror and need to know how much disk you need upfront. Mordred would be ebetter person to ask as he has thought about this problem space a lot more20:45
pabelangerianw: want to +1 workflow too?20:45
clarkbsdake: we will probably also ask that you don't limit yourself to rpms/fedora20:45
sdaguezaro: neat about the gerrit hash tags, what is the state in 2.11? (you said they weren't fully baked)20:45
pabelangerianw: now that we have 2 +220:45
clarkbsdake: if you want to do the owrk we will want it for ubuntu too20:45
sdakeclarkb oh ya we want deb too sorry didnt' mention that20:45
anteayaand I think krotscheck has already stood up an npm mirror for something20:45
*** skylerberg has quit IRC20:45
ianwpabelanger: i'll just wait for jenkins20:45
clarkbanteaya: ya its mostly done I think we just need to redo the apache setup for pypi so they can live side by side20:45
pabelangerianw: Perfect20:46
anteayaclarkb: on the same server you mean?20:46
clarkbanteaya: yes20:46
anteayacool, thanks20:46
sdakeclarkb i dont know a thing about puppet but I can see if I can find someone that does :)20:46
*** bswartz has quit IRC20:46
sdakeand after that I can babysit the mirrors for our stuff20:46
clarkbanteaya: so we would hvae http://mirror.dfw.openstack.org/pypi and http://mirror.dfw.openstack.org/npm20:46
*** baoli has quit IRC20:46
clarkbanteaya: whereas right no / is pypi20:47
sdakei'll speak to mordred, but it sounds like what we are after is in the realm of possibility20:47
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources: Summit Application API  https://review.openstack.org/22196420:47
anteayaclarkb: which 404 right now20:47
clarkbs/right no/right now/20:47
*** baoli has joined #openstack-infra20:47
anteayaclarkb: ah yeah, okay cool, thanks for explaining20:47
clarkbsdake: also at least in rax you can use their local mirror20:47
*** achanda has joined #openstack-infra20:47
clarkbif you want something for now20:47
sdakeclarkb the mirrors are for things like rdo, ceph, etc20:48
openstackgerritDavid Shrewsbury proposed openstack-infra/shade: Normalize user information  https://review.openstack.org/23777420:48
sdakethe distro mirrors work well enough for us already20:48
clarkbsdake: I don't think we would want to make super specialized mirrors20:49
*** sergmelikyan has quit IRC20:49
sdakeits the things that dont hae any mirrors at all20:49
clarkbsdake: we would just want generic distro mirrors20:49
clarkbsdake: oh you mean pulling from rdo repo itself?20:49
sdakethe problem is alot of mirrors we need are not in distro20:49
*** baoli has quit IRC20:49
sdakeclarkb roger we do20:49
*** heyongli has quit IRC20:49
sdakeclarkb half our gates install every bit of files in the rdo repos20:49
*** baoli has joined #openstack-infra20:49
sdakeand we install all the ceph files20:50
sdakeand a few other non-distro mirrors20:50
sdakethat are properly licensed20:50
SamYapledoes rdo package ceph sdake?20:50
*** sergmelikyan has joined #openstack-infra20:50
*** heyongli has joined #openstack-infra20:50
sdakerdo does not package ceph20:50
SamYaplei know cloud-archive does for ubuntu20:50
clarkbsdake: fwiw the mirrors would be expected to mirror the distros20:50
nibalizerokay im getting ready to do the zuul restart clarkb fungi20:50
openstackgerritSean Dague proposed openstack-infra/devstack-gate: use cp always instead of mv / cp logic  https://review.openstack.org/23690120:50
nibalizerw/ zuul-changes do I have to get queues other than gate and check? http://docs.openstack.org/infra/zuul/zuul.html#starting-zuul20:51
clarkbsdake: adding extra stuff probably isn't a problem too, but we defnitely want that base20:51
sdakei'm good with mirroring all of rdo, is that what you mean?20:51
clarkbnibalizer: I normally just do check and gate20:51
sdagueclarkb: hey, so how bad would it be to just do the cp all the time in d-g for cache as proposed here - https://review.openstack.org/23690120:51
sdakeor you mean all of epel ?20:51
clarkbsdague: no I mean all of fedora20:51
*** dkranz has quit IRC20:51
clarkbfor x86_6420:51
sdakeok well that is a much bigger disk chunk but i'd be good with that20:51
sdakeif we could mirror *everything* we depend on I'd love that20:52
sdakethat way if the mirror breaks, and our gate breaks, we can fix  it in one place20:52
clarkbsdake: any idea how big f22 x86_64 is?20:52
sdakewell we use centos20:52
sdakebut I am not sure how large the mirror is no20:52
clarkbsdake: ok centos7 x86_64 then20:52
nibalizerokay, got the queues20:52
nibalizerstoppign zuul20:52
sdakeI would expect very large however20:52
sdake200gb perhaps20:53
*** sridhar_ram1 has quit IRC20:53
sdakebut just  atotal random guess20:53
SamYaplesdake: just the binary side, i dont think its that large20:53
sdakei can find out if it would help20:53
*** baoli has quit IRC20:53
*** achanda has quit IRC20:53
greghaynessdake: clarkb http://dl.fedoraproject.org/pub/DIRECTORY_SIZES.txt20:53
SamYapleclarkb: just to be clear, does this mean you are also good with mirroring ubuntu-trusty?20:53
*** pauloewerton has quit IRC20:53
*** eharney has quit IRC20:53
SamYaplethis will affect the work20:53
*** baoli has joined #openstack-infra20:53
clarkbsdake: well we just need to proprely size the volume ahead of time20:53
sdakeclarkb ack, so there isn't reallly a cost issue on the disk size then?20:54
clarkbSamYaple: yes I think if we mirroed anything we would want to do it properly for ubuntu and centos/fedora20:54
SamYaplesounds good to me20:54
clarkbsdake: I mean they are donated resources so we try to us them properly20:54
*** rfolco_ has quit IRC20:54
sdakeyes we want to mirror atleast ubuntu and centos20:54
sdakeand i'd be good with fedora too20:54
*** baoli has quit IRC20:55
*** sergmelikyan has quit IRC20:55
greghaynesFWIW dib *really* wants fedora mirros because that is about 90% of our failures20:55
*** sergmelikyan has joined #openstack-infra20:55
SamYapleclarkb: looks like ubuntu is 777GB for a full mirror...20:55
SamYaplebut thats probably source packages too20:55
nibalizerokay its back on, but the 'enqueue' commands are not working20:55
sdakefedora 20 looks like 90gb20:55
clarkbSamYaple: we don't want fuul20:56
*** baoli has joined #openstack-infra20:56
clarkbSamYaple: we are x86_64 only20:56
clarkband we only use 1 release ish20:56
clarkbsame with cnetos we wouldn't care about 6 only 720:56
SamYapleoh sure. let me figure that out for x86_64 and no source20:56
clarkband for fedora we wouldn't care about 21 just 22 etc20:56
nibalizernvm I think I figured it out20:56
*** eharney has joined #openstack-infra20:56
clarkbnibalizer: oyu may need sudo powers20:56
*** sridhar_ram has joined #openstack-infra20:57
nibalizerya, zuul needed its configs20:57
nibalizerand -c /etc/zuul/zuul.conf didn't work20:57
sdakeclarkb right I was just using it as a guess on size20:57
clarkbnibalizer: also, generally we do gate before check but not a big deal20:57
sdakeso about 100gb for fedora20:57
*** baoli has quit IRC20:57
nibalizerstill running, but i see things appearing in the status page20:57
clarkbsdake: according to the link greghaynes posted f22 is 24GB20:57
*** alivigni has quit IRC20:58
clarkbsdague: looking20:58
sdakefor updates and x86_64?20:58
*** baoli has joined #openstack-infra20:58
*** dramalho has quit IRC20:58
clarkboh there is an everything path thats 72GB20:59
clarkbso 72GB for f2220:59
*** bdemers has quit IRC20:59
sdakeeverything isn't the updates sdir tho20:59
sdakethats separate i think20:59
sdakeso its everything + updates20:59
SamYapleyea i dont think 72gb is right20:59
*** heyongli has quit IRC20:59
clarkbwow I don't think fedora knows how english works20:59
sdakeI could be wrong :)21:00
sdakewouldn't be first or last time21:00
jesusauruswhy did zuul need a restart?21:00
clarkbjesusaurus: pick up a bugfix21:00
*** sergmelikyan has quit IRC21:00
*** baoli has quit IRC21:00
*** sergmelikyan has joined #openstack-infra21:00
*** heyongli has joined #openstack-infra21:00
*** baoli has joined #openstack-infra21:01
clarkbsdague: ya  Ithink it was for space reasons, we would run out of disk on / just doing openstack things if we didn't clean up behind ourselves21:01
sdagueclarkb: ok, but that doesn't seem to be the case atm21:01
SamYapleclarkb: im not going to guess. im going to do the mirror and du it. ill get back to you on the size21:02
sdagueespecially now that we don't go and slice off the lvm partition21:02
clarkbsdague: the lvm partition was always on a different device anyways21:02
clarkbsdague: the problem is that / runs out21:02
clarkbSamYaple: cool21:02
sdagueclarkb: ok, so / is where /opt is?21:02
*** dzamboni has quit IRC21:03
clarkbsdague: no /opt is remounted ephemeral drive if it exists and / is the root device of the VM21:03
SamYaplenvm clarkb apt-mirror is clever. 82.6 GiB will be downloaded into archive.21:03
sdagueclarkb: ah, so the concern is to delete the cache after we're good?21:03
clarkbsdague: yes21:03
clarkbsdague: so that myslq and other things not using /opt are happy21:03
*** weshay has quit IRC21:03
*** regXboi has quit IRC21:03
SamYapleok did we get an exact number on centos 7?21:04
*** burgerk_ has quit IRC21:04
SamYapleubuntu is ~83GB21:04
sdagueclarkb: ok, has this failed like this any time in recent memory?21:04
sdaguebecause this did pass all the tests21:04
clarkbsdague: no because we clean up the disk properly21:04
sdakeapevec says rdo is about 600mb, so i think with 5 gb of storage there we wouldn't run out anytime soon21:04
*** aeng has joined #openstack-infra21:04
sdagueclarkb: right, but when I moved to a straight cp, it passes all the tests as well21:04
sdakeclarkb so the process should be for us to produce a list of mirrors we need and disk space usage, then submit that perhaps to the mailing lsit?21:05
*** serg_melikyan has joined #openstack-infra21:05
clarkbSamYaple: sdake so if we think large and figure 120GB per distro release + fudge factor for smaller repos we would be good21:05
*** sergmelikyan has quit IRC21:05
SamYapleclarkb: that sounds very good21:05
clarkbsdake: yes but did they run on both clouds? what about OVH with only 40GB of disk total?21:05
SamYaplethe smaller repos are tiny, like 2-300mb most of the time21:05
sdakewhat is ovh?21:05
clarkbsdake: er sorry that was for sdague21:05
nibalizerits a new cloud provider21:05
clarkbtab complete fail21:06
sdakeoh got it :)21:06
sdakeget that alot21:06
*** aysyd has quit IRC21:06
sdagueclarkb: so... how about if I mv to /opt/cache early21:06
sdaguethen cp -l from there21:06
clarkbsdague: well I am trying to understand why grenade fails at all21:06
clarkbsdague: because I distinctly remember fixing grenade21:06
clarkbwe would do the cp to old then the mv to new21:06
clarkbthats why that flag exists21:06
sdagueclarkb: nope, there is some code to do that, but apparently things keep getting refactored and the flags get dropped21:07
clarkbsdague: so why not just fix that?21:07
sdaguebecause it's just going to get broken again21:07
clarkbit doesn't have to21:07
*** achanda has joined #openstack-infra21:07
clarkbsdague: I am just concerned that 2 weeks from now we will hvae a lot of flaky tests and no one will understand why21:08
clarkband its because we reverted a fix to a known problem without proper testing21:08
clarkbrather than just fixing the fix to not break21:08
clarkbSamYaple: sdake: the other thing that mordred and jeblair were interested in was maybe using AFS for the mirrors but I don't think thats a strict requirements because both pypi and npm mirrors are just local disk to the cloud region21:09
clarkbSamYaple: sdake but probably good to get their input as a result21:09
nibalizertaron: did you rubber-duck to victory or do you still need me on hound?21:09
sdakeafs is ?21:09
clarkbsdake: its like global nfs21:09
SamYaplelets let mordred work on shade.... :)21:09
nibalizerandrew filesystem21:09
sdakeoh andrew file system?21:09
nibalizerits from the before times21:10
krotschecksdake: Sorry, came late to this.21:10
*** heyongli has quit IRC21:10
sdakeyes I am from before time :)21:10
sdagueclarkb: ok, but did you understand my next fix21:10
sdaguewhich is mv to /opt/cache, so we end up off of /21:10
sdaguethen cp -l from there21:10
sdaguewhich means we can get the same results without a flag that will get lost21:10
*** heyongli has joined #openstack-infra21:11
clarkbsdague: the problem is that not all clouds have a separate disk for /opt21:11
sdaguehttps://review.openstack.org/#/c/177017/28/devstack-vm-gate-wrap.sh,cm - patch that merged in May21:11
clarkbsdague: so that will work fine for rax and hpcloud but not ovh21:11
clarkb(not sure about bluebox cloud)21:11
krotscheckDid someone already mention that we want to rename pypi.<region>.openstack.org to mirrors.<region>.openstack.org?21:11
sdagueclarkb: it will be exactly the same impact21:11
sdagueas the "fixed" code21:11
clarkbsdague: no you will have two copies under /opt21:11
sdagueno, I won't21:11
sdague*cp -l*21:11
clarkboh you want to hardlink21:12
*** baoli has quit IRC21:12
sdagueI actually tried that as the first attempt, but the second filesystem got in the way21:12
clarkbhardlinking would be fine if we mv to /opt first as that should work if separate device exists and if not21:12
*** baoli has joined #openstack-infra21:12
*** jsavak has quit IRC21:12
sdaguewell, mv it unconditionally21:12
sdaguethere is no harm in moving if it's the same fs, it will just be a rename and be fast21:13
openstackgerritCyril Roelandt proposed openstack-infra/project-config: Add a gate-dsvm-functional-py34 job to Neutron  https://review.openstack.org/23317721:14
clarkbinstead of cp -l could we just teach grenade and devstack to look at one location for the files?21:14
*** trown is now known as trown|outttypeww21:14
openstackgerritgreghaynes proposed openstack-infra/nodepool: Decouple nodepool db and config from builders  https://review.openstack.org/22239821:14
*** achanda has quit IRC21:15
*** abetterlie_ has quit IRC21:15
*** changbl has quit IRC21:15
clarkbit looks like the copycache flag was removed as part of the ansible refactor, so that was missed21:16
clarkbI am happy to just add the flag back or do a hardlink that is device aware. But I don't think we should make >1 copy if we can avoid it21:17
clarkbhardlinking has the benefit of always being one copy regardless21:17
openstackgerritSean Dague proposed openstack-infra/devstack-gate: use cp always instead of mv / cp logic  https://review.openstack.org/23690121:18
*** tiswanso has quit IRC21:18
sdagueright, hardlinking seems better for something like this21:18
*** mwagner_ has quit IRC21:19
sdagueok, so lets see if that's usable21:19
*** jamielennox is now known as jamielennox|away21:20
*** heyongli has quit IRC21:20
*** tiswanso has joined #openstack-infra21:20
*** heyongli has joined #openstack-infra21:21
clarkbSamYaple: StevenK is the person to talk to about ubuntu mirrors too21:23
clarkbsince he runs at least one set of mirrors and is ubuntuy21:23
*** skylerberg has joined #openstack-infra21:23
SamYapleyea i run one myself, but im not quite qure of the integration point with infra here21:23
openstackgerritSean Dague proposed openstack-infra/devstack-gate: use cp always instead of mv / cp logic  https://review.openstack.org/23690121:23
nibalizerclarkb: any objection or commentary on me running  https://git.openstack.org/cgit/openstack-infra/system-config/tree/launch/README to build out the openstack-health server for mtreinish21:24
sdagueclarkb: updated the commit message to reflect the latest version of the code21:24
sdakeclarkb thanks a bunch for working with us on this - means alot for the kolla community21:25
*** tiswanso has quit IRC21:25
sdakeour inability to have a voting gate really drags down our productivity21:25
clarkbSamYaple: currently we have a mirror host in each region that is configured by puppet so we would update the pupept there to add in the extra stuff21:26
clarkbSamYaple: and an infra root will hve to make sure we have enough disk21:26
clarkbbut like I said would be good to get mordred to sanity check as I know he has thought about the problem a lot more than me21:26
SamYaplewill do clarkb thanks for the infos21:26
sdakewe spend all kinds of time bisecting broken changes21:26
openstackgerritMerged openstack-infra/shade: Replace Bunch with compatible fork Munch  https://review.openstack.org/23707321:27
clarkbnibalizer: nope, the only thing I would keep in mind is that you want to make sure you boot a trusty host and I am not sure what flavor we think we need21:27
SamYapleI suppose ill finally have to dig into infra....21:27
clarkbSamYaple: start in openstack-infra/system-config/manifests/site.pp21:27
SamYaplewill do21:27
*** rossella_s has joined #openstack-infra21:27
mtreinishclarkb, nibalizer: fungi and I were briefly discussing it yesterday, we probably can start small21:27
mtreinishif we end up having load issues we can resize up to something bigger21:27
clarkbmtreinish: nibalizer cool, I would do maybe 4GB performance then21:27
openstackgerritMerged openstack-infra/project-config: Match on dsvm-fedora-22 and dsvm-f21 nodes  https://review.openstack.org/23777121:27
nibalizerso many firsts today21:28
clarkbnibalizer: that will require you set both the flavor and image explicitly21:28
*** jamielennox|away is now known as jamielennox21:28
*** nadya has quit IRC21:28
*** thorst1 has quit IRC21:29
clarkbnibalizer: feel free to run commands or whatever by me if you have questions21:30
nibalizerclarkb:  http://paste.openstack.org/show/476920/ i get that just trying to flavor-list, what gives?21:30
*** heyongli has quit IRC21:30
nibalizershould i be using a novaclient from a venv in my users homedir?21:30
taronnibalizer: I think I rubber ducked to victory but see earlier highlights in here21:31
taroneither way I probably won't need to poke you before devops21:31
*** heyongli has joined #openstack-infra21:31
clarkbnibalizer: nice I hvaen't seen that before21:31
clarkbnibalizer: let me try21:32
nibalizertaron: ok!21:32
nibalizerI didn't check your work but the logic was sound, if you have a refreshonly exec and nothing notifying it, it will never fire21:32
taronwell, I have my problem then.21:32
* taron will see how well they can write puppet whilst half awake in cs25021:33
clarkbnibalizer: the nova in /home/clarkb/venv/bin/nova works21:33
clarkbnibalizer: so I think novaclient has made a non backward compatible change for older clouds21:33
*** salv-orlando has quit IRC21:33
clarkbnibalizer: or maybe we need newer21:33
clarkblet me compare versions21:33
nibalizerill try a venv and see where I get21:33
clarkbnibalizer: ya my venv has 2.26.0 which works but 2.32.0 does not21:34
clarkbsdague: ^ fyi21:34
*** nadya has joined #openstack-infra21:34
*** lykinsb__ has quit IRC21:34
*** salv-orlando has joined #openstack-infra21:34
nibalizerclarkb: fyi openstack 1.8.0 works21:35
nibalizeraka latest on pypi21:35
*** achanda has joined #openstack-infra21:35
clarkbsdake: your d-g patch is mostly ok except the first find mv moves to the wrong path21:35
clarkber sdague ^21:35
clarkbnibalizer: what about novaclient21:35
nibalizeri have 2.32.021:36
nibalizerhaven't tried downgrading it21:36
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create network-checker project  https://review.openstack.org/23582221:38
clarkbianw: for https://review.openstack.org/#/c/236814/ isthere another change that adds bashate to the d-g job list?21:39
clarkbianw: I am not seeing it in the list of check josb21:39
pabelangerianw: might be back later tonight. Not sure yet. Want to try fedora-22 stuff for system-config21:39
*** hashar has joined #openstack-infra21:40
*** heyongli has quit IRC21:40
*** baoli has quit IRC21:41
*** baoli has joined #openstack-infra21:41
*** heyongli has joined #openstack-infra21:42
*** nadya has quit IRC21:44
*** dizquierdo has joined #openstack-infra21:44
*** bpokorny has joined #openstack-infra21:46
*** alex_xu has quit IRC21:46
*** baoli has quit IRC21:47
*** achanda has quit IRC21:47
sdakeclarkb getting not registered images on centos builds21:47
*** alex_xu has joined #openstack-infra21:48
*** baoli has joined #openstack-infra21:48
clarkbsdake: I think they need to be 'centos7'21:48
clarkbare those new jobs?21:48
sdakethey are old21:48
sdake3-4 months21:48
sdakethere was a special override for them at one point someone added21:49
sdakeI think you iirc21:49
sdakebut it could have been someone else21:49
sdakeit was a "buid-images" override21:49
sdakethe override was designed to make it honor the distro name in the job rather then guess21:49
openstackgerritgreghaynes proposed openstack-infra/nodepool: Decouple nodepool db and config from builders  https://review.openstack.org/22239821:50
sdakei suspected someone would end up breaking it :)21:50
*** dkranz has joined #openstack-infra21:50
*** heyongli has quit IRC21:51
openstackgerritgreghaynes proposed openstack-infra/nodepool: Decouple builders from nodepool instance  https://review.openstack.org/23213521:51
*** heyongli has joined #openstack-infra21:52
openstackgerritgreghaynes proposed openstack-infra/nodepool: Builders can be run on their own  https://review.openstack.org/23225121:52
*** amuller has quit IRC21:52
openstackgerritEmma Barber proposed openstack-infra/system-config: Add codesearch.o.o node and module  https://review.openstack.org/21654921:53
*** bpokorny has quit IRC21:54
*** ociuhandu has quit IRC21:54
sdakeclarkb the only thing i can find that might be relevant is 29a714741b24c1247dfaf9fd2863fce3d843860221:56
*** achanda has joined #openstack-infra21:57
sdakebut I dont know how that kolla_image_re thing works21:57
sdakenot the faintest idea :)21:57
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create network-checker project  https://review.openstack.org/23582221:57
*** stevemar_ has quit IRC21:58
*** stevemar_ has joined #openstack-infra21:58
*** edmondsw has quit IRC21:59
*** hashar has quit IRC21:59
clarkbsdake: do you have links to jobs that worked?22:01
*** heyongli has quit IRC22:01
*** Piet has quit IRC22:01
sdakeclarkb yes i can find one22:01
*** bpokorny has joined #openstack-infra22:01
*** achanda has quit IRC22:02
sdakenote the gates sdont quite work - but atleast there isnt' a non-registered error :)22:02
sdakethey fail because our gate is a leaky sieve22:02
*** heyongli has joined #openstack-infra22:02
sdake(hence the need for a mirror of our repos we depend on:)22:02
*** stevemar_ has quit IRC22:02
*** dkranz has quit IRC22:02
*** eharney has quit IRC22:03
sdakeSamYaple said he did a recheck and zuul shows the jobs running22:03
sdakemaybe enodepool went out to lunch for a short period?22:03
clarkbsdake: thats what I am beginning to think22:03
clarkbsdake: zuul was restarted which restarts gearman22:04
sdakei'll just wait then, i'll be in contct if we get stuck22:04
sdakeoh right a restart22:04
sdakethat does it :)22:04
clarkbsdake: and if nodepool didn't register centos7 build hosts prior to your patch then NOT_REGISTERED22:04
clarkbsdake: so I would try a recheck and see if that makes it better22:04
sdakeyup thanks22:04
sdakei thought maybe omeone broke that special regex logic we have22:04
sdakebut soudns like no t :)22:04
*** ociuhandu has joined #openstack-infra22:05
SamYaplejobs are going as of about 10 minutes ago after a recheck22:05
sdakethe reason we have the regex xlogic is we want to add a gate-kolla-build-images- oraclelinux22:05
SamYaplei wouldnt worry22:05
*** claudiub has quit IRC22:05
sdakeand we wan't to use centos7 for that22:05
*** doug-fish has quit IRC22:05
*** serg_melikyan has quit IRC22:05
sdakebut build oracleliux with it22:05
sdakeso we ignore the distro name in our job name22:06
anteayakevinbenton: sorry for the long delay22:06
sdakeclarkb ^^ if that makees sense22:06
anteayakevinbenton: was away yesterday, doing all the returning home things22:06
clarkbsdake: yup  Iremember that now22:06
openstackgerritMerged openstack/requirements: Updated from generate-constraints  https://review.openstack.org/23742022:06
sdakebut wit hthe renames, we didn't get around to adding oralcelinux just yet22:06
clarkband I have reread things and it should be working22:07
sdakeI guess if I wasn't so jammed for time i'd do it now22:07
clarkbzuul restart is the culprit I think22:07
sdakecool thanks22:07
anteayakevinbenton: what do you need to merge this patch? https://review.openstack.org/#/c/237741/22:07
anteayakevinbenton: who else has +2 on that repo?22:07
*** baoli has quit IRC22:08
openstackgerritgreghaynes proposed openstack-infra/nodepool: Decouple nodepool db and config from builders  https://review.openstack.org/22239822:08
*** baoli has joined #openstack-infra22:08
openstackgerritgreghaynes proposed openstack-infra/nodepool: Decouple builders from nodepool instance  https://review.openstack.org/23213522:08
funginibalizer: zuul restart went without a hitch?22:08
*** ZZelle has quit IRC22:08
openstackgerritgreghaynes proposed openstack-infra/nodepool: Builders can be run on their own  https://review.openstack.org/23225122:08
fungiare we getting job stats again?22:08
anteayakevinbenton: your rename patch looks good to me, and it is listed in the infra meeting agenda - well done, now we just have to schedule a rename window after summit22:10
clarkbfungi: it seems to have worked fine, haven't checked graphite yet and sdake did notice that centos7 jobs were NOT_REGISTERED for a bit22:10
clarkbfungi: also there is scrollback about distro mirros if you are interested22:10
clarkbfungi: also novaclient on puppetmaster is too new for rackspace22:11
anteayaclarkb fungi I'm going to request reviews on zaro's patches which create a 2.11.4 branch for him to work off of: https://review.openstack.org/#/q/status:open+project:openstack-infra/gerrit+branch:openstack/2.11.4+topic:gerrit-upgrade,n,z22:11
clarkbfungi: the novaclient (2.26.0) in my homedir venv does work22:11
nibalizerfungi: yep22:11
*** heyongli has quit IRC22:11
anteayathey already have nibalizer's +2s22:11
anteayathank you nibalizer22:11
*** achanda has joined #openstack-infra22:12
*** heyongli has joined #openstack-infra22:12
*** austin81 has left #openstack-infra22:13
openstackgerritOpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements  https://review.openstack.org/23735122:13
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create network-checker project  https://review.openstack.org/23582222:14
openstackgerritMerged openstack/diskimage-builder: Fix tests/test_functions.bash  https://review.openstack.org/22933422:15
*** ZZelle has joined #openstack-infra22:17
anteayaclarkb: thank you22:17
ianwclarkb: no, that's todo once it passes22:17
fungiclarkb: i want to say this is not the first time latest novaclient has stopped working on rackspace. what's the issue this time--do you know?22:17
clarkbfungi: microversions are not available in rax22:18
clarkbfungi: so I think its related to the old issue where it tries to list the versions andhits an api that is blocked or non existsant22:18
*** sdake has quit IRC22:18
fungiclarkb: and latest novaclient breaks anywhere with no microversion support?22:18
*** gildub has joined #openstack-infra22:18
clarkbfungi: it should probably fall back to base 2.0 api instead but I am not sure of all the specifics22:18
clarkbfungi: I guess? I only tried against rax after nibalizer ran into it22:18
*** sdake has joined #openstack-infra22:18
fungijohnthetubaguy: i know it's late there, but heads up ^22:19
fungijroll: you might be interested too, at least tangentially22:19
*** armax has quit IRC22:19
mtreinishfungi, clarkb: the microversion fixes should all be landed now22:19
mtreinishthe latest should work22:19
clarkbalso 2.32.0 is what failed22:20
fungilatest not yet released?22:20
jrollseems like novaclient should handle servers without microversion support...22:20
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create network-checker project  https://review.openstack.org/23582222:20
jrollwere nova microversions released in liberty or kilo?22:20
clarkbmtreinish: latest is what failed22:20
mtreinishfungi: latest released, sdague was in firedrill mode after the release using version discovery broke everything22:20
jroll(and why doesn't rackspace have it if it was released in kilo)22:20
clarkbwas released 27 hours ago22:20
*** rossella_s has quit IRC22:21
clarkbjroll: its a load balancer/l7 firewall isse iiec22:21
jrolloh joy22:21
* jroll assumes repose22:21
clarkbmtreinish: so I think we rebroke it22:21
*** nelsnelson has quit IRC22:21
*** heyongli has quit IRC22:21
*** asettle has joined #openstack-infra22:22
mtreinishmelwitt: ^^^ have you head about this?22:22
*** heyongli has joined #openstack-infra22:22
openstackgerritSridhar Ramaswamy proposed openstack-infra/project-config: Add tacker tarballs and python jobs  https://review.openstack.org/23560222:22
openstackgerritVladimir Kozhukalov proposed openstack-infra/project-config: Create fuel-menu project  https://review.openstack.org/23517722:23
*** skylerberg has quit IRC22:23
*** tonytan4ever has quit IRC22:23
openstackgerritMerged openstack-infra/irc-meetings: Update tripleo meeting to weekly  https://review.openstack.org/23760922:24
openstackgerritRamy Asselin proposed openstack-infra/puppet-elastic_recheck: Use the new --conf option to pass in values  https://review.openstack.org/23779622:24
clarkbjroll: nova has had the version url thing forever but classic security response is dont publish your version22:25
clarkbjroll: and everything breaks22:25
jrollclarkb: yeah, I was just surprised I guess22:25
jrolldunno why22:25
anteayatonyb: I see I am the only reviewer so far on the latest patchset for getting the skipped meetings feature into yaml2ical: https://review.openstack.org/#/c/232312/22:26
mtreinishclarkb: I wonder if this is part of the problem: https://review.openstack.org/#/c/230024/22:26
mtreinishthat was included in 2.32.0 which released recently22:27
openstackgerritRamy Asselin proposed openstack-infra/puppet-elastic_recheck: Use the new --conf option to pass in values  https://review.openstack.org/23779622:27
tonybanteaya: Yeah, this is true.  ttx is busy and he needs to at least +2 them.22:27
fungimtreinish: clarkb: where "recently" means... yesterday22:27
tonybanteaya: then there is the process of releaseing yaml2ical.22:27
anteayatonyb: okay so I guess it won't happen before this summit22:28
anteayatonyb: thank you for listening to me and for trying22:28
anteayatonyb: you are awesome22:28
anteayatonyb: well we will have them for christmas then maybe22:28
tonybanteaya: it's looking that way.22:28
mtreinishfungi: did nodepool get updated though? because if this was an issue I'm surprised we're not seeing it there22:28
anteayatonyb: yup22:28
clarkbmtreinish: no it hasnt22:29
anteayatonyb: just wanted to say how much I appreciate your work here22:29
anteayatonyb: thank you22:29
clarkbwould require nodepool patch to merge and restart the service22:29
tonybanteaya: You're very welcome.22:29
anteayatonyb: :)22:29
mtreinishclarkb: ah ok. That's probably a good policy22:29
*** jamielennox is now known as jamielennox|away22:31
*** heyongli has quit IRC22:32
*** heyongli has joined #openstack-infra22:33
melwittmtreinish: have not heard about it until now22:34
*** baoli has quit IRC22:34
* nibalizer runs launch_node 22:35
nibalizer(using novaclinet 2.26.0)22:35
clarkbnibalizer: you set trusty PVHVM image and 4GB performance flavor?22:35
*** baoli has joined #openstack-infra22:35
clarkbmelwitt: basically it looks like novaclient is looking to negotiate versions again even if the cloud doesn't allow that22:35
clarkbmelwitt: I can probably get --debug output if that will help22:35
nibalizeractually, I didn't22:35
clarkbnibalizer: :)22:36
nibalizerhttps://git.openstack.org/cgit/openstack-infra/system-config/tree/launch/README#n15 doesn't have an 'image' setting in the rackspace area22:36
clarkbnibalizer: it does in the next section22:36
nibalizerhwoever, launch_node failed on not having paramiko available to import, so I'm saved22:36
*** jongchoi has joined #openstack-infra22:36
clarkbnibalizer: so you want to use the dotfile from the first section but invoke launch_node like the second section to specify flavor and image22:36
*** jongchoi has quit IRC22:36
nibalizerIMAGE='Ubuntu 14.04 LTS (Trusty Tahr) (PVHVM)' seem right to you?22:36
clarkbI wonder why we pretend that defaults are ever what we want22:36
clarkbnibalizer: yes22:36
clarkbPVHVM is the important modifier22:37
openstackgerritHongbin Lu proposed openstack-infra/project-config: Magnum: run functional test in liberty branch  https://review.openstack.org/23779822:37
clarkbnibalizer: and flavor-list should igve you a list of flavors you want performance22:37
nibalizerits working!22:37
clarkbnibalizer: and I think 4GB22:37
nibalizerand urrlib3 loves to errror22:37
*** gordc has quit IRC22:37
clarkbwhich may actually be the default22:37
nibalizerI have a flavor set22:37
melwittclarkb: hm, okay. and 2.31.0 was working fine?22:38
nibalizerthe insecure platform setting stuff is really quite irritating22:38
clarkbmelwitt: sdague did a bunch of work to make sure some recent release worked. I will confirm 2.31.0 works or doesn22:38
clarkbnibalizer: yes22:38
*** annegentle has quit IRC22:39
clarkbmelwitt: confirmed that 2.31.0 works against rax22:39
*** dimtruck is now known as zz_dimtruck22:40
melwittclarkb: okay, thanks. looking to see what could have caused it22:40
anteayafungi: your patch merged, bigswitch lives again: http://git.openstack.org/cgit/stackforge/networking-bigswitch/tree/22:41
nibalizerpuppet running on hte new node22:41
*** heyongli has quit IRC22:42
*** bpokorny has quit IRC22:42
anteayanibalizer: \o/22:43
*** heyongli has joined #openstack-infra22:43
*** sabeen has quit IRC22:44
*** nadya has joined #openstack-infra22:45
*** ddieterl_ has quit IRC22:45
clarkbnibalizer: at the end it will print out a list of commands to run to set up dns, I personally like to ssh into the ip address first and do some initial poking around before commiting to the dns records22:46
clarkbnibalizer: simplifies cleanup if something went horribly wrong22:46
nibalizerso the dns is an api but not designate?22:46
clarkbnibalizer: correct22:46
*** baoli has quit IRC22:46
nibalizeroh wow 'rackdns rdns-create' ya thats janky22:48
nibalizerwell at least there is an api!22:48
*** sdake_ has joined #openstack-infra22:48
*** sigmavirus24 is now known as sigmavirus24_awa22:49
*** sdake has quit IRC22:49
nibalizerok dns 'created'22:49
*** nadya has quit IRC22:49
nibalizerI don't want to bust my cache when it wont work but I'll verify in a few minutes22:50
nibalizernow doing the host key dance22:50
nibalizermtreinish: errors: http://health.openstack.org/ !22:51
nibalizerbut we have them22:51
*** EricGonczer_ has joined #openstack-infra22:51
mtreinishnibalizer: cool22:51
mtreinishso that's actually a bug in openstack-health there isn't a response on /22:52
*** heyongli has quit IRC22:52
*** EricGonc_ has joined #openstack-infra22:53
mtreinishbut it doesn't look like it's setup properly: http://health.openstack.org/tests22:53
mtreinishstill returns a 40422:53
*** heyongli has joined #openstack-infra22:53
clarkbnibalizer: first plcae I would look is /var/log/apache2/error_log22:54
clarkbthen the site specific error_log22:54
*** marcusvrn_ has quit IRC22:55
openstackgerritSpencer Krum proposed openstack-infra/system-config: Use the 40char fingerprint of the puppetlabs key  https://review.openstack.org/23780222:55
*** EricGonczer_ has quit IRC22:56
clarkbnibalizer: ^ I thought we already made that change/22:56
nibalizerclarkb: thats possible?22:56
nibalizeri'm still getting the error so22:56
nibalizeri made the change22:56
*** shashank_hegde has quit IRC22:56
clarkbok I will have to look closer22:57
*** jhesketh has quit IRC23:00
*** denisra has quit IRC23:01
clarkbnibalizer: Target WSGI script not found or unable to stat: /opt/openstack-health/.venv/bin/openstack-health23:02
*** shashank_hegde has joined #openstack-infra23:02
nibalizeralso our puppet code isn't converging23:02
*** heyongli has quit IRC23:02
*** heyongli has joined #openstack-infra23:03
mtreinishclarkb: hmm, it should be installing into a venv. I didn't write that puppet, let me check23:03
*** jhesketh has joined #openstack-infra23:03
nibalizerthanks for your help today clark with zuul and launching nodes23:04
clarkbthere is a .venv there23:05
clarkbso guessing that it just didn't install right?23:05
mtreinishclarkb: yeah, from nibalizer's paste it doesn't look like the install ever got triggered23:06
openstackgerritIan Wienand proposed openstack-infra/project-config: Add bashate test to devstack-gate  https://review.openstack.org/23780323:06
*** salv-orlando has quit IRC23:06
*** yamahata__ has quit IRC23:11
mtreinishnibalizer: hmm, from that paste it doesn't look like it cloned properly23:11
*** alex_xu has quit IRC23:11
mtreinishbzr wasn't the right choice :)23:11
*** isq has joined #openstack-infra23:12
clarkbthe repo dir did exist23:12
*** alex_xu has joined #openstack-infra23:12
melwittclarkb: what error do you see from novaclient when you run it against rax? I think the patch mtreinish linked earlier is probably responsible but I don't yet see how23:13
*** heyongli has quit IRC23:13
mtreinishclarkb: hmm, then I dunno23:13
clarkbmelwitt: ERROR (UnsupportedVersion): Server doesn't support microversions23:13
*** heyongli has joined #openstack-infra23:14
*** bpokorny has joined #openstack-infra23:14
clarkbI can paste a traceback if yo uwant it too23:15
melwittclarkb: okay, that's what I expected. and I see why that change caused it23:15
*** xyang1 has quit IRC23:15
melwittclarkb: yeah, we need to open a bug about this anyway. can put the trace in there23:16
clarkbmelwitt: ok I will start that23:17
mtreinishclarkb, nibalizer: actually the install in the venv says it's running, it's an exec I didn't realize that23:17
openstackgerritMerged openstack-infra/devstack-gate: Remove jeepyb and pypi-mirror from PROJECTS  https://review.openstack.org/23550623:17
melwittclarkb: thanks23:18
mtreinishalthough that probably should have a subscribe on the vcsrepo to trigger re-installing on new commits. But, that's probably a follow on fix23:18
clarkbmelwitt: https://bugs.launchpad.net/python-novaclient/+bug/150824423:19
openstackLaunchpad bug 1508244 in python-novaclient "Version 2.32.0 doesn't work against rackspace" [Undecided,New]23:19
openstackgerritgreghaynes proposed openstack-infra/nodepool: Add dib deletion test  https://review.openstack.org/21527523:21
openstackgerritgreghaynes proposed openstack-infra/nodepool: Trigger image delete using gearman  https://review.openstack.org/19811423:21
openstackgerritgreghaynes proposed openstack-infra/nodepool: Trigger image uploads using gearman  https://review.openstack.org/19772223:21
openstackgerritgreghaynes proposed openstack-infra/nodepool: Builders can be run on their own  https://review.openstack.org/23225123:21
openstackgerritgreghaynes proposed openstack-infra/nodepool: Decouple builders from nodepool instance  https://review.openstack.org/23213523:21
openstackgerritgreghaynes proposed openstack-infra/nodepool: Make secure.conf optional and backwards-compat  https://review.openstack.org/23280023:21
openstackgerritgreghaynes proposed openstack-infra/nodepool: Move nodepool config loading into module  https://review.openstack.org/23159223:21
openstackgerritgreghaynes proposed openstack-infra/nodepool: Trigger image builds using gearman  https://review.openstack.org/19517523:21
openstackgerritgreghaynes proposed openstack-infra/nodepool: Decouple nodepool db and config from builders  https://review.openstack.org/22239823:21
*** armax has joined #openstack-infra23:22
*** heyongli has quit IRC23:23
*** heyongli has joined #openstack-infra23:24
*** EricGonc_ has quit IRC23:25
*** EricGonczer_ has joined #openstack-infra23:26
openstackgerritSridhar Ramaswamy proposed openstack-infra/project-config: Add tacker tarballs and python jobs  https://review.openstack.org/23560223:26
*** yamamoto has joined #openstack-infra23:28
sridhar_raminfra-team: trying line up things to push pypi packages for tacker project. This will be our first time pushing pkg. I'm looking for the minimal recipe required to do this. I gave an attempt using https://review.openstack.org/#/c/235602/ but I'm not 100%. Looking for your guidance & help. Thanks!23:29
mordredclarkb, melwitt: should we just ban 2.32.0 in our things for now?23:29
nibalizermtreinish: 'using bzr' its just puppet being confused, it can be ignored23:30
nibalizervery old bug23:30
mordrednibalizer: aren't we switching to a git module at some point?23:31
*** angdraug has quit IRC23:31
mtreinishnibalizer: yeah, clarkb said the repo was there23:32
crinklethe upstream vcsrepo fixed that bug btw23:32
mtreinishnibalizer: do you have any ideas on what is going wrong?23:32
nibalizermordred: already started: https://review.openstack.org/#/c/235389/23:32
*** jamielennox|away is now known as jamielennox23:32
mordredcinerama: :)23:33
mordredcrinkle: :)23:33
openstackgerritMonty Taylor proposed openstack-infra/shade: Support private address override in inventory  https://review.openstack.org/23780923:33
openstackgerritMonty Taylor proposed openstack-infra/shade: Use OCC to create clouds in inventory  https://review.openstack.org/23781023:33
mordredcinerama: you can also have a :)23:33
*** heyongli has quit IRC23:33
*** heyongli has joined #openstack-infra23:34
*** yamamoto has quit IRC23:34
nibalizermordred: looking23:34
*** jaypipes has quit IRC23:36
*** ZZelle has quit IRC23:37
melwittmordred: I wasn't sure if we should revert and release a new version vs ban 2.32.023:37
openstackgerritMonty Taylor proposed openstack-infra/shade: Support private address override in inventory  https://review.openstack.org/23780923:37
nibalizerso yea we're refenrecening a wsgi file that doesn't exist23:37
nibalizerI don't see where in the module it sets up the wsgi script23:37
*** Sukhdev has joined #openstack-infra23:38
mtreinishnibalizer: which file?23:38
nibalizer  WSGIScriptAlias / /opt/openstack-health/.venv/bin/openstack-health23:38
nibalizerthat file doesn't exist23:38
pabelangerany core have some spare time to troubleshoot grafana.o.o and see if there are puppet errors?23:38
nibalizermaybe it was supposed to be created by pip install?23:38
mtreinishnibalizer: that should be installed when the python package gets installed23:38
mordredmelwitt: good question. I may put a !2.32.0 in shade anyway just on general principle23:39
mtreinishopenstack-health is the wsgi_script entrypoint that pbr will automagically create for us: http://git.openstack.org/cgit/openstack/openstack-health/tree/setup.cfg#n3023:39
openstackgerritMonty Taylor proposed openstack-infra/shade: python-novaclient does not work against rackspace  https://review.openstack.org/23781123:40
*** skylerberg has joined #openstack-infra23:40
openstackgerritIvan Udovichenko proposed openstack-infra/project-config: Adds the 1st initial package for upstream infra packaging  https://review.openstack.org/21629223:40
nibalizerNotice: /Stage[main]/Openstack_health::Api/Exec[package-application]/returns: executed successfully23:40
nibalizerlet me try the pip install manually23:41
*** bpokorny has quit IRC23:41
*** fitoduarte has joined #openstack-infra23:42
nibalizerah I think its currently23:42
nibalizer  WSGIScriptAlias / /opt/openstack-health/.venv/bin/openstack-health23:42
nibalizerand it should be23:42
nibalizer  WSGIScriptAlias / /opt/openstack-health/.venv/bin/openstack-health23:42
nibalizerer /opt/openstack-health/.venv/bin/openstack-health-api23:42
*** dizquierdo has quit IRC23:42
melwittmordred: okay. I'll post on the ML about it to get some input about a revert23:42
tlbrcould you please merge https://review.openstack.org/#/c/216292 we rebased it too many times23:43
mtreinishnibalizer: openstack-health-api is something different23:43
tlbri'm just getting desperate23:43
nibalizeroh :(23:43
mtreinishit's just for local debugging, it's not a real wsgi script23:43
mtreinishnibalizer: oh, what is the pbr version in the venv?23:43
*** heyongli has quit IRC23:43
mtreinishI wonder if pbr is too old, so it doesn't know the wsgi_scripts stuff23:44
*** heyongli has joined #openstack-infra23:44
pabelangerCannot remember, are updates to zuul/openstack_functions.py realtime or does zuul need to be restarted?23:44
*** dims_ has quit IRC23:45
mordredI would now like to announce that all of the old OpenStack modules in ansible have been derecated and replaced with new shade-based ones23:45
nibalizermordred: looks like 1.8.123:45
pabelangerAh crap23:45
pabelangerI know why23:45
*** dims has joined #openstack-infra23:46
openstackgerritPaul Belanger proposed openstack-infra/system-config: Run puppet-apply tests with fedora 22  https://review.openstack.org/23168123:46
mtreinishnibalizer: hmm that's the latest23:46
pabelangerokay, maybe not23:46
mtreinishI guess it's not that23:46
nibalizermtreinish: i verified locally that pip install of the head of openstakc-health produces the openstack-health file in bin23:46
*** camunoz_gone is now known as camunoz23:46
pabelangerianw: seems zuul thinks gate-infra-puppet-apply-dsvm-fedora-22 still not registered\23:47
pabelangercan only think the update for openstack_functions.py is not live yet23:47
ianwdid we all miss somthing in the regex ...?23:47
mtreinishnibalizer: :/ I wonder why it's not working in that venv23:48
ianwpabelanger: or, there was a restart, maybe test got hit by that (https://review.openstack.org/#/c/236901/ had issues with centos7 too)23:48
pabelangerianw: hmm, curious to see what zuul logs report23:49
mtreinishnibalizer: nuke the venv and try again? :)23:49
nibalizerso I just made a new venv, as my nibz user23:50
nibalizerand installed from /opt/openstack-health and it didn't produce the wsgi script23:50
nibalizerwhats the official name? wsgi script? wsgi entrypoint?23:50
openstackgerritDave McCowan proposed openstack-infra/project-config: Add Project Marshal  https://review.openstack.org/23453023:50
openstackgerritSridhar Ramaswamy proposed openstack-infra/project-config: Add tacker tarballs and python jobs  https://review.openstack.org/23560223:51
*** sridhar_ram has quit IRC23:51
mtreinishnibalizer: dunno, I call it the wsgi_script entrypoint23:51
mtreinishnibalizer: hmm, is the repo up to date?23:51
*** thorst has joined #openstack-infra23:52
mtreinishnibalizer: the entrypoint was added a while ago: http://git.openstack.org/cgit/openstack/openstack-health/commit/?id=0dcbd6e583df294e690a013c1788968e58e3082c23:52
nibalizerya so I just ran git clone openstack-health; pip install openstack-health on my machine and it produced the wsgi entrypoint23:53
nibalizerthen I ran the same set of commands on the health node (as my user) and it didn't produce the wsgi entrypoin23:54
nibalizerwhich is pretty confusing23:54
*** heyongli has quit IRC23:54
mtreinishnibalizer: yeah, that is bizarre23:54
nibalizercan you try, and give me another datapoint?23:54
mtreinishnibalizer: check the pip version in the virtualenv maybe?23:54
*** heyongli has joined #openstack-infra23:55
mtreinishI'm wondering if the venv is using the ubuntu package which will be older everything23:55
mtreinishnibalizer: it works locally for me, I've been playing with things locally for a couple of weeks23:55
nibalizer7.1.0 on the host, 7.0.1 on my laptop23:55
mtreinishnibalizer: in the health.o.o venv? virtualenv will bundle pip23:56
nibalizerya by host i mean a venv I created on health.o.o23:56
*** alivigni has joined #openstack-infra23:57
mtreinishhmm, I'm running 7.1.2 locally, but none of those are old enough to be the cause23:57
mtreinishmordred: ^^^ do you have any ideas?23:57
mtreinishweird python packaging behavior is your cup of tea23:57
nibalizeralso the python/virtualenv on openstack systems are provided by get-pip I think23:57
nibalizerso we're likely not (directly) hitting a 'trusty is old' issue23:58
*** Swami has quit IRC23:58
mtreinishnibalizer: well I was thinking maybe the puppet modules had a dep on the apt package or something23:58
mtreinishI didn't know where the virtualenv module came from23:58
openstackgerritLucas Alvares Gomes proposed openstack/diskimage-builder: Install IPA in a virtual enviroment  https://review.openstack.org/23584323:59
nibalizermtreinish: http://paste.openstack.org/show/476935/23:59
ianwpabelanger: yeah, devstack experimental not working either.  regex seems ok23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!