| *** Sukhdev has quit IRC | 00:00 | |
| *** Sukhdev has joined #openstack-infra | 00:03 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/storyboard: Add example commands for the Story api https://review.openstack.org/332946 | 00:04 |
|---|---|---|
| *** rajinir has quit IRC | 00:04 | |
| *** salv-orl_ has quit IRC | 00:06 | |
| *** sflanigan has joined #openstack-infra | 00:08 | |
| fungi | i missed the ansible-test-patching excitement. sorry, got dragged off to do evening errands/chores | 00:09 |
| fungi | looks like it was more fun in here | 00:10 |
| *** rcernin has quit IRC | 00:12 | |
| *** raunak has joined #openstack-infra | 00:13 | |
| *** rdx_matt has joined #openstack-infra | 00:16 | |
| pabelanger | Shrews: jeblair: looks like we have a failure already: http://logs.openstack.org/08/333508/1/check/gate-puppet-openstack-integration-4-scenario002-tempest-ubuntu-xenial-nv/c66c434/_zuul_ansible/ansible_log.txt | 00:18 |
| *** bhavik has joined #openstack-infra | 00:19 | |
| mmedvede | wznoinsk: _alastor_ : I got ci-watch instance running at http://54.205.162.64:5000 . Not much data there yet to be useful | 00:21 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Handle locales install on Fedora 24 https://review.openstack.org/333118 | 00:22 |
| openstackgerrit | Spencer Krum proposed openstack-infra/project-config: Move vinz jobs to javascript from python https://review.openstack.org/333634 | 00:24 |
| *** jamesmcarthur has joined #openstack-infra | 00:24 | |
| *** thorst_away has joined #openstack-infra | 00:24 | |
| clarkb | nibalizer: re ^ there is a separate vinz-ui or something project that is the javascript one | 00:25 |
| clarkb | the streams didn't cross did they? | 00:25 |
| nibalizer | um? | 00:26 |
| nibalizer | i mean both repos are empty | 00:26 |
| nibalizer | so there is a vinz, vinz-client, and puppet-vinz | 00:26 |
| *** mriedem1 has joined #openstack-infra | 00:26 | |
| clarkb | nibalizer: ya vinz-client is the one I am thinking of, should alrady be configured to run js test things | 00:26 |
| nibalizer | all it has is something called gate-vinz-webclient-js-draft | 00:27 |
| nibalizer | i think this was architected at some point... and now that architecture is lost | 00:28 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Handle locales install on Fedora 24 https://review.openstack.org/333118 | 00:28 |
| nibalizer | so my thought was dump things in 'vinz' until we know where we want to go | 00:28 |
| *** mriedem has quit IRC | 00:28 | |
| clarkb | gotcha | 00:28 |
| *** jamesmcarthur has quit IRC | 00:29 | |
| openstackgerrit | Merged openstack-infra/project-config: Move gate-storyboard-js-integration to experimental https://review.openstack.org/333604 | 00:29 |
| *** kzaitsev_mb has quit IRC | 00:29 | |
| jeblair | nibalizer: i think the arch was 'vinz is a web service that talks to gerrit, vinz-ui is a javascript client'. basically, exactly like storyboard. i think that all predates gerrit being its own web ui, so we may not need 2 components now. if we only need a js thingy, then i see no reason it shouldn't be called 'vinz'. :) | 00:31 |
| jeblair | er | 00:31 |
| jeblair | correction: 'predates gerrit being its own rest server' | 00:32 |
| nibalizer | ah | 00:32 |
| jeblair | pabelanger, Shrews, mordred: here is the key line from that log: | 00:32 |
| jeblair | 2016-06-24 00:16:03,321 p=26980 u=zuul | fatal: [node]: FAILED! => {"changed": false, "failed": true, "msg": "async task did not complete within the requested time; {'failed': True, 'module_stderr': u'', 'module_stdout': u'', 'msg': 'MODULE FAILURE', 'invocation': {'module_name': u'async_status'}, 'parsed': False}"} | 00:32 |
| nibalizer | jeblair: yea my current production architecture idea is we use apache to take over what happens when you hit review.openstack.org/vinz and ship some js to the user, which then hits the gerrit api and makes go a code review service | 00:33 |
| jeblair | that's the content of async_result for debugging | 00:33 |
| jeblair | nibalizer: you said "go" ;) | 00:34 |
| nibalizer | hah | 00:34 |
| jeblair | nibalizer: that sounds nice -- the authentication bit is fuzzy for me | 00:36 |
| jeblair | that might be something to not kick too far down the road | 00:36 |
| nibalizer | well to start, you auth with the same way you auth now, then head over to urls starting with /vinz | 00:36 |
| nibalizer | but really i think you can just hit /login and it gets-it-done | 00:37 |
| jeblair | gotcha | 00:37 |
| nibalizer | yep, thats exactly true | 00:37 |
| nibalizer | once you're on a page ... you're authed essentially | 00:37 |
| openstackgerrit | Merged openstack-infra/storyboard: Allow users to be filtered by openid and email https://review.openstack.org/333567 | 00:37 |
| clarkb | ya basically the way the current UI uses rest and auth I think | 00:37 |
| openstackgerrit | Spencer Krum proposed openstack-infra/project-config: Move vinz jobs to javascript from python https://review.openstack.org/333634 | 00:37 |
| jeblair | cool | 00:37 |
| clarkb | vinz should be able to piggy back on that | 00:37 |
| *** vhosakot has quit IRC | 00:38 | |
| jeblair | i have to forage for food now | 00:38 |
| *** daemontool has quit IRC | 00:38 | |
| nibalizer | yah clark has it right | 00:41 |
| fungi | nibalizer: at your recommendation, i've added hurgleburgler to vinz-core | 00:42 |
| nibalizer | fantastic! | 00:42 |
| fungi | phschwartz: do you have any interest in reviewing vinz-related changes? do you happen to know if klmitch does? | 00:42 |
| nibalizer | my first cut of vinz was cssless and sad, it also was as i've been told 'bad javascript' | 00:42 |
| nibalizer | fungi: i have it on good authority that phschwartz has been a father for approx 12 hours, and is not going to be around for a while | 00:43 |
| fungi | nibalizer: ooh, i write some really terrible javascript! unfortunately i'm busy at the moment writing terrible python and puppet instead | 00:43 |
| nibalizer | fungi: yes, hurgleburgler added the css and fixed the js | 00:43 |
| fungi | phschwartz: ignore my question. in fact, if you actually _see_ my question, i'll be disappointed in you | 00:44 |
| openstackgerrit | Spencer Krum proposed openstack-infra/vinz: Add requirements.txt to flask-dev app https://review.openstack.org/333640 | 00:45 |
| *** mriedem1 is now known as mriedem | 00:45 | |
| *** ffrank has quit IRC | 00:46 | |
| *** rdx_matt has quit IRC | 00:50 | |
| *** lvdongbing has joined #openstack-infra | 00:51 | |
| *** zhurong has joined #openstack-infra | 00:57 | |
| *** dimtruck is now known as zz_dimtruck | 00:58 | |
| *** samP has joined #openstack-infra | 00:58 | |
| jamielennox | hey, have we seen ImportError: No module named packaging.specifiers on requirements check jobs? | 00:59 |
| *** SumitNaiksatam has joined #openstack-infra | 01:01 | |
| *** Sukhdev has quit IRC | 01:01 | |
| fungi | jamielennox: yeah, someone was in here a little bit ago mentioning a patch they had proposed to the reqs repo to fix that | 01:02 |
| *** ayoung has joined #openstack-infra | 01:02 | |
| fungi | i directed them to the new #openstack-requirements channel where the new requirements team hangs out | 01:02 |
| jamielennox | fungi: thanks, mostly wanted to check if i needed to dig into it - and seems like no | 01:03 |
| *** Sukhdev has joined #openstack-infra | 01:04 | |
| *** pvaneck has quit IRC | 01:08 | |
| *** gluytium has joined #openstack-infra | 01:08 | |
| *** julim has joined #openstack-infra | 01:08 | |
| *** salv-orlando has joined #openstack-infra | 01:08 | |
| *** Sukhdev has quit IRC | 01:08 | |
| *** shashank_hegde has quit IRC | 01:11 | |
| *** salv-orlando has quit IRC | 01:15 | |
| *** Jeffrey4l has joined #openstack-infra | 01:16 | |
| *** Apoorva has quit IRC | 01:17 | |
| *** tphummel has quit IRC | 01:17 | |
| samP | AJaeger: yolanda: Thank you for adding masakari repo (https://review.openstack.org/#/c/330370/). Could you please add {"Sampath Priyankara", "sam47priya@gmail.com"} (myself) to masakari-core and masakari-release groups on gerrit? | 01:19 |
| samP | Or is this the correct pace to make this request? or ML? | 01:20 |
| *** zul has quit IRC | 01:22 | |
| *** zul_ has joined #openstack-infra | 01:22 | |
| *** flwang has quit IRC | 01:22 | |
| *** sdague has quit IRC | 01:23 | |
| *** kzaitsev_mb has joined #openstack-infra | 01:26 | |
| *** Daisy has joined #openstack-infra | 01:27 | |
| *** docaedo has quit IRC | 01:29 | |
| *** xarses has joined #openstack-infra | 01:31 | |
| *** tqtran has quit IRC | 01:31 | |
| *** vhosakot has joined #openstack-infra | 01:32 | |
| *** gluytium has quit IRC | 01:33 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/storyboard: Add example commands for the Story api https://review.openstack.org/332946 | 01:34 |
| *** nwkarsten has quit IRC | 01:38 | |
| *** nwkarsten has joined #openstack-infra | 01:38 | |
| *** zz_dimtruck is now known as dimtruck | 01:39 | |
| *** amotoki has joined #openstack-infra | 01:39 | |
| *** kzaitsev_mb has quit IRC | 01:44 | |
| *** nwkarsten has quit IRC | 01:46 | |
| *** rdx_matt has joined #openstack-infra | 01:47 | |
| *** PollyZ has joined #openstack-infra | 01:47 | |
| *** docaedo has joined #openstack-infra | 01:50 | |
| *** yanyanhu has joined #openstack-infra | 01:50 | |
| *** gouthamr_ is now known as gouthamr | 01:54 | |
| *** baoli has joined #openstack-infra | 02:01 | |
| *** nwkarsten has joined #openstack-infra | 02:02 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Don't use devuser for zuul-worker https://review.openstack.org/333653 | 02:04 |
| *** sflanigan is now known as sflanigan_afk | 02:06 | |
| *** nwkarsten has quit IRC | 02:07 | |
| *** vhosakot has quit IRC | 02:07 | |
| *** mriedem has quit IRC | 02:09 | |
| *** zhurong has quit IRC | 02:09 | |
| *** woodster_ has quit IRC | 02:09 | |
| *** zhurong has joined #openstack-infra | 02:09 | |
| *** salv-orlando has joined #openstack-infra | 02:15 | |
| *** thorst_away has quit IRC | 02:16 | |
| *** baoli has quit IRC | 02:16 | |
| *** thorst_away has joined #openstack-infra | 02:17 | |
| *** PollyZ has quit IRC | 02:18 | |
| *** rdx_matt has quit IRC | 02:19 | |
| *** sflanigan_afk is now known as sflanigan | 02:24 | |
| *** yamahata has quit IRC | 02:24 | |
| *** thorst_away has quit IRC | 02:25 | |
| *** Na3iL has joined #openstack-infra | 02:26 | |
| *** salv-orlando has quit IRC | 02:27 | |
| *** tqtran has joined #openstack-infra | 02:29 | |
| *** raunak has quit IRC | 02:30 | |
| *** tqtran has quit IRC | 02:34 | |
| *** Daisy has quit IRC | 02:39 | |
| *** Daisy has joined #openstack-infra | 02:40 | |
| *** kzaitsev_mb has joined #openstack-infra | 02:41 | |
| *** kzaitsev_mb has quit IRC | 02:45 | |
| *** vhosakot has joined #openstack-infra | 02:47 | |
| *** Daisy has quit IRC | 02:48 | |
| *** esberglu has quit IRC | 02:49 | |
| *** gyee has quit IRC | 02:51 | |
| *** gouthamr has quit IRC | 02:55 | |
| *** yuanying has quit IRC | 02:55 | |
| *** bswartz has joined #openstack-infra | 02:56 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/storyboard: Add example commands for the Story api https://review.openstack.org/332946 | 02:56 |
| *** vhosakot has quit IRC | 02:58 | |
| *** ramishra has joined #openstack-infra | 02:59 | |
| *** raunak has joined #openstack-infra | 03:01 | |
| *** ramishra has quit IRC | 03:01 | |
| greghaynes | nibalizer: jeblair for vinz auth - as long as you can serve up is at the same domain as gerrit the auth is doable. One way is to Ajax to / and parse out the auth token, I hope gerrit has a better way, though | 03:03 |
| greghaynes | Trying to look in to that is what led me to the xss deal recently ;) | 03:03 |
| greghaynes | Ah, seems like nibalizer knows what's up | 03:04 |
| *** ramishra has joined #openstack-infra | 03:07 | |
| *** tphummel has joined #openstack-infra | 03:09 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-dev/hacking: Updated from global requirements https://review.openstack.org/333670 | 03:09 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements https://review.openstack.org/333673 | 03:10 |
| *** PollyZ has joined #openstack-infra | 03:13 | |
| *** tphummel has quit IRC | 03:16 | |
| *** ramishra has quit IRC | 03:16 | |
| *** rdx_matt has joined #openstack-infra | 03:16 | |
| Srinu | hi | 03:19 |
| *** PollyZ has quit IRC | 03:21 | |
| *** ramishra has joined #openstack-infra | 03:22 | |
| *** rm_work has quit IRC | 03:22 | |
| *** thorst_away has joined #openstack-infra | 03:23 | |
| *** cody-somerville has quit IRC | 03:23 | |
| *** zhurong has quit IRC | 03:23 | |
| *** rm_work has joined #openstack-infra | 03:25 | |
| *** amotoki has quit IRC | 03:26 | |
| *** cody-somerville has joined #openstack-infra | 03:26 | |
| *** cody-somerville has joined #openstack-infra | 03:26 | |
| *** lvdongbing has quit IRC | 03:27 | |
| *** lvdongbing has joined #openstack-infra | 03:28 | |
| *** salv-orlando has joined #openstack-infra | 03:29 | |
| *** thorst_away has quit IRC | 03:30 | |
| *** Na3iL has quit IRC | 03:31 | |
| *** zhurong has joined #openstack-infra | 03:31 | |
| *** zhurong has quit IRC | 03:33 | |
| *** rcernin has joined #openstack-infra | 03:34 | |
| *** ramishra has quit IRC | 03:37 | |
| *** salv-orlando has quit IRC | 03:37 | |
| *** zhurong has joined #openstack-infra | 03:39 | |
| notmorgan | nibalizer, crinkle, mtreinish: any of you have x1c w/ a WWAN card installed? | 03:40 |
| notmorgan | nibalizer, crinkle, mtreinish: trying to determine if linux is causing an issue or the WWAN card/sim slot on mine is bad - I can't get it to register the SIM is inserted. | 03:40 |
| notmorgan | Shrews: ^ cc | 03:40 |
| notmorgan | jeblair: ^ | 03:41 |
| notmorgan | though i think Shrews and jeblair aren't running this type of setup. | 03:41 |
| *** kzaitsev_mb has joined #openstack-infra | 03:42 | |
| *** gongysh has joined #openstack-infra | 03:42 | |
| *** coolsvap is now known as coolsvap_ | 03:43 | |
| *** ramishra has joined #openstack-infra | 03:43 | |
| *** amotoki has joined #openstack-infra | 03:44 | |
| *** kzaitsev_mb has quit IRC | 03:47 | |
| *** yamamot__ has joined #openstack-infra | 03:47 | |
| *** rdx_matt has quit IRC | 03:49 | |
| *** ramishra has quit IRC | 03:49 | |
| *** yuanying has joined #openstack-infra | 03:50 | |
| *** ramishra has joined #openstack-infra | 03:54 | |
| openstackgerrit | Yanyan Hu proposed openstack-infra/project-config: Add senlin-tempest-functional test job https://review.openstack.org/333719 | 03:54 |
| *** shashank_hegde has joined #openstack-infra | 03:55 | |
| openstackgerrit | Yanyan Hu proposed openstack-infra/project-config: Add senlin-tempest-functional test job https://review.openstack.org/333719 | 03:57 |
| openstackgerrit | Yanyan Hu proposed openstack-infra/project-config: Add senlin-tempest-functional test job https://review.openstack.org/333719 | 03:59 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Don't use devuser for zuul-worker https://review.openstack.org/333653 | 03:59 |
| *** amotoki has quit IRC | 04:00 | |
| *** sdake_ has joined #openstack-infra | 04:00 | |
| *** sdake has quit IRC | 04:02 | |
| *** amotoki has joined #openstack-infra | 04:03 | |
| *** Sukhdev has joined #openstack-infra | 04:07 | |
| *** markvoelker has quit IRC | 04:11 | |
| *** amotoki has quit IRC | 04:18 | |
| *** david-lyle has quit IRC | 04:20 | |
| *** david-lyle has joined #openstack-infra | 04:20 | |
| *** sdake_ has quit IRC | 04:22 | |
| *** Qiming has quit IRC | 04:22 | |
| *** Qiming has joined #openstack-infra | 04:23 | |
| *** thorst_away has joined #openstack-infra | 04:28 | |
| *** baoli has joined #openstack-infra | 04:28 | |
| *** sflanigan has quit IRC | 04:31 | |
| *** tqtran has joined #openstack-infra | 04:31 | |
| *** sflanigan has joined #openstack-infra | 04:33 | |
| *** sflanigan has joined #openstack-infra | 04:33 | |
| *** baoli has quit IRC | 04:33 | |
| *** ramishra has quit IRC | 04:34 | |
| *** thorst_away has quit IRC | 04:35 | |
| *** tqtran has quit IRC | 04:35 | |
| *** Daisy has joined #openstack-infra | 04:36 | |
| *** psachin has joined #openstack-infra | 04:36 | |
| *** links has joined #openstack-infra | 04:37 | |
| *** sdake has joined #openstack-infra | 04:38 | |
| *** amotoki has joined #openstack-infra | 04:38 | |
| *** salv-orlando has joined #openstack-infra | 04:39 | |
| *** Daisy has quit IRC | 04:41 | |
| *** ramishra has joined #openstack-infra | 04:41 | |
| *** _Fdaisuke_ has quit IRC | 04:43 | |
| *** kzaitsev_mb has joined #openstack-infra | 04:43 | |
| *** rdx_matt has joined #openstack-infra | 04:46 | |
| *** kzaitsev_mb has quit IRC | 04:47 | |
| *** rdx_matt has quit IRC | 04:50 | |
| *** links has quit IRC | 04:50 | |
| *** links has joined #openstack-infra | 04:50 | |
| *** ramishra has quit IRC | 04:50 | |
| *** coolsvap_ has quit IRC | 04:51 | |
| *** salv-orlando has quit IRC | 04:51 | |
| *** yamamot__ has quit IRC | 04:53 | |
| *** ramishra has joined #openstack-infra | 04:56 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Don't use devuser for zuul-worker https://review.openstack.org/333653 | 05:01 |
| *** Daisy has joined #openstack-infra | 05:06 | |
| *** julim has quit IRC | 05:10 | |
| *** markvoelker has joined #openstack-infra | 05:11 | |
| *** shahid__ has joined #openstack-infra | 05:13 | |
| *** _nadya_ has joined #openstack-infra | 05:13 | |
| *** salv-orlando has joined #openstack-infra | 05:14 | |
| *** markvoelker has quit IRC | 05:16 | |
| *** ramishra has quit IRC | 05:17 | |
| *** bhavik has quit IRC | 05:18 | |
| *** Daisy has quit IRC | 05:18 | |
| *** Daisy has joined #openstack-infra | 05:18 | |
| *** ramishra has joined #openstack-infra | 05:18 | |
| *** kzaitsev_mb has joined #openstack-infra | 05:19 | |
| *** zeih has joined #openstack-infra | 05:20 | |
| *** coolsvap has joined #openstack-infra | 05:21 | |
| *** Daisy has quit IRC | 05:23 | |
| *** links has quit IRC | 05:25 | |
| coolsvap | any background work happening with gerrit, seems very slow | 05:26 |
| zaro | coolsvap: seems ok to me. which part is slow for you? | 05:27 |
| coolsvap | zaro, in general, i am the review open time, time to change between files in review | 05:28 |
| coolsvap | its taking more than usual | 05:28 |
| coolsvap | just added one comment | 05:29 |
| *** rdx_matt has joined #openstack-infra | 05:30 | |
| *** maishsk_ has joined #openstack-infra | 05:31 | |
| zaro | coolsvap: which project? i just added a comment to https://review.openstack.org/#/c/236150 and it wasn't slow for me. | 05:31 |
| coolsvap | the comment in review is not uploaded instead i got server unavailable | 05:31 |
| coolsvap | https://review.openstack.org/#/c/333732/1/doc/quickstart.rst | 05:31 |
| coolsvap | zaro, ^^ | 05:32 |
| *** sdake has quit IRC | 05:32 | |
| *** maishsk has quit IRC | 05:33 | |
| *** maishsk_ is now known as maishsk | 05:33 | |
| *** thorst_away has joined #openstack-infra | 05:33 | |
| *** sdake has joined #openstack-infra | 05:34 | |
| zaro | coolsvap: just added a comment to that change, didn't seem slow either. | 05:34 |
| zaro | maybe your internet connection? | 05:34 |
| coolsvap | zaro, i though the same, restarted everthing, but everything else is working fine mail, vpn n all | 05:35 |
| coolsvap | will check after some time | 05:35 |
| *** rcernin has quit IRC | 05:35 | |
| *** rdx_matt has quit IRC | 05:36 | |
| *** links has joined #openstack-infra | 05:39 | |
| *** Sukhdev has quit IRC | 05:40 | |
| *** thorst_away has quit IRC | 05:40 | |
| *** prabu_ has joined #openstack-infra | 05:42 | |
| *** ramishra has quit IRC | 05:42 | |
| *** tlian has quit IRC | 05:44 | |
| *** yamamot__ has joined #openstack-infra | 05:46 | |
| *** ramishra has joined #openstack-infra | 05:47 | |
| *** dimtruck is now known as zz_dimtruck | 05:48 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Start fleshing out ansible-launcher tests https://review.openstack.org/317482 | 05:52 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Clean up results returned from ansible-launcher https://review.openstack.org/317481 | 05:52 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add jjb and zmq to requirements https://review.openstack.org/316752 | 05:52 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Prepare files for testing zuulv25 https://review.openstack.org/316753 | 05:52 |
| *** hashar has joined #openstack-infra | 06:00 | |
| *** YorikSar has quit IRC | 06:01 | |
| *** Daisy has joined #openstack-infra | 06:01 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Switch to odd, update chairs https://review.openstack.org/331863 | 06:01 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Don't use devuser for zuul-worker https://review.openstack.org/333653 | 06:03 |
| *** YorikSar has joined #openstack-infra | 06:03 | |
| *** signed8bit_Zzz is now known as signed8bit | 06:04 | |
| *** signed8bit has quit IRC | 06:05 | |
| *** links has quit IRC | 06:05 | |
| *** Daisy has quit IRC | 06:06 | |
| *** raunak has quit IRC | 06:08 | |
| *** Na3iL has joined #openstack-infra | 06:09 | |
| *** YorikSar has quit IRC | 06:10 | |
| *** Nikolay_St has joined #openstack-infra | 06:11 | |
| *** rcernin has joined #openstack-infra | 06:12 | |
| *** YorikSar has joined #openstack-infra | 06:12 | |
| *** ccamacho has joined #openstack-infra | 06:12 | |
| *** _nadya_ has quit IRC | 06:12 | |
| *** markvoelker has joined #openstack-infra | 06:13 | |
| *** _nadya_ has joined #openstack-infra | 06:13 | |
| *** Na3iL has quit IRC | 06:13 | |
| *** ramishra has quit IRC | 06:16 | |
| *** ramishra has joined #openstack-infra | 06:17 | |
| *** ccamacho has quit IRC | 06:17 | |
| *** ccamacho|Zzz has quit IRC | 06:17 | |
| *** _nadya_ has quit IRC | 06:17 | |
| *** markvoelker has quit IRC | 06:17 | |
| *** pgadiya has joined #openstack-infra | 06:17 | |
| *** kzaitsev_mb has quit IRC | 06:17 | |
| *** linuxmaniac_ has joined #openstack-infra | 06:17 | |
| *** javeriak has joined #openstack-infra | 06:18 | |
| *** linuxmaniac has quit IRC | 06:18 | |
| *** links has joined #openstack-infra | 06:18 | |
| *** aeng has quit IRC | 06:20 | |
| *** ramishra has quit IRC | 06:22 | |
| *** javeriak has quit IRC | 06:23 | |
| *** javeriak has joined #openstack-infra | 06:23 | |
| *** shashank_hegde has quit IRC | 06:25 | |
| *** ramishra has joined #openstack-infra | 06:27 | |
| *** salv-orlando has quit IRC | 06:28 | |
| *** ramishra has quit IRC | 06:28 | |
| *** shashank_hegde has joined #openstack-infra | 06:29 | |
| *** Daisy has joined #openstack-infra | 06:30 | |
| *** ramishra has joined #openstack-infra | 06:30 | |
| *** tqtran has joined #openstack-infra | 06:32 | |
| *** rdx_matt has joined #openstack-infra | 06:32 | |
| *** rdx_matt has quit IRC | 06:32 | |
| *** hashar has quit IRC | 06:34 | |
| *** andreas_s has joined #openstack-infra | 06:35 | |
| *** tqtran has quit IRC | 06:37 | |
| *** links has quit IRC | 06:37 | |
| *** _nadya_ has joined #openstack-infra | 06:38 | |
| *** thorst_away has joined #openstack-infra | 06:38 | |
| *** thorst_away has quit IRC | 06:45 | |
| *** links has joined #openstack-infra | 06:49 | |
| *** sarob has joined #openstack-infra | 06:50 | |
| *** sarob has quit IRC | 06:55 | |
| *** javeriak has quit IRC | 06:55 | |
| *** javeriak has joined #openstack-infra | 06:55 | |
| *** tesseract- has joined #openstack-infra | 06:57 | |
| *** rdx_matt has joined #openstack-infra | 07:00 | |
| openstackgerrit | Wayne Warren proposed openstack-infra/jenkins-job-builder: Rename "parallelize" decorator to "concurrent" https://review.openstack.org/333780 | 07:01 |
| *** pkoniszewski has quit IRC | 07:03 | |
| *** yolanda has joined #openstack-infra | 07:04 | |
| *** javeriak_ has joined #openstack-infra | 07:06 | |
| *** rdx_matt has quit IRC | 07:07 | |
| *** e0ne has joined #openstack-infra | 07:07 | |
| *** maishsk_ has joined #openstack-infra | 07:08 | |
| *** javeriak has quit IRC | 07:09 | |
| *** maishsk has quit IRC | 07:10 | |
| *** maishsk_ is now known as maishsk | 07:10 | |
| *** matrohon has joined #openstack-infra | 07:11 | |
| *** sdake_ has joined #openstack-infra | 07:12 | |
| *** pkoniszewski has joined #openstack-infra | 07:12 | |
| *** _nadya_ has quit IRC | 07:13 | |
| *** ifarkas has joined #openstack-infra | 07:13 | |
| *** markvoelker has joined #openstack-infra | 07:13 | |
| *** sdake has quit IRC | 07:15 | |
| *** markvoelker has quit IRC | 07:19 | |
| *** flepied1 has quit IRC | 07:25 | |
| *** arxcruz has joined #openstack-infra | 07:26 | |
| *** ihrachys has joined #openstack-infra | 07:28 | |
| openstackgerrit | Hisashi Osanai proposed openstack-infra/project-config: Fix Monasca Analytics python27 job https://review.openstack.org/331535 | 07:29 |
| *** amoralej|off is now known as amoralej | 07:29 | |
| *** esikachev has joined #openstack-infra | 07:30 | |
| *** shardy has joined #openstack-infra | 07:31 | |
| *** jlanoux has joined #openstack-infra | 07:31 | |
| *** Na3iL has joined #openstack-infra | 07:31 | |
| *** yaume has joined #openstack-infra | 07:34 | |
| *** yamahata has joined #openstack-infra | 07:35 | |
| *** shardy_ has joined #openstack-infra | 07:35 | |
| *** salv-orlando has joined #openstack-infra | 07:36 | |
| *** ramishra has quit IRC | 07:37 | |
| *** ramishra has joined #openstack-infra | 07:38 | |
| *** jpich has joined #openstack-infra | 07:40 | |
| *** jlanoux_ has joined #openstack-infra | 07:41 | |
| *** Daisy has quit IRC | 07:41 | |
| yolanda | good morning | 07:41 |
| *** Daisy has joined #openstack-infra | 07:41 | |
| *** thorst_away has joined #openstack-infra | 07:43 | |
| *** jlanoux has quit IRC | 07:44 | |
| *** shardy has quit IRC | 07:45 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config: Make gate-tempest-dsvm-neutron-full-ssh voting https://review.openstack.org/333794 | 07:45 |
| *** javeriak has joined #openstack-infra | 07:46 | |
| *** javeria__ has joined #openstack-infra | 07:47 | |
| *** javeriak_ has quit IRC | 07:47 | |
| samP | yolanda: good morning, Thank you for adding masakari repo (https://review.openstack.org/#/c/330370/) | 07:48 |
| *** Na3iL has quit IRC | 07:48 | |
| yolanda | hi samP , glad to help | 07:49 |
| *** ihrachys has quit IRC | 07:50 | |
| *** javeriak has quit IRC | 07:50 | |
| samP | yolanda: I would like to ask one more help, which is add me to gerrit group masakari-core and masakari-review | 07:50 |
| *** javeriak has joined #openstack-infra | 07:50 | |
| *** thorst_away has quit IRC | 07:50 | |
| *** ramishra has quit IRC | 07:50 | |
| yolanda | sure | 07:50 |
| *** ramishra has joined #openstack-infra | 07:51 | |
| openstackgerrit | Hisashi Osanai proposed openstack-infra/project-config: Fix Monasca Analytics python27 job https://review.openstack.org/331535 | 07:51 |
| samP | yolanda: me (Sampath Priyankara, sam47priya@gmail.com), what info you need? | 07:51 |
| yolanda | i added you | 07:52 |
| openstackgerrit | Merged openstack-infra/project-config: start a readme describing the release tools https://review.openstack.org/332945 | 07:52 |
| samP | yolanda: excellent, thank you very much. | 07:52 |
| *** javeria__ has quit IRC | 07:53 | |
| *** markmcd has quit IRC | 07:55 | |
| chem | hi people, on the Xenial image is there the dbus.service Unit or should I install it ? | 07:56 |
| *** YorikSar has quit IRC | 07:56 | |
| *** YorikSar has joined #openstack-infra | 07:57 | |
| *** flepied has joined #openstack-infra | 07:57 | |
| openstackgerrit | Merged openstack-infra/project-config: Upgrade experimental searchlight ES 2.3.2 https://review.openstack.org/333525 | 07:57 |
| *** rcernin has quit IRC | 07:59 | |
| openstackgerrit | Merged openstack-infra/project-config: Add senlin-tempest-functional test job https://review.openstack.org/333719 | 07:59 |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| chem | or a simpler question, what is the image used for xenial vm, is it the official one or another ? | 08:00 |
| *** Daisy_ has joined #openstack-infra | 08:00 | |
| *** sdake_ has quit IRC | 08:02 | |
| *** lucas-afk is now known as lucasagomes | 08:02 | |
| *** dizquierdo has joined #openstack-infra | 08:03 | |
| *** rdx_matt has joined #openstack-infra | 08:03 | |
| *** rdx_matt has quit IRC | 08:04 | |
| *** Daisy has quit IRC | 08:04 | |
| *** rdx_matt has joined #openstack-infra | 08:04 | |
| openstackgerrit | Merged openstack-infra/project-config: Run identity v3 only job in heat https://review.openstack.org/315214 | 08:04 |
| *** eezhova has joined #openstack-infra | 08:05 | |
| *** _nadya_ has joined #openstack-infra | 08:07 | |
| *** shashank_hegde has quit IRC | 08:09 | |
| *** rdx_matt has quit IRC | 08:09 | |
| openstackgerrit | Alexander Evseev proposed openstack-infra/zuul: Add ability to use SSH instead of HTTP to get refs https://review.openstack.org/246433 | 08:09 |
| openstackgerrit | Alexander Evseev proposed openstack-infra/zuul: Report job status 'QUEUED' for queued jobs https://review.openstack.org/272967 | 08:10 |
| openstackgerrit | Alexander Evseev proposed openstack-infra/zuul: Create Merger Git repos using connection names https://review.openstack.org/282660 | 08:10 |
| *** abregman has joined #openstack-infra | 08:10 | |
| openstackgerrit | Alexander Evseev proposed openstack-infra/zuul: Allow using numeric change number in addition to ID in Depends-On tag https://review.openstack.org/215553 | 08:10 |
| *** sflanigan has quit IRC | 08:13 | |
| *** _degorenko|afk is now known as degorenko | 08:13 | |
| *** arif-ali has joined #openstack-infra | 08:13 | |
| *** dtantsur|afk is now known as dtantsur | 08:14 | |
| *** dmk0202 has joined #openstack-infra | 08:17 | |
| *** daemontool has joined #openstack-infra | 08:17 | |
| *** rossella_s has joined #openstack-infra | 08:19 | |
| *** ramishra has quit IRC | 08:20 | |
| openstackgerrit | Madhuri Kumari proposed openstack-infra/project-config: Add new project Zunclient to openstack repo https://review.openstack.org/317699 | 08:20 |
| *** acoles_ is now known as acoles | 08:21 | |
| *** HeOS has quit IRC | 08:23 | |
| *** markmcd has joined #openstack-infra | 08:24 | |
| *** javeriak has quit IRC | 08:27 | |
| *** jamesmcarthur has joined #openstack-infra | 08:29 | |
| *** shardy_ is now known as shardy_afk | 08:30 | |
| *** binbincong has joined #openstack-infra | 08:30 | |
| *** kzaitsev_mb has joined #openstack-infra | 08:30 | |
| *** ramishra has joined #openstack-infra | 08:30 | |
| wznoinsk | mmedvede, superslow but content looks ok | 08:30 |
| *** markmcd has quit IRC | 08:31 | |
| *** electrofelix has joined #openstack-infra | 08:32 | |
| *** jamesmcarthur has quit IRC | 08:33 | |
| openstackgerrit | Bilal proposed openstack-infra/python-storyboardclient: Updated Worklist API https://review.openstack.org/333810 | 08:34 |
| *** rdx_matt has joined #openstack-infra | 08:34 | |
| openstackgerrit | Bilal proposed openstack-infra/python-storyboardclient: Updated Worklist API https://review.openstack.org/333810 | 08:36 |
| *** matrohon has quit IRC | 08:38 | |
| *** rdx_matt has quit IRC | 08:39 | |
| *** denisra has quit IRC | 08:41 | |
| *** tosky has joined #openstack-infra | 08:42 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config: Make gate-tempest-dsvm-neutron-full-ssh voting https://review.openstack.org/333794 | 08:45 |
| *** BobBall_AWOL is now known as BobBall | 08:46 | |
| *** d0ugal has quit IRC | 08:46 | |
| *** derekh has joined #openstack-infra | 08:46 | |
| *** d0ugal has joined #openstack-infra | 08:46 | |
| *** d0ugal has quit IRC | 08:46 | |
| *** d0ugal has joined #openstack-infra | 08:46 | |
| *** BobBall is now known as Guest74267 | 08:46 | |
| *** gongysh has quit IRC | 08:47 | |
| openstackgerrit | Bilal proposed openstack-infra/python-storyboardclient: Updated Worklist API https://review.openstack.org/333817 | 08:47 |
| *** thorst_away has joined #openstack-infra | 08:48 | |
| *** sarob has joined #openstack-infra | 08:51 | |
| *** kzaitsev_mb has quit IRC | 08:51 | |
| *** dizquierdo has quit IRC | 08:54 | |
| *** yanyanhu has quit IRC | 08:54 | |
| *** thorst_away has quit IRC | 08:55 | |
| *** trash_ is now known as trash | 08:55 | |
| openstackgerrit | Bilal proposed openstack-infra/python-storyboardclient: Updated Worklist API https://review.openstack.org/333820 | 08:55 |
| *** sarob has quit IRC | 08:56 | |
| *** Na3iL has joined #openstack-infra | 08:57 | |
| openstackgerrit | Merged openstack-infra/system-config: Add compute088.useast metadata for better tracking https://review.openstack.org/330401 | 08:58 |
| *** jordanP has joined #openstack-infra | 08:59 | |
| *** jyuso1 has quit IRC | 09:00 | |
| *** markusry has joined #openstack-infra | 09:00 | |
| openstackgerrit | Bilal proposed openstack-infra/python-storyboardclient: Updated Worklist API https://review.openstack.org/333828 | 09:00 |
| *** ramishra has quit IRC | 09:01 | |
| *** javeriak has joined #openstack-infra | 09:02 | |
| *** markusry has quit IRC | 09:02 | |
| *** markusry has joined #openstack-infra | 09:02 | |
| *** ramishra has joined #openstack-infra | 09:03 | |
| *** Daisy_ has quit IRC | 09:04 | |
| *** Daisy has joined #openstack-infra | 09:04 | |
| *** javeriak has quit IRC | 09:06 | |
| *** javeriak has joined #openstack-infra | 09:07 | |
| *** Daisy has quit IRC | 09:09 | |
| openstackgerrit | Bilal proposed openstack-infra/python-storyboardclient: Updated Worklist API https://review.openstack.org/333828 | 09:09 |
| *** markusry has quit IRC | 09:09 | |
| *** markusry has joined #openstack-infra | 09:09 | |
| *** javeriak has quit IRC | 09:13 | |
| *** javeriak has joined #openstack-infra | 09:13 | |
| *** HeOS has joined #openstack-infra | 09:16 | |
| *** markvoelker has joined #openstack-infra | 09:16 | |
| *** sheeprine has quit IRC | 09:19 | |
| *** sheeprine has joined #openstack-infra | 09:19 | |
| *** javeriak has quit IRC | 09:19 | |
| *** javeriak has joined #openstack-infra | 09:20 | |
| *** markvoelker has quit IRC | 09:20 | |
| *** hashar has joined #openstack-infra | 09:21 | |
| *** aarefiev has quit IRC | 09:21 | |
| *** amotoki has quit IRC | 09:22 | |
| *** tosky has quit IRC | 09:22 | |
| *** amotoki has joined #openstack-infra | 09:23 | |
| *** tosky has joined #openstack-infra | 09:24 | |
| *** markusry has quit IRC | 09:25 | |
| *** shardy_afk is now known as shardy | 09:27 | |
| *** samP has quit IRC | 09:29 | |
| *** markusry has joined #openstack-infra | 09:30 | |
| *** amotoki_ has joined #openstack-infra | 09:34 | |
| *** markusry has quit IRC | 09:35 | |
| *** rdx_matt has joined #openstack-infra | 09:35 | |
| *** markusry has joined #openstack-infra | 09:35 | |
| *** rdx_matt has quit IRC | 09:35 | |
| *** amotoki has quit IRC | 09:35 | |
| *** rdx_matt has joined #openstack-infra | 09:38 | |
| *** markusry has quit IRC | 09:38 | |
| *** sambetts|afk is now known as sambetts | 09:39 | |
| *** markusry has joined #openstack-infra | 09:39 | |
| *** dizquierdo has joined #openstack-infra | 09:41 | |
| *** fawadkhaliq has joined #openstack-infra | 09:42 | |
| *** fawadkhaliq has quit IRC | 09:42 | |
| *** markusry has quit IRC | 09:43 | |
| *** ramishra has quit IRC | 09:45 | |
| derekh | so nodepool is now spinning up instances on RH2 \o/ , but they're not being spun up with any network attached, the nodepool config specifies the "private" network, any ideas about whats going wrong http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml#n632 | 09:48 |
| *** kzaitsev_mb has joined #openstack-infra | 09:49 | |
| *** ramishra has joined #openstack-infra | 09:52 | |
| *** thorst_away has joined #openstack-infra | 09:52 | |
| *** markusry has joined #openstack-infra | 09:54 | |
| *** markvoelker has joined #openstack-infra | 09:54 | |
| *** thorst_away has quit IRC | 10:00 | |
| *** zhurong has quit IRC | 10:06 | |
| *** yamahata has quit IRC | 10:07 | |
| *** eezhova has quit IRC | 10:08 | |
| *** sdague has joined #openstack-infra | 10:09 | |
| *** pcaruana has joined #openstack-infra | 10:09 | |
| *** javeriak has quit IRC | 10:11 | |
| *** ramishra has quit IRC | 10:13 | |
| *** ramishra has joined #openstack-infra | 10:13 | |
| *** ccamacho has joined #openstack-infra | 10:13 | |
| *** salv-orlando has quit IRC | 10:14 | |
| *** amotoki_ has quit IRC | 10:19 | |
| *** ccamacho has quit IRC | 10:20 | |
| *** javeriak has joined #openstack-infra | 10:24 | |
| *** claudiub has joined #openstack-infra | 10:28 | |
| *** ramishra has quit IRC | 10:30 | |
| *** ramishra has joined #openstack-infra | 10:30 | |
| *** dtantsur is now known as dtantsur|brb | 10:32 | |
| *** kzaitsev_mb has quit IRC | 10:34 | |
| *** lvdongbing has quit IRC | 10:35 | |
| *** javeriak has quit IRC | 10:35 | |
| *** Guest74267 is now known as BobBall | 10:36 | |
| *** ccamacho has joined #openstack-infra | 10:36 | |
| *** salv-orlando has joined #openstack-infra | 10:36 | |
| openstackgerrit | Bin Hu proposed openstack-infra/project-config: New project openstack/gluon https://review.openstack.org/333843 | 10:37 |
| *** javeriak has joined #openstack-infra | 10:43 | |
| *** gouthamr has joined #openstack-infra | 10:43 | |
| *** eezhova has joined #openstack-infra | 10:43 | |
| *** javeriak has quit IRC | 10:45 | |
| *** javeriak has joined #openstack-infra | 10:45 | |
| *** craige has quit IRC | 10:46 | |
| *** bauzas is now known as bauwser | 10:46 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:47 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Fix alignment of long-running simple events https://review.openstack.org/333225 | 10:49 |
| *** aarefiev has joined #openstack-infra | 10:51 | |
| *** sarob has joined #openstack-infra | 10:52 | |
| *** craige has joined #openstack-infra | 10:54 | |
| *** acabot has quit IRC | 10:54 | |
| *** sarob has quit IRC | 10:56 | |
| *** thorst_away has joined #openstack-infra | 10:58 | |
| *** salv-orl_ has joined #openstack-infra | 10:59 | |
| *** ramishra has quit IRC | 11:01 | |
| *** salv-orlando has quit IRC | 11:02 | |
| *** ramishra has joined #openstack-infra | 11:02 | |
| *** abregman is now known as abregman|afk | 11:02 | |
| *** kzaitsev_mb has quit IRC | 11:03 | |
| *** thorst_away has quit IRC | 11:05 | |
| *** acabot has joined #openstack-infra | 11:06 | |
| *** thorst_away has joined #openstack-infra | 11:07 | |
| openstackgerrit | Bilal proposed openstack-infra/python-storyboardclient: Updated Worklist API https://review.openstack.org/333828 | 11:10 |
| *** weshay has joined #openstack-infra | 11:12 | |
| *** ramishra has quit IRC | 11:14 | |
| *** thorst_away is now known as thorst_ | 11:15 | |
| *** Srinu has quit IRC | 11:16 | |
| *** kzaitsev_mb has joined #openstack-infra | 11:18 | |
| *** cody-somerville has quit IRC | 11:20 | |
| *** shardy has quit IRC | 11:23 | |
| *** azvyagintsev has quit IRC | 11:23 | |
| openstackgerrit | Merged openstack-infra/infra-ansible: Make services optional https://review.openstack.org/331036 | 11:24 |
| *** shardy has joined #openstack-infra | 11:24 | |
| openstackgerrit | Alexander Chadin proposed openstack-infra/project-config: Add rally job for Watcher service and support for rally-gate https://review.openstack.org/333860 | 11:30 |
| *** ramishra has joined #openstack-infra | 11:32 | |
| *** rhallisey has joined #openstack-infra | 11:32 | |
| yamamot__ | can anyone consider merging this? https://review.openstack.org/#/c/333097/ this is blocking midonet dev badly. | 11:32 |
| *** dprince has joined #openstack-infra | 11:33 | |
| *** tqtran has joined #openstack-infra | 11:34 | |
| openstackgerrit | vinay kumar muddu proposed openstack/diskimage-builder: Add support for building images capable of UEFI https://review.openstack.org/287784 | 11:35 |
| *** ramishra has quit IRC | 11:36 | |
| *** jordanP has quit IRC | 11:37 | |
| *** tqtran has quit IRC | 11:39 | |
| *** javeriak has quit IRC | 11:40 | |
| *** javeriak has joined #openstack-infra | 11:40 | |
| *** alaski is now known as lascii | 11:41 | |
| AJaeger | yamamot__: this needs a neutron infra liason +1 first - ask dougwig, armax or mestery for review, please | 11:42 |
| yamamot__ | AJaeger: ok thank you | 11:43 |
| *** andreas_s has quit IRC | 11:44 | |
| *** zhurong has joined #openstack-infra | 11:44 | |
| *** ldnunes has joined #openstack-infra | 11:45 | |
| *** sarob has joined #openstack-infra | 11:50 | |
| *** ddieterly has joined #openstack-infra | 11:53 | |
| *** sarob has quit IRC | 11:55 | |
| *** prabu_ has quit IRC | 11:57 | |
| *** ddieterly has quit IRC | 11:59 | |
| *** psilvad_ has joined #openstack-infra | 12:00 | |
| *** ccamacho is now known as ccamacho|lunch | 12:00 | |
| *** jtomasek_ has joined #openstack-infra | 12:01 | |
| openstackgerrit | vinay kumar muddu proposed openstack/diskimage-builder: Add support for building images capable of UEFI https://review.openstack.org/287784 | 12:01 |
| Shrews | notmorgan: i do not have such a setup | 12:02 |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-httpd: Update context on vhost for selinux https://review.openstack.org/333873 | 12:03 |
| *** Na3iL has quit IRC | 12:04 | |
| *** rcernin has joined #openstack-infra | 12:04 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/release-tools: Automate retiring old branches https://review.openstack.org/333875 | 12:07 |
| *** rfolco has joined #openstack-infra | 12:08 | |
| *** yamamot__ has quit IRC | 12:08 | |
| *** pradk has joined #openstack-infra | 12:09 | |
| *** ddieterly has joined #openstack-infra | 12:11 | |
| openstackgerrit | vinay kumar muddu proposed openstack/diskimage-builder: Add support for building images capable of UEFI https://review.openstack.org/287784 | 12:13 |
| *** liusheng has quit IRC | 12:13 | |
| openstackgerrit | Bin Hu proposed openstack-infra/project-config: New project openstack/gluon https://review.openstack.org/333843 | 12:14 |
| *** jordanP has joined #openstack-infra | 12:15 | |
| asselin | rcarrillocruz, let me know when you're around. I'm having trouble find the dependent ansible roles referenced in your patch: https://review.openstack.org/#/c/320466/ | 12:15 |
| *** dtardivel has joined #openstack-infra | 12:15 | |
| olaph | notmorgan: I have a x250, and have never gotten the wwan to work | 12:15 |
| *** liusheng has joined #openstack-infra | 12:16 | |
| openstackgerrit | Daniel Mellado proposed openstack-infra/project-config: Fix simple typo https://review.openstack.org/333880 | 12:16 |
| rcarrillocruz | asselin: you need the depends-on and https://review.openstack.org/#/q/topic:cloud_launcher | 12:17 |
| *** javeriak has quit IRC | 12:17 | |
| asselin | rcarrillocruz, I added the depends-on patch. Which ones from that topic query? all of the open ones? | 12:18 |
| *** matt-borland has joined #openstack-infra | 12:18 | |
| *** tpsilva has joined #openstack-infra | 12:19 | |
| esikachev | hey, folks! how you prepare nodes for running system-configs puppets? whitch source.lists? | 12:20 |
| *** aysyd has joined #openstack-infra | 12:21 | |
| rcarrillocruz | https://review.openstack.org/#/q/status:open+project:openstack-infra/system-config+branch:master+topic:cloud_launcher | 12:21 |
| rcarrillocruz | asselin: ^ | 12:21 |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-httpd: Update context on vhost for selinux https://review.openstack.org/333873 | 12:22 |
| asselin | rcarrillocruz, ok i see...so it's not a patch chain? I'll cherry-pick them and try again. | 12:23 |
| rcarrillocruz | bear in mind that approach is a replacement for the current launch-node, i.e. create just one server with values passed in on the command line | 12:23 |
| rcarrillocruz | asselin: right, not a patch chain, just cherry pick and you should be good | 12:23 |
| *** chem has quit IRC | 12:24 | |
| asselin | ok will give it a shot | 12:24 |
| rcarrillocruz | you can just create a resources.yml and feed to the role with -e "@resources.yml" | 12:25 |
| rcarrillocruz | you can put whatever resources you want in there | 12:25 |
| *** ddieterly has quit IRC | 12:26 | |
| openstackgerrit | Alexander Chadin proposed openstack-infra/project-config: Add rally job for Watcher service and support for rally-gate https://review.openstack.org/333860 | 12:26 |
| *** binbincong has quit IRC | 12:29 | |
| *** kdas_ has joined #openstack-infra | 12:30 | |
| *** jcoufal has joined #openstack-infra | 12:30 | |
| *** kdas_ is now known as kushal | 12:32 | |
| *** binbincong has joined #openstack-infra | 12:32 | |
| *** lucasagomes is now known as lucas-hungry | 12:32 | |
| *** kushal has quit IRC | 12:32 | |
| *** kushal has joined #openstack-infra | 12:32 | |
| *** Na3iL has joined #openstack-infra | 12:35 | |
| *** psachin has quit IRC | 12:36 | |
| *** gordc has joined #openstack-infra | 12:37 | |
| *** psachin has joined #openstack-infra | 12:38 | |
| *** jtomasek_ has quit IRC | 12:40 | |
| openstackgerrit | vinay kumar muddu proposed openstack/diskimage-builder: Build wholedisk images capable of secure boot https://review.openstack.org/333888 | 12:44 |
| *** zul_ is now known as zul | 12:44 | |
| *** erlon has joined #openstack-infra | 12:45 | |
| *** jtomasek has quit IRC | 12:46 | |
| *** alex_xu has quit IRC | 12:46 | |
| *** ddieterly has joined #openstack-infra | 12:46 | |
| asselin | rcarrillocruz, thanks for explaining -e usage. I got it talking to my cloud. hitting the ssl issue again, but I know how to work around that. thank you! | 12:47 |
| *** jtomasek has joined #openstack-infra | 12:47 | |
| *** rlandy has joined #openstack-infra | 12:48 | |
| *** alex_xu has joined #openstack-infra | 12:48 | |
| *** chem has joined #openstack-infra | 12:49 | |
| rcarrillocruz | ++ | 12:52 |
| openstackgerrit | Bin Hu proposed openstack-infra/project-config: New project openstack/gluon https://review.openstack.org/333843 | 12:52 |
| *** coolsvap has quit IRC | 12:52 | |
| *** edmondsw has joined #openstack-infra | 12:55 | |
| *** trown|outtypewww is now known as trown | 12:56 | |
| *** yamamoto has joined #openstack-infra | 12:57 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/infra-ansible: Set hostnames before the common play https://review.openstack.org/333893 | 12:58 |
| *** baoli has joined #openstack-infra | 12:58 | |
| *** links has quit IRC | 13:00 | |
| *** andymaier has joined #openstack-infra | 13:01 | |
| *** dizquierdo has quit IRC | 13:01 | |
| *** ddieterly has quit IRC | 13:02 | |
| *** amitgandhinz has joined #openstack-infra | 13:02 | |
| *** baoli_ has joined #openstack-infra | 13:02 | |
| *** infra-red has joined #openstack-infra | 13:04 | |
| *** infra-red has quit IRC | 13:05 | |
| *** baoli has quit IRC | 13:05 | |
| *** infra-red has joined #openstack-infra | 13:06 | |
| *** binbincong has quit IRC | 13:09 | |
| *** akshai has joined #openstack-infra | 13:14 | |
| *** pgadiya has quit IRC | 13:14 | |
| *** denisra has joined #openstack-infra | 13:15 | |
| *** dtantsur|brb is now known as dtantsur | 13:15 | |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: DNM: experimenting with local.conf https://review.openstack.org/326585 | 13:16 |
| AJaeger | jhesketh: thanks for retirng kilo | 13:16 |
| *** mdrabe has joined #openstack-infra | 13:16 | |
| jhesketh | no worries, sorry it took so long | 13:16 |
| *** amotoki has joined #openstack-infra | 13:16 | |
| mtreinish | notmorgan: I didn't pay for the modem, but it still has the sim slot (and antennas I think) If I wanted to use it I'm pretty sure I'd have to put a cellular modem in first | 13:18 |
| mtreinish | notmorgan: but I've not actually tried it | 13:18 |
| *** Na3iL has quit IRC | 13:18 | |
| *** infra-red has quit IRC | 13:19 | |
| *** esberglu has joined #openstack-infra | 13:19 | |
| AJaeger | jhesketh: did you also sign up for removing kilo from zuul/layout? | 13:20 |
| *** sdake has joined #openstack-infra | 13:21 | |
| jhesketh | AJaeger: nope, but I can | 13:21 |
| AJaeger | great! Please rember to remove now unused job definition as well - those that *only* run on kilo... | 13:22 |
| jhesketh | rgr | 13:22 |
| *** esberglu has quit IRC | 13:24 | |
| *** nwkarsten has joined #openstack-infra | 13:26 | |
| *** akshai_ has joined #openstack-infra | 13:26 | |
| *** piet has joined #openstack-infra | 13:26 | |
| *** amoralej is now known as amoralej|lunch | 13:29 | |
| openstackgerrit | Merged openstack-infra/python-storyboardclient: Updated Worklist API https://review.openstack.org/333828 | 13:29 |
| *** akshai has quit IRC | 13:30 | |
| *** xyang1 has joined #openstack-infra | 13:30 | |
| *** mriedem has joined #openstack-infra | 13:31 | |
| *** dmk0202 has quit IRC | 13:31 | |
| *** rbrndt_ has joined #openstack-infra | 13:32 | |
| *** ddieterly has joined #openstack-infra | 13:33 | |
| *** bin_ has joined #openstack-infra | 13:33 | |
| bin_ | Hi, I submitted patch for creating a new project "openstack/gluon": https://review.openstack.org/#/c/333843/ | 13:34 |
| bin_ | There is a build error: Gerritbot can only handle 120 channels but found 121. http://logs.openstack.org/43/333843/3/check/gate-project-config-irc-access/1ec2b49/console.html | 13:34 |
| bin_ | Looks like it cannot handle my IRC channel "#openstack-gluon" that I added in "accessbot/channels.yaml" and "gerritbot/channels.yaml" even if the access is ok "INFO:checkaccess:openstackinfra access ok on #openstack-gluon" | 13:35 |
| AJaeger | bin_: in this case: Do not touch gerritbot file. Accessbot change is fine. | 13:35 |
| bin_ | OK. I will remove the added lines in gerritbot file | 13:36 |
| bin_ | thank you | 13:36 |
| *** woodster_ has joined #openstack-infra | 13:37 | |
| openstackgerrit | Merged openstack-infra/project-config: Fix simple typo https://review.openstack.org/333880 | 13:37 |
| bin_ | BTW, can I also add " meetbots: - open_stack" in accessbot file? | 13:37 |
| *** dprince has quit IRC | 13:38 | |
| bin_ | does it mean the meetbots will be added to my channel "#openstack-gluon"? | 13:38 |
| *** Ravikiran_K has joined #openstack-infra | 13:38 | |
| *** esberglu has joined #openstack-infra | 13:39 | |
| *** dmellado_ is now known as dmellado | 13:40 | |
| *** ddieterly is now known as ddieterly[away] | 13:40 | |
| openstackgerrit | Bin Hu proposed openstack-infra/project-config: New project openstack/gluon https://review.openstack.org/333843 | 13:43 |
| *** Daisy has joined #openstack-infra | 13:45 | |
| *** piet has quit IRC | 13:46 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/project-config: Remove kilo jobs https://review.openstack.org/333910 | 13:46 |
| jhesketh | AJaeger: ^ there's a good chance I've missed something, but it's late and I'm going to bed sorry | 13:46 |
| *** lucas-hungry is now known as lucasagomes | 13:46 | |
| *** ddieterly[away] is now known as ddieterly | 13:47 | |
| *** Daisy has quit IRC | 13:49 | |
| *** amitgandhinz has quit IRC | 13:50 | |
| *** inc0 has joined #openstack-infra | 13:51 | |
| *** amitgandhinz has joined #openstack-infra | 13:51 | |
| bin_ | Do I need to specifically add reviewers? or as long as Gerritbot announced it this channel "Bin Hu proposed openstack-infra/project-config: New project openstack/gluon https://review.openstack.org/333843", it is set. and I just need to wait? | 13:51 |
| *** HeOS_ has quit IRC | 13:51 | |
| SamYaple | bin_: you just need to wait, give it a few days/week and if you dont have any reviews yet ask the channel here | 13:52 |
| *** dprince has joined #openstack-infra | 13:52 | |
| *** hongbin has joined #openstack-infra | 13:53 | |
| bin_ | Sure, I will just wait. Thank you | 13:54 |
| *** shahid__ has quit IRC | 13:56 | |
| *** jordanP has quit IRC | 13:56 | |
| *** matrohon has joined #openstack-infra | 13:59 | |
| pabelanger | morning | 14:01 |
| AJaeger | jhesketh: thanks, will review when I'm fully awake (post-lunch trauma right now) | 14:01 |
| *** afred312 has quit IRC | 14:05 | |
| *** eharney has joined #openstack-infra | 14:07 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add new db api method to get recent successes by metadata https://review.openstack.org/333460 | 14:10 |
| *** julim has joined #openstack-infra | 14:16 | |
| *** yaume has quit IRC | 14:19 | |
| *** ikhudoshyn has joined #openstack-infra | 14:21 | |
| *** amoralej|lunch is now known as amoralej | 14:22 | |
| *** maishsk has quit IRC | 14:24 | |
| *** tonytan4ever has joined #openstack-infra | 14:24 | |
| *** yamahata has joined #openstack-infra | 14:27 | |
| openstackgerrit | Sergey Skripnick proposed openstack-infra/project-config: Add pytest_results.html to test-results publisher https://review.openstack.org/333930 | 14:28 |
| *** bhavik has joined #openstack-infra | 14:29 | |
| *** YorikSar has quit IRC | 14:30 | |
| *** inc0 has quit IRC | 14:30 | |
| *** inc0 has joined #openstack-infra | 14:31 | |
| openstackgerrit | Mark Hamzy proposed openstack/diskimage-builder: dmidecode does not exist for ppc64/ppc64el https://review.openstack.org/333932 | 14:32 |
| *** psachin has quit IRC | 14:33 | |
| Shrews | jeblair: pabelanger: mordred and I have another patch to try: https://github.com/Shrews/ansible-modules-core/commit/8763f0c3f5f71d99f9e136f3fca5502c17c48952 | 14:33 |
| *** lvdongbing has joined #openstack-infra | 14:33 | |
| Shrews | jeblair: pabelanger: tl;dr, removes another "reading from the same file we are currently writing to" scenario | 14:34 |
| pabelanger | looking | 14:34 |
| pabelanger | Shrews: Ah, nice fined | 14:35 |
| pabelanger | find* | 14:36 |
| *** markusry has quit IRC | 14:36 | |
| *** inc0 has quit IRC | 14:36 | |
| Shrews | pabelanger: mordred's patch, i just did the grunt work | 14:36 |
| *** inc0 has joined #openstack-infra | 14:37 | |
| *** mdrabe has quit IRC | 14:37 | |
| *** rossella_s has quit IRC | 14:37 | |
| *** rossella_s has joined #openstack-infra | 14:38 | |
| openstackgerrit | Bilal proposed openstack-infra/python-storyboardclient: Add support for WorklistItems https://review.openstack.org/333939 | 14:38 |
| *** rook has quit IRC | 14:39 | |
| *** YorikSar has joined #openstack-infra | 14:39 | |
| *** dingyichen has quit IRC | 14:39 | |
| *** rook has joined #openstack-infra | 14:40 | |
| *** inc0 has quit IRC | 14:40 | |
| *** gongysh has joined #openstack-infra | 14:40 | |
| *** inc0 has joined #openstack-infra | 14:41 | |
| openstackgerrit | Bilal proposed openstack-infra/python-storyboardclient: Removed whitespace for PEP8 test https://review.openstack.org/333939 | 14:41 |
| openstackgerrit | Bilal proposed openstack-infra/python-storyboardclient: Add support for WorklistItems https://review.openstack.org/333939 | 14:42 |
| *** mdrabe has joined #openstack-infra | 14:44 | |
| *** julim has quit IRC | 14:44 | |
| *** dprince has quit IRC | 14:44 | |
| jeblair | Shrews, mordred, pabelanger: good morning! i've been catching up on #ansible-devel | 14:46 |
| *** eezhova has quit IRC | 14:46 | |
| pabelanger | I should do the same, let me get some coffee | 14:47 |
| *** Hal has joined #openstack-infra | 14:49 | |
| *** Guest25417 has quit IRC | 14:49 | |
| *** dprince has joined #openstack-infra | 14:49 | |
| *** Hal is now known as Guest38466 | 14:50 | |
| *** bhavik has quit IRC | 14:50 | |
| *** Guest38466 has quit IRC | 14:50 | |
| *** rook has quit IRC | 14:50 | |
| *** zz_dimtruck is now known as dimtruck | 14:51 | |
| *** eezhova has joined #openstack-infra | 14:51 | |
| *** ajmiller has joined #openstack-infra | 14:53 | |
| *** rook has joined #openstack-infra | 14:53 | |
| *** xenogear has quit IRC | 14:55 | |
| *** bhavik has joined #openstack-infra | 14:55 | |
| *** mrunge has quit IRC | 14:55 | |
| openstackgerrit | Antoine Cabot proposed openstack-infra/project-config: Add specs entry for Watcher specs https://review.openstack.org/333949 | 14:56 |
| jeblair | Shrews: that patch lgtm. i also really want to do something about communicate, but also can't get past the fact that at the end of the day we're going to have 'result' in memory in order to modify it... so... :) | 14:56 |
| *** infra-red has joined #openstack-infra | 14:56 | |
| *** ddieterly is now known as ddieterly[away] | 14:58 | |
| openstackgerrit | Antoine Cabot proposed openstack-infra/project-config: Add specs entry for Watcher specs https://review.openstack.org/333949 | 14:58 |
| jeblair | Shrews: i guess that's more of an ansible-module protocol issue -- like, if it were a multi-line protocol or something (like "result={json}\nstderr=string\nfuture_thing=foo") we wouldn't need to store anything in memory. | 14:58 |
| tristanC | fungi: about your openid provider question, red hat uses saml. google auth is probably a good candidate to act as an openid proxy for saml identity provider such as red hat | 14:59 |
| *** hurgleburgler has joined #openstack-infra | 14:59 | |
| *** mhickey has joined #openstack-infra | 15:00 | |
| fungi | tristanC: saml for staff authentication or for community authentication? | 15:01 |
| tristanC | and software-factory uses an apache module called cauth to abstract authentication (supporting local user, openid, ldap, keystone) and it relies on the remote_user header | 15:01 |
| fungi | i guess after a very long time of not using red hat i'm unclear on what/where their "community" is | 15:01 |
| pabelanger | derekh: http://paste.openstack.org/show/521959/ that is the exception we see in nodepool | 15:02 |
| tristanC | fungi: oh well, saml is for @redhat user, e.g. all employees | 15:02 |
| openstackgerrit | Merged openstack-infra/project-config: stop merging tags from stable branches into master https://review.openstack.org/333421 | 15:02 |
| fungi | tristanC: so anyway, the question was more about teh fact that the rdo project's gerrit has the ability to authenticate with github (openid) and it seemed like a strange choice since the fedora project has an openid provider up and running which all fedora project community members presumably have accounts in | 15:03 |
| *** zhurong has quit IRC | 15:03 | |
| fungi | but it's mostly my ignorance. i have no idea if fedora and red hat are at all tightly interrelated, or if red hat is a company with employees who turn fedora into a product | 15:04 |
| Shrews | jeblair: also, that output is limited to async_status output, so maybe that won't ever get too large? | 15:04 |
| *** bhavik has quit IRC | 15:04 | |
| tristanC | fedora community would use id.fedoraproject.org, and rdo picked github because most contributor already exist there... cauth query for user ssh key and inject them in gerrit account | 15:04 |
| *** jamesmcarthur has joined #openstack-infra | 15:04 | |
| *** Nikolay_St has quit IRC | 15:05 | |
| *** dizquierdo has joined #openstack-infra | 15:05 | |
| fungi | my last serious experiences with red hat stopped around red hat linux 7.3 and rhel 2.1 | 15:05 |
| *** devananda has joined #openstack-infra | 15:06 | |
| *** acoles is now known as acoles_ | 15:06 | |
| fungi | and fedora didn't exist back then, hence my lack of clarity on the community relationships | 15:06 |
| derekh | pabelanger: instance on rh2 were going to an error state, I'm currently investigating why, so I've disabled nodepools access to rh2 while I do that | 15:06 |
| *** tphummel has joined #openstack-infra | 15:06 | |
| derekh | pabelanger: I think the problem is that the images are > 5GB and as a result chunked in swift | 15:07 |
| jeblair | Shrews: but that would include stdout from, say, a shell module invocation, right? | 15:07 |
| tristanC | fungi: i really don't think red hat intent is to turn fedora into a product, and the community is self hosted | 15:07 |
| pabelanger | fungi: tristanC: to be fair, user ssh keys do exist on fedora FAS: https://admin.fedoraproject.org/accounts/user/view/pabelanger too | 15:07 |
| jeblair | Shrews: it won't be too big for us at least, since we're only running zuul_runner | 15:07 |
| *** vhosakot has joined #openstack-infra | 15:07 | |
| jeblair | Shrews: more of a general-case (or possibly zuulv3) concern | 15:07 |
| derekh | pabelanger: so I'm trying to find out if we have problems with images when they are split into segments | 15:08 |
| pabelanger | derekh: ya, we cache a lot of things on them | 15:08 |
| pabelanger | derekh: thanks for the heads up | 15:08 |
| *** rhochmuth has joined #openstack-infra | 15:08 | |
| Shrews | jeblair: i suppose so. good point | 15:08 |
| tristanC | pabelanger: then using fedora openid is probably something we can investigate for rdo | 15:09 |
| fungi | tristanC: i meant take the output from the fedora community, stabilize and productize it as a downstream/derivative distribution | 15:09 |
| fungi | sort of like what ubuntu does with debian | 15:09 |
| pabelanger | derekh: if you don't get things fixed for the weekend, do you mind setting tripleo-test-cloud-rh2 to -1 in nodepool.yaml? So were not continuing to launch servers | 15:09 |
| derekh | pabelanger: np, once I sort it out I'll re enable | 15:09 |
| derekh | pabelanger: lets set it know | 15:09 |
| derekh | pabelanger: *now | 15:09 |
| *** e0ne has quit IRC | 15:10 | |
| *** e0ne has joined #openstack-infra | 15:10 | |
| openstackgerrit | Bin Hu proposed openstack-infra/project-config: New project openstack/gluon, which brings a model-driven, extensible framework for L3 Networking Services. When a telecom service provider needs to provides its customers with a new L3 Networking Service, the new service is modeled in a YAML file. Gluon f https://review.openstack.org/333843 | 15:10 |
| pabelanger | derekh: great, thanks | 15:10 |
| fungi | tristanC: thanks. my main questions were around how much cross-over there is (or isn't) between the fedora developer community and the red hat developer community, or if red hat even has such a thing as a "developer community" vs simply "employees" | 15:11 |
| derekh | pabelanger: so do you want a patch to set "min-ready: -1" ? or are you setting it somewhere else? | 15:11 |
| *** rajinir has joined #openstack-infra | 15:11 | |
| *** number80 has joined #openstack-infra | 15:11 | |
| pabelanger | derekh: yes, a patch please | 15:11 |
| *** eharney has quit IRC | 15:12 | |
| fungi | tristanC: like, do interested users contribute patches to rhel, or does red hat tell them to go stick it in fedora and they'll pull from there later? | 15:12 |
| *** tesseract- has quit IRC | 15:12 | |
| *** matrohon has quit IRC | 15:13 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/project-config: Disable instances on rh2 https://review.openstack.org/333956 | 15:13 |
| derekh | pabelanger: ^ | 15:13 |
| *** dansmith is now known as superdan | 15:13 | |
| *** infra-red has quit IRC | 15:14 | |
| *** rook has quit IRC | 15:14 | |
| *** _nadya_ has quit IRC | 15:16 | |
| *** tonytan4ever has quit IRC | 15:19 | |
| *** salv-orl_ has quit IRC | 15:20 | |
| tristanC | fungi: i believe things need to happen in fedora first before it gets into rhel. | 15:21 |
| fungi | tristanC: i guess that includes rdo? rdo packages are developed in fedora and then adopted by rhel later? | 15:22 |
| *** bhavik has joined #openstack-infra | 15:22 | |
| fungi | just trying to understand how many of the people using the rdo gerrit are fedora community people | 15:23 |
| tristanC | fungi: that is correct, rdo is acting like fedora for the rhel openstack distribution | 15:23 |
| rajinir | Need some help with publishing console and devstack-logs. Devstack-logs , how do I setup? | 15:24 |
| fungi | rajinir: are you asking how to make a webserver? | 15:24 |
| tristanC | and it's likely we used github authentication for review.rdoproject.org simply because it's working out of the box in software-factory | 15:24 |
| *** rook has joined #openstack-infra | 15:24 | |
| *** arxcruz has quit IRC | 15:24 | |
| tristanC | well that upstream/downstream interaction is the true challenge of having both a community and corporate hat | 15:25 |
| rajinir | fungi: Have a log server and able to copy the logs. But missing the devstack-logs, so I guess, I'm missing some setup intructions | 15:25 |
| fungi | tristanC: got it, so that was one of the canned openid authentication options in software factory? it's not configurable to allow you to just specify an openid provider of your choice? | 15:25 |
| *** d0ugal has quit IRC | 15:25 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Update query for ceph scheduling bug 1475333 https://review.openstack.org/333963 | 15:25 |
| openstack | bug 1475333 in devstack "gate-tempest-dsvm-full-ceph fails with Insufficient free space for volume creation on host ubuntu@ceph#ceph (requested / avail): 1/0.0" [High,Confirmed] https://launchpad.net/bugs/1475333 | 15:25 |
| tristanC | fungi: it's likely id.fedora or google auth will also work out of the box, though from my understanding there are some variable to take into account when changing openid provider. | 15:26 |
| fungi | rajinir: if you're using devstack-gate, one of its final steps is to copy the logs into one spot on the job node so that your runner )jenkins or whatever) can retrieve those and publish them somewhere | 15:26 |
| *** eharney has joined #openstack-infra | 15:27 | |
| fungi | tristanC: cool. i think that answers my questions. i didn't know if that deployment wasn't using the fedora community idp because of compatibility issues betwene gerrit and ipsilon, or for unrelated reasons | 15:27 |
| tristanC | for example launchpad doesn't need any action, while github requires you to create an "application" | 15:27 |
| rajinir | fungi: You mean the logs are copied to the node to node where Jenkins is installed or copied to a location on the slave devstack node | 15:28 |
| fungi | rajinir: if you're using jenkins, its publishers pull logs off the jenkins slave back to the jenkins master and then push them somewhere else | 15:28 |
| *** javeriak has joined #openstack-infra | 15:29 | |
| rajinir | fungi: May be I'm missing the spot on the jenkins master where they are published I guessed | 15:29 |
| fungi | rajinir: so devstack-gate collects various devstack and tempest and other kinds of logs on the slave into a "logs" subdirectory of the job workspace and we have a jenkins publisher defined which publishes those | 15:29 |
| *** arxcruz has joined #openstack-infra | 15:30 | |
| tristanC | fungi: differences in authentication providers is the reason we developped cauth, so that all software factory services relies on a unified remote_user header mechanism | 15:30 |
| fungi | rajinir: the publisher definition we're using is this one http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n930 | 15:30 |
| fungi | tristanC: oh, right, so cauth is taking care of the openid auth anyway, and gerrit doesn't know about openid there | 15:30 |
| fungi | tristanC: thanks, my question was actually irrelevant in that case ;) | 15:31 |
| *** xdcc has joined #openstack-infra | 15:31 | |
| tristanC | fungi: exactly, well cauth will call an internal service (called managesf) that will inject user and ssh key per service databases | 15:31 |
| tristanC | fungi: you're welcome, glad i could clarify that question :) | 15:32 |
| fungi | rajinir: basically an scp publisher which acts on the contents of the logs directory on the slave after the job completes and uploads it to a server we're maintaining called static.openstack.org | 15:32 |
| *** ddieterly[away] is now known as ddieterly | 15:32 | |
| rajinir | fungi: only console-logs are in the job workspace on the jenkins master. I'll have to check why the logs from the slave node are not published | 15:33 |
| fungi | rajinir: oh, no console logs are special. are you getting all your logs _except_ the console logs showing up? | 15:33 |
| rajinir | fungi: only the console logs , devstack-logs are missing. May be I have some wrong logserver settings in the template | 15:34 |
| rajinir | will check | 15:34 |
| fungi | rajinir: console logs are actually stored in memory on the jenkins master, not a file anywhere. we fake copying that by retrieving it from memory with this publisher http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n919 | 15:34 |
| fungi | rajinir: and it depends on a custom feature clarkb hacked into the upstream scp publisher some years ago, so requires a fork of that plugin to get the console log from memory | 15:35 |
| openstackgerrit | Merged openstack-infra/project-config: Disable instances on rh2 https://review.openstack.org/333956 | 15:36 |
| sdague | fungi: is there a way to make a job fail but capture the logs? my local.conf d-g change is stalling on subnode and I'm 100% not sure why with the telnet console log | 15:36 |
| fungi | rajinir: also in fairness i should mention that all of this is (recently) historical, we no longer use jenkins as of a couple weeks ago and are doing basically the same things with ansible and a new daemon in zuul now | 15:36 |
| sdague | telnet://104.130.204.167:8088 | 15:36 |
| *** lvdongbing has quit IRC | 15:36 | |
| *** tqtran has joined #openstack-infra | 15:36 | |
| clarkb | fungi: its not a fork anymore but is on unreleased master of plugin iirc | 15:36 |
| fungi | sdague: hrm, i could probably ssh into it and kill something to abort the job payload early | 15:36 |
| rajinir | fungi: ok. Thanks for the info. I would like to make this work before we get rif of jenkins. will check the log server setup. | 15:36 |
| fungi | rajinir: oh, i'm not suggesting getting rid of jenkins, just mentioning that this is how it worked for us in the past, but i'm not implying that we're doing it the same way any longer | 15:37 |
| sdague | oh, wait, I might see the issue in the code | 15:37 |
| rajinir | fungi: ok, got it | 15:37 |
| sdague | stupid missing . | 15:37 |
| fungi | sdague: sometimes it's amazing how something so small as a dot can create a bug | 15:38 |
| *** sdake has quit IRC | 15:38 | |
| *** sdake has joined #openstack-infra | 15:38 | |
| *** sdake has quit IRC | 15:38 | |
| *** sdake has joined #openstack-infra | 15:38 | |
| *** pcaruana has quit IRC | 15:38 | |
| *** gouthamr has quit IRC | 15:39 | |
| mordred | jeblair: I also made this: https://github.com/emonty/ansible/commit/11bfa671246601c01767914a4f8b28f434fa80a0 | 15:39 |
| mordred | jeblair: which I think is likely not needed in reality with the async_module patch | 15:39 |
| mordred | but may be decent belt-and-suspenders | 15:39 |
| clarkb | fungi: rajinir also thwy arw stored on disk but not in html or text. they are encoded base64 or something. One thibg that plugin does is reencode into utf8 and wrap in html | 15:39 |
| *** gouthamr has joined #openstack-infra | 15:39 | |
| *** rcernin has quit IRC | 15:39 | |
| *** jlanoux_ has quit IRC | 15:40 | |
| *** tqtran has quit IRC | 15:40 | |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: support local.conf instead of localrc https://review.openstack.org/326585 | 15:40 |
| *** salv-orlando has joined #openstack-infra | 15:40 | |
| sdague | clarkb: ^^^ I think is remarkably close to working | 15:40 |
| *** roxanaghe has joined #openstack-infra | 15:41 | |
| jeblair | mordred: yeah, though, do we want to apply that now, as it might hide what we're tracking down? | 15:41 |
| sdague | also, why is d-g still running kilo jobs? :) | 15:42 |
| *** roxanaghe has quit IRC | 15:42 | |
| mordred | jeblair: no - I think we should see if the async_module patches fix it | 15:42 |
| *** jamesmcarthur has quit IRC | 15:42 | |
| mordred | jeblair: then if they do, make sure that the ansible patch doens't break it :) | 15:42 |
| *** xdcc has left #openstack-infra | 15:43 | |
| jeblair | mordred, Shrews, pabelanger: so i will now apply today's patch from Shrews instead of yesterdays patch. sound good? | 15:43 |
| *** jamesmcarthur has joined #openstack-infra | 15:43 | |
| pabelanger | no objection from me | 15:43 |
| Shrews | yup | 15:43 |
| *** Daisy has joined #openstack-infra | 15:45 | |
| jeblair | Shrews: https://github.com/Shrews/ansible-modules-core/commit/8763f0c3f5f71d99f9e136f3fca5502c17c48952 right? | 15:45 |
| *** ams__ has joined #openstack-infra | 15:45 | |
| Shrews | jeblair: correct | 15:46 |
| *** pt_15 has quit IRC | 15:46 | |
| *** arxcruz has quit IRC | 15:46 | |
| rajinir | clarkb, fungi ok, thx | 15:46 |
| *** zeih has quit IRC | 15:46 | |
| *** zul has quit IRC | 15:47 | |
| *** ccamacho|lunch is now known as ccamacho|away | 15:47 | |
| fungi | sdague: i think jhesketh was proposing (or proposed) a change today to remove kilo now that he's deleted the branches | 15:49 |
| *** e0ne has quit IRC | 15:49 | |
| *** itisha has quit IRC | 15:50 | |
| *** Daisy has quit IRC | 15:50 | |
| *** xarses has quit IRC | 15:50 | |
| *** daemontool_ has joined #openstack-infra | 15:52 | |
| *** baoli has joined #openstack-infra | 15:53 | |
| *** lvdongbing has joined #openstack-infra | 15:53 | |
| *** baoli_ has quit IRC | 15:53 | |
| *** daemontool has quit IRC | 15:54 | |
| *** ddieterly is now known as ddieterly[away] | 15:55 | |
| *** baoli has quit IRC | 15:55 | |
| *** alex_xu has quit IRC | 15:55 | |
| *** baoli has joined #openstack-infra | 15:55 | |
| *** raunak has joined #openstack-infra | 15:56 | |
| *** baoli_ has joined #openstack-infra | 15:57 | |
| *** ddieterly[away] is now known as ddieterly | 15:58 | |
| *** andymaier has quit IRC | 15:58 | |
| *** alex_xu has joined #openstack-infra | 15:58 | |
| jeblair | funny story -- ansible and the core modules repo share a history so you can push one to the other. whoops. | 15:59 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove kilo jobs https://review.openstack.org/333910 | 16:00 |
| *** baoli has quit IRC | 16:01 | |
| clarkb | jeblair: as a fast forward? awesome | 16:01 |
| jeblair | clarkb: well, not as a fast-forward -- i can't ff these branches anyway since i'm rewinding a commit | 16:02 |
| jeblair | okay, i think i have them all staged now | 16:03 |
| *** cody-somerville has joined #openstack-infra | 16:03 | |
| jeblair | so i will do the stop/install/start thing | 16:03 |
| *** vhosakot has quit IRC | 16:03 | |
| *** vhosakot has joined #openstack-infra | 16:04 | |
| *** kushal has quit IRC | 16:04 | |
| asselin | rcarrillocruz, so I'm noticing that with the current cloud-launcher, floating ip times out per ansible, but it succeeds after the timeout. Running ansible again and it complains there's already an ip. http://paste.openstack.org/show/521978/ | 16:04 |
| *** Nikolay_St has joined #openstack-infra | 16:05 | |
| *** degorenko is now known as _degorenko|afk | 16:06 | |
| *** esikachev has quit IRC | 16:06 | |
| *** gongysh has quit IRC | 16:07 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Gerritbot: Remove obsolete kilo conditions https://review.openstack.org/333977 | 16:08 |
| jeblair | okay, i've got this in a playbook | 16:09 |
| jeblair | so i'm about to run an ansible command that will end up aborting all running jobs in fairly quick succession | 16:09 |
| *** esikachev has joined #openstack-infra | 16:09 | |
| mordred | jeblair: neat! sledgehammers! | 16:09 |
| jeblair | (and they should all get restarted immediately) | 16:09 |
| jeblair | http://paste.openstack.org/show/521981/ | 16:09 |
| jeblair | mordred, pabelanger, clarkb: ^ that look right? | 16:10 |
| jeblair | (it's basically a copy of the graceful restart playbook, but without the graceful, and with the ansible install) | 16:10 |
| *** piet has joined #openstack-infra | 16:10 | |
| jeblair | the timeout should actually be more like 30 seconds than 3 hours, but hey | 16:10 |
| *** akshai_ has quit IRC | 16:11 | |
| mordred | jeblair: yah. lgtm | 16:11 |
| pabelanger | jeblair: Ya, looks good from what I see | 16:12 |
| rcarrillocruz | asselin: that's not an issue with the role, but an issue with your cloud | 16:12 |
| rcarrillocruz | i had similar issues with other HOS clouds | 16:12 |
| pabelanger | jeblair: poll could be reduced from 60s, but the is a nit | 16:12 |
| rcarrillocruz | check on a devstack, you will see it just works | 16:12 |
| asselin | rcarrillocruz, how to work around? | 16:12 |
| jeblair | pabelanger: oh, yeah, i'll bump it down to 5 | 16:12 |
| mordred | rcarrillocruz: if it fails on HOS but succeeds on devstack, we should see if shade can do a better job of dealing with that perhaps? | 16:13 |
| rcarrillocruz | yeah, iirc it was some issue with the sync between neutron and nova | 16:13 |
| jeblair | oh, i formatted the chdir with shell wrong | 16:13 |
| rcarrillocruz | where | 16:13 |
| rcarrillocruz | it would should show in some neutron table the floating ip was there | 16:13 |
| pabelanger | Ah, right, I think that need args | 16:13 |
| rcarrillocruz | but nova didn't have that synced | 16:13 |
| rcarrillocruz | and the call waits | 16:13 |
| rcarrillocruz | that was a while ago | 16:14 |
| mordred | rcarrillocruz: yah. this is a bug we've seen in older openstack versions for sure | 16:14 |
| *** bin_ has quit IRC | 16:14 | |
| rcarrillocruz | asselin: try reproducing with the corresponding shade call | 16:14 |
| mordred | rcarrillocruz: I've got a patch somewhere to make shade disregard nova's view of floating ips and to use neutron's instead | 16:14 |
| rcarrillocruz | feel free to open a bug or ping me | 16:14 |
| rcarrillocruz | mordred: right, i even remember pinging you about it at some point about this | 16:14 |
| mordred | since it doesn't actually matter if nova knows about the FIP at all, as long as neutron has it wired up to NAT to the right thing | 16:14 |
| rcarrillocruz | ++ | 16:14 |
| mordred | rcarrillocruz: https://review.openstack.org/#/c/315713/ | 16:15 |
| rcarrillocruz | asselin: i remember i had to circument that but not using os_server auto_ip, but use os_floating_ip, and then reference the IP created to the os_server task | 16:15 |
| rcarrillocruz | but obviously, that's not in the launcher | 16:15 |
| *** esikachev has quit IRC | 16:15 | |
| mordred | rcarrillocruz: maybe we should get that patch into shape | 16:16 |
| rcarrillocruz | that workaround i tell you predates the cloud launcher, it was my first baby steps with ansible openstack modules | 16:16 |
| asselin | ok yeah...but auto_ip should check if there's already an ip, no? how does it know when to stop? | 16:16 |
| *** roxanaghe has joined #openstack-infra | 16:16 | |
| rcarrillocruz | asselin: the problem is that you have the FIP in two tables | 16:16 |
| rcarrillocruz | nova | 16:16 |
| rcarrillocruz | and neutron | 16:16 |
| *** xarses has joined #openstack-infra | 16:16 | |
| rcarrillocruz | so you may have the FIP created, yet the sahde call waiting | 16:16 |
| mordred | asselin: yah. it's an openstack issue - and one of the many reasons that clouds that require floating ips are horrible to work with | 16:17 |
| rcarrillocruz | mordred: happy to work that ticket | 16:17 |
| rcarrillocruz | asselin: do this, try to reproduce with a shade call | 16:17 |
| rcarrillocruz | i'll get an account in your HOS clouds | 16:17 |
| rcarrillocruz | cloud | 16:17 |
| jeblair | mordred, pabelanger, Shrews: restarts complete | 16:17 |
| rcarrillocruz | and i'll work it with monty's patch | 16:17 |
| asselin | i see....so this is a openstack issue shade doesn't (yet) make it transparent | 16:17 |
| *** tphummel has quit IRC | 16:18 | |
| *** weshay is now known as weshay_brb | 16:18 | |
| asselin | mordred, nice commit message :) | 16:18 |
| clarkb | jeblair: ya that seems fine to me, maybe add it to a tools repo in zuul? | 16:18 |
| pabelanger | jeblair: what do you think about adding an optional argument to graceful, which is the max time we want to wait? If the timeout is reached, we move to stop() regardless if self.builds is None | 16:19 |
| Shrews | jeblair: where does the patching part come in? | 16:20 |
| Shrews | is that done manually? | 16:20 |
| rcarrillocruz | asselin: let me know how the reprdouce goes, feel free to open a story on storyboard about it | 16:20 |
| rcarrillocruz | i'll pick up | 16:20 |
| *** amitgandhinz has quit IRC | 16:20 | |
| *** abregman|afk has quit IRC | 16:21 | |
| *** amitgandhinz has joined #openstack-infra | 16:21 | |
| *** abregman|afk has joined #openstack-infra | 16:21 | |
| jeblair | Shrews: yeah, i have not playbooked that yet. it's a bunch of git commands which i have already done. | 16:21 |
| *** abregman_ has joined #openstack-infra | 16:21 | |
| asselin | rcarrillocruz, so shade storyboard? | 16:21 |
| jeblair | Shrews: so the /root/ansible repo is already prepped on all the hosts | 16:21 |
| rcarrillocruz | asselin: yep | 16:21 |
| Shrews | jeblair: ah, ok | 16:22 |
| jeblair | pabelanger: that could be nice. a sort of *mostly* graceful. | 16:22 |
| jeblair | pabelanger: i tried to add an argument to the command pipe stuff recently, it's not trivial. :) | 16:22 |
| *** mriedem has quit IRC | 16:22 | |
| clarkb | pabelanger: jeblair we can all it the tripleo-timeout | 16:22 |
| pabelanger | jeblair: Ya, maybe in cases like this, we can buffer 10mins to give jobs that are close to the end time to finish, then abort | 16:22 |
| *** abregman_ has quit IRC | 16:23 | |
| jeblair | clarkb: just what i was thinking | 16:23 |
| pabelanger | clarkb: :D | 16:23 |
| *** gouthamr has quit IRC | 16:23 | |
| *** abregman has joined #openstack-infra | 16:23 | |
| *** zul has joined #openstack-infra | 16:23 | |
| jeblair | pabelanger: if you attempt this -- start with the argparse part of the change -- that's the hardest part :) | 16:23 |
| *** piet has quit IRC | 16:24 | |
| jeblair | oh, and there's the spike visible on the node graph | 16:24 |
| fungi | also sometimes the ironic timeout | 16:25 |
| *** Apoorva has joined #openstack-infra | 16:25 | |
| *** eharney has quit IRC | 16:25 | |
| *** abregman|afk has quit IRC | 16:26 | |
| openstackgerrit | Milan Kováčik proposed openstack-infra/project-config: Introduce experimental Inspector grenade job https://review.openstack.org/331001 | 16:26 |
| clarkb | jeblair: Shrews' patch didn't fix the job fails? Did we at least get more info from the extra logging? | 16:27 |
| asselin | rcarrillocruz, https://storyboard.openstack.org/#!/story/2000659. will try to figure out the shade reproducer next. | 16:27 |
| jeblair | clarkb: we did get info from the logging and identified another similar/related potential failure, so we just installed v2 of Shrews patch that corrects that | 16:27 |
| pabelanger | jeblair: sure, I'll see about giving it a try this afternoon | 16:27 |
| clarkb | jeblair: with the finally block? | 16:28 |
| jeblair | did anyone else's irc just look like this? http://paste.openstack.org/show/521992/ | 16:28 |
| clarkb | jeblair: no | 16:28 |
| *** akshai has joined #openstack-infra | 16:28 | |
| clarkb | jeblair: try a ^R | 16:28 |
| jeblair | clarkb: and communicate() | 16:28 |
| *** weshay_brb is now known as weshay | 16:28 | |
| pabelanger | no, I got the full line correctly too | 16:29 |
| jeblair | clarkb: i'm in screen; i just switched screens to refresh and it's still like that | 16:29 |
| pabelanger | jeblair: can I delete the currently help nodes in nodepool? | 16:29 |
| clarkb | huh, I see two separate lines fungi | also sometimes the ironic timeout | 16:29 |
| jeblair | eavesdrop also saw things correctly. that's very weird. | 16:29 |
| jeblair | pabelanger: please do | 16:29 |
| *** hashar has quit IRC | 16:29 | |
| pabelanger | jeblair: done | 16:30 |
| fungi | weechat didn't do anything weird on my end, but that could be explained by the first comment being mine so different echo | 16:30 |
| pabelanger | I have to admit, I've had to purge 0 leaked nodes in nodepool since switching to zuul-launcher. It is very nice | 16:31 |
| *** yamahata has quit IRC | 16:31 | |
| fungi | i'm guessing it's something to do with some terminals' rendering of the name "Milan Kováčik" | 16:31 |
| *** javeriak has quit IRC | 16:31 | |
| *** ajmiller has quit IRC | 16:31 | |
| fungi | maybe one of those causes the cursor to move up one line? | 16:31 |
| *** ajmiller has joined #openstack-infra | 16:32 | |
| fungi | presumably the č | 16:32 |
| fungi | which seems to have been interpreted in your terminal as Ä<up cursor> | 16:32 |
| *** alex_xu has quit IRC | 16:32 | |
| jeblair | aha! | 16:32 |
| jeblair | fungi: you just made my terminal go wonky | 16:33 |
| *** ajmiller has quit IRC | 16:33 | |
| fungi | hah, i can recommend some setting for traditional xterm! ;) | 16:33 |
| *** ajmiller has joined #openstack-infra | 16:33 | |
| jeblair | so i think i did get all the data in the correct order | 16:33 |
| jeblair | the gerritbot line just ended up doing some cursor movement | 16:33 |
| *** eezhova has quit IRC | 16:34 | |
| fungi | i spent a long time playing around with making sure utf-8 encoded japanese rendered correctly over telnet, tmux, ssh, et cetera | 16:34 |
| jeblair | fungi: can you make it go back now? | 16:34 |
| jeblair | [16:34] [jeblair(+Zi)] [3:freenode/#openstack-infra(+cnt)] [Act: 2,4,5,6<up cursor> | 16:34 |
| jeblair | fungi: ^ | 16:34 |
| pabelanger | ha ha ha | 16:34 |
| fungi | wow | 16:34 |
| pabelanger | awesome | 16:34 |
| *** dmk0202 has joined #openstack-infra | 16:34 | |
| pabelanger | ascii injections! | 16:34 |
| jeblair | what's weird is i thought i just got this thing working right with utf8 | 16:34 |
| asselin | rcarrillocruz, mordred fwiw that patch gets past my error | 16:34 |
| *** jamesmcarthur has quit IRC | 16:34 | |
| jeblair | pabelanger: seriously | 16:34 |
| *** alex_xu has joined #openstack-infra | 16:35 | |
| openstackgerrit | Alexander Chadin proposed openstack-infra/project-config: Add rally job for Watcher service and support for rally-gate https://review.openstack.org/333860 | 16:35 |
| jeblair | EmilienM: we have a new version of the patch intended to fix the ansible problem in place | 16:36 |
| fungi | actually, it looks like i've since been able to undo most of my tweaks to xterm configuration (except for needing to keep eightBitInput disabled) | 16:36 |
| *** jpich has quit IRC | 16:36 | |
| fungi | so really, no clue there | 16:37 |
| fungi | but yes, that is certainly disturbing behavior | 16:37 |
| fungi | what terminal emulator are you using? rxvt if memory serves? | 16:37 |
| clarkb | Shrews: jeblair the github "change" lgtm. Also thats a fun race, everyone read and write to the same file all at once | 16:37 |
| *** eharney has joined #openstack-infra | 16:38 | |
| *** jamesmcarthur has joined #openstack-infra | 16:38 | |
| clarkb | I am also impressed that the stderr and stdout were going to the same place but we were expecting valid json in the end | 16:38 |
| fungi | i guess now i have enough info to be able to engineer ansi animation over irc for rxvt users ;) | 16:38 |
| *** javeriak has joined #openstack-infra | 16:39 | |
| openstackgerrit | Milan Kováčik proposed openstack-infra/project-config: Introduce experimental Inspector grenade job https://review.openstack.org/331001 | 16:39 |
| clarkb | just keep getting Milan to push more patches | 16:40 |
| openstackgerrit | Milan Kováčik proposed openstack-infra/project-config: Introduce experimental Inspector grenade job https://review.openstack.org/331001 | 16:40 |
| clarkb | :) | 16:40 |
| jeblair | oh good grief | 16:40 |
| clarkb | jeblair: do you use urxvt-256color? | 16:41 |
| jeblair | clarkb: i use urxvt, but i think the problem is screen | 16:41 |
| mordred | asselin: my shade patch for the FIPs? that's good to know | 16:41 |
| clarkb | ah | 16:41 |
| pabelanger | clarkb: almost like you planned that | 16:41 |
| clarkb | right screen needs to be started with -U and compiled with that support in place | 16:41 |
| asselin | mordred, yes https://review.openstack.org/#/c/315713/1 | 16:41 |
| mordred | asselin: cool! | 16:42 |
| *** tphummel has joined #openstack-infra | 16:42 | |
| asselin | shade can handle schrodinger's cat | 16:43 |
| *** ddieterly has quit IRC | 16:44 | |
| *** sarob has joined #openstack-infra | 16:44 | |
| *** rbrndt_ has quit IRC | 16:44 | |
| *** gouthamr has joined #openstack-infra | 16:44 | |
| mordred | :) | 16:45 |
| *** dtantsur is now known as dtantsur|afk | 16:45 | |
| *** derekh has quit IRC | 16:45 | |
| clarkb | just not floating IPs | 16:46 |
| *** sarob has quit IRC | 16:47 | |
| *** zeih has joined #openstack-infra | 16:47 | |
| *** sarob has joined #openstack-infra | 16:47 | |
| clarkb | semi random probably terrible idea | 16:47 |
| *** lucasagomes is now known as lucas-afk | 16:47 | |
| Shrews | jeblair: mosh. try it. live it. love it | 16:48 |
| clarkb | what if we ran the infracloud control plane somewhere a bit more static, like in a cloud | 16:48 |
| clarkb | then whenever HP or whoever has hardware we set them up as compute hosts against that control plane | 16:48 |
| clarkb | then infracloud never goes away, just the compute hosts do | 16:48 |
| nibalizer | 16:48 | |
| nibalizer | herp a derp! | 16:48 |
| bkero | What advantage does that provide? You can quantify how many compute hosts you lack for the services? | 16:49 |
| nibalizer | clarkb: don't we need pretty strong links between controller and compute? | 16:49 |
| clarkb | ttx: ^ was it you that was talking to an openstack deployer that does similar | 16:49 |
| *** Nikolay_St has quit IRC | 16:49 | |
| clarkb | nibalizer: I think ttx was saying someone is doing something similar with geographically distant hosting locations | 16:49 |
| clarkb | bkero: the advantage is not needing to reroll all of the services every 2 months | 16:49 |
| nibalizer | clarkb: i think thats a capital idea though! since we'd then be able to keep the control plane (essentially pets) online and updated | 16:50 |
| clarkb | bkero: compute hosts are relatively simple | 16:50 |
| *** rbrndt_ has joined #openstack-infra | 16:50 | |
| *** acabot has quit IRC | 16:50 | |
| nibalizer | then we just semi dynamically spin up and down computes | 16:50 |
| bkero | It sounds like a good ide aif you expect that your compute resources are going to shift out from under you | 16:50 |
| nibalizer | bkero: thats the reality | 16:50 |
| bkero | That's been the reality, yeah. | 16:50 |
| bkero | Will it always be? and if it isn't anymore, does the added complexity make it worth it? | 16:51 |
| jeblair | nibalizer: pets are the ones you euthanize when sick, right? | 16:51 |
| *** rossella_s has quit IRC | 16:51 | |
| *** zeih has quit IRC | 16:51 | |
| clarkb | bkero: I think we expected it to be a one and done situation, then they decided to do it again. So while I can't say it will always be this way all signs point to yes | 16:51 |
| nibalizer | jeblair: haha | 16:51 |
| nibalizer | i do love the jeblair-corollary to the pets-cattle metaphor | 16:52 |
| bkero | clarkb: then having a lightweight control plane set up somewhere else makes sense | 16:52 |
| bkero | What's the bandwidth between compute and control plane though? | 16:52 |
| Shrews | jeblair: mordred: looks like maybe no love on the patch? http://logs.openstack.org/61/318961/13/check/gate-puppet-openstack-integration-4-scenario002-tempest-ubuntu-xenial-nv/d24c4fd/_zuul_ansible/ansible_log.txt | 16:52 |
| rcarrillocruz | El mñn | 16:52 |
| clarkb | bkero: the real hurt there is going to be our test images | 16:52 |
| bkero | exactly | 16:53 |
| clarkb | since they are huge and transfering from hosts in eg rax to hardware in HP might not be happyness | 16:53 |
| bkero | 11gb over commodity internet for every new test image? | 16:53 |
| clarkb | bkero: ~5GB since we could qcow2 | 16:53 |
| bkero | still | 16:53 |
| clarkb | anyways if its something we want to investigate we can take that into account and determine if there are ways of dealing with that | 16:53 |
| bkero | That's okay if it's line speed, but I doubt it'll be line speed | 16:53 |
| nibalizer | so for ephmeral, as i understand it, the cinder controller pushes a copy of each glance image on to each compute | 16:54 |
| nibalizer | so that would be sorta painful | 16:54 |
| mordred | Shrews: FAILED! => {"changed": false, "failed": true, "msg": "async task did not complete within the requested time; {'failed': True, 'module_stderr': u'', 'module_stdout': u'', 'msg': 'MODULE FAILURE', 'invocation': {'module_name': u'async_status'}, 'parsed': False}"} _EXCELLENT_ | 16:54 |
| jeblair | pabelanger: ^ | 16:54 |
| clarkb | nibalizer: no cinder is involved | 16:54 |
| nibalizer | but i think clarks point is a) we could solve that problem and b) less painful than reloading the control plane every X weeks? | 16:54 |
| nibalizer | oh sorry maybe I meant glance? | 16:55 |
| clarkb | nibalizer: qemu/nova/libvirt cache the qcow2 on the hypervisors | 16:55 |
| jeblair | mordred: does that tell you more about the problem? | 16:55 |
| mordred | jeblair: nope | 16:55 |
| mordred | jeblair: it's the same error as before | 16:55 |
| clarkb | nibalizer: so essentially we would have large network bw needs once a day, or we would need to tackle this differneyl | 16:55 |
| Shrews | are we 100% certain that zl* node has the patch? | 16:56 |
| Shrews | b/c ugh | 16:56 |
| clarkb | its probably not the sanest idea, and there are likely real challenges involved with it. Just trying to think of ways we can be productive without wondering/waiting for next move | 16:56 |
| *** Goneri has joined #openstack-infra | 16:56 | |
| jeblair | mordred: oh so that was sarcasm :) | 16:57 |
| *** ajmiller has quit IRC | 16:57 | |
| jeblair | Shrews: i'll go hunting | 16:57 |
| *** ajmiller has joined #openstack-infra | 16:57 | |
| nibalizer | clarkb: yah | 16:58 |
| Shrews | jeblair: we've got to devise a better way to test things like this, because this is pretty disruptive and difficult | 16:58 |
| *** trown is now known as trown|lunch | 16:58 | |
| nibalizer | for the midcycle, did we settle on topics? I don't see topics on https://wiki.openstack.org/wiki/Sprints/QAInfraNewtonSprint | 16:59 |
| *** salv-orl_ has joined #openstack-infra | 16:59 | |
| nibalizer | i heard infracloud and zuulv3, but I don't see it written down | 16:59 |
| jeblair | Shrews: i'm open to suggestions. though, to be honest, i would prefer that happen in the ansible context :) | 17:00 |
| *** rkukura has quit IRC | 17:00 | |
| anteaya | nibalizer: there has been no decision that I have seen | 17:00 |
| *** inc0 has quit IRC | 17:00 | |
| anteaya | nibalizer: you could write them down as suggestions, then at least they are written down | 17:00 |
| *** inc0 has joined #openstack-infra | 17:01 | |
| nibalizer | ok i will create a topics section | 17:01 |
| *** acabot has joined #openstack-infra | 17:01 | |
| *** rkukura has joined #openstack-infra | 17:01 | |
| *** rkukura has quit IRC | 17:02 | |
| jeblair | Shrews: http://paste.openstack.org/show/521994/ | 17:02 |
| *** bhavik1 has joined #openstack-infra | 17:02 | |
| nibalizer | we shoul try to figure out what are 'accepted' topics though pretty soon so the relevant personnel can get travel authorized | 17:02 |
| *** akshai has quit IRC | 17:02 | |
| *** salv-orlando has quit IRC | 17:02 | |
| Shrews | *sigh* | 17:02 |
| *** akshai has joined #openstack-infra | 17:02 | |
| openstackgerrit | Alexander Chadin proposed openstack-infra/project-config: Add rally job for Watcher service and support for rally-gate https://review.openstack.org/333860 | 17:03 |
| Shrews | all of the centos jobs succeeded. this is xenial only | 17:03 |
| Shrews | maybe we should dig into the differences there | 17:03 |
| clarkb | xenial uses python3 by default, maybe ansible is using that instead of 2? | 17:04 |
| *** piet has joined #openstack-infra | 17:04 | |
| Shrews | orly? hrm | 17:04 |
| clarkb | Shrews: `python` is still python2 though | 17:04 |
| clarkb | so depends on how ansible invokes the remote python processes | 17:04 |
| Shrews | clarkb: it will use the system python unless you give it an explicit one in the host file | 17:05 |
| *** bhavik has quit IRC | 17:05 | |
| *** bhavik1 is now known as bhavik | 17:05 | |
| Shrews | ansible_python_interpreter or such | 17:05 |
| anteaya | nibalizer: I think that is a reasonable position | 17:05 |
| openstackgerrit | Milan Kováčik proposed openstack-infra/project-config: Introduce experimental Inspector grenade job https://review.openstack.org/331001 | 17:06 |
| mordred | nibalizer: ++ | 17:06 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove kilo jobs https://review.openstack.org/333910 | 17:07 |
| AJaeger | I've just updated jhesketh 's change ^ | 17:07 |
| nibalizer | okay created a topics section, with proposed and accepted subsections https://wiki.openstack.org/wiki/Sprints/QAInfraNewtonSprint | 17:07 |
| *** sdake_ has joined #openstack-infra | 17:07 | |
| nibalizer | might be a thing that's easier done on an etherpad | 17:07 |
| AJaeger | project-config cores, I think this is fine now - but it's a larger change, so quick reviews - instead of endless rebases - would be appreciated for 333910. | 17:07 |
| *** sdake has quit IRC | 17:09 | |
| *** lvdongbing has quit IRC | 17:12 | |
| AJaeger | EmilienM: could you double check the puppet branches, please? There're still some juno and icehouse there that surprised me - see 333910 | 17:13 |
| rcarrillocruz | asselin: cool, I will take a look and assign myself | 17:13 |
| *** stajkowski has quit IRC | 17:14 | |
| *** jed56 has quit IRC | 17:15 | |
| *** dmk0202 has quit IRC | 17:15 | |
| clarkb | AJaeger: jhesketh I am reviewing the kilo removal change in project-config now | 17:15 |
| *** ifarkas has quit IRC | 17:15 | |
| *** dmk0202 has joined #openstack-infra | 17:16 | |
| *** dtardivel has quit IRC | 17:17 | |
| *** yamahata has joined #openstack-infra | 17:17 | |
| clarkb | AJaeger: also looks like networking-odl has juno and icehouse rules in zuul's layout file | 17:17 |
| EmilienM | Ajaeger: currently on my phone, will look later. I'm on holiday today | 17:17 |
| *** flepied has quit IRC | 17:18 | |
| openstackgerrit | Merged openstack-infra/infra-specs: Priority specs cleanup/update for Newton cycle https://review.openstack.org/331903 | 17:19 |
| mordred | Shrews: Python 2.7.11+ (default, Apr 17 2016, 14:00:29) | 17:19 |
| *** kzaitsev_mb has quit IRC | 17:19 | |
| mordred | Shrews: that's the python that shows up with /usr/bin/python on the xenial nodes | 17:19 |
| notmorgan | mordred: ++ | 17:20 |
| strigazi | AJaeger: hi, magnum's job for the install-guide is expected to published here, right? http://docs.openstack.org/project-install-guide/container-infrastructure-management/draft/ | 17:20 |
| *** piet has quit IRC | 17:20 | |
| notmorgan | mordred: i think latest fedora and the rolling releases are the only ones defaulting to py3 | 17:20 |
| Shrews | mordred: jeblair: can we add my pub key to one of the held xenial nodes and let me run a few tests against it? | 17:20 |
| clarkb | mistral has a juno rule too | 17:20 |
| notmorgan | mordred: for "/usr/bin/python" | 17:20 |
| clarkb | notmorgan: fedora shouldn't use py3 in /usr/bin/python either | 17:21 |
| AJaeger | EmilienM: double check it next week then - we cna do more cleanjup as needed... | 17:21 |
| clarkb | notmorgan: defaulting to python3 means python2 is not installed but `python` should always give you a python2 unless you arch | 17:21 |
| clarkb | or gentto | 17:21 |
| AJaeger | clarkb, thanks for reviewing. There's still a branch in git for networking-odl. | 17:21 |
| clarkb | AJaeger: as long as we don't have to keep old test platforms alive for them its probably fine | 17:21 |
| Shrews | i'm curious if even a simple async playbook will work correctly | 17:21 |
| AJaeger | clarkb: I hope not ;) | 17:21 |
| AJaeger | clarkb: I'm patching jenkins/jobs/ now for kilo... | 17:22 |
| notmorgan | clarkb: it also in some cases depends on the order the packages are installed =/ I ran into this on a VM i had | 17:22 |
| asselin | rcarrillocruz, i'm seeing an issue though with launch node. So on subsequent runs it seems to use a different ssh key instead of the one that's registered | 17:22 |
| clarkb | notmorgan: python upstreams says `python` is always python2, most distros are at least attempting to honor that | 17:23 |
| *** SumitNaiksatam has quit IRC | 17:23 | |
| *** electrofelix has quit IRC | 17:23 | |
| notmorgan | clarkb: silly distros... =/ | 17:23 |
| notmorgan | clarkb: that don't honor it that is | 17:23 |
| notmorgan | not that do. | 17:23 |
| notmorgan | i am guessing i'd feel the same way with perl and perl6 :P if i used perl still. | 17:24 |
| *** roxanaghe has quit IRC | 17:24 | |
| *** cbader has quit IRC | 17:24 | |
| *** roxanaghe has joined #openstack-infra | 17:24 | |
| rcarrillocruz | that is not an issue, but expected. The key generated in the playbook is created with a random file | 17:24 |
| *** dprince has quit IRC | 17:24 | |
| rcarrillocruz | If you want idempotency, just use the role with a resources.yaml with the keypair there | 17:25 |
| mordred | Shrews: totes. one sec | 17:25 |
| *** sabari has left #openstack-infra | 17:25 | |
| asselin | rcarrillocruz, ok | 17:25 |
| *** tosky has quit IRC | 17:26 | |
| mordred | Shrews: is https://launchpad.net/%7Edshrews/+sshkeys still valid? | 17:26 |
| *** amitgandhinz has quit IRC | 17:26 | |
| Shrews | mordred: aye | 17:26 |
| mordred | Shrews: jenkins@104.130.203.55 | 17:26 |
| Shrews | danke | 17:27 |
| clarkb | AJaeger: jhesketh I have approved that change, will keep an eye on it for fallout if we missed something | 17:27 |
| *** cody-somerville has quit IRC | 17:27 | |
| *** cody-somerville has joined #openstack-infra | 17:27 | |
| *** cody-somerville has quit IRC | 17:27 | |
| *** cody-somerville has joined #openstack-infra | 17:27 | |
| *** amitgandhinz has joined #openstack-infra | 17:27 | |
| jeblair | mordred, Shrews: that node ran http://logs.openstack.org/39/333839/3/check/gate-puppet-openstack-integration-3-scenario001-tempest-ubuntu-xenial-nv/197db2e/ | 17:27 |
| AJaeger | thanks, clarkb ! | 17:27 |
| asselin | rcarrillocruz, you don't need it inside launch_node? | 17:27 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Jenkins/jobs: Kilo cleanup https://review.openstack.org/334008 | 17:28 |
| AJaeger | clarkb: and one more ^ | 17:28 |
| AJaeger | project-config cores, 334008 and https://review.openstack.org/333977 are the rest for the kilo cleanup | 17:28 |
| asselin | rcarrillocruz, also it's set here: { role: cloud-launcher, generate_keypair: yes } | 17:28 |
| jeblair | mordred, Shrews: i think that might be a legit failure (eg, not ansible) | 17:28 |
| jeblair | http://logs.openstack.org/39/333839/3/check/gate-puppet-openstack-integration-3-scenario001-tempest-ubuntu-xenial-nv/197db2e/console.html#_2016-06-24_16_55_30_345118 | 17:29 |
| jeblair | yep | 17:29 |
| *** grassass has quit IRC | 17:29 | |
| *** tqtran has joined #openstack-infra | 17:30 | |
| mordred | jeblair: oh. piddle | 17:30 |
| jeblair | mordred, Shrews: node 1975977 looks like an ansible failure: http://logs.openstack.org/06/333506/1/check/gate-puppet-openstack-integration-3-scenario001-tempest-ubuntu-xenial-nv/86e3b82/_zuul_ansible/ansible_log.txt | 17:30 |
| clarkb | AJaeger: wow no wonder upgrading openstack is so difficult :) | 17:30 |
| mordred | jeblair: k. I'll stick Shrews' key on that one too | 17:31 |
| *** shashank_hegde has joined #openstack-infra | 17:31 | |
| rcarrillocruz | right, that flag is to generate a keypair on the fly to use for server creation | 17:31 |
| clarkb | AJaeger: 334008 is a bit scarier but good cleanup too | 17:31 |
| rcarrillocruz | Just dont use it | 17:31 |
| *** claudiub|2 has joined #openstack-infra | 17:31 | |
| rcarrillocruz | And create your named keypair on the resources.yml file | 17:31 |
| *** inc0 has quit IRC | 17:32 | |
| mordred | Shrews: 172.99.106.31 is the node jeblair mentions above - and you also have access to jenkins@ on it | 17:32 |
| rcarrillocruz | Check the sample on the role repo | 17:32 |
| *** grassass has joined #openstack-infra | 17:32 | |
| *** HeOS has quit IRC | 17:33 | |
| AJaeger | clarkb: I did this manually, hope I got it right ;) | 17:33 |
| asselin | rcarrillocruz, which sample? | 17:33 |
| clarkb | AJaeger: just more thinking if something depends on old kilo behavior that they shouldn't | 17:33 |
| clarkb | AJaeger: +2 we have to keep moving forward | 17:34 |
| *** dprince has joined #openstack-infra | 17:34 | |
| rcarrillocruz | resources.yml.sample :-) | 17:34 |
| *** daemontool_ has quit IRC | 17:34 | |
| AJaeger | clarkb, thanks. | 17:35 |
| *** claudiub has quit IRC | 17:35 | |
| rcarrillocruz | Infra ansible repo I think it is | 17:35 |
| *** dmk0202 has quit IRC | 17:35 | |
| AJaeger | Any other project-config reviewers for 334008, if possible, please? | 17:36 |
| *** mriedem has joined #openstack-infra | 17:36 | |
| *** rkukura has joined #openstack-infra | 17:37 | |
| *** roxanaghe has quit IRC | 17:38 | |
| nibalizer | AJaeger: on it | 17:38 |
| *** woodster_ has quit IRC | 17:39 | |
| AJaeger | thanks, nibalizer ! | 17:39 |
| *** dmk0202 has joined #openstack-infra | 17:39 | |
| pleia2 | reviewed, nibalizer can +A if he thinks it looks good | 17:39 |
| pleia2 | lots of good cleanup there \o/ | 17:40 |
| pleia2 | thanks AJaeger | 17:40 |
| *** cbader has joined #openstack-infra | 17:40 | |
| Shrews | mordred: jeblair: this is not a race | 17:40 |
| Shrews | the async file was successfully written 6 minutes AFTER async_wrapper gave up | 17:41 |
| *** roxanaghe has joined #openstack-infra | 17:42 | |
| AJaeger | thanks, pleia2 | 17:42 |
| AJaeger | if you're in kilo review mode, there's also https://review.openstack.org/333977 - for gerritbot... | 17:43 |
| *** ajmiller has quit IRC | 17:45 | |
| * pleia2 has a look | 17:45 | |
| *** amitgandhinz has quit IRC | 17:46 | |
| pleia2 | good good | 17:46 |
| *** amitgandhinz has joined #openstack-infra | 17:46 | |
| *** inc0 has joined #openstack-infra | 17:47 | |
| *** baoli_ has quit IRC | 17:47 | |
| AJaeger | thanks, pleia2 | 17:48 |
| *** baoli has joined #openstack-infra | 17:48 | |
| *** akshai has quit IRC | 17:48 | |
| *** zeih has joined #openstack-infra | 17:48 | |
| *** akshai has joined #openstack-infra | 17:49 | |
| *** sambetts is now known as sambetts|afk | 17:49 | |
| *** chem is now known as chem|week-end | 17:50 | |
| *** baoli has quit IRC | 17:50 | |
| Shrews | i dunno. i'm stumped. mordred: done with those nodes. i can't glean anything new from them | 17:51 |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder: Refactor: block-device handling (local loop) https://review.openstack.org/319591 | 17:53 |
| notmorgan | ooh hmm. i think i found an issue with the opportunistic testing in nodepool (mysql). seems like it has cases where it didn't cleanup the dynamic users/grants =/ | 17:54 |
| *** zeih has quit IRC | 17:54 | |
| notmorgan | i should really use a docker container for this instead of local mysql. | 17:54 |
| mordred | Shrews, jeblair: at this point, I think I would accept _any_ explanation if there was actually an explanation | 17:54 |
| mordred | this whole "oops, sometimes it doesn't work" is extra weird | 17:55 |
| notmorgan | mordred: ++ i don't like "oops, it doesn't work sometimes" | 17:55 |
| pabelanger | Shrews: mordred: jeblair: I'm starting to feel, this condition should be broken into 2 different checks. https://github.com/ansible/ansible/blob/devel/lib/ansible/executor/task_executor.py#L588 If finshed: break, thats straightforward. If any other failure, we need to decided abort the poll process early or continue regardless until we reach our poll timeout. | 17:56 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove kilo jobs https://review.openstack.org/333910 | 17:56 |
| * AJaeger had to remove an extra ")" to fix the regex ^ | 17:56 | |
| notmorgan | AJaeger: yay REGEX! | 17:56 |
| *** rbrndt_ has quit IRC | 17:56 | |
| pabelanger | personally, I feel we should continue until poll has timeout, then error "async didn't complete within requested time' | 17:57 |
| AJaeger | notmorgan: I shouldn't have removed the extra () that jhesketh wisely left in ;( | 17:57 |
| *** yamamoto has quit IRC | 17:57 | |
| * AJaeger tests locally first | 17:57 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Jenkins/jobs: Kilo cleanup https://review.openstack.org/334008 | 17:58 |
| pabelanger | I think switching to 328649 could give us better control of that process | 17:59 |
| openstackgerrit | Merged openstack-infra/project-config: Gerritbot: Remove obsolete kilo conditions https://review.openstack.org/333977 | 17:59 |
| AJaeger | clarkb: do you want to review my one ")" removal in 333910 again, please? The job passed locally now | 18:01 |
| clarkb | looking | 18:01 |
| *** Ravikiran_K has quit IRC | 18:02 | |
| clarkb | AJaeger: lgtm you cna probably approve since yo uonly changed one character | 18:02 |
| *** zul has quit IRC | 18:02 | |
| AJaeger | clarkb: will do, thanks | 18:02 |
| *** dizquierdo has quit IRC | 18:03 | |
| *** openstackgerrit has quit IRC | 18:03 | |
| *** openstackgerrit has joined #openstack-infra | 18:03 | |
| *** sdake_ has quit IRC | 18:04 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Switch to async_status for zuul_runner task https://review.openstack.org/328649 | 18:04 |
| *** e0ne has joined #openstack-infra | 18:05 | |
| *** jamesmca_ has joined #openstack-infra | 18:06 | |
| *** inc0 has quit IRC | 18:06 | |
| *** flepied has joined #openstack-infra | 18:06 | |
| AJaeger | according to our calendar, there's no meeting going on - what about merging https://review.openstack.org/#/c/330017/ (channels have been renamed) and https://review.openstack.org/329570 and https://review.openstack.org/332881 - all three for system-config | 18:06 |
| *** flepied1 has joined #openstack-infra | 18:07 | |
| *** inc0 has joined #openstack-infra | 18:07 | |
| jpmaxman | fungi mordred so it has been a few days and the wiki seems to be holding steady. Do you want me to officially submit the changes to enable recaptcha as a patch? | 18:07 |
| fungi | jpmaxman: yes, i've heard no complaints with it so far | 18:07 |
| *** jamesmc__ has joined #openstack-infra | 18:07 | |
| fungi | jpmaxman: please do! | 18:07 |
| mordred | ++ | 18:08 |
| jpmaxman | ok sounds good - I might keep an eye on it over the weekend just to be sure then do it on Monday if tha's ok | 18:08 |
| *** ajmiller has joined #openstack-infra | 18:08 | |
| mordred | jpmaxman: sounds awesome | 18:08 |
| pleia2 | AJaeger: good idea, reviewing now | 18:09 |
| AJaeger | thanks, pleia2 ! | 18:09 |
| jpmaxman | great I think when everything gets updated to the latest version it will also include the new recaptcha that looks a little nicer. | 18:09 |
| fungi | jpmaxman: sounds great. thanks again | 18:09 |
| *** jamesmcarthur has quit IRC | 18:09 | |
| clarkb | are new accounts able to be made now? | 18:09 |
| fungi | clarkb: that's not been turned back on yet, no | 18:10 |
| *** flepied has quit IRC | 18:10 | |
| fungi | i expect that's the next step is to see if the spammers creating new accounts (assuming there's no option to enforce captchas on account creation as well) are at all thwarted from making edits with this harder captcha implementation | 18:11 |
| jpmaxman | by the way - pleia2 on the new server are you using trusty or xenial? | 18:11 |
| fungi | jpmaxman: the current plan is to do trusty | 18:11 |
| jpmaxman | I've seen some suspicious accounts created | 18:11 |
| *** jamesmca_ has quit IRC | 18:11 | |
| fungi | we're wary of adding xenial servers for heavily-used services this early in its lifecycle | 18:12 |
| jpmaxman | but so far none have made any edits or additions | 18:12 |
| jpmaxman | ok sounds good trusty is trusty :) | 18:12 |
| clarkb | jpmaxman: all new accounts should only happen when we flip thr bit temproarily, is there an example you can share that we can check against that with? | 18:13 |
| dougwig | clarkb: i think that's a good idea for the controller/db/rabbit, but i'm more worried about how much pain you're going to have with the neutron router being that far away. | 18:13 |
| fungi | jpmaxman: oh, indeed there were quite a few created on wednesday it looks like. was new account creation back on for a while? https://wiki.openstack.org/w/index.php?title=Special%3AListUsers&username=&group=&creationSort=1&desc=1&limit=50 | 18:13 |
| dougwig | clarkb: which underlay network is at play? vxlan? provider hardware? | 18:14 |
| clarkb | dougwig: provider nets | 18:14 |
| clarkb | dougwig: so that shouldnt be an issue | 18:14 |
| jpmaxman | https://wiki.openstack.org/wiki/Special:Log/newusers | 18:14 |
| fungi | and yeah, it looks like the only one of those created in that timeframe with contribs is toby (who is on the board) editing the release name proposals pages | 18:15 |
| dougwig | clarkb: well, the neutron node still provides DHCP and metadata for those. | 18:15 |
| jpmaxman | yes people can create new accounts | 18:15 |
| jpmaxman | that was part of the test to see if this was working :) | 18:15 |
| *** eharney has quit IRC | 18:15 | |
| clarkb | dougwig: on the compute host though I thought | 18:16 |
| fungi | jpmaxman: i still see /srv/mediawiki/Settings.php has "$wgGroupPermissions['*']['createaccount'] = false;" so what's working around that currently? | 18:16 |
| AJaeger | dougwig: the midonet folks asked, could you review their change https://review.openstack.org/333097 , please? | 18:16 |
| dougwig | clarkb: no, it comes from a namespace on the neutron node, which needs to be plugged into the same provider net. but, a local neutron node and distant controller/db still moves a lot of the setup complexity. | 18:17 |
| jpmaxman | blorg - well actually it looks like my changes reverted | 18:18 |
| dougwig | AJaeger: done | 18:18 |
| jpmaxman | including the recaptcha | 18:18 |
| jpmaxman | is it still being overwritten with the puppet config | 18:18 |
| jpmaxman | I guess we still know nothing :( | 18:18 |
| jpmaxman | as to if it worked or not | 18:18 |
| mordred | boo! | 18:18 |
| fungi | jpmaxman: hrm, yeah it looks like puppet is getting run there according to the syslog | 18:19 |
| fungi | i'm tracking that down now | 18:19 |
| mordred | infra-root: all of the groups in the generated groups file are empty | 18:19 |
| *** amitgandhinz has quit IRC | 18:19 | |
| *** amitgandhinz has joined #openstack-infra | 18:19 | |
| jpmaxman | ok thanks :) I'll reapply on Monday and keep an eye on it | 18:20 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-mediawiki: Require users to confirm their email address before they can edit https://review.openstack.org/334029 | 18:20 |
| AJaeger | dougwig: thanks. Will rebase and approve after the kilo eol-change is in... | 18:20 |
| jpmaxman | so no super hurry | 18:20 |
| fungi | mordred: i guess that means including the emergency/disable groups? | 18:20 |
| jpmaxman | don't want to make any changes before the weekend hits | 18:20 |
| mordred | fungi: emergency groups are there | 18:20 |
| pabelanger | fungi: we should consider ^ for wiki.o.o too | 18:20 |
| pabelanger | confirm email address for edit | 18:20 |
| fungi | jpmaxman: agreed, sorry about that. digging into it now obviously | 18:20 |
| mordred | fungi: but the larger set of generated groups not so much | 18:20 |
| openstackgerrit | Merged openstack-infra/system-config: Rename IRC channel of container service https://review.openstack.org/330017 | 18:21 |
| mordred | fungi: I'd lke to run expand-groups.sh - any concerns with that? | 18:21 |
| fungi | mordred: oh, so any of the ones we have set in public git are not getting set now? | 18:21 |
| mordred | fungi: yes. that's correct | 18:21 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-mediawiki: Require users to confirm their email address before they can edit https://review.openstack.org/334029 | 18:21 |
| jpmaxman | I created a test account in launchpad and am pretty sure I had to confirm my e-mail before I could login to the wiki | 18:21 |
| *** trown|lunch is now known as trown | 18:21 | |
| jpmaxman | I can try it again to be sure | 18:21 |
| pabelanger | ya, I am not sure if it matters with openid | 18:22 |
| pabelanger | but since we still will allow mobile users, I think we want that | 18:22 |
| fungi | pabelanger: jpmaxman: yeah, i'm pretty sure lp already requires account confirmation, so it's unlikely to be thwarting anyone if we do the same on the wiki | 18:22 |
| pabelanger | since spammers used mobile IIRC | 18:22 |
| jpmaxman | I think mobile was disabled by impossible captcha - not sure if that's still the case | 18:22 |
| fungi | pabelanger: nope, the spammers were using lp until we changed teh captcha to something guessable and then they started using the mobile account creation | 18:22 |
| jpmaxman | ahh | 18:23 |
| fungi | pabelanger: we previously had mobile account creation "disabled" by making that captcha impossible to answer as jpmaxman mentioned | 18:23 |
| fungi | but since we wanted to add the captcha to edits we needed to make it answerable, which in turn allowed spammers to create accounts on the wiki without going through lp openid | 18:24 |
| *** e0ne has quit IRC | 18:24 | |
| *** _nadya_ has joined #openstack-infra | 18:25 | |
| *** zul has joined #openstack-infra | 18:25 | |
| *** sarob has quit IRC | 18:25 | |
| *** sarob has joined #openstack-infra | 18:26 | |
| *** rbrndt_ has joined #openstack-infra | 18:26 | |
| *** SumitNaiksatam has joined #openstack-infra | 18:26 | |
| mordred | fungi: I have re-run expand groups | 18:26 |
| fungi | i have a (very vague) recollection that the impossible captcha workaround was something Ryan_Lane added so that we could enable the mobile view extension even though the openid extension didn't apply to it | 18:26 |
| mordred | and the generated-groups file is now correct | 18:26 |
| fungi | mordred: should we have to run that manually under normal circumstances? i thought our run_all.sh was supposed to be doing it automatically | 18:27 |
| fungi | i guess we changed something which subtly broke it? | 18:28 |
| mordred | fungi: we shoudl not. also, it was definitely _run_ - but something broke it whenever it was | 18:28 |
| mordred | fungi: unfortunately, I cannot find logs to indicate what broken | 18:28 |
| mordred | fungi: the file had all of the group headings in it - just no content in each grouo | 18:28 |
| fungi | oh, ew | 18:28 |
| openstackgerrit | Merged openstack-infra/system-config: Add #openstack-fwaas to statusbot/meetbot https://review.openstack.org/329570 | 18:29 |
| openstackgerrit | Merged openstack-infra/system-config: Add #openstack-pkg to statusbot & meetbot_channels https://review.openstack.org/332881 | 18:29 |
| fungi | aside from causing stuff we had in the disabled group in the public groups file to get puppeted anyway, what else could that have broken i wonder? | 18:29 |
| *** shardy has quit IRC | 18:29 | |
| openstackgerrit | Bin Hu proposed openstack-infra/project-config: New project of a Model-Driven Extensible Framework https://review.openstack.org/333843 | 18:29 |
| fungi | mordred: was the wiki getting puppeted even though it was in the public disabled list what led you to this discovery, or did you spot something else broken too? | 18:30 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: move the release tools that need to run on secure nodes https://review.openstack.org/332448 | 18:31 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: networking-midonet: Make ml2 job non-voting https://review.openstack.org/333097 | 18:31 |
| *** kushal has joined #openstack-infra | 18:31 | |
| mordred | fungi: yes. it was puppet breaking jpmaxman | 18:32 |
| fungi | okay, so we haven't spotted anything else this impacted (yet). that's good at least | 18:32 |
| AJaeger | I rebased 333097, any project-config core wants to review and approve to unblock them, please? | 18:33 |
| * AJaeger wishes everybody a great weekend | 18:33 | |
| mordred | fungi: \o/ | 18:34 |
| *** eharney has joined #openstack-infra | 18:34 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: add a "ptl" command to show info about how to contact a PTL https://review.openstack.org/333462 | 18:34 |
| fungi | have a great weekend AJaeger | 18:34 |
| fungi | i need to step away for a bit and do some yardwork while the weather is still fair | 18:35 |
| fungi | before my lawn develops a taste for human flesh | 18:35 |
| anteaya | AJaeger: why do you create a new rule for job? why not just have voting: false under the current rule that defines test branches? http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n1709 | 18:36 |
| anteaya | fungi: enjoy yardwork | 18:37 |
| openstackgerrit | Merged openstack-infra/project-config: Remove kilo jobs https://review.openstack.org/333910 | 18:38 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/nodepool: Add alembic migration support https://review.openstack.org/136589 | 18:38 |
| *** kushal has quit IRC | 18:39 | |
| dougwig | AJaeger: you cool with the 3 hour timeout on https://review.openstack.org/#/c/330926/ ? i thought those were getting shrunk. | 18:39 |
| *** kushal has joined #openstack-infra | 18:39 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Add addTestResults to GitHub pull request builder trigger https://review.openstack.org/321300 | 18:39 |
| notmorgan | mordred, jeblair, ^ this is just a rebase/update of the initial migration script, next step is the auto-run. | 18:40 |
| notmorgan | mordred, jeblair: just an FYI. | 18:40 |
| anteaya | dougwig: thanks for asking, I'm not okay with a 3 hour timeout | 18:40 |
| *** openstackstatus has quit IRC | 18:40 | |
| *** openstack has joined #openstack-infra | 18:42 | |
| *** openstackstatus has joined #openstack-infra | 18:42 | |
| *** ChanServ sets mode: +v openstackstatus | 18:42 | |
| notmorgan | mordred: sorry, yeah . irc lagging for me =/ | 18:42 |
| anteaya | dougwig: I agree with you, the timeout is off | 18:42 |
| notmorgan | mordred: though it was *very* easy to create the initial revision. woo :) | 18:42 |
| notmorgan | i like the "automatic" migration script creation | 18:42 |
| notmorgan | mordred: so my plan is: on an unmanaged DB, just hook into the sql-a meta.create_all() and stamp it as "latest" - if the db is not under management but has data, just stamp it with "latest", and if it is under management but not latest, perform the upgrade | 18:44 |
| notmorgan | mordred: on startup. | 18:44 |
| notmorgan | first thing there was "unmanaged and empty db" sorry | 18:44 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Update config file provider docs to have default options https://review.openstack.org/333393 | 18:44 |
| mordred | ++ | 18:45 |
| *** r-mibu has quit IRC | 18:45 | |
| openstackgerrit | Ben Nemec proposed openstack/diskimage-builder: Clean more from ironic-agent ramdisk image https://review.openstack.org/334041 | 18:45 |
| *** r-mibu has joined #openstack-infra | 18:45 | |
| openstackgerrit | Ben Nemec proposed openstack/diskimage-builder: Don't create an image just to delete it https://review.openstack.org/334042 | 18:45 |
| *** claudiub|2 has quit IRC | 18:45 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci https://review.openstack.org/324777 | 18:45 |
| *** amitgandhinz has quit IRC | 18:46 | |
| *** amitgandhinz has joined #openstack-infra | 18:46 | |
| notmorgan | mordred: lol you already tell it to upgrade on connection :P | 18:48 |
| notmorgan | mordred: in NodeDatabase | 18:48 |
| notmorgan | mordred: you did the magic work for that already ^_^ | 18:48 |
| *** salv-orl_ has quit IRC | 18:49 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Update sonar plugin https://review.openstack.org/333482 | 18:49 |
| *** zeih has joined #openstack-infra | 18:50 | |
| clarkb | ya all teh work is done except for conflict resolution and updating to current db state iirc | 18:50 |
| *** gouthamr has quit IRC | 18:51 | |
| *** amitgandhinz has quit IRC | 18:51 | |
| *** amitgandhinz has joined #openstack-infra | 18:52 | |
| *** hashar has joined #openstack-infra | 18:52 | |
| sdague | corrupt mirror? | 18:53 |
| sdague | http://logs.openstack.org/72/326572/3/check/gate-grenade-dsvm-neutron/90219fb/logs/grenade.sh.txt.gz#_2016-06-24_16_44_19_040 | 18:53 |
| *** kushal has quit IRC | 18:54 | |
| AJaeger | anteaya: the change is based on another one that removes the lines you quote | 18:55 |
| *** zeih has quit IRC | 18:55 | |
| *** ikhudoshyn has quit IRC | 18:55 | |
| *** amotoki has quit IRC | 18:55 | |
| anteaya | AJaeger: patchset one looked fine to me | 18:56 |
| anteaya | where is the change that removes the rule? | 18:56 |
| AJaeger | anteaya: just merged... | 18:57 |
| AJaeger | anteaya: I12adc42629da0089b8520a6f06d3f59d9522fc02 | 18:57 |
| AJaeger | https://review.openstack.org/#/c/333910/ | 18:57 |
| *** yamamoto has joined #openstack-infra | 18:58 | |
| anteaya | AJaeger: okay thank you | 18:59 |
| *** gouthamr has joined #openstack-infra | 19:00 | |
| *** yamamoto has quit IRC | 19:00 | |
| *** ajmiller has quit IRC | 19:00 | |
| *** yamamoto has joined #openstack-infra | 19:00 | |
| *** ajmiller has joined #openstack-infra | 19:00 | |
| *** javeriak has quit IRC | 19:01 | |
| *** rvba has quit IRC | 19:02 | |
| *** csomerville has joined #openstack-infra | 19:02 | |
| *** _nadya_ has quit IRC | 19:03 | |
| *** sdake has joined #openstack-infra | 19:03 | |
| *** sarob has quit IRC | 19:03 | |
| *** cody-somerville has quit IRC | 19:06 | |
| *** Sukhdev has joined #openstack-infra | 19:06 | |
| *** abregman has quit IRC | 19:08 | |
| openstackgerrit | Merged openstack-infra/project-config: networking-midonet: Make ml2 job non-voting https://review.openstack.org/333097 | 19:10 |
| *** adreznec has quit IRC | 19:12 | |
| *** adreznec has joined #openstack-infra | 19:13 | |
| *** baoli has joined #openstack-infra | 19:15 | |
| *** salv-orlando has joined #openstack-infra | 19:16 | |
| jeblair | sdague: when i manually d/l that file i get a correct zip file. suspect that's pip actually saying "network hiccup; incomplete transmission" but not able to find the right words | 19:19 |
| rcarrillocruz | asselin: you sorted out? | 19:19 |
| rcarrillocruz | so | 19:19 |
| rcarrillocruz | with the cloud launcher, you feed a file which you have your layout of resources | 19:19 |
| rcarrillocruz | servers | 19:19 |
| rcarrillocruz | networks | 19:19 |
| rcarrillocruz | subnets | 19:19 |
| rcarrillocruz | etc | 19:19 |
| rcarrillocruz | *and* you can also keypairs | 19:19 |
| rcarrillocruz | so you can perfectly put there a 'bootstrap' key which you use to create servers | 19:20 |
| rcarrillocruz | and ofc, because ansible modules are idempotent | 19:20 |
| rcarrillocruz | running the role against that resources.yml will be too | 19:20 |
| rcarrillocruz | now, the launch-node playbook with the launcher | 19:20 |
| rcarrillocruz | uses the feature of generating a keypair on the fly | 19:20 |
| rcarrillocruz | to have a feature parity with the current launch-node. In the current launch-node, they don't use a 'static' keypair to create servers | 19:20 |
| rcarrillocruz | they create a keypair on each run | 19:20 |
| rcarrillocruz | which is what the 'generate_keypair' flag does, feature parity ^ | 19:21 |
| sdague | jeblair: ok | 19:21 |
| rcarrillocruz | hope it makes sense now, i was off home and was hard to write from my phone | 19:21 |
| *** ajmiller has quit IRC | 19:22 | |
| *** ajmiller has joined #openstack-infra | 19:22 | |
| *** csomerville has quit IRC | 19:23 | |
| sdague | jeblair: maybe rax ord network issues again? | 19:25 |
| *** njohnston|afk is now known as njohnston | 19:25 | |
| rcarrillocruz | asselin: besides the resources.yml.sample from infra-ansible, you can also check out http://rcarrillocruz.com/deploying-multiple-openstack-clouds-with-ansible-in-a-data-driven-fashion/ | 19:25 |
| sdague | https://review.openstack.org/#/c/326585/ - tempest full is a net fail on ansible setup | 19:25 |
| rcarrillocruz | it has the bootstrap key resource i refer about | 19:26 |
| *** esberglu has quit IRC | 19:26 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/nodepool: Add alembic migration support https://review.openstack.org/136589 | 19:28 |
| notmorgan | mordred, jeblair, Shrews, nibalizer: ^ alembic migration support. it may need some tweaking still, but pushing it up for eyes. | 19:28 |
| *** _nadya_ has joined #openstack-infra | 19:29 | |
| *** yamamoto has quit IRC | 19:29 | |
| notmorgan | mordred, jeblair, Shrews, nibalizer: I am inclined to make this another "hard" break upgrade where dropping the schema is best (or require a direct alembic migration by cli) rather than just assuming the DB is "up-to-date" if it is populated. | 19:29 |
| nibalizer | oooo | 19:30 |
| mordred | nibalizer: as a person on the receiving end of a hard-break upgrade - which of the above do you prefer? | 19:31 |
| *** Jeffrey4l_ has joined #openstack-infra | 19:31 | |
| nibalizer | um | 19:31 |
| *** xarses has quit IRC | 19:32 | |
| mordred | one more hard-break to get onto the gravy train of no more hard breaks ... or assume that the current schema is up to date and just stamp it as good and move on | 19:33 |
| *** jswarren has joined #openstack-infra | 19:33 | |
| notmorgan | i mean... i could try and get REALLY fancy and handle conditional generated schema upgrades based upon reflection... but it makes my skin crawl | 19:33 |
| notmorgan | and i know it's doable... | 19:33 |
| notmorgan | just... please lets not do that :P | 19:33 |
| nibalizer | jswarren: moravec dunno how much scrollback you have | 19:34 |
| nibalizer | but notmorgan has posted a path https://review.openstack.org/136589 | 19:34 |
| jswarren | I have none, unfortunately | 19:34 |
| nibalizer | to allow database migrations for CDing nodepool, which would help us | 19:34 |
| *** Jeffrey4l has quit IRC | 19:34 | |
| *** esberglu has joined #openstack-infra | 19:34 | |
| openstackgerrit | Merged openstack-infra/puppet-ulimit: Fix deprecated warnings https://review.openstack.org/331445 | 19:34 |
| nibalizer | so there is some question about the behaviour post-patch, basically if we need to have a hard break | 19:34 |
| *** ccarmack has joined #openstack-infra | 19:34 | |
| *** _nadya_ has quit IRC | 19:35 | |
| notmorgan | mordred: the big note here is empty DBs use reflection to create the schema, so if our migration scripts diverge from the actual models... we get in an icky place [but we'd be there anyway] for upgrades | 19:35 |
| mordred | jswarren: tl;dr - two options - assume existing db is up to date, stamp it good so that it's set up for migrations in the future - downside, if your db is _not_ currently up to date this will lead to weird problems in the future | 19:35 |
| notmorgan | jswarren: weird, brokeness or not... or who knows what will end up happening | 19:35 |
| mordred | jswarren: or the other option is to make this one more hard break that requires you as a consumer to do a manual task | 19:35 |
| notmorgan | it will be weird. | 19:35 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: have the ptl command parse the liaison wiki page https://review.openstack.org/334067 | 19:36 |
| *** sdake has quit IRC | 19:36 | |
| *** woodster_ has joined #openstack-infra | 19:36 | |
| jswarren | I'm thinking a manual, one-time task would be OK. | 19:36 |
| *** sdake has joined #openstack-infra | 19:36 | |
| *** amitgandhinz has quit IRC | 19:36 | |
| notmorgan | since historically we've said "drop the schema" and let it be recreated -- i think we're better off continuing with that. | 19:36 |
| notmorgan | but like i said... i could do something fancy here if it *really* matters. | 19:37 |
| mordred | cool. that's the safest. and then moving forward after that it'll just magically work | 19:37 |
| mordred | notmorgan: related - do we need the nodepool-db-manage command? I know it's there, but with the magic auto-upgrade in place, I'm not sure it's a needed thing anymore | 19:37 |
| notmorgan | ok let me revise this to raise an exception saying OMG NO, if the DB is populated. | 19:37 |
| *** amitgandhinz has joined #openstack-infra | 19:37 | |
| mordred | notmorgan: ++ | 19:37 |
| notmorgan | mordred: i figure it never hurts to let people off-line migrate for safety | 19:37 |
| mordred | kk | 19:37 |
| notmorgan | mordred: i wouldn't take it away tbh. | 19:38 |
| * notmorgan thinks other openstack projects should do the auto-upgrade on start if needed [*cough* keystone *cough*] | 19:38 | |
| mordred | notmorgan: keystone maybe ... probably not nova | 19:38 |
| notmorgan | but i know that is not a "good plan"(tm) to spring on people | 19:38 |
| notmorgan | mordred: eh, if it has the proper no-downtime-upgrade code -- impact would be minimal | 19:39 |
| jswarren | notmorgan: is there a risk of orphaned resources if the DB is just wiped out? | 19:39 |
| mtreinish | jeblair: I finally got around to testing your patch and it got past the stacktrace. Now it's working through the ~1400 thing sync queue (kinda slowly, but that's probably just the coffeeshop wifi) | 19:39 |
| notmorgan | mordred: ^ defer to you on that one. | 19:39 |
| notmorgan | i ... i don't know -- maybe? | 19:39 |
| *** ajmiller has quit IRC | 19:39 | |
| *** mhickey has quit IRC | 19:39 | |
| mordred | jswarren: if they're there, they'll show up in nodepool alien-list ... but i'm going to re-defer to nibalizer | 19:39 |
| *** baoli has quit IRC | 19:40 | |
| * mordred hands nibalizer the conch | 19:40 | |
| notmorgan | ahhhh alien-list | 19:40 |
| jswarren | Will we need to manually destroy VMs, I assume images will need to be recreated. | 19:40 |
| nibalizer | im a fan of simple not fancy | 19:40 |
| mordred | so - you could also manually add the alembic migrations table | 19:40 |
| mordred | since you happen to konw you're up to date | 19:40 |
| jswarren | Oh OK | 19:41 |
| nibalizer | oh yea btw jswarren and moravec are 2 of hte folk i work with at ibm | 19:41 |
| nibalizer | they are the people who actually had nodepool die on them, my role is more advisory i guess | 19:41 |
| jswarren | pschwar is our nodepool guy, but he's out at the moment. | 19:42 |
| mordred | notmorgan: maybe put the "add the migration code if it's not there" logic into nodepool-db-manage? that way an admin who knows what's up can opt-in to running the code to update it | 19:42 |
| mordred | which can happen while nodepool is running and then a nodepool restart would be just a normal restart | 19:42 |
| jswarren | I guess he's phschwartz on here | 19:42 |
| nibalizer | i mean running a nodepool (especially against a derpy cloud) puts you in the position of occasionally having to clean it out, thats just standard operating procedure | 19:42 |
| mordred | notmorgan: like nodepool-db-manage bootstrap --yes-i-mean-it or something | 19:43 |
| notmorgan | mordred: i mean you could just create the table in the DB directly | 19:43 |
| notmorgan | mordred: and populate it | 19:43 |
| nibalizer | I don't really have an opinion on how to design it, it makes sense to me that we'd have another breaking change to add alembic support. | 19:44 |
| pabelanger | I've only see tripleo-test-cloud-rh1 require manual deleting recently. I haven't had to do it on our other clouds in a while | 19:44 |
| nibalizer | pabelanger: thats good | 19:44 |
| notmorgan | nibalizer: in many cases i wouldn't advocate a breaking change. but right now, we have historically just broken things to add new schema stuff | 19:44 |
| nibalizer | yah | 19:44 |
| notmorgan | nibalizer: this fits the pattern we have adhered to until alembic | 19:44 |
| notmorgan | nibalizer: so... i feel better just making this the "last breaking change" | 19:44 |
| *** csomerville has joined #openstack-infra | 19:45 | |
| nibalizer | yea | 19:45 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/nodepool: Add alembic migration support https://review.openstack.org/136589 | 19:45 |
| nibalizer | and jswarren is the operator I came to defend yesterday, so if he's cool with it im cool with it | 19:45 |
| nibalizer | thanks a ton notmorgan and the others who have worked on that patch | 19:45 |
| notmorgan | ^ | 19:46 |
| sdague | is there a patch up yet to stop doing grenade on liberty? | 19:46 |
| *** ajmiller has joined #openstack-infra | 19:46 | |
| notmorgan | mordred: i mean, it is trivial to add "alembic_command.stamp('head') | 19:46 |
| notmorgan | mordred: if you feel even vaguely strong about that, i'm happy to do it...but i also like this breaking change because it forces consistency that may have been lost if people did manual sql-updates for say contraint/constraing-names that don't impact runtime but would make future migrations icky | 19:47 |
| nibalizer | jesusaur: you also run a downstream nodepool, opinions? | 19:48 |
| * jesusaur reads some backscroll | 19:48 | |
| pabelanger | notmorgan: ++ | 19:50 |
| *** zeih has joined #openstack-infra | 19:51 | |
| *** ajmiller_ has joined #openstack-infra | 19:53 | |
| *** csomerville has quit IRC | 19:54 | |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: Don't run grenade on liberty https://review.openstack.org/334072 | 19:54 |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: remove all reference to partial-ncpu job https://review.openstack.org/334073 | 19:54 |
| sdague | liberty is wedged until the first of those is handled | 19:54 |
| *** csomerville has joined #openstack-infra | 19:55 | |
| *** zeih has quit IRC | 19:56 | |
| jesusaur | nibalizer: notmorgan: i agree that we shouldnt assume that a populated database is up-to-date | 19:56 |
| *** amitgandhinz has quit IRC | 19:56 | |
| *** ajmiller has quit IRC | 19:57 | |
| jesusaur | this seems like a reasonable path forward | 19:57 |
| anteaya | sdague: +2 | 19:57 |
| *** Nikolay_St has joined #openstack-infra | 19:57 | |
| *** amitgandhinz has joined #openstack-infra | 19:57 | |
| anteaya | sdague: do say if you don't get any other takers in the next 15 minutes | 19:57 |
| *** ajmiller__ has joined #openstack-infra | 19:58 | |
| *** ccarmack has left #openstack-infra | 19:59 | |
| sdague | ok, well I'm about to head to beer o'clock. Hopefully if people want liberty working again, they'll notice :) | 20:01 |
| *** ajmiller_ has quit IRC | 20:02 | |
| anteaya | sdague: okay cool, have a great weekend | 20:03 |
| sdague | you too | 20:03 |
| nibalizer | im gonna be sad when liberty is eol and all these double meanings go away | 20:03 |
| *** Apoorva has quit IRC | 20:03 | |
| anteaya | what double meanings? | 20:03 |
| nibalizer | nevermind | 20:03 |
| anteaya | I'll be glad to see it gone | 20:04 |
| anteaya | maybe next time summit is in Canada we can actually honour Canada when selecting the name | 20:04 |
| *** Apoorva has joined #openstack-infra | 20:04 | |
| anteaya | I hold onto the hope in any case | 20:05 |
| russellb | canada doesn't like liberty? | 20:05 |
| mtreinish | anteaya: there's no liberty in canada? | 20:05 |
| russellb | mtreinish: i win | 20:05 |
| *** kzaitsev_mb has joined #openstack-infra | 20:05 | |
| mtreinish | I guess there is still a queen :) | 20:05 |
| mtreinish | russellb: haha | 20:06 |
| mtreinish | you do | 20:06 |
| anteaya | mtreinish: it is an american concept | 20:06 |
| anteaya | to the extent is a brand and slogan | 20:07 |
| clarkb | sdague: is gate-greande-dsvm-multinode not a job anymore? | 20:07 |
| mordred | anteaya: well, we did rework the naming process in protest | 20:08 |
| nibalizer | imm having a hard time parsing 334072 tbh | 20:08 |
| anteaya | mordred: I saw that and I understand that vancouver is booked for a summit in the future with dates to be determined, hence my hope | 20:09 |
| * rcarrillocruz loved Vancouver summit, best venue | 20:10 | |
| anteaya | rcarrillocruz: it really was an amazingly designed building for acoustics | 20:10 |
| rcarrillocruz | being able to sit in a table with those views... | 20:11 |
| anteaya | but 8 straight days of sunshine, well, don't expect that to happen twice in a row | 20:11 |
| rcarrillocruz | heh | 20:11 |
| anteaya | yes I do love the mountains | 20:11 |
| anteaya | vancouver never gets 8 straight days of sunshine, that weather was extremely unusual | 20:11 |
| *** ajmiller__ is now known as ajmiller | 20:11 | |
| clarkb | sdague: I am trying to figure out why you removed the grenade multinode rules | 20:12 |
| clarkb | oh is it because it has the same prefix as the other grenade jobs so the names will match in both cases | 20:13 |
| *** HeOS has joined #openstack-infra | 20:15 | |
| *** xarses has joined #openstack-infra | 20:16 | |
| fungi | mordred: jswarren: the alien list is less relevant now that nodepool can automatically clean up leaked instances (identified by the metadata it stashes on them). if you completely wipe out the db, you'll probably have to delete some leaked _images_ but no leaked instances--nodepool should take care of them | 20:17 |
| clarkb | and even if you don't delete the db you will likely have leaked images, bceause it turns out we leak images | 20:18 |
| fungi | somebody should really call a plumber | 20:18 |
| clarkb | make sure they specialize in glance | 20:18 |
| * nibalizer gets a wrench | 20:18 | |
| clarkb | it has to do with the way the glance api works, lack of documentation, and bad assumptions | 20:19 |
| fungi | with all the leaks we've been dealing with at the house these past couple weeks, i've been starting to joke that buying a house on the water is almost like buying a boat that goes nowhere | 20:19 |
| clarkb | basically to make an image is a multistep process and if you fail you need to properly unroll any prior successful bits, and none of the clients do that (shade might now though) | 20:19 |
| clarkb | fungi: ouch | 20:20 |
| *** Nikolay_St has quit IRC | 20:20 | |
| anteaya | fungi: wow | 20:20 |
| anteaya | is part of your house below the water line? | 20:20 |
| *** vhosakot has quit IRC | 20:21 | |
| fungi | not quite, but *close* | 20:21 |
| *** burgerk has quit IRC | 20:21 | |
| anteaya | oh my | 20:21 |
| fungi | but these leaks are above-ground. one in the flashing around one of the drain vents up on the roof, a missing patch of roof shingles, a cracked primary condensate tray in the hvac, a leaked supply to one of the toilets, a cracked exterior line out to the dock where we didn't realize one of the valves didn't properly winterize... | 20:22 |
| fungi | oh, and the water which was streaming down our chimney in every storm | 20:22 |
| clarkb | sdague: did you want to write the d-g patch to remove the partial ncpu logic? | 20:22 |
| anteaya | oh okay | 20:22 |
| openstackgerrit | Merged openstack-infra/project-config: Don't run grenade on liberty https://review.openstack.org/334072 | 20:22 |
| anteaya | sounds like those things could happen regardless of how close you were to the ocean | 20:22 |
| anteaya | still not fun | 20:22 |
| fungi | so it was sort of _like_ being in a boat. water coming in everywhere | 20:22 |
| anteaya | ah | 20:23 |
| *** akshai has quit IRC | 20:23 | |
| anteaya | have you figured out the water down the chimney issue yet? | 20:23 |
| anteaya | like coming down the inside of the chimney? | 20:23 |
| fungi | yeah, we just had them cap off the chimney since we're using propane logs in it | 20:23 |
| clarkb | we had a largeish amount of rain yesterday and I did not feel like I was in a boat so yay | 20:23 |
| fungi | that put a stop to it | 20:23 |
| anteaya | good to hear it | 20:24 |
| anteaya | clarkb: yay! | 20:24 |
| anteaya | clarkb: how is the house shaping yup? | 20:24 |
| anteaya | any boxless spots yet? | 20:24 |
| anteaya | can you sit in a chair at all? | 20:24 |
| fungi | it does always feel like victory when you have a piece of furniture you can finally sit down on | 20:25 |
| anteaya | yes | 20:25 |
| clarkb | anteaya: so many things to do, but we are living in it. The kitchen and living room ar emostly unpacked | 20:25 |
| clarkb | the dining area has a table in it surrounded by many boxes | 20:26 |
| clarkb | most of my office is in a closet | 20:26 |
| *** Daisy has joined #openstack-infra | 20:26 | |
| clarkb | but that is because I sort of want to rearrange how I utilize various pieces of hardware, old desktop will be new server box, old htpc will be new desktop etc | 20:26 |
| *** sdake has quit IRC | 20:26 | |
| anteaya | yay unpacked kitchen! | 20:27 |
| *** amitgandhinz has quit IRC | 20:27 | |
| anteaya | yup | 20:27 |
| sc` | moving. so fun. i can't wait until i move out of my apartment. have to move a 42u cab out, among other things | 20:27 |
| anteaya | so many things to do, but we are living in it describes anyone living in a house that I've seen | 20:27 |
| *** amitgandhinz has joined #openstack-infra | 20:27 | |
| *** hashar has quit IRC | 20:27 | |
| rcarrillocruz | sc`: i hear ya, i have a 31U cab, and I may eventually move in some months :/ | 20:28 |
| *** sarob has joined #openstack-infra | 20:28 | |
| sc` | rcarrillocruz: mine is pure torture. 90s era steel sub microsystems cab :) | 20:28 |
| sc` | sun | 20:29 |
| sc` | not sub | 20:29 |
| rcarrillocruz | lol | 20:29 |
| fungi | sc`: i lugged one around myself for years. finally decided it was no longer owrth the effort to care for (and power, and cool) that much equipment. i was sort of sad to see it go though | 20:29 |
| rcarrillocruz | mine are repurposed old hw crammed on 4U cases :/ | 20:29 |
| *** yamamoto has joined #openstack-infra | 20:30 | |
| *** sarob has quit IRC | 20:30 | |
| sc` | fungi: i had a late 80s cab i carried with me across texas. as soon as i moved out of texas, i gave it to someone i knew, fully loaded | 20:30 |
| clarkb | fungi: I am glad to have decided that early on, also I have moved on average once a year for the last 10 years or so | 20:30 |
| clarkb | super excited to live in one place for at least 5 years | 20:30 |
| sc` | i like the sun cab just fine, but it may go away if i move more than across town | 20:30 |
| *** Daisy has quit IRC | 20:31 | |
| sc` | or across the bay, as it might be, here | 20:31 |
| openstackgerrit | Merged openstack-infra/project-config: remove all reference to partial-ncpu job https://review.openstack.org/334073 | 20:31 |
| clarkb | sdague: one suggestion for the localconf change posted | 20:32 |
| sc` | really liking zuul v3 so far. i get my streaming logs back, which is infinitely helpful debugging integration gates | 20:32 |
| fungi | sc`: yeah, i gave mine up when we moved out to the beach. to save up before buying we moved into a sub-1ksqft townhouse and were using one of the two bedrooms purely as a storage unit for our stuff. made some pretty drastic choices about what was worth keeping and what i could part with | 20:35 |
| fungi | sc`: that's actually zuul v2.5 (really just zuul with added features to allow you to use not-jenkins). but i expect that's a feature which is coming in v3 too | 20:35 |
| sc` | ah. thought i read it was v3 somewhere. my bad. i still like that i get my streaming logs back | 20:35 |
| *** ajmiller has quit IRC | 20:36 | |
| fungi | v3 is still in its own separate feature branch | 20:36 |
| jeblair | mordred, pabelanger, Shrews: what's the sitrep on the thing? | 20:36 |
| sc` | no waiting forever for the failure message if it's already failed | 20:36 |
| fungi | krotscheck: any chance you're already looking into the npm-run job issue mentioned on the -dev ml? | 20:37 |
| fungi | i was going to take a peek for obvious causes if not | 20:37 |
| anteaya | clarkb: yay living in the same place for multiple years in a row \o/ | 20:38 |
| *** ajmiller has joined #openstack-infra | 20:38 | |
| fungi | clarkb: i'm feeling a similar rush. now that i own a house i'm actually inclined to, like, paint rooms maybe without worrying that i'll have to repaint when my lease is up | 20:39 |
| *** roxanaghe has quit IRC | 20:39 | |
| Shrews | jeblair: i'm stumped. i think we should try mordred's patch and pabelanger's change (probably separately) | 20:39 |
| fungi | feels kinda like https://xkcd.com/905/ sometimes | 20:39 |
| jeblair | fungi: in america, house owns you | 20:40 |
| fungi | too true!!! | 20:40 |
| jeblair | Shrews: mordred's change is the 'be resilient' change, right? what's pabelanger's? | 20:41 |
| *** piet has joined #openstack-infra | 20:41 | |
| fungi | i hope pabelanger's is called "be vigilant" | 20:41 |
| clarkb | jeblair: the logstash backlog is erratic and we have at least one worker that stopped doing stuff on the 16th with http://paste.openstack.org/show/522037/ appears related to gear, that look familiar to you at all? | 20:41 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci https://review.openstack.org/324777 | 20:41 |
| *** yamamoto has quit IRC | 20:42 | |
| clarkb | jeblair: looks like underlying network connectivity caused some gear reconnects but then it wait into a wait state that I don't understand | 20:43 |
| anteaya | fungi: in a kind of extrapolation that comic also describes any city person that comes to where I live | 20:43 |
| Shrews | jeblair: https://review.openstack.org/328649 | 20:43 |
| anteaya | what all this vibrant nature, all these beautiful animals, all this peace and quiet? | 20:44 |
| anteaya | let's destroy that | 20:44 |
| Shrews | jeblair: mordred's was https://github.com/emonty/ansible/commit/11bfa671246601c01767914a4f8b28f434fa80a0 | 20:44 |
| jeblair | clarkb: i don't see a 'grab_job' packet | 20:45 |
| *** rvba has joined #openstack-infra | 20:45 | |
| *** rvba has quit IRC | 20:45 | |
| *** rvba has joined #openstack-infra | 20:45 | |
| *** dmk0202 has quit IRC | 20:46 | |
| clarkb | notmorgan: btw http://logs.openstack.org/18/291318/16/check/gate-keystone-python27-db/b7d2a5b/console.html?level=INFO exploded spectacularly, not sure if a major issue in keystone unittests but probably worth a look | 20:46 |
| *** jamesmc__ has quit IRC | 20:46 | |
| clarkb | jeblair: I just hosed my browser opening ^ so will be a sec before I can refer back to that paste | 20:46 |
| clarkb | jeblair: I wonder if when the worker goes down on GRAB_JOB if it doesn't resend if servers flap under it? | 20:48 |
| clarkb | maybe not in all cases? | 20:48 |
| jeblair | pabelanger, Shrews: with 649, we will emit quite a large number of log lines | 20:49 |
| jeblair | they'll be uploaded to logs.o.o too | 20:50 |
| fungi | just to refresh my memory on how builders are supposed to work. for a shell builder, unless you set errorexit on a multi-line builder, the builder "fails" if the very last command run exits nonzero, right? | 20:50 |
| clarkb | I am going to restart one of the 4 workers on that host, and see if it persists. Possibly a firewall issue | 20:50 |
| fungi | i _think_ that's why the npm-run builder is silently passing stuff which fails `npm lint` but we have an awful lot of multi-line builders which don't set errorexit and might benefit from it if that's the case | 20:51 |
| *** zeih has joined #openstack-infra | 20:51 | |
| jeblair | fungi: they are "#!/bin/bash -x" by default | 20:52 |
| clarkb | oh we also need to update these workers to deal with new zuul worker console logs | 20:52 |
| fungi | got it. so no -e | 20:52 |
| jeblair | fungi: i'm pretty sure we matched that to jenkins, but it is ... difficult for me to check that at this point | 20:52 |
| clarkb | jeblair: do the zuul launchers continue to copy logs after the zmq finished event is sent like we did with jenkins console logs? or did that get corrected? | 20:52 |
| fungi | jeblair: yeah, that's my recollection from jenkins days, just trying to be sure | 20:53 |
| jeblair | clarkb: zmq completion event will only happen after log copying is complete | 20:53 |
| clarkb | jeblair: woot thanks | 20:54 |
| fungi | i guess these npm jobs don't generate subunit streams so i'm not going to find them in the health dashboard. time for graphite | 20:54 |
| jeblair | Shrews, pabelanger, mordred: i could get behind adding mordred's patch now | 20:54 |
| jeblair | Shrews, pabelanger, mordred: i want someone to say "i have thought about the log explosion issue and i think this is the best thing to do and everything will be fine" before pabelanger's patch though. :) | 20:55 |
| *** rfolco has quit IRC | 20:55 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Don't retry console log downloads https://review.openstack.org/334090 | 20:55 |
| clarkb | jeblair: ^ is the change to take advantage of that new behavior | 20:55 |
| *** zeih has quit IRC | 20:56 | |
| jeblair | clarkb: that is a complicated change | 20:56 |
| jeblair | i mean, for a friday | 20:56 |
| jeblair | is there a good way for me to 'git fetch' this? https://github.com/emonty/ansible/commit/11bfa671246601c01767914a4f8b28f434fa80a0 | 20:57 |
| clarkb | jeblair: git fetch github.com/emonty/ansible 11bfa671246601c01767914a4f8b28f434fa80a0 | 20:58 |
| jeblair | oh that works? | 20:58 |
| jeblair | neat | 20:58 |
| Shrews | jeblair: when one of the nodes is held, would it be possible to re-run the same job from the launcher, but by hand and with ANSIBLE_DEBUG=1 turned on? | 20:58 |
| clarkb | it should, you might need to add an https:// | 20:58 |
| fungi | jeblair: similar to the fetch syntax gerrit suggests for changes... `git fetch https://git.openstack.org/openstack/entropy refs/changes/18/118818/1` (just substitute the repo url and refname) | 20:59 |
| jeblair | Shrews: we can download the ansible directory that the job used and run it from anywhere (including the launcher). though most jobs (and especially these) are not idempotent, so i expect it to fail quickly | 20:59 |
| jeblair | fungi: yeah, i just didn't think you could fetch a git sha for some reason | 20:59 |
| fungi | oh, yeah you definitely can. done it quite a few times | 21:00 |
| mordred | jeblair: here's hoping my bandaid works with the other bandaids | 21:00 |
| jeblair | error: no such remote ref 11bfa671246601c01767914a4f8b28f434fa80a0 | 21:00 |
| mordred | and bandaids over the still-unknown-failure | 21:00 |
| Shrews | jeblair: poo. would be nice if we could capture that somehow | 21:00 |
| jeblair | that may be why i thought that | 21:00 |
| fungi | might be refs/11bfa671246601c01767914a4f8b28f434fa80a0 | 21:00 |
| jeblair | not so much | 21:00 |
| mordred | jeblair: I would do get fetch http://github.com/emonty/ansible resilient-async | 21:01 |
| mordred | which is the branch it's in | 21:01 |
| jeblair | mordred: neat. that should work, but i wish i knew how to come up with that on my own | 21:01 |
| mordred | jeblair: I went here: https://github.com/emonty/ansible/commit/11bfa671246601c01767914a4f8b28f434fa80a0 | 21:01 |
| *** dprince has quit IRC | 21:01 | |
| mordred | and then at the bottom of the commit message there is a little branch icon with the name of the branch | 21:01 |
| jeblair | oh there it is, thanks | 21:02 |
| mordred | jeblair: sure! I also find the ui very confusing - I only know this because I went through the same thing a few hours ago grabbing Shrews change | 21:02 |
| *** sarob has joined #openstack-infra | 21:02 | |
| jeblair | okay, i am going to try to write a playbook to rsync this repo onto the launchers | 21:03 |
| jeblair | i think rsync will be way better than a bunch of weird git commands | 21:03 |
| fungi | huh, i could have sworn i've fetched a sha before. this is weird | 21:04 |
| fungi | i wonder what i did | 21:04 |
| anteaya | I've seen you fetch a sha | 21:04 |
| anteaya | you have done so before | 21:04 |
| jeblair | i never have | 21:04 |
| anteaya | never? | 21:04 |
| jeblair | nope | 21:04 |
| fungi | digging in git manpages to try and see | 21:04 |
| anteaya | I would have considered you the kind of person that would | 21:04 |
| jeblair | mind you, i like fungi's world better and would like to live in it | 21:04 |
| jeblair | anteaya: well, in my world, it's not possible | 21:05 |
| anteaya | jeblair: ah | 21:05 |
| fungi | yeah, me too. if i could find my world again i'll send you an invite | 21:05 |
| anteaya | ha ha ha | 21:05 |
| *** Daisy has joined #openstack-infra | 21:05 | |
| anteaya | wear rubber boots | 21:05 |
| jeblair | maybe you can only fetch git shas one day every 100 years | 21:05 |
| *** sarob has quit IRC | 21:07 | |
| *** trown is now known as trown|outtypewww | 21:07 | |
| jeblair | i *know* there are people in this channel that got that. | 21:07 |
| *** esberglu has quit IRC | 21:07 | |
| *** sarob_ has joined #openstack-infra | 21:07 | |
| *** sarob_ has quit IRC | 21:08 | |
| *** sarob_ has joined #openstack-infra | 21:08 | |
| mordred | jeblair: did lin-manuel write that musical too? | 21:08 |
| clarkb | ya pretty sure you can fetch sha1s | 21:08 |
| clarkb | because all a ref does is point to a sha1 | 21:08 |
| * fungi thought of davey jones | 21:08 | |
| fungi | clarkb: now the question is how to make the syntax | 21:09 |
| jeblair | mordred: thank you | 21:09 |
| *** Daisy has quit IRC | 21:09 | |
| jeblair | "fungi lives in brigadoon" is the answer we were looking for | 21:09 |
| anteaya | ha ha ha | 21:09 |
| fungi | oh, indeed. been forever since i've seen that | 21:10 |
| *** yamamoto has joined #openstack-infra | 21:11 | |
| *** mriedem has quit IRC | 21:11 | |
| jeblair | mordred: i want a playbook that does 1 task on all launchers, then does several more tasks on all the launchers (only after the first task has been done everywhere successfully) | 21:13 |
| jeblair | mordred: what does that look like? | 21:13 |
| clarkb | jeblair: iirc you just add a new top level type section after the first one and specify your hosts again and all that | 21:14 |
| *** esberglu has joined #openstack-infra | 21:14 | |
| jeblair | so the yaml 'file' thing: --- | 21:14 |
| jeblair | oh | 21:14 |
| mordred | you don't need th e--- | 21:14 |
| clarkb | you might not even need that its just another list entry | 21:14 |
| jeblair | no the playbook is a list already | 21:14 |
| clarkb | ya | 21:14 |
| mordred | yah | 21:14 |
| *** esberglu has quit IRC | 21:14 | |
| jeblair | so just another element in the top level list | 21:15 |
| jeblair | got it | 21:15 |
| mordred | each of the things in the list is a "play" | 21:15 |
| *** yamamoto has quit IRC | 21:15 | |
| *** piet has quit IRC | 21:15 | |
| *** nwkarsten has quit IRC | 21:18 | |
| *** ldnunes has quit IRC | 21:18 | |
| *** inc0 has quit IRC | 21:18 | |
| *** jbernard has quit IRC | 21:19 | |
| *** nwkarsten has joined #openstack-infra | 21:19 | |
| fungi | okay, apparently i'm more insane than i realized. it's not possible without git >=2.5 on the server and uploadpack.allowReachableSHA1InWant set | 21:19 |
| clarkb | huh | 21:19 |
| clarkb | clearly we should all upgrade | 21:19 |
| jeblair | we should upgrade github | 21:20 |
| clarkb | jeblair: yes that, oh wait | 21:20 |
| fungi | ftp://www.kernel.org/pub/software/scm/git/docs/junio-gpg-pub/RelNotes/2.5.0.txt | 21:20 |
| jeblair | clarkb: to be fair, external parties *have* upgraded github before | 21:20 |
| *** weshay is now known as weshay_afk | 21:20 | |
| fungi | so i'm quite sure i've not interacted with a git 2.5+ server often enough to have been likely to have ever successfully done that thing | 21:20 |
| jeblair | something about ruby iirc | 21:20 |
| *** ajmiller_ has joined #openstack-infra | 21:21 | |
| fungi | [new in 2.5] "git upload-pack" that serves "git fetch" can be told to serve commits that are not at the tip of any ref, as long as they are reachable from a ref, with uploadpack.allowReachableSHA1InWant configuration variable. | 21:22 |
| *** persia has quit IRC | 21:22 | |
| fungi | so prior to 2.5, fetch only groks ref tips | 21:22 |
| *** nwkarsten has quit IRC | 21:23 | |
| fungi | or rather, the server only knows how to hand them out | 21:23 |
| fungi | if they're ref tips (tags, heads, et cetera) | 21:23 |
| openstackgerrit | Merged openstack-infra/system-config: Don't retry console log downloads https://review.openstack.org/334090 | 21:23 |
| *** lascii is now known as alaski | 21:23 | |
| fungi | okay, my curiosity is satisfied. the likelihood that i've been hopping between almost identical parallel dimensions increases another notch | 21:24 |
| *** persia has joined #openstack-infra | 21:24 | |
| clarkb | what if the sha1 is a ref tip? I guess it would've been in jeblairs case | 21:25 |
| *** ajmiller has quit IRC | 21:25 | |
| *** aysyd has quit IRC | 21:25 | |
| notmorgan | hmm... looks like i can't "check" current revision of the db like that... hhmmm | 21:25 |
| *** ajmiller_ has quit IRC | 21:27 | |
| clarkb | with logstash sorted for the moment /me looks into that keystone derp | 21:28 |
| *** rlandy has quit IRC | 21:29 | |
| clarkb | jeblair: we likely want to put a sanity check into our file uploads, I am up to 175MB and counting on this console log download | 21:29 |
| clarkb | now over 200MB | 21:29 |
| clarkb | I am impressed with keystones ability to do this | 21:29 |
| jeblair | clarkb: did we have that with jenkins? i know we put some stuff in the run-tox scripts... | 21:30 |
| clarkb | jeblair: I am not sure we did with jenkins | 21:30 |
| clarkb | 300MB | 21:30 |
| jeblair | i'm going to hard-restart all the zuul launchers with mordred's change now | 21:30 |
| jeblair | well first | 21:31 |
| jeblair | http://paste.openstack.org/show/522046/ | 21:31 |
| jeblair | how does that playbook look? | 21:31 |
| jeblair | clarkb, mordred: ^ | 21:31 |
| *** ajmiller has joined #openstack-infra | 21:32 | |
| jeblair | (i ran the first play separately, and the resulting git repo lgtm | 21:32 |
| stevemar | clarkb: what did we do now? :) | 21:32 |
| fungi | stevemar: manufacture very large job console logs? | 21:33 |
| clarkb | jeblair: yes lgtm | 21:33 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/nodepool: Add alembic migration support https://review.openstack.org/136589 | 21:33 |
| clarkb | stevemar: http://logs.openstack.org/18/291318/16/check/gate-keystone-python27-db/b7d2a5b/console.html?level=INFO | 21:33 |
| clarkb | stevemar: be warned its over 500MB now | 21:33 |
| jeblair | here goes | 21:33 |
| * clarkb is happy he doesn't have a bw quota to worry about | 21:33 | |
| stevemar | =\ | 21:33 |
| clarkb | stevemar: not sure if it is only that change or if larger scale | 21:34 |
| clarkb | but in general if your unittests produce that much output its unhappytimes | 21:34 |
| clarkb | still waiting for my download to finish so I can dig in and see what happened | 21:34 |
| stevemar | theres a lot of https://review.openstack.org/#/c/333109/5 | 21:35 |
| stevemar | err | 21:35 |
| clarkb | http://paste.openstack.org/show/522047/ | 21:35 |
| clarkb | its that create grant for miles | 21:35 |
| stevemar | File "keystone/assignment/backends/base.py", line 458, in create_grant domain_id, project_id, inherited_to_projects) | 21:35 |
| clarkb | 576MB | 21:35 |
| stevemar | clarkb: god damn revoke api | 21:36 |
| * stevemar closed the tab | 21:36 | |
| *** gouthamr has quit IRC | 21:36 | |
| clarkb | stevemar: this change does modify that file | 21:36 |
| clarkb | https://review.openstack.org/#/c/291318/22/keystone/assignment/backends/base.py | 21:36 |
| jeblair | doesn't python have a 1000-frame stack limit? | 21:36 |
| jeblair | oh, is it one of those for every test? | 21:36 |
| stevemar | lol | 21:37 |
| stevemar | jeblair: maybe not *every* | 21:37 |
| clarkb | jeblair: ya the actual error is the stack limit being hit | 21:37 |
| clarkb | so every one of the failed tests writs ~2k lines | 21:37 |
| stevemar | clarkb: slapped a -2 on it, can you kill the job? | 21:38 |
| jeblair | pabelanger, mordred, Shrews, clarkb, EmilienM: i have restarted all the zuul launchers with mordred's ansible change (which we also expect to reduce incidences of the error). i will delete any held nodes now. | 21:38 |
| clarkb | https://review.openstack.org/#/c/291318/16/keystone/assignment/backends/base.py is the ps that exploded | 21:38 |
| clarkb | stevemar: oh the job is long dead | 21:38 |
| clarkb | stevemar: and the explosion happened on an older ps | 21:38 |
| stevemar | clarkb: ah | 21:38 |
| stevemar | yeah, i see that now, 22 vs 16 | 21:38 |
| clarkb | stevemar: what I am trying to figure out is if this is more persistent but probably isn't, so if we can see that latest ps works then maybe we are ok | 21:39 |
| clarkb | oh lol I see the bug ya its related to that ps | 21:39 |
| clarkb | there is no check to stop recursing so its an infinite loop (except for stack limits) | 21:39 |
| * clarkb looks at latest ps | 21:39 | |
| fungi | playing around with graphite, it looks like the last time gate-horizon-npm-run-lint failed in the check pipeline was late on june 17, and i don't see any obvious changes to their job configuration around that timeframe | 21:40 |
| clarkb | stevemar: ok I thin kwe are fine now, probably do want to have a sanity check somewhere for this though | 21:40 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/nodepool: Add alembic migration support https://review.openstack.org/136589 | 21:40 |
| EmilienM | jeblair: thanks for the update | 21:41 |
| * clarkb wonders why people don't run tox locally first | 21:41 | |
| fungi | so around june 18th it seems like that job started passing always even when it should fail | 21:41 |
| fungi | clarkb: or `npm lint` in this case | 21:42 |
| fungi | (it doesn't use tox) | 21:42 |
| clarkb | fungi: ya thinking of the infinite recursion make half a gig of log data case | 21:42 |
| fungi | oh, hah | 21:42 |
| fungi | yep | 21:42 |
| fungi | so that's what was going on with the huge keystone log? | 21:43 |
| fungi | a loop/recursion? | 21:43 |
| *** amitgandhinz has quit IRC | 21:43 | |
| fungi | fun stuff | 21:43 |
| *** matt-borland has quit IRC | 21:43 | |
| stevemar | fungi: we excel in fun | 21:43 |
| clarkb | yup there was a recursive call with no break, it hit stack limits which made massive tracebacks which happened over and over in many tests | 21:43 |
| clarkb | its unfortunate that Turing guy figured out we can't test for this | 21:43 |
| *** claudiub|2 has joined #openstack-infra | 21:44 | |
| clarkb | we could run a gate-halting on all projects | 21:44 |
| *** PollyZ has joined #openstack-infra | 21:44 | |
| fungi | yeah, someone should really do something about him | 21:44 |
| stevemar | we should create a test after him, a Turing test maybe | 21:45 |
| jeblair | fungi: june 16 was no-more-jenkins day | 21:46 |
| fungi | jeblair: yep, i don't think the npm job situation is related because your message went out late on the 16th and we had failures through late the next day according to graphite | 21:47 |
| jeblair | fungi: so if you are certain the correct behavior manifested between june 16 and june 18, that should exclude zuul-launcher behavior change as a potential cause | 21:47 |
| clarkb | wow its only 2:45pm, this waking up at 3am business has me all confused | 21:47 |
| jeblair | 21:47 < clarkb> wow its only 2:45pm, this waking up at 3am business has me all confused | 21:47 |
| pabelanger | jeblair: re: 649. Let me look into our logging issue more. And possibly propose a change | 21:48 |
| anteaya | clarkb: did you wake up at 3am again today? | 21:48 |
| jeblair | that's some lag :) | 21:48 |
| clarkb | anteaya: no but everything is wonky | 21:48 |
| fungi | er, i was misreading the graph, so failures up to nearly noon utc on the 18th i think? http://graphite.openstack.org/render/?width=800&height=400&target=None&target=stats_counts.zuul.pipeline.check.job.gate-horizon-npm-run-lint.FAILURE&from=-24days | 21:48 |
| anteaya | clarkb: ah | 21:48 |
| jeblair | fungi: agreed | 21:49 |
| *** thorst_ has quit IRC | 21:49 | |
| fungi | anyway, i need to go make pizza. this dough isn't gonna toss itself | 21:49 |
| fungi | back in a while | 21:49 |
| *** thorst has joined #openstack-infra | 21:50 | |
| anteaya | happy pizza | 21:50 |
| *** mdrabe has quit IRC | 21:50 | |
| *** bauwser is now known as bauza | 21:52 | |
| *** bauza is now known as bauzas | 21:52 | |
| *** zeih has joined #openstack-infra | 21:52 | |
| *** gouthamr has joined #openstack-infra | 21:53 | |
| *** gouthamr_ has joined #openstack-infra | 21:55 | |
| *** roxanaghe has joined #openstack-infra | 21:56 | |
| *** zeih has quit IRC | 21:56 | |
| *** roxanaghe has quit IRC | 21:56 | |
| *** gouthamr has quit IRC | 21:58 | |
| *** Apoorva has quit IRC | 21:58 | |
| *** Apoorva has joined #openstack-infra | 21:58 | |
| *** thorst has quit IRC | 21:58 | |
| *** roxanaghe has joined #openstack-infra | 21:59 | |
| clarkb | ok I think logstash setup is happyness now | 22:00 |
| anteaya | yay happyness | 22:03 |
| *** dimtruck is now known as zz_dimtruck | 22:05 | |
| *** csomerville has quit IRC | 22:06 | |
| anteaya | zaro: are you around today at all? I'm wondering if you have any thoughts on SotK's last comment on this patch: https://review.openstack.org/#/c/332946/17 | 22:07 |
| anteaya | zaro: you had said you are way next week, I wonder if you are here today or gone already | 22:07 |
| *** sdague has quit IRC | 22:08 | |
| anteaya | s/way/away | 22:11 |
| *** pradk has quit IRC | 22:12 | |
| *** ajmiller has quit IRC | 22:13 | |
| *** harlowja has quit IRC | 22:17 | |
| *** tpsilva has quit IRC | 22:18 | |
| pabelanger | jeblair: it looks like FAILED - RETRYING: TASK: command (1 retries left) loggings lines are removed in the latest version of ansible. So, long term, I don't think it is an issue for us. https://github.com/ansible/ansible/issues/14558 was the reported issue. | 22:19 |
| clarkb | doing more digging into the logstash stuff we may want to consider adding the zuul launcher info to the event (or maybe its there and I just have to inject it into ES) | 22:19 |
| clarkb | and I should probably stop using type == jenkins :) | 22:19 |
| anteaya | clarkb: meaning zuul launcher id number? | 22:20 |
| pabelanger | jeblair: I'll test in a bit with latest ansible to confirm | 22:20 |
| clarkb | anteaya: the hostname or something like that ya | 22:20 |
| clarkb | also build_status is now a json dict rather than just SUCCESS or FAILURE | 22:20 |
| clarkb | jeblair: ^ I think that is a slight api change in the zmq events | 22:20 |
| jeblair | clarkb: what's build_status? | 22:21 |
| clarkb | jeblair: was SUCCESS, FAILURE, ABORTED before iirc | 22:21 |
| jeblair | event['build']['status']='SUCCESS' | 22:21 |
| jeblair | clarkb: oh hrm, | 22:22 |
| anteaya | that sounds reasonable to me | 22:22 |
| jeblair | clarkb: it may end up as a dict | 22:22 |
| *** rhochmuth has quit IRC | 22:22 | |
| clarkb | jeblair: fields["build_status"] = event["build"].get("status", "UNKNOWN") is what we do to get it from zmq | 22:23 |
| clarkb | jeblair: and {"result": "SUCCESS"} is what we get from that | 22:23 |
| *** ajmiller has joined #openstack-infra | 22:24 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/storyboard: Add example commands for the Story api https://review.openstack.org/332946 | 22:26 |
| *** rbrndt_ has quit IRC | 22:26 | |
| *** PollyZ_ has joined #openstack-infra | 22:28 | |
| *** harlowja has joined #openstack-infra | 22:30 | |
| *** PollyZ has quit IRC | 22:32 | |
| *** edmondsw has quit IRC | 22:32 | |
| zaro | fungi: this should have turned off puppet on review-dev correct? https://review.openstack.org/#/c/332606/ | 22:34 |
| zaro | fungi: i'm seeing files getting reverted again. was wondering how to verify that it's off? | 22:35 |
| zaro | anteaya: i'm here | 22:36 |
| *** PollyZ_ has quit IRC | 22:36 | |
| *** raunak has quit IRC | 22:37 | |
| anteaya | zaro: ah thank you | 22:38 |
| anteaya | zaro: the bit we cant' seem to agree on is the response examples, are you okay if I leave that for now so we can merge the command examples? | 22:39 |
| anteaya | zaro: I'd like to get this first section merged then move onto other sections | 22:39 |
| anteaya | we can discuss the response examples later in follow up patches? | 22:39 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: clean up result handling https://review.openstack.org/334104 | 22:40 |
| clarkb | woo e-r grpahs are all ancient /me looks at that next | 22:40 |
| jeblair | clarkb: ^ | 22:40 |
| jeblair | zaro: there was an error in ansible this morning that would have caused that, i believe it has been corrected | 22:40 |
| clarkb | jeblair: danke | 22:41 |
| *** mriedem has joined #openstack-infra | 22:41 | |
| jeblair | clarkb: will the log processor handle {"build": {"status": null}} ? | 22:41 |
| clarkb | jeblair: I think it will, if that json parses null into None then we will set it to the strin "UNKNOWN" | 22:42 |
| jeblair | clarkb: yes, json will do that | 22:42 |
| clarkb | if it parses it into the string "null" then we will set it to "null: | 22:42 |
| clarkb | cool so that should be fine | 22:42 |
| jeblair | w00t | 22:42 |
| fungi | zaro: yeah, all the servers we disabled ansible/puppet for in git got updated when something caused ansible to not expand our host groups (and "disabled" is one of our groups) | 22:42 |
| jeblair | clarkb: oh wait, but it looks for 'ABORTED' right? | 22:43 |
| clarkb | thats curious we have a bunch of flock processes with hour long waits that are way older than an hour | 22:43 |
| clarkb | jeblair: it will ignore ABORTED ya | 22:44 |
| jeblair | clarkb: we don't send 'ABORTED' anymore -- it'll be None instead in those cases | 22:44 |
| clarkb | ok, we can update the workers | 22:44 |
| jeblair | clarkb: (basically, a zuul abort is indistinguishable from an operator-triggered abort) | 22:44 |
| zaro | anteaya: sure, i'm flexible as comment on the change | 22:44 |
| *** rajinir has quit IRC | 22:44 | |
| jeblair | (and they both result in None) | 22:44 |
| anteaya | zaro: oh thank you | 22:45 |
| *** mrhillsman is now known as mrhillsman_ | 22:46 | |
| *** yamahata has quit IRC | 22:47 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-log_processor: Handle zuul launcher aborted jobs https://review.openstack.org/334107 | 22:47 |
| clarkb | jeblair: ^ that should do it | 22:47 |
| *** Goneri has quit IRC | 22:48 | |
| jeblair | clarkb: er, actually, i think build_status will be None there | 22:49 |
| zaro | well, that was good to know. thanks. | 22:49 |
| jeblair | clarkb: because .get will succesfully return the value in the dict -- None | 22:49 |
| clarkb | jeblair: oh huh | 22:50 |
| clarkb | right .get only returns if the key doesn't exist | 22:50 |
| clarkb | in this case the key does exist | 22:50 |
| jeblair | clarkb: we could have zuul launcher not return a status instead of saying 'null' | 22:50 |
| jeblair | or we could check for that in the log processor | 22:50 |
| clarkb | the current thing is fine, and we can just check for the serialized "None" | 22:50 |
| jeblair | it doesn't really matter; the interface here is vague and zuul scheduler handles both | 22:51 |
| jeblair | ok | 22:51 |
| clarkb | since the gearman stuff should serialize it to None | 22:51 |
| clarkb | er "None" | 22:51 |
| *** ajmiller has quit IRC | 22:51 | |
| *** ajmiller has joined #openstack-infra | 22:52 | |
| clarkb | oh wait no because its json again | 22:52 |
| *** raunak has joined #openstack-infra | 22:52 | |
| *** ajmiller has quit IRC | 22:52 | |
| jeblair | yeah, we should end up with None in both places i think | 22:52 |
| clarkb | ya job = gear.Job(b'push-log', json.dumps(output).encode('utf8')) | 22:53 |
| *** zeih has joined #openstack-infra | 22:53 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-log_processor: Handle zuul launcher aborted jobs https://review.openstack.org/334107 | 22:54 |
| clarkb | I hvae killed the oldest (from june 9th) flocked process on status.o.o hoping that the e-r things will start flowing now | 22:55 |
| *** gordc has quit IRC | 22:55 | |
| *** thorst has joined #openstack-infra | 22:57 | |
| *** Apoorva has quit IRC | 22:57 | |
| *** zeih has quit IRC | 22:57 | |
| *** rvba has quit IRC | 22:58 | |
| *** salv-orl_ has joined #openstack-infra | 22:58 | |
| *** salv-orlando has quit IRC | 23:01 | |
| *** markvoelker has quit IRC | 23:01 | |
| *** yamahata has joined #openstack-infra | 23:02 | |
| *** markvoelker has joined #openstack-infra | 23:02 | |
| *** thorst has quit IRC | 23:04 | |
| *** ams__ has quit IRC | 23:04 | |
| clarkb | jeblair: ok http://status.openstack.org/elastic-recheck/ is working agian and you can see the dicts there, but those will filter out over the next 10 days or so once we get these changes in | 23:08 |
| jeblair | cool | 23:09 |
| clarkb | jeblair: one neat thing is the lack of jenkins slaves went offline :) | 23:09 |
| jeblair | \o/ | 23:10 |
| clarkb | git connectivity seems to be our biggest issue in the env according to e-r now | 23:10 |
| *** amitgandhinz has joined #openstack-infra | 23:14 | |
| anteaya | clarkb: do we need more git servers in the farm? | 23:15 |
| clarkb | anteaya: no I think it is the rax ord connectivity issues | 23:15 |
| anteaya | ah | 23:15 |
| anteaya | :( | 23:15 |
| *** amitgandhinz has quit IRC | 23:18 | |
| *** nwkarsten has joined #openstack-infra | 23:21 | |
| *** hongbin has quit IRC | 23:22 | |
| *** nwkarsten has quit IRC | 23:26 | |
| *** markvoelker has quit IRC | 23:26 | |
| rm_work | Hey, can the job timeout for our gate jobs be configured on an individual job basis? | 23:35 |
| rm_work | My understanding was that it wasn't something we were able to change on our own | 23:35 |
| rm_work | Is there a reason we can't just set our job timeout to, say, 120 minutes? | 23:36 |
| fungi | rm_work: yeah, the timeout is set per job or job-template in the jenkins/jobs yaml configuration files | 23:37 |
| rm_work | OK, so there's no reason we couldn't turn it up a bit? | 23:37 |
| rm_work | (like, a political reason, if not a technical one) | 23:37 |
| *** superdan is now known as dansmith | 23:38 | |
| rm_work | Any policy that says "don't use over <X> timeout value"? | 23:38 |
| fungi | well, there's always the "why are our jobs taking longer? dunno... let's just up the timeout!" factor | 23:38 |
| rm_work | heh, yes | 23:38 |
| rm_work | but if we KNOW why, and need the timeout higher, it's possible? | 23:38 |
| rm_work | I wonder why I thought we couldn't do that | 23:39 |
| rm_work | I swear we investigated it and were told we couldn't do it | 23:39 |
| fungi | like, you should _understand_ why your jobs take that long and be sure you can't improve your job performance before you consider just letting them run longer | 23:39 |
| JayF | There' | 23:40 |
| JayF | There's also the "how long do you want to wait for things to check / merge" factor | 23:40 |
| fungi | also if your job is instantiated from a job-template shared by other jobs/projects you may need to copy that template to a slightly different name | 23:40 |
| JayF | part of why we did the work to stabilize ironic jobs before going with grenade, b/c they already take a while and rechecks hurt | 23:40 |
| rm_work | Yeah | 23:41 |
| fungi | and if your project has these jobs running in a shared gate queue with other projects, having jobs take too long somewhat slows down gating for other projects | 23:42 |
| rm_work | This is related to "octavia jobs need to spin up nested VMs and there's no vt-x support in the gate" | 23:42 |
| rm_work | we split into like 5 jobs to run everything in parallel | 23:42 |
| *** sdague has joined #openstack-infra | 23:43 | |
| rm_work | but even so, some of those are getting precariously close to timeout, and we don't even have many tests in each yet | 23:43 |
| JayF | similar to the problems ironic has had, we just had to chip away at it and do some optimizations to make it faster | 23:43 |
| JayF | I'm not familiar with octavia much, but I know using the virtio network driver in qemu and using ipxe for most pxe booting helped save us a little bit of time | 23:44 |
| rm_work | hmm | 23:44 |
| JayF | seems like that might not be relevant for you all though, doesn't look liek you do much pxe booting | 23:44 |
| rm_work | probably worth looking into | 23:44 |
| rm_work | no, we do not | 23:44 |
| rm_work | the virtio maybe? | 23:45 |
| JayF | if you're just testing an lb, is there any way to fake out the servers instead of booting them? | 23:45 |
| rm_work | does that increase boot-time? | 23:45 |
| rm_work | no | 23:45 |
| rm_work | these are our functional tests | 23:45 |
| *** thingee has quit IRC | 23:45 | |
| rm_work | we need to make sure that the configs are generated and shipped correctly, AND that the agent inside the vm sets them up right, and that haproxy actually starts and loadbalances | 23:45 |
| JayF | Ah. Well good luck then, sounds like you have a similar but non-overlapping problem to Ironic. | 23:45 |
| JayF | could you use something like a container instead? | 23:45 |
| rm_work | We'd LOVE to do that | 23:46 |
| JayF | That would likely move a bit faster than a full VM, if that's what you're having to do. | 23:46 |
| rm_work | lol | 23:46 |
| JayF | in Ironic we have to spawn the vm because pxe booting and other shenanigans | 23:46 |
| rm_work | then we'd need support for containers, which is... not there yet | 23:46 |
| rm_work | but also that wouldn't test our VM driver :/ | 23:46 |
| fungi | an option to avoid the slows-down-gating-for-other-projects factor (if that's an issue) is to run those jobs _only_ in the check pipeline | 23:46 |
| *** roxanaghe has quit IRC | 23:46 | |
| jeblair | rm_work: yeah, we've had a 60 minute target for a while | 23:46 |
| rm_work | yeah fungi i think that's probably reasonable | 23:46 |
| fungi | but that's usually a bad idea for letting deadlocks slip through, so best avoided if it can be | 23:47 |
| *** salv-orl_ has quit IRC | 23:47 | |
| rm_work | jeblair: i remember the global default was lowered a while back from like 80? | 23:47 |
| jeblair | rm_work: i beleve the *default* is 30 minutes | 23:47 |
| rm_work | I might look at bumping ours back to like 90m though | 23:47 |
| rm_work | lol ok | 23:47 |
| rm_work | it takes 30m just for devstack to fully spin up IME | 23:48 |
| jeblair | well, that's for all jobs, only about 50% of which are devstack | 23:48 |
| rm_work | I always laugh at the "this could take up to five minutes" line >_< | 23:48 |
| clarkb | rm_work: devstack is down to about 1 minutes now | 23:48 |
| clarkb | er 10 | 23:48 |
| *** woodster_ has quit IRC | 23:49 | |
| rm_work | well, we have to build a VM image as part of our stack plugin, which means an extra 10m or so | 23:49 |
| rm_work | and we rely on many projects so they have to install too | 23:49 |
| fungi | for example, if your pep8 job runs longer than the default 30-minute timeout, you might be thankful we don't let that keep going | 23:49 |
| rm_work | our gate installs like ... nova, glance, keystone, neutron, neutron-lbaas, octavia, barbican and maybe a couple other things i'm not remembering from the base | 23:49 |
| *** adam_g has quit IRC | 23:50 | |
| rm_work | heh yeah i think pep8 is much lower ^_^ | 23:50 |
| fungi | you can also trim down what services are started in d-g if that helps at all | 23:50 |
| *** thingee has joined #openstack-infra | 23:51 | |
| clarkb | also does novas container stuff not work? | 23:52 |
| clarkb | in theory you could have a container flavor and boot those and you wouldnt know the difference from an api perspective | 23:52 |
| fungi | as in nova-docker? | 23:53 |
| rm_work | I thought that wasn't ... "official" | 23:54 |
| *** zeih has joined #openstack-infra | 23:54 | |
| rm_work | I think it might do network plugging differently? which affects us | 23:54 |
| rm_work | we do manual networking stuff | 23:54 |
| rm_work | and the agent is fairly aware of what system it's running inside, so we'd need a slightly different driver for it | 23:54 |
| clarkb | either nova docker or nova lxc | 23:54 |
| rm_work | we'd LOVE to have containers working, but it's not a simple undertaking | 23:55 |
| rm_work | it's definitely on our roadmap though | 23:55 |
| clarkb | you do more than just listen on interface with haproxy and check another service somewhere? | 23:55 |
| rm_work | we have an agent that runs inside the Amphora | 23:56 |
| rm_work | VM/container/whatever it | 23:56 |
| rm_work | *it is | 23:56 |
| clarkb | ya but why does it care about the external networking | 23:56 |
| clarkb | if you have tcp/udp it shpuld be happu no? | 23:56 |
| * fungi loves the word "amphora" | 23:56 | |
| rm_work | we plug/unplug networks with neutron | 23:56 |
| rm_work | and use namespaces internally | 23:56 |
| rm_work | the agent facilitates that | 23:57 |
| rm_work | well, along with changes in our custom image | 23:57 |
| rm_work | i don't know how the images work for containers | 23:57 |
| rm_work | I'm guessing we'd need to build a different type of image for it to work as a container image? | 23:57 |
| clarkb | you shouldnt need to | 23:58 |
| fungi | not necessarily, no | 23:58 |
| *** sdake has joined #openstack-infra | 23:58 | |
| rm_work | so we build an ubuntu 14.04 image with DIB, we could just boot that as a container? | 23:58 |
| *** zeih has quit IRC | 23:58 | |
| clarkb | yes | 23:58 |
| clarkb | it will just ignore ypur kernel | 23:58 |
| fungi | the main difference is that the kernel/bootloader in your image is irrelevant in a container, i think? | 23:58 |
| *** Sukhdev has quit IRC | 23:58 | |
| rm_work | interesting, i wonder how difficult it'd be to test that | 23:58 |
| rm_work | if i can just ... enable the lxc/docker flavor easily and switch the flavor tag we use.... | 23:59 |
| rm_work | I *feel* like there's more to it, but | 23:59 |
| *** SumitNaiksatam has quit IRC | 23:59 | |
| rm_work | i'm willing to try it out, for sure | 23:59 |
| * clarkb wonder why you need to muck with neutron in theory you just need tcp/udp and or other l3 connectivity to lb | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!