| *** mrmartin has quit IRC | 00:00 | |
| *** esker has joined #openstack-infra | 00:02 | |
| *** yamamoto has quit IRC | 00:03 | |
| *** camunoz has quit IRC | 00:03 | |
| *** otter768 has quit IRC | 00:03 | |
| *** pahuang has quit IRC | 00:03 | |
| *** tlian2 has joined #openstack-infra | 00:04 | |
| *** dchen has joined #openstack-infra | 00:04 | |
| *** ybathia has quit IRC | 00:04 | |
| *** bdemers has joined #openstack-infra | 00:06 | |
| *** tlian has quit IRC | 00:07 | |
| *** shashank_hegde has quit IRC | 00:07 | |
| *** bdemers has quit IRC | 00:08 | |
| *** bdemers has joined #openstack-infra | 00:09 | |
| openstackgerrit | Merged openstack-infra/shade: Clarify future changes in docs https://review.openstack.org/209055 | 00:09 |
|---|---|---|
| *** zul has joined #openstack-infra | 00:09 | |
| *** yamamoto has joined #openstack-infra | 00:09 | |
| *** achanda_ has quit IRC | 00:11 | |
| *** tim_o has quit IRC | 00:12 | |
| *** ivar-lazzaro has quit IRC | 00:14 | |
| *** yamamoto has quit IRC | 00:14 | |
| *** pahuang has joined #openstack-infra | 00:16 | |
| openstackgerrit | Davide Agnello proposed openstack-infra/project-config: Adding RALLY_SCENARIO venv for Cue job https://review.openstack.org/201285 | 00:16 |
| *** yamamoto has joined #openstack-infra | 00:16 | |
| *** thorst has quit IRC | 00:20 | |
| *** esker has quit IRC | 00:20 | |
| *** camunoz has joined #openstack-infra | 00:20 | |
| *** esker has joined #openstack-infra | 00:21 | |
| *** weshay has quit IRC | 00:22 | |
| morganfainberg | lifeless: question for you - re: supporting something like uwsgi in openstack (devstack) | 00:27 |
| morganfainberg | lifeless: should I add this to globalreqs? long term I'm expecting to make it the default deployment in devstack, but I'm not sure if it should be in keystonerequirements.txt or test-requirements.txt here | 00:27 |
| morganfainberg | or is this an extras like pip install keystone[uwsgi] | 00:28 |
| morganfainberg | ? | 00:28 |
| *** jasondotstar is now known as jasondotstar|awa | 00:28 | |
| *** jasondotstar|awa is now known as jasondotstar|afk | 00:28 | |
| *** pvaneck has quit IRC | 00:29 | |
| *** Guest5314 is now known as med_ | 00:29 | |
| *** med_ has quit IRC | 00:29 | |
| *** med_ has joined #openstack-infra | 00:29 | |
| mtreinish | morganfainberg: how was mod_wsgi installed? | 00:30 |
| morganfainberg | mtreinish: mod_wsgi is an apache module | 00:30 |
| morganfainberg | uwsgi is python / pip installable | 00:30 |
| mtreinish | morganfainberg: I thought mod_wsgi was just python code too | 00:30 |
| morganfainberg | and is managed separate from the associated apache_module | 00:30 |
| morganfainberg | nope | 00:30 |
| morganfainberg | mod_uwsgi is a direct apache module which would work like mod_wsgi does | 00:31 |
| mtreinish | morganfainberg: google failed me: https://pypi.python.org/pypi/mod_wsgi/4.4.13 | 00:31 |
| morganfainberg | those are both c/c++ apache/bucketbrigade | 00:31 |
| clarkb | if you do that you have to switch horizon too or use different ports right? | 00:31 |
| morganfainberg | oh it might be cython | 00:31 |
| morganfainberg | i've *never* pip installed mod_wsgi | 00:31 |
| morganfainberg | ever | 00:31 |
| mtreinish | morganfainberg: but that's what I was asking we have to be getting it from somewhere, are we just relying on the ubuntu package for it? | 00:32 |
| clarkb | in any case it has to be on global reqs before you can use it anywhere properly so do that. then I think I would extras it | 00:32 |
| morganfainberg | ubuntu package atm | 00:32 |
| morganfainberg | clarkb: well we could also just do the system package | 00:32 |
| clarkb | no please dont :) | 00:32 |
| morganfainberg | but with uwsgi it looks like we may want to track faster | 00:32 |
| mtreinish | morganfainberg: yeah dont | 00:32 |
| clarkb | that circumvents the shiny constraints stuff | 00:32 |
| morganfainberg | and it would break my end goal of supporting venvs | 00:32 |
| morganfainberg | the other big win here is this should eliminate the apache can't stop error | 00:33 |
| morganfainberg | since uwsgi is run separate from apache itself. | 00:33 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Throw an exception on a server without an IP https://review.openstack.org/208105 | 00:33 |
| mordred | Shrews: ^^ what about that | 00:33 |
| morganfainberg | whereas mod_wsgi doesn't do that afaict [plus ini-style configs = win compared to icky apache vhost configs] | 00:33 |
| SpamapS | morganfainberg: the apache can't stop error? | 00:34 |
| morganfainberg | SpamapS: it's fairly transient but we can't do an apache restart | 00:34 |
| morganfainberg | apache is SLOW to restart with mod_wsgi processes if there are active connections | 00:34 |
| mtreinish | SpamapS: iirc, it's why we don't use keystone in apache on grenade | 00:34 |
| morganfainberg | that is since apache is directly managing the wsgi processes. | 00:35 |
| SpamapS | also isn't uwsgi going to give some things up compared to Apache for the simple fact that it's not the 25 year optimized/matured apache daemon? | 00:35 |
| morganfainberg | mtreinish: it's one of the reasons | 00:35 |
| morganfainberg | SpamapS: we would be using uwsgi as jus the process manager in these cases. | 00:35 |
| mtreinish | morganfainberg: well I was just giving you the benefit of the doubt on the eventlet -> apache thing :) | 00:35 |
| SpamapS | so uwsgi does some kind of socket that mod_uwsgi talks over? | 00:35 |
| morganfainberg | mtreinish: lol don't ever give me that benefit, always ask hard questions [cause i like giving answers like eventlet sucks) | 00:35 |
| morganfainberg | SpamapS: that is one mode, it also can act as a stand-alone http server (migration path from eventlet anyone?) | 00:36 |
| morganfainberg | SpamapS: also mod_wsgi doesn't easily (or even when it does, doesn't do it well) support venvs under one apache | 00:36 |
| morganfainberg | so uwsgi is a good alternative as it can live in the services venv [this isn't devstack relevant, but will help some deployment methodologies] | 00:37 |
| SpamapS | no thats all very interesting. | 00:37 |
| morganfainberg | i'm also fine being told "oh god no" :) hence why bringing up the convo :) | 00:37 |
| SpamapS | I'm kind of thinking in the case where you care about the difference in optimization that uwsgi vs. mod_wsgi offers.. you just use containers or chroots anyway. | 00:38 |
| *** tsekiyam_ has joined #openstack-infra | 00:38 | |
| SpamapS | I mean, you're talking about probably 0.1% latency at worst given that the real pain is running crypto operations. | 00:38 |
| morganfainberg | uwsgi is pretty impressive fwiw | 00:38 |
| SpamapS | Yeah it's got a good enough reputation that I think it may not even be worse. :) | 00:39 |
| morganfainberg | hehe | 00:39 |
| morganfainberg | looked at the code and was impressed | 00:39 |
| SpamapS | since php5 came out and claimed to be thread-safe, I've been fighting to try and get app servers to run in apache entirely. | 00:39 |
| morganfainberg | but it's not hard to be better at that type of code than I am (easily impressed) | 00:39 |
| SpamapS | It just doesn't work. :-P | 00:39 |
| mtreinish | morganfainberg: I'm still concerned about evoking ragnarok if we move to it | 00:39 |
| clarkb | anyways for python deos they should all be listed in global reqs so we have full control of the transitive install set | 00:40 |
| morganfainberg | mtreinish: that is always a concern | 00:40 |
| *** bharath has quit IRC | 00:40 | |
| flaper87 | FWIW, we're also looking forward to using uwsgi in Zaqar. We've different transports that could be easily served through uwsgi and it'd be cool to serve them using just 1 tool. | 00:40 |
| clarkb | if we start losing that control things can go wonkt | 00:40 |
| mordred | stackalytics runs in uwsgi in prod, fwiw | 00:40 |
| SpamapS | fcgi was so much faster and more resilient.. there was just no point in ever trying mod_php5 again. And python appears to be the same. | 00:40 |
| * flaper87 gets a link | 00:40 | |
| morganfainberg | clarkb: ok thats good enough for me. | 00:40 |
| clarkb | *wonky | 00:40 |
| mordred | and when I tried running a copy in mod_wsgi, it was way less gooder | 00:40 |
| morganfainberg | clarkb: i'll toss it in g-r and then flaper87 can dogpile and +1 the heck out of it | 00:40 |
| flaper87 | https://review.openstack.org/#/c/202432/ | 00:40 |
| mordred | SUPER unscientific | 00:40 |
| asselin_ | nibalizer, crinkle what's better: use heira for all vars or mix them? https://review.openstack.org/#/c/200330/3/manifests/sample_ci.pp | 00:40 |
| flaper87 | morganfainberg: go for it, I'll +1 | 00:41 |
| morganfainberg | mordred: another nice data point [even if unscientific] | 00:41 |
| *** mtanino has quit IRC | 00:41 | |
| clarkb | mordred gooder is a scientific unit right? | 00:41 |
| flaper87 | that patch is probably not in its best form but you get the idea | 00:41 |
| *** tsekiyama has quit IRC | 00:41 | |
| SpamapS | I think the problem with mod_{interpreter} is that you don't actually need a whole interpreter for some of the more expensive pieces of serving HTTP, but if you run mod_{interpreter}, python is there, wasting memory, while you're parsing HTTP. | 00:41 |
| morganfainberg | clarkb: last question: should I floor at the latest release of uwsgi (or latest major)? | 00:41 |
| morganfainberg | in g-r? | 00:41 |
| mtreinish | morganfainberg: I always get yelled at to set a min for a new req in that list | 00:41 |
| SpamapS | meanwhile with a uwsgi, the same is true, but you don't have all of the Apache<->interpreter coupling to get working well. | 00:41 |
| mtreinish | even if there is only 1 release | 00:41 |
| morganfainberg | mtreinish: sold I'll set a nice safe minimum | 00:42 |
| morganfainberg | SpamapS: thats what I totally dig about it | 00:42 |
| *** tsekiyam_ has quit IRC | 00:42 | |
| clarkb | morganfainberg if you know that older works set that else latest as we will test latest | 00:42 |
| morganfainberg | then again I always liked fast_cgi/scgi modes | 00:42 |
| morganfainberg | clarkb: i think we want 2.x+ | 00:42 |
| morganfainberg | clarkb: i'll do some quick research before placing the review up | 00:42 |
| *** yamamoto has quit IRC | 00:43 | |
| *** prad has quit IRC | 00:43 | |
| Shrews | mordred: love it. just a couple of minor tweaks, plz | 00:44 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/puppet-openstackci: WIP: Initial start at creating sample ci https://review.openstack.org/200330 | 00:46 |
| nibalizer | asselin_: you should make that into a parametrized class | 00:46 |
| nibalizer | which means you have implicit hiera lookups | 00:46 |
| nibalizer | so you dont need to do any explicitly in your openstackci::master class | 00:46 |
| asselin_ | nibalizer, how does that work? | 00:47 |
| asselin_ | the implicit heira lookups | 00:47 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Throw an exception on a server without an IP https://review.openstack.org/208105 | 00:47 |
| mordred | Shrews: done | 00:47 |
| nibalizer | https://docs.puppetlabs.com/hiera/1/puppet.html | 00:47 |
| mordred | asselin_: it's magic | 00:48 |
| nibalizer | mordred: so question, where is the script that generates an ansible inventory from 'puppet cert list' | 00:48 |
| asselin_ | thanks | 00:48 |
| *** tiswanso has joined #openstack-infra | 00:48 | |
| Shrews | mordred: +2, for reals this time | 00:48 |
| clarkb | nibalizee in the puppet ansible module iirc | 00:48 |
| nibalizer | according to infra-manual, infra-cloud will have The machines will all be added to a manual ansible inventory file adjacent | 00:49 |
| nibalizer | to the dynamic inventory that ansible currently uses to run puppet. | 00:49 |
| nibalizer | erp that pasted badly but my point is clear i hope | 00:49 |
| *** ivar-lazzaro has joined #openstack-infra | 00:49 | |
| mordred | nibalizer: nope. I cannot read the text if the whitespace is off | 00:49 |
| mordred | nibalizer: -1 | 00:49 |
| nibalizer | clarkb: aha! | 00:49 |
| nibalizer | thanks | 00:49 |
| asselin_ | nibalizer, very neat, so in this case, I will remove all explicit heira lookups | 00:51 |
| *** tiswanso has quit IRC | 00:51 | |
| nibalizer | asselin_: ya | 00:51 |
| *** tiswanso has joined #openstack-infra | 00:52 | |
| nibalizer | and take all variables as parameters to your class | 00:52 |
| mordred | nibalizer: I intend to replace it with an inventory that makes the inventory from nova, btw | 00:52 |
| clarkb | does hiera scope to class in those cases? | 00:52 |
| clarkb | how to deal with conflicts otherwise? | 00:52 |
| mordred | nibalizer: but that was part of the puppet-apply work in my brainhole | 00:52 |
| mordred | clarkb: you can make a hiera value that's like openstack_project.gerrit.sysadmins iirc, right nibalizer ? | 00:53 |
| mordred | s/value/key/ | 00:53 |
| nibalizer | clarkb: yea trigger warning: may make your eyes bleed https://docs.puppetlabs.com/hiera/1/puppet.html#automatic-parameter-lookup | 00:54 |
| nibalizer | mordred: yea I am plotting for puppet apply | 00:54 |
| mordred | nibalizer: woot | 00:54 |
| openstackgerrit | Morgan Fainberg proposed openstack/requirements: Add uwsgi to global-requirements https://review.openstack.org/209300 | 00:55 |
| *** puranamr has quit IRC | 00:55 | |
| *** stevemar has joined #openstack-infra | 00:55 | |
| morganfainberg | clarkb: ^ that is the latest release of uwsgi - rather than fighting bug fix "do we need this" just grabbed the latest knowing it meets the 2.x needs and is less likely to have some wonkyness | 00:56 |
| morganfainberg | SpamapS: ^ cc | 00:56 |
| *** geoffarnold has quit IRC | 00:57 | |
| *** armax has quit IRC | 00:58 | |
| *** zxiiro has joined #openstack-infra | 00:58 | |
| *** stevemar has quit IRC | 00:58 | |
| *** changbl has quit IRC | 00:59 | |
| *** salv-orlando has joined #openstack-infra | 00:59 | |
| mtreinish | clarkb: so I'm debugging an issue with run-docs.sh on tempest where a command in tox -edocs isnt being run. Did tox -l always only show job defs in the envlist? | 01:00 |
| mtreinish | because that's what the issue was | 01:00 |
| clarkb | mtreinish not sure | 01:01 |
| clarkb | but maybe? we do tox -evenv -- python setup.py build_sphinx with jenkins | 01:01 |
| mtreinish | clarkb: well looking at run-docs.sh it does a tox -l to figure out if there is a job named docs | 01:02 |
| mtreinish | if so it uses that | 01:02 |
| mtreinish | otherwise it does the venv thing | 01:02 |
| clarkb | wat | 01:02 |
| *** alexpilotti has quit IRC | 01:03 | |
| mtreinish | clarkb: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/run-docs.sh#n12 | 01:03 |
| clarkb | we still havent made docspart of the interface have we? anyways I have no knowledge of that | 01:03 |
| *** salv-orlando has quit IRC | 01:04 | |
| mtreinish | hmm, ok. We added a command before the sphinx build to generate a sample config, I guess I'll just fix the if check there to do a tox --showconfig and grep that | 01:04 |
| *** ashuk has quit IRC | 01:04 | |
| *** abitha has quit IRC | 01:05 | |
| *** armax has joined #openstack-infra | 01:05 | |
| openstackgerrit | li,chen proposed openstack-infra/project-config: A Devstack-plugin that installs a HDFS backend https://review.openstack.org/207050 | 01:06 |
| *** amotoki has joined #openstack-infra | 01:07 | |
| *** warpc__ has quit IRC | 01:08 | |
| *** alexsyip has quit IRC | 01:09 | |
| *** spzala has joined #openstack-infra | 01:11 | |
| asselin_ | nibalizer, clarkb any reason this patch needs to wait until puppet-os-client_config has content defined? https://review.openstack.org/#/c/204313/ | 01:12 |
| *** armax has quit IRC | 01:12 | |
| asselin_ | when we did the moduel-split, this was one of the steps | 01:13 |
| asselin_ | boiler-plate | 01:13 |
| *** skylerberg has quit IRC | 01:15 | |
| *** tiswanso has quit IRC | 01:19 | |
| *** tiswanso has joined #openstack-infra | 01:19 | |
| *** david-lyle has joined #openstack-infra | 01:19 | |
| *** jyuso1 has quit IRC | 01:20 | |
| clarkb | asselin_ I was just trying to figure out what we were doing there first. I have questions about general puppetisms I was hoping to get cleared up | 01:21 |
| *** ashuk has joined #openstack-infra | 01:21 | |
| Nakato | Interesting, I've got a tox job that specifies "basepython = python2.7", but it's pulling in system libraries from python3.4. | 01:21 |
| clarkb | in particular why go yaml (hiera) -> puppet dsl-> yaml | 01:21 |
| clarkb | Nakato are you on Arch? | 01:21 |
| asselin_ | clarkb, understood, but that's more for implementation of the puppet module. Are you suggesting perhaps we don't need that module at all? | 01:22 |
| *** esmute_ has quit IRC | 01:22 | |
| clarkb | yes | 01:22 |
| clarkb | we can just drop a yaml file trivially | 01:22 |
| asselin_ | true | 01:23 |
| asselin_ | I think the reason it was made into a module was to hold the cloud credentials | 01:23 |
| *** erlon has quit IRC | 01:23 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Fix search for docs tox job in run-docs.sh https://review.openstack.org/209305 | 01:23 |
| mtreinish | clarkb: ^^^ | 01:23 |
| asselin_ | which would be stored in heira | 01:24 |
| * mtreinish points out he sucks at regex | 01:24 | |
| clarkb | asselin_ you can do that witg a file resource too | 01:24 |
| asselin_ | clarkb, so how do you hide the secrets? | 01:26 |
| clarkb | mtreinish I think its always [testenv:docs] | 01:26 |
| clarkb | asselin_ put them in hiera | 01:26 |
| asselin_ | and how do they get into the yaml file? | 01:26 |
| clarkb | then either use a template or verbatim file content | 01:26 |
| asselin_ | ok, so the template gets the secrets its needs from heira | 01:27 |
| Nakato | clarkb: yea | 01:27 |
| clarkb | I just worry we have way overengineered the thing if yaml -> puppet -> yaml is how it works | 01:28 |
| mtreinish | clarkb: well in the normal case, but I don't think anything says it has to be | 01:28 |
| *** changbl has joined #openstack-infra | 01:28 | |
| clarkb | Nakato python is broken on arch and their `python` is python3 | 01:28 |
| clarkb | mtreinish thats built into tox iirc | 01:28 |
| clarkb | I thought testenv is specific | 01:29 |
| asselin_ | clarkb, seems to be discouraged: https://docs.puppetlabs.com/hiera/1/puppet.html#using-the-lookup-functions-from-templates | 01:29 |
| mtreinish | clarkb: testenv is special, but I thought you could build a job which doesn't inherit from it | 01:29 |
| mtreinish | you just have to define all the required vars in that definition | 01:29 |
| *** ashuk has quit IRC | 01:30 | |
| mtreinish | but I could be wrong | 01:30 |
| *** olaph has joined #openstack-infra | 01:30 | |
| Nakato | clarkb: Won't that mean it'll break as systems switch python3 to being the default python, or has something been done terribly wrong in Arch? | 01:30 |
| *** dank_ is now known as dan | 01:31 | |
| mtreinish | clarkb: nope you're right, it'll just be treated as an env definition unless you set testenv | 01:31 |
| mtreinish | that's what lets you call it | 01:32 |
| mtreinish | I'll respin the patch | 01:32 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/puppet-openstackci: WIP: Initial start at creating sample ci https://review.openstack.org/200330 | 01:32 |
| *** gildub has quit IRC | 01:32 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Fix search for docs tox job in run-docs.sh https://review.openstack.org/209305 | 01:33 |
| *** spzala has quit IRC | 01:33 | |
| clarkb | Nakato arch is horribly wrong. `python` is always python2 and python3 is python3 according to python | 01:34 |
| *** tiswanso has quit IRC | 01:34 | |
| *** tqtran-afk has quit IRC | 01:34 | |
| Nakato | clarkb: thanks. I'll just stop looking into that then. | 01:34 |
| *** tiswanso has joined #openstack-infra | 01:34 | |
| clarkb | so default python 3 just means you have `python3` and no `python` | 01:34 |
| mtreinish | clarkb: ugh, yeah don't get me started on arch's default. It drives me mad | 01:35 |
| mtreinish | it just makes things more difficult for no reason | 01:35 |
| clarkb | hey Nakato started it | 01:35 |
| openstackgerrit | Merged openstack-infra/subunit2sql: Adds method to get metadata by key/value pair https://review.openstack.org/208614 | 01:36 |
| clarkb | mtreinish tumbleweed | 01:37 |
| mtreinish | clarkb: hehe, sorry I actually was looking at something else :) | 01:37 |
| * mtreinish shakes fist at Nakato for bringing it up | 01:37 | |
| *** crc32 has joined #openstack-infra | 01:38 | |
| *** yamamoto has joined #openstack-infra | 01:41 | |
| *** yamamoto has quit IRC | 01:42 | |
| *** aukhan has quit IRC | 01:42 | |
| *** Somay has joined #openstack-infra | 01:42 | |
| *** MarkAtwood has quit IRC | 01:42 | |
| *** baoli has joined #openstack-infra | 01:44 | |
| openstackgerrit | Steve Kowalik proposed openstack-infra/project-config: No longer ignore exit status of zanata-cli push https://review.openstack.org/209309 | 01:45 |
| *** sergmelikyan has joined #openstack-infra | 01:45 | |
| *** gildub has joined #openstack-infra | 01:48 | |
| *** unicell1 has quit IRC | 01:48 | |
| Nakato | Just as I wanted a bit curious I found the pep. https://www.python.org/dev/peps/pep-0394/ :) | 01:50 |
| *** julim has joined #openstack-infra | 01:51 | |
| *** tjones has quit IRC | 01:52 | |
| openstackgerrit | Merged openstack-infra/subunit2sql: Edits to_dict() to return correct dict https://review.openstack.org/208756 | 01:53 |
| StevenK | Ubuntu at least, is also trying to make /usr/bin/python be Python 3, but they aren't there yet | 01:54 |
| *** tjones has joined #openstack-infra | 01:54 | |
| *** ddieterly has joined #openstack-infra | 02:01 | |
| *** ub has joined #openstack-infra | 02:03 | |
| *** sdake_ has quit IRC | 02:04 | |
| *** ub2 has quit IRC | 02:04 | |
| *** dims has quit IRC | 02:04 | |
| *** sdake has joined #openstack-infra | 02:04 | |
| *** zz_dimtruck is now known as dimtruck | 02:04 | |
| *** otter768 has joined #openstack-infra | 02:04 | |
| *** salv-orlando has joined #openstack-infra | 02:05 | |
| *** achanda has joined #openstack-infra | 02:07 | |
| *** olaph has quit IRC | 02:07 | |
| *** olaph has joined #openstack-infra | 02:08 | |
| *** otter768 has quit IRC | 02:09 | |
| *** baoli has quit IRC | 02:09 | |
| *** zul has quit IRC | 02:09 | |
| Nakato | StevenK: That's a good note. | 02:11 |
| *** sabeen has quit IRC | 02:11 | |
| *** salv-orlando has quit IRC | 02:12 | |
| *** baoli has joined #openstack-infra | 02:13 | |
| *** achanda has quit IRC | 02:14 | |
| *** Somay has quit IRC | 02:15 | |
| *** puranamr has joined #openstack-infra | 02:17 | |
| *** puranamr has quit IRC | 02:21 | |
| *** annegentle has quit IRC | 02:22 | |
| *** tlian2 has quit IRC | 02:22 | |
| *** pal has joined #openstack-infra | 02:22 | |
| *** annegentle has joined #openstack-infra | 02:23 | |
| *** Somay has joined #openstack-infra | 02:23 | |
| *** sabeen1 has joined #openstack-infra | 02:26 | |
| *** Somay has quit IRC | 02:27 | |
| *** achanda has joined #openstack-infra | 02:28 | |
| *** baoli has quit IRC | 02:29 | |
| *** yamamoto has joined #openstack-infra | 02:31 | |
| *** gyee has quit IRC | 02:35 | |
| *** pahuang has quit IRC | 02:36 | |
| *** rfolco has quit IRC | 02:40 | |
| *** pal has quit IRC | 02:41 | |
| openstackgerrit | Merged openstack/diskimage-builder: debian: properly configure interfaces https://review.openstack.org/205036 | 02:41 |
| *** sabeen1 has quit IRC | 02:43 | |
| *** aukhan has joined #openstack-infra | 02:46 | |
| *** pahuang has joined #openstack-infra | 02:49 | |
| *** sabeen has joined #openstack-infra | 02:51 | |
| *** ajmiller has joined #openstack-infra | 02:53 | |
| *** annegentle has quit IRC | 02:54 | |
| *** achanda_ has joined #openstack-infra | 02:54 | |
| *** achanda has quit IRC | 02:55 | |
| *** achanda has joined #openstack-infra | 03:03 | |
| *** achanda_ has quit IRC | 03:03 | |
| EmilienM | anteaya: hey, I don't understand https://review.openstack.org/#/c/206693/ | 03:04 |
| *** btully has quit IRC | 03:04 | |
| EmilienM | anteaya: I just want to run the publisher on a non voting job | 03:04 |
| anteaya | EmilienM: from the commit message "Because this job is non-voting and experimental" | 03:05 |
| EmilienM | yeah | 03:05 |
| *** baoli has joined #openstack-infra | 03:05 | |
| anteaya | the job isnt' in the experimental pipeline | 03:05 |
| EmilienM | the upgrade jobs are non voting from the beginning | 03:05 |
| *** shashank_hegde has joined #openstack-infra | 03:05 | |
| anteaya | it is in the check pipeline | 03:05 |
| EmilienM | no and we don't want | 03:05 |
| EmilienM | yeah | 03:05 |
| EmilienM | it's like this for our puppet jobs | 03:05 |
| anteaya | you told crinkle it is in the experimental pipeline | 03:05 |
| anteaya | and it isn't | 03:05 |
| EmilienM | nope, it's an experimental job | 03:06 |
| EmilienM | that means: | 03:06 |
| EmilienM | non voting | 03:06 |
| anteaya | it is in the check pipeline | 03:06 |
| anteaya | no | 03:06 |
| anteaya | an experimental job means it is in the experimental pipeline | 03:06 |
| EmilienM | it's a vocabulary issue I guess (for me) | 03:06 |
| EmilienM | ok sorry | 03:06 |
| anteaya | yes | 03:06 |
| EmilienM | but anyway, the patch is good I guess | 03:06 |
| anteaya | experimental job has a meaning | 03:06 |
| anteaya | the patch is fine | 03:06 |
| EmilienM | ok | 03:06 |
| anteaya | the commit message about job pipeline is misleading | 03:07 |
| EmilienM | and Ill loose a +2 and wait again :( | 03:07 |
| anteaya | to those who know that experimental job means it is in the experimental pipeline | 03:07 |
| EmilienM | I'm just stuck to make progress on this one and I really hope have it quickly | 03:07 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: jenkins: implement copy-puppet-logs publisher https://review.openstack.org/206693 | 03:07 |
| anteaya | if crinkle wan't to ack that she knows this is a non-voting check job I'm fine with +A | 03:08 |
| EmilienM | done | 03:08 |
| EmilienM | now I have to wait again :) | 03:08 |
| anteaya | okay | 03:08 |
| EmilienM | anteaya: I'm waiting for this patch because we *need* logs | 03:08 |
| EmilienM | and this is a first step only | 03:08 |
| EmilienM | I'll send a patch later this week to enable the publisher for all our puppet jobs | 03:08 |
| EmilienM | ianw: if you around, can you +2 again please ? | 03:08 |
| anteaya | please remove # Experimental job now line 1404 from zuul/layout.yaml please? http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n1404 | 03:09 |
| EmilienM | anteaya: this is not related to this patch... | 03:09 |
| anteaya | since this is a non-voting check job | 03:09 |
| anteaya | I know but the comment is incorrect | 03:09 |
| anteaya | so please fix with a different patch | 03:09 |
| *** fitoduarte has quit IRC | 03:11 | |
| anteaya | did you want to address ianw's shell wrapper comment from patchset 14 while you are at it? | 03:11 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/layout: drop 'experimental' from Puppet comments https://review.openstack.org/209328 | 03:12 |
| *** fitoduarte has joined #openstack-infra | 03:12 | |
| EmilienM | anteaya: done | 03:12 |
| EmilienM | anteaya: no | 03:12 |
| anteaya | thank you | 03:12 |
| EmilienM | anteaya: later | 03:12 |
| EmilienM | anteaya: iteration is cool | 03:12 |
| EmilienM | anteaya: I want to test it first and of course I'll improve the script and such | 03:12 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/layout: drop 'experimental' from Puppet comments https://review.openstack.org/209328 | 03:13 |
| EmilienM | anteaya: all fine for you ? | 03:14 |
| anteaya | looking | 03:14 |
| anteaya | very slow wifi | 03:14 |
| EmilienM | well, I just updated the commit message | 03:14 |
| *** dimtruck is now known as zz_dimtruck | 03:15 | |
| *** dmorita_ has quit IRC | 03:16 | |
| anteaya | reading | 03:16 |
| anteaya | reviewed | 03:19 |
| *** hdd has joined #openstack-infra | 03:19 | |
| anteaya | thanks EmilienM | 03:19 |
| EmilienM | anteaya: thanks! I go to bed now | 03:19 |
| anteaya | yup, enjoy sleep | 03:20 |
| *** sergmelikyan has quit IRC | 03:21 | |
| *** doug-fish has joined #openstack-infra | 03:24 | |
| *** rlandy has joined #openstack-infra | 03:27 | |
| *** hdd has quit IRC | 03:28 | |
| *** annegentle has joined #openstack-infra | 03:29 | |
| crinkle | anteaya: EmilienM I am fine with the job being a non-voting check job | 03:29 |
| anteaya | crinkle: thanks | 03:29 |
| anteaya | EmilienM respun if you want to re-review | 03:29 |
| crinkle | anteaya: will do | 03:30 |
| *** pahuang has quit IRC | 03:30 | |
| anteaya | it is just taht "experimental job" means something very specific | 03:30 |
| anteaya | and I wanted to ensure we were all clear | 03:30 |
| anteaya | thanks | 03:30 |
| *** MarkAtwood has joined #openstack-infra | 03:31 | |
| *** pal has joined #openstack-infra | 03:36 | |
| *** MarkAtwood has quit IRC | 03:36 | |
| *** rlandy has quit IRC | 03:36 | |
| *** tiswanso has quit IRC | 03:37 | |
| *** Nakato has quit IRC | 03:38 | |
| *** achanda has quit IRC | 03:38 | |
| *** annegentle has quit IRC | 03:38 | |
| *** ivar-lazzaro has quit IRC | 03:40 | |
| *** Nakato has joined #openstack-infra | 03:40 | |
| *** pahuang has joined #openstack-infra | 03:46 | |
| lifeless | morganfainberg: its a it of both | 03:47 |
| morganfainberg | lifeless: ack | 03:47 |
| lifeless | morganfainberg: but why? doesn't offer anything over mod_wsgi does it? | 03:47 |
| morganfainberg | it does in a number of ways. we can avoid using apache-directly managed processes, and handles venvs (not really a devstack concern) | 03:48 |
| *** baoli has quit IRC | 03:49 | |
| *** baoli has joined #openstack-infra | 03:49 | |
| morganfainberg | it makes apache more of the surface HTTPD server and lets uwsgi handle the service management. for devstack this is a win since we move away from lumping everything into apache and requiring heavy/slow/painful restarts which has resulted in us consistently hitting some specific apache errors. it is also better than directly writing / managing | 03:50 |
| morganfainberg | wsgi-http-socket code in our apps that we wouldn't want in production | 03:50 |
| lifeless | morganfainberg: hmmm, so this doesn't match my prior experience w/mod_wsgi | 03:52 |
| lifeless | morganfainberg: its a shame you didn't chat to graeme at pyconau - he's the mod_wsgi author | 03:52 |
| morganfainberg | lifeless: i went to...and he was gone | 03:52 |
| morganfainberg | got to him too late | 03:52 |
| morganfainberg | the issue we tend to hit is an apache specific error | 03:52 |
| morganfainberg | the reason we do a stop && sleep && start | 03:53 |
| lifeless | why ? | 03:53 |
| lifeless | I mean, apache2ctl graceful is the thing to do | 03:53 |
| morganfainberg | when apache is managing the unerlying processes (in any form) it gets slow/cranky about restarts. it's been a lingering issue for ... well I hit this back in 2005 and it's just been around | 03:53 |
| *** baoli has quit IRC | 03:54 | |
| *** coolsvap|away is now known as coolsvap | 03:54 | |
| morganfainberg | graceful likewise hits a very slow reload time. it's just how apache manages connections to the open processes. in the case of mod_proxy and similar it is less worried about dropping things on the floor | 03:54 |
| morganfainberg | in production, yes graceful. | 03:54 |
| SpamapS | this is interesting... | 03:54 |
| SpamapS | apache _is_ the open processes.. | 03:54 |
| morganfainberg | SpamapS: yes. | 03:55 |
| lifeless | so look, I don't want to get in the way. uwsgi - sure; global-requirements, plus extras where you want to surface it optionally. | 03:55 |
| lifeless | plus we need to tweak the sync rules around use of extras | 03:55 |
| *** stevemar has joined #openstack-infra | 03:55 | |
| SpamapS | so I wonder if it's code that is ignoring/blocking signal reception, or worse, code that is blocking the signal handler itself. | 03:55 |
| morganfainberg | lifeless: oh i'm happy to work more with mod_wsgi. this is really about opening the conversation | 03:55 |
| morganfainberg | lifeless: thats why i asked instead of just doing :) | 03:55 |
| lifeless | I just would be focusing on mod_wsgi if I was pushing on this space | 03:55 |
| morganfainberg | lifeless: i want to find the best direction. | 03:55 |
| lifeless | based on my prior production experience | 03:56 |
| SpamapS | morganfainberg: has anybody gone as far as installing symbols/code and attaching with gdb? | 03:56 |
| lifeless | morganfainberg: I'll leave you in SpamapS ... tender.... hands | 03:56 |
| morganfainberg | SpamapS: I've done it in the past it's a mess back when i did it. | 03:56 |
| morganfainberg | SpamapS: have not done it recently. | 03:56 |
| SpamapS | morganfainberg: do you have a sense for where it was stuck? stuff in libapr or things in libpython? | 03:56 |
| morganfainberg | SpamapS: mostly in apache handoff iirc | 03:57 |
| morganfainberg | where apache was waiting for connections to close down that were ebing handed off via bucketbrigade (perhaps) or whichever similar tech was used | 03:57 |
| morganfainberg | and cranking the timeouts down low enough caused other issues | 03:57 |
| morganfainberg | mod_wsgi is fairly opaque beyond the basics as well from the apache standpoint | 03:58 |
| *** stevemar has quit IRC | 03:58 | |
| morganfainberg | it's fairly rich if you dive in from the other side. | 03:58 |
| * morganfainberg has always liked split concerns "manage process", "manage http connections and dispatching" | 03:59 | |
| morganfainberg | but thats a design *i* like, it may not be the most correct for our gate | 03:59 |
| *** ddieterly has quit IRC | 03:59 | |
| SpamapS | morganfainberg: I've not ever seen apache get tripped up on that. Especially not with 'restart', because it basically just forcibly closes everything and then the child dies whether the socket is actually closed or not. | 04:01 |
| morganfainberg | SpamapS: it's dependant on how busy the system is. | 04:01 |
| SpamapS | morganfainberg: with graceful I definitely have seen that, since thats what graceful is supposed to do. :) | 04:01 |
| morganfainberg | we hit a "cannot restart apache" a lot of times before due to this issue. | 04:01 |
| morganfainberg | and back in $webhosting_platform_days$ we came to realize it was just a known quantity | 04:01 |
| morganfainberg | we hit = devstack/gate | 04:02 |
| SpamapS | oh and this is prefork yes? | 04:02 |
| openstackgerrit | Merged openstack-infra/project-config: jenkins: implement copy-puppet-logs publisher https://review.openstack.org/206693 | 04:02 |
| morganfainberg | i don't think we use prefork anymore anywhere | 04:02 |
| morganfainberg | i think we're worker but not event | 04:02 |
| morganfainberg | (default iirc is worker | 04:02 |
| morganfainberg | for apache these days. | 04:02 |
| morganfainberg | but i've seen this in both cases. have not checked event, event might be way less prone to it | 04:03 |
| SpamapS | ah yeah mod_wsgi and python are thread safe enough for worker. | 04:03 |
| *** geoffarnold has joined #openstack-infra | 04:04 | |
| *** tjones has quit IRC | 04:05 | |
| *** gnuoy has quit IRC | 04:05 | |
| *** jamespage has quit IRC | 04:05 | |
| *** otter768 has joined #openstack-infra | 04:05 | |
| SpamapS | morganfainberg: still, makes me wonder about GIL things and whether this is a problem in prefork. | 04:05 |
| *** jamespage has joined #openstack-infra | 04:06 | |
| *** gnuoy has joined #openstack-infra | 04:06 | |
| morganfainberg | SpamapS: in 2005 with mod_* things running, we saw similar | 04:06 |
| *** soren has joined #openstack-infra | 04:06 | |
| morganfainberg | but prefork was core of the prefork process closing (similar to how the wsgi processes are forked off) | 04:06 |
| *** geoffarnold has quit IRC | 04:06 | |
| morganfainberg | it looks (afaict) to be a very related edge case. but if we hit it 1 in 1000, it is more than a blip in infra | 04:07 |
| SpamapS | morganfainberg: I'm certain there's an explanation. Solution.... not so much ;) | 04:08 |
| morganfainberg | in production, the main ask (and i'll be documenting uwsgi for this in either case) is venv support that is much better than mod_wsgi | 04:08 |
| morganfainberg | if it ends up in gate or not, i'm less concerned. but if we can reduce special case code for restarting apache and associated things too, might be worthwhile | 04:09 |
| *** geoffarnold has joined #openstack-infra | 04:09 | |
| *** jasondotstar|afk has quit IRC | 04:09 | |
| *** otter768 has quit IRC | 04:10 | |
| *** salv-orlando has joined #openstack-infra | 04:10 | |
| morganfainberg | not that mod_wsgi misses the mark in any particular way. just alternate options is good. | 04:10 |
| *** harlowja_at_home has joined #openstack-infra | 04:11 | |
| morganfainberg | lifeless: in either case i expect to make uwsgi optional (extras) for keystone for sure. beyond that is more open conversation :) | 04:12 |
| *** deepakcs has joined #openstack-infra | 04:13 | |
| *** maishsk has joined #openstack-infra | 04:14 | |
| sdake | when is the next project rename date? | 04:15 |
| *** MarkAtwood has joined #openstack-infra | 04:16 | |
| *** geoffarnold has quit IRC | 04:16 | |
| SpamapS | morganfainberg: so looking at how it handles HUP, it's still slave to python and mod_wsgi stopping, which may be subject to GIL problems and likely is too conservative | 04:16 |
| *** pal has quit IRC | 04:16 | |
| morganfainberg | SpamapS: yeah | 04:17 |
| sdake | hey spamaps kolla big tent proposal was w+1 today :) | 04:17 |
| SpamapS | sdake: fantastic.. we need to talk a bit about Magnum. I'm afraid it's going down a dark road tying so tightly to Heat. :-P | 04:18 |
| sdake | what else should it tie with? | 04:18 |
| SpamapS | the API's of OpenStack. :) | 04:18 |
| *** amotoki has quit IRC | 04:18 | |
| sdake | isn't that what heat does well ? :) | 04:18 |
| SpamapS | no | 04:18 |
| SpamapS | IMO it does it terribly, which is why I proposed convergence. Still quite a bit away from doing that well. :) | 04:19 |
| sdake | we have our midcycle tomorrow and the next day | 04:19 |
| sdake | your welcome to add yoru agenda item to it if you want to bring it up - we hve 2 or 3 unallocated sessions | 04:19 |
| SpamapS | Yeah I might. :) | 04:19 |
| *** monester has joined #openstack-infra | 04:19 | |
| harlowja_at_home | SpamapS, do u have any idea how much it uses heat, and really does it have to? | 04:20 |
| sdake | it does reallly ahve to use heat or we have to reinevent a bunch of waht heat does | 04:20 |
| sdake | (which we dont want to do) | 04:20 |
| sdake | i dont quit eunderstand the concern unless its a scaling problem | 04:21 |
| harlowja_at_home | my read is that's what SpamapS is saying | 04:21 |
| harlowja_at_home | (without convergence stuff it is a scaling problem?) | 04:21 |
| sdake | ya thats how I read it as well | 04:21 |
| morganfainberg | harlowja_at_home: i see you are at home :P [there is nothing else I really had to say, but figured it would entertain me to point that out] | 04:21 |
| *** salv-orlando has quit IRC | 04:22 | |
| SpamapS | Scaling, resilience, flexibility.. | 04:22 |
| sdake | we are really locked into heat | 04:22 |
| harlowja_at_home | morganfainberg, highly likely ;) | 04:22 |
| SpamapS | Solve the first 2 and you still have a really rigid language. | 04:22 |
| SpamapS | Magnum has _python_ | 04:22 |
| SpamapS | use _python_ | 04:22 |
| sdake | doing all this stuff without heat would take 6+ months of rework | 04:22 |
| harlowja_at_home | :-/ | 04:22 |
| harlowja_at_home | i like da python | 04:22 |
| SpamapS | sdake: which part do you need Heat for? | 04:22 |
| sdake | I can't support that work when we have major problems to solve atm with magnum implementation | 04:22 |
| sdake | we use it to setup neutron networks vm booting etc | 04:22 |
| sdake | ironic booting | 04:23 |
| sdake | cinder volumes | 04:23 |
| sdake | pretty much everywhere | 04:23 |
| sdake | what we do is make a coe (container orchestration engine) heat template | 04:23 |
| harlowja_at_home | morganfainberg, are u at home?? | 04:23 |
| sdake | and that does the heavy lifting | 04:23 |
| morganfainberg | harlowja_at_home: I am in a coffee shope in brisbane throwing things and flaper87 | 04:23 |
| harlowja_at_home | morganfainberg, well good cause i'm in your home | 04:24 |
| harlowja_at_home | hahahaha | 04:24 |
| harlowja_at_home | j/k | 04:24 |
| morganfainberg | s/and/at | 04:24 |
| harlowja_at_home | never said which home i'm at :-P | 04:24 |
| morganfainberg | harlowja_at_home: eh, while you're there, could you put some oil on the bike chains? I want to go for a ride when I get home and feel lazy. | 04:24 |
| *** fawadkhaliq has joined #openstack-infra | 04:25 | |
| *** ajmiller has quit IRC | 04:25 | |
| harlowja_at_home | sure, i also tweaked your brakes, and made sure your tire is correctly inflated | 04:25 |
| flaper87 | morganfainberg: if those things are wine, don't stop! | 04:25 |
| flaper87 | :D | 04:25 |
| *** ajmiller has joined #openstack-infra | 04:25 | |
| openstackgerrit | Merged openstack-infra/project-config: zuul/layout: drop 'experimental' from Puppet comments https://review.openstack.org/209328 | 04:25 |
| morganfainberg | harlowja_at_home: sweet. | 04:25 |
| * harlowja_at_home also mountain bikes :-P | 04:25 | |
| harlowja_at_home | if u road bike, i'm sorry, can't be BFFs with u | 04:26 |
| morganfainberg | harlowja_at_home: triathlon | 04:26 |
| harlowja_at_home | ok, guess thats ok, idk though | 04:27 |
| morganfainberg | but mostly road. sorry. | 04:27 |
| morganfainberg | because tri isn't MTB | 04:27 |
| harlowja_at_home | ya, tri is road, swim, run around or something right? | 04:27 |
| SpamapS | sdake: perhaps my experience trying to boot whole datacenters with Heat has colored my opinion of it a bit. :-P | 04:27 |
| morganfainberg | harlowja_at_home: swim, ride, run | 04:27 |
| harlowja_at_home | ya, thats it | 04:27 |
| harlowja_at_home | morganfainberg, so obvious question, sram or shimano, haha | 04:28 |
| harlowja_at_home | although i think road bikes are more dominated by sram, can't remember | 04:28 |
| morganfainberg | harlowja_at_home: SRAM Red 22 for road bike, and Shimano Di2 for tri bike | 04:29 |
| *** stevemar has joined #openstack-infra | 04:29 | |
| harlowja_at_home | cool | 04:29 |
| harlowja_at_home | red 22, is that the crazy expensive carbon all over stuff | 04:29 |
| morganfainberg | harlowja_at_home: it's more shimano for road, but i like the really mechanical feel of SRAM. but electronic on a bike is sliiiiiick | 04:29 |
| *** zeih has joined #openstack-infra | 04:29 | |
| morganfainberg | harlowja_at_home: red 22 is carbon-y but not crazy pricy... like $1200 for the whole groupset new | 04:29 |
| morganfainberg | the "force 22" [next step down] is like $800-900ish | 04:30 |
| harlowja_at_home | kk, that stuff gets crazy expensive, i like metal still, lol | 04:30 |
| *** larainema has quit IRC | 04:30 | |
| harlowja_at_home | take my metal from my dead hands | 04:30 |
| morganfainberg | i spend more on my frame and wheels than on my groupset | 04:30 |
| *** larainema has joined #openstack-infra | 04:30 | |
| morganfainberg | lots and lots and lots of carbon fiber | 04:30 |
| morganfainberg | [hey i have a CF laptop!] | 04:30 |
| harlowja_at_home | :-P | 04:30 |
| morganfainberg | most of the groupset is metal, i mean... CF doesn't hold up for the cassette, chain, chainrings, spider, pedal body, etc | 04:31 |
| harlowja_at_home | ya, especially on mountain bikes imho, lol | 04:31 |
| morganfainberg | CF MTB is totally doable though | 04:32 |
| morganfainberg | some slick downhill MTB frames | 04:32 |
| clarkb | just dont forget your lights light the ninja bike currently riding double wide in bike lane next to me | 04:32 |
| harlowja_at_home | ya, frames, but idk, with the falls i take, i feel like i'd poke through the frame, or shatter it, lol | 04:32 |
| morganfainberg | but honestly, CX looks more interesting than MTB | 04:32 |
| clarkb | *lights like | 04:32 |
| *** fifieldt has joined #openstack-infra | 04:32 | |
| morganfainberg | to me at least. | 04:32 |
| harlowja_at_home | https://i.imgur.com/inV6ZQ0.jpg (my super bike @ tahoe), lol | 04:32 |
| morganfainberg | clarkb: there are always those who have deathwishes. | 04:32 |
| morganfainberg | harlowja_at_home: Nice! | 04:33 |
| harlowja_at_home | :-P | 04:33 |
| morganfainberg | Giant makes good bikes, ESP MTB | 04:33 |
| harlowja_at_home | ya, so far its been good to me (the brand in general) | 04:33 |
| * morganfainberg rides cannondale and specialized | 04:33 | |
| harlowja_at_home | cools, cannondale and mtb idk about they seem to do weird stuff in there higher end MTB | 04:34 |
| harlowja_at_home | like make there own forks and crap | 04:34 |
| harlowja_at_home | always seemed odd that they did that but idk | 04:34 |
| *** zeih has quit IRC | 04:34 | |
| harlowja_at_home | p.vitalmtb.com/photos/stories/2014/03/26/max_jekyll_1024x1024_153180.jpg (ie that stuff) | 04:34 |
| harlowja_at_home | ^ also looks, umm, ya, not my style, lol | 04:35 |
| morganfainberg | again road bike, so... | 04:35 |
| harlowja_at_home | :-P | 04:35 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Turn anvil venv trusty into a voting/gating job https://review.openstack.org/209263 | 04:35 |
| morganfainberg | i'd not do the weird single fork thing canondale does on an MTB | 04:35 |
| morganfainberg | i'd go sworks or giant | 04:36 |
| harlowja_at_home | yup | 04:36 |
| *** ajmiller has quit IRC | 04:36 | |
| harlowja_at_home | making the custom fork think makes it harder to do maintaince, change parts ... all the things | 04:36 |
| harlowja_at_home | *custom fork thing | 04:36 |
| harlowja_at_home | so morganfainberg u want me to start cross-project spec :-P | 04:37 |
| harlowja_at_home | another super-fun one, lol | 04:37 |
| morganfainberg | harlowja_at_home: sounds good. | 04:37 |
| harlowja_at_home | k | 04:37 |
| morganfainberg | harlowja_at_home: i agree with flaper87, i like the plan | 04:37 |
| morganfainberg | and opinionated is a good starting place imo | 04:37 |
| harlowja_at_home | fair enough | 04:37 |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config: midonet: Skip jobs where appropriate https://review.openstack.org/209338 | 04:38 |
| *** julim has quit IRC | 04:38 | |
| *** maishsk has quit IRC | 04:39 | |
| harlowja_at_home | is redhat ready to package zookeeper ;) | 04:39 |
| harlowja_at_home | (dun dun dun) | 04:40 |
| *** fedexo has joined #openstack-infra | 04:40 | |
| *** pal has joined #openstack-infra | 04:40 | |
| openstackgerrit | Catherine Diep proposed openstack-infra/project-config: Move Refstack project from stackforge to openstack-infra. https://review.openstack.org/205777 | 04:40 |
| flaper87 | harlowja_at_home: LOL | 04:40 |
| harlowja_at_home | https://apps.fedoraproject.org/packages/zookeeper/overview/ i guess does exist, just don't think i've ever seen it in RHEL | 04:40 |
| flaper87 | I see what you did there | 04:40 |
| harlowja_at_home | ;) | 04:41 |
| flaper87 | harlowja_at_home: I don't that's really a problem. I want what's best for OpenStack | 04:41 |
| * harlowja_at_home i'm just an innocent child | 04:41 | |
| harlowja_at_home | lol | 04:41 |
| harlowja_at_home | flaper87, agreed | 04:41 |
| *** achanda has joined #openstack-infra | 04:41 | |
| *** maishsk has joined #openstack-infra | 04:43 | |
| openstackgerrit | Merged openstack-infra/project-config: Add separate requirements validation job https://review.openstack.org/205291 | 04:45 |
| *** yolanda has joined #openstack-infra | 04:45 | |
| *** michchap has joined #openstack-infra | 04:47 | |
| *** garyk has joined #openstack-infra | 04:48 | |
| *** rkukura has quit IRC | 04:49 | |
| clarkb | fifieldt: PHP Parse error: syntax error, unexpected T_STRING in /srv/vhosts/groups.openstack.org/slot0/profiles/groups/modules/commons/commons_events/commons_events.views_default.inc on line 300 | 04:49 |
| openstackgerrit | Merged openstack-infra/project-config: Add python3 non-voting jobs to python-designateclient https://review.openstack.org/208935 | 04:49 |
| clarkb | fifieldt: that appears to be the 500 error from groups.openstack.org based on the apache error logs | 04:50 |
| harlowja_at_home | morganfainberg, i've also seen quite a few of https://pypi.python.org/pypi/distconfig (and similar on pypi); probably useful for keystone policy sharing stuffs that i've heard about | 04:51 |
| * morganfainberg nods | 04:52 | |
| morganfainberg | i expect to piggyback on the DLM choice for other things in keystone | 04:52 |
| harlowja_at_home | well then i guess i better not recommend file locks on a netapp filer somewhere, lol | 04:53 |
| flaper87 | harlowja_at_home: don't even.. I mean... arrghh | 04:54 |
| harlowja_at_home | ha | 04:54 |
| *** ildikov has quit IRC | 04:54 | |
| clarkb | fifieldt: I am being called by babies to do other thing sbut hopefully that gives you and others enough to work on | 04:55 |
| harlowja_at_home | i'm actually surprised that neutron folks aren't using this for port mappings and all that, I thought most SDN's required something like it to be able to quickly do similar configuration management stuff | 04:55 |
| harlowja_at_home | to be able to reconfigure all the things when needed | 04:56 |
| *** maishsk has quit IRC | 04:56 | |
| harlowja_at_home | morganfainberg, https://github.com/Nextdoor/ndserviceregistry#nextdoor-service-registry (also probably in your idea-list, or something like it) | 04:57 |
| morganfainberg | yeah similar | 04:57 |
| harlowja_at_home | * https://engblog.nextdoor.com/2013/02/19/nextdoor-service-registry/ (just found that) | 04:57 |
| *** fawadkhaliq has quit IRC | 04:57 | |
| *** shashank_hegde has quit IRC | 04:58 | |
| *** fawadkhaliq has joined #openstack-infra | 04:58 | |
| openstackgerrit | Ramy Asselin proposed openstack-infra/puppet-openstackci: WIP: Initial start at creating sample ci https://review.openstack.org/200330 | 04:58 |
| *** rkukura has joined #openstack-infra | 04:58 | |
| *** pcaruana has quit IRC | 04:59 | |
| openstackgerrit | Merged openstack/diskimage-builder: Correct URL in ironic-agent README https://review.openstack.org/209017 | 05:00 |
| *** ddieterly has joined #openstack-infra | 05:00 | |
| *** puranamr has joined #openstack-infra | 05:01 | |
| *** ub has quit IRC | 05:01 | |
| *** abregman has joined #openstack-infra | 05:01 | |
| *** shashank_hegde has joined #openstack-infra | 05:04 | |
| *** bharath_ has joined #openstack-infra | 05:04 | |
| *** ddieterly has quit IRC | 05:05 | |
| *** yfried_ has joined #openstack-infra | 05:05 | |
| *** puranamr has quit IRC | 05:05 | |
| openstackgerrit | Steven Dake proposed openstack-infra/project-config: Move Kolla to the openstack git namespace https://review.openstack.org/209343 | 05:06 |
| *** bharath_ has quit IRC | 05:06 | |
| *** abregman has quit IRC | 05:06 | |
| *** shashank_hegde has quit IRC | 05:07 | |
| *** harlowja_at_home has quit IRC | 05:08 | |
| *** yfried__ has joined #openstack-infra | 05:08 | |
| *** pahuang has quit IRC | 05:09 | |
| *** dchen has quit IRC | 05:09 | |
| *** yfried_ has quit IRC | 05:10 | |
| *** camunoz has quit IRC | 05:10 | |
| *** camunoz has joined #openstack-infra | 05:12 | |
| *** crc32 has quit IRC | 05:16 | |
| *** BharatK has joined #openstack-infra | 05:17 | |
| *** BharatK_ has joined #openstack-infra | 05:20 | |
| *** dchen has joined #openstack-infra | 05:21 | |
| *** salv-orlando has joined #openstack-infra | 05:21 | |
| openstackgerrit | Feodor Tersin proposed openstack-infra/project-config: Add Neutron vpn service for ec2api Neutron gating job https://review.openstack.org/194670 | 05:21 |
| *** pahuang has joined #openstack-infra | 05:22 | |
| openstackgerrit | Merged openstack-infra/subunit2sql: Add migration to add indexes on common search patterns https://review.openstack.org/196194 | 05:23 |
| fifieldt | clarkb, sorry, I was noming | 05:23 |
| *** salv-orlando has quit IRC | 05:23 | |
| *** BharatK has quit IRC | 05:24 | |
| fifieldt | Based on that, I'm going to guess that one of the events imported from meetup.com had some quotes in it that were not properly escaped. | 05:24 |
| openstackgerrit | Merged openstack-infra/project-config: Enforce python3 support in Anchor https://review.openstack.org/208261 | 05:25 |
| *** fitoduarte has quit IRC | 05:26 | |
| openstackgerrit | Merged openstack-infra/project-config: Turn anvil venv trusty into a voting/gating job https://review.openstack.org/209263 | 05:27 |
| *** dchen has quit IRC | 05:28 | |
| openstackgerrit | Merged openstack-infra/project-config: No longer ignore exit status of zanata-cli push https://review.openstack.org/209309 | 05:29 |
| *** fedexo has quit IRC | 05:39 | |
| *** dchen has joined #openstack-infra | 05:41 | |
| *** ildikov has joined #openstack-infra | 05:42 | |
| *** Somay has joined #openstack-infra | 05:42 | |
| *** mrmartin has joined #openstack-infra | 05:44 | |
| *** bharath_ has joined #openstack-infra | 05:46 | |
| *** nadya has joined #openstack-infra | 05:48 | |
| *** nadya has quit IRC | 05:49 | |
| openstackgerrit | Marton Kiss proposed openstack-infra/groups: Fix of #208953 https://review.openstack.org/209350 | 05:55 |
| *** flepied1 has quit IRC | 05:58 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/209356 | 06:01 |
| *** ddieterly has joined #openstack-infra | 06:01 | |
| *** esker has quit IRC | 06:01 | |
| *** jyuso1 has joined #openstack-infra | 06:01 | |
| openstackgerrit | Merged openstack-infra/groups: Fix of #208953 https://review.openstack.org/209350 | 06:03 |
| *** maishsk has joined #openstack-infra | 06:03 | |
| *** esker has joined #openstack-infra | 06:03 | |
| *** ParsectiX has joined #openstack-infra | 06:04 | |
| *** ddieterly has quit IRC | 06:05 | |
| *** otter768 has joined #openstack-infra | 06:06 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/208820 | 06:06 |
| *** jasondotstar|afk has joined #openstack-infra | 06:10 | |
| *** otter768 has quit IRC | 06:11 | |
| *** Somay has quit IRC | 06:12 | |
| *** mrunge has joined #openstack-infra | 06:12 | |
| *** puranamr has joined #openstack-infra | 06:12 | |
| *** fitoduarte has joined #openstack-infra | 06:12 | |
| *** jasondotstar|afk has quit IRC | 06:15 | |
| *** funzo has quit IRC | 06:23 | |
| *** andrey-mp has joined #openstack-infra | 06:25 | |
| *** puranamr has quit IRC | 06:25 | |
| *** monester has quit IRC | 06:28 | |
| *** shashank_hegde has joined #openstack-infra | 06:29 | |
| *** flepied has joined #openstack-infra | 06:32 | |
| *** yolanda has quit IRC | 06:34 | |
| *** nadya has joined #openstack-infra | 06:34 | |
| *** yfried__ has quit IRC | 06:35 | |
| *** marzif_ has quit IRC | 06:35 | |
| *** yfried__ has joined #openstack-infra | 06:35 | |
| *** pcaruana has joined #openstack-infra | 06:36 | |
| *** yfried has joined #openstack-infra | 06:37 | |
| *** Somay has joined #openstack-infra | 06:40 | |
| *** yfried__ has quit IRC | 06:40 | |
| *** b10n1k has joined #openstack-infra | 06:40 | |
| *** stevemar has quit IRC | 06:40 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/209356 | 06:40 |
| *** salv-orlando has joined #openstack-infra | 06:41 | |
| *** stevemar has joined #openstack-infra | 06:41 | |
| *** MarkAtwood has quit IRC | 06:41 | |
| *** ub has joined #openstack-infra | 06:41 | |
| *** achanda has quit IRC | 06:42 | |
| *** ihrachyshka has joined #openstack-infra | 06:42 | |
| *** rcarrillocruz has quit IRC | 06:42 | |
| *** vlaza has joined #openstack-infra | 06:43 | |
| *** stevemar has quit IRC | 06:43 | |
| *** adduarte has joined #openstack-infra | 06:44 | |
| *** PinkFreud has quit IRC | 06:45 | |
| *** zeih has joined #openstack-infra | 06:45 | |
| *** PinkFreud has joined #openstack-infra | 06:46 | |
| *** fitoduarte has quit IRC | 06:48 | |
| *** scheuran has joined #openstack-infra | 06:48 | |
| openstackgerrit | Merged openstack-infra/project-config: midonet: Skip jobs where appropriate https://review.openstack.org/209338 | 06:50 |
| *** zeih has quit IRC | 06:50 | |
| *** sputnik13 has joined #openstack-infra | 06:52 | |
| *** kushal has joined #openstack-infra | 06:53 | |
| *** unicell has joined #openstack-infra | 06:54 | |
| *** adduarte has quit IRC | 06:54 | |
| *** ricky1 has joined #openstack-infra | 06:56 | |
| *** fitoduarte has joined #openstack-infra | 06:57 | |
| *** fitoduarte has quit IRC | 06:58 | |
| *** ddieterly has joined #openstack-infra | 07:02 | |
| *** afazekas has joined #openstack-infra | 07:02 | |
| *** sputnik13 has quit IRC | 07:03 | |
| *** ifarkas has joined #openstack-infra | 07:04 | |
| *** Somay has quit IRC | 07:04 | |
| *** ddieterly has quit IRC | 07:06 | |
| *** nadya has quit IRC | 07:08 | |
| *** Somay has joined #openstack-infra | 07:11 | |
| *** Ala has joined #openstack-infra | 07:12 | |
| *** marzif_ has joined #openstack-infra | 07:13 | |
| *** jcoufal has joined #openstack-infra | 07:13 | |
| *** marzif__ has joined #openstack-infra | 07:16 | |
| *** markus_z has joined #openstack-infra | 07:16 | |
| *** marzif_ has quit IRC | 07:18 | |
| *** macjack has joined #openstack-infra | 07:19 | |
| *** ricky1 has quit IRC | 07:19 | |
| *** monester has joined #openstack-infra | 07:20 | |
| *** yamamoto has quit IRC | 07:21 | |
| *** nadya has joined #openstack-infra | 07:23 | |
| *** shashank_hegde has quit IRC | 07:24 | |
| *** funzo has joined #openstack-infra | 07:24 | |
| *** ricky1 has joined #openstack-infra | 07:25 | |
| *** yolanda has joined #openstack-infra | 07:25 | |
| *** yamamoto has joined #openstack-infra | 07:26 | |
| *** nadya has quit IRC | 07:27 | |
| *** sshnaidm has joined #openstack-infra | 07:28 | |
| *** arxcruz has joined #openstack-infra | 07:29 | |
| *** hamzy has quit IRC | 07:30 | |
| *** yamahata has quit IRC | 07:31 | |
| *** witek has joined #openstack-infra | 07:33 | |
| *** nadya has joined #openstack-infra | 07:34 | |
| *** krtaylor has quit IRC | 07:35 | |
| *** jschwarz has joined #openstack-infra | 07:36 | |
| *** jlanoux has joined #openstack-infra | 07:38 | |
| *** salv-orlando has quit IRC | 07:38 | |
| *** b10n1k has quit IRC | 07:39 | |
| *** b10n1k has joined #openstack-infra | 07:39 | |
| *** sobersabre has joined #openstack-infra | 07:42 | |
| *** funzo has quit IRC | 07:42 | |
| *** hamzy has joined #openstack-infra | 07:42 | |
| *** e0ne has joined #openstack-infra | 07:47 | |
| *** amotoki has joined #openstack-infra | 07:48 | |
| *** yfried_ has joined #openstack-infra | 07:48 | |
| *** krtaylor has joined #openstack-infra | 07:49 | |
| *** Somay has left #openstack-infra | 07:49 | |
| *** e0ne has quit IRC | 07:52 | |
| *** yfried has quit IRC | 07:53 | |
| *** HeOS has quit IRC | 07:54 | |
| Ng | SpamapS: so, FTR, I am removing... cd-undercloud.tripleo.org. A138.35.77.3, ci-overcloud.hp1.tripleo.org. A138.35.77.6, cd-overcloud.hp1.tripleo.org. A138.35.77.5, ci-overcloud.tripleo.org. A138.35.77.5, cd-overcloud.tripleo.org. A138.35.77.4, and undercloud.hp1.tripleo.org. A138.35.77.4 | 07:54 |
| Ng | SpamapS: and changed bastion.hp1.tripleo.org from 138.35.77.2 to 15.184.52.2 | 07:54 |
| *** e0ne has joined #openstack-infra | 07:56 | |
| openstackgerrit | Joseph Lanoux proposed openstack-infra/project-config: Add non-voting job with ssh validation https://review.openstack.org/208481 | 07:57 |
| *** dtantsur|afk is now known as dtantsur | 07:57 | |
| openstackgerrit | Merged openstack-infra/project-config: fwaas: Enable python3 job https://review.openstack.org/207927 | 07:58 |
| *** Somay has joined #openstack-infra | 07:59 | |
| *** atuvenie has joined #openstack-infra | 08:02 | |
| *** ddieterly has joined #openstack-infra | 08:03 | |
| *** e0ne has quit IRC | 08:04 | |
| *** fawadkhaliq has quit IRC | 08:06 | |
| *** otter768 has joined #openstack-infra | 08:07 | |
| *** ddieterly has quit IRC | 08:07 | |
| *** gildub has quit IRC | 08:08 | |
| *** fhubik has joined #openstack-infra | 08:10 | |
| *** otter768 has quit IRC | 08:12 | |
| *** abregman has joined #openstack-infra | 08:12 | |
| *** fhubik is now known as fhubik_afk | 08:12 | |
| *** jistr has joined #openstack-infra | 08:14 | |
| *** Somay has quit IRC | 08:14 | |
| vponomaryov | Guys, from core team, please, look at this small change - https://review.openstack.org/#/c/209067/ Manila project needs it to remove merge-blocker | 08:16 |
| *** andrey-mp has quit IRC | 08:20 | |
| openstackgerrit | Nadya Shakhat proposed openstack-infra/project-config: Make gate-ceilometer-dsvm-functional-mongodb voting https://review.openstack.org/209398 | 08:28 |
| *** fawadkhaliq has joined #openstack-infra | 08:28 | |
| *** boris-42 has quit IRC | 08:30 | |
| *** mdbooth has quit IRC | 08:30 | |
| *** mdbooth has joined #openstack-infra | 08:30 | |
| *** jcoufal has quit IRC | 08:31 | |
| *** jcoufal has joined #openstack-infra | 08:33 | |
| *** ihrachyshka has quit IRC | 08:35 | |
| openstackgerrit | Nadya Shakhat proposed openstack-infra/project-config: Make gate-ceilometer-dsvm-functional-mongodb voting https://review.openstack.org/209398 | 08:37 |
| *** soren has quit IRC | 08:39 | |
| *** salv-orlando has joined #openstack-infra | 08:39 | |
| *** stevemar has joined #openstack-infra | 08:41 | |
| *** mhu has joined #openstack-infra | 08:43 | |
| *** stevemar has quit IRC | 08:45 | |
| *** pal has quit IRC | 08:46 | |
| *** ricky1 is now known as rcarrillocruz | 08:48 | |
| *** jasondotstar|afk has joined #openstack-infra | 08:48 | |
| *** dchen has quit IRC | 08:51 | |
| openstackgerrit | Nadya Shakhat proposed openstack-infra/project-config: Make gate-ceilometer-dsvm-functional-mongodb voting https://review.openstack.org/209398 | 08:51 |
| *** fhubik_afk is now known as fhubik | 08:51 | |
| *** HeOS has joined #openstack-infra | 08:51 | |
| *** dingyichen has quit IRC | 08:52 | |
| *** pal has joined #openstack-infra | 08:53 | |
| *** jasondotstar|afk is now known as jasondotstar | 08:54 | |
| *** sabeen has quit IRC | 08:55 | |
| *** nadya_ has joined #openstack-infra | 08:56 | |
| *** nadya has quit IRC | 08:59 | |
| *** ddieterly has joined #openstack-infra | 09:04 | |
| *** jcoufal has quit IRC | 09:05 | |
| *** ddieterly has quit IRC | 09:08 | |
| *** rguillebert has joined #openstack-infra | 09:09 | |
| *** funzo has joined #openstack-infra | 09:12 | |
| *** jcoufal has joined #openstack-infra | 09:18 | |
| *** pilgrimstack has quit IRC | 09:19 | |
| *** jcoufal has quit IRC | 09:19 | |
| *** pilgrimstack has joined #openstack-infra | 09:20 | |
| *** yushiro has joined #openstack-infra | 09:21 | |
| *** mika has quit IRC | 09:21 | |
| *** KasDev has joined #openstack-infra | 09:22 | |
| yushiro | sdague, hi. Thank you for your review https://review.openstack.org/#/c/206019/ | 09:25 |
| yushiro | sdague, clarkb Would you please review it again? https://review.openstack.org/#/c/206019/ | 09:27 |
| KasDev | hello everyone! | 09:27 |
| KasDev | Is it possible to pares in arguments from command line when running the "jenkins job builder"? Eg. "jenkins-job update jobfile.yaml arg1 arg2 arg3" . | 09:29 |
| *** e0ne has joined #openstack-infra | 09:31 | |
| openstackgerrit | Anastasia Kuznetsova proposed openstack-infra/project-config: Add Mistral gate that run unit tests and use postgresql https://review.openstack.org/207961 | 09:34 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Fix translation sync with Transifex https://review.openstack.org/209419 | 09:35 |
| *** salv-orlando has quit IRC | 09:36 | |
| *** e0ne has quit IRC | 09:38 | |
| openstackgerrit | Merged openstack-infra/puppet-iptables: Add Gemfile and puppet 4 checks https://review.openstack.org/208755 | 09:40 |
| *** flepied has quit IRC | 09:41 | |
| *** stevemar has joined #openstack-infra | 09:42 | |
| openstackgerrit | Feodor Tersin proposed openstack-infra/project-config: Add VPNaaS to ec2api Neutron gating jobs https://review.openstack.org/194670 | 09:43 |
| *** cdent has joined #openstack-infra | 09:45 | |
| *** stevemar has quit IRC | 09:46 | |
| *** andrey-mp has joined #openstack-infra | 09:49 | |
| Zara | hi, is there a protocol for people putting things on the infra meeting agenda (eg: review requests) when they are unable to attend the meeting? We (Storyboard) couldn't make it yesterday, but we have a worklists and boards spec that we'd like some eyes on: https://review.openstack.org/#/c/202989/ | 09:49 |
| *** yamamoto has quit IRC | 09:52 | |
| *** flepied has joined #openstack-infra | 09:54 | |
| *** jcoufal has joined #openstack-infra | 09:56 | |
| *** dtantsur is now known as dtantsur|brb | 09:58 | |
| *** nadya_ has quit IRC | 09:58 | |
| rcarrillocruz | Zara: you can add items on the agenda wiki page | 09:59 |
| rcarrillocruz | https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting | 10:00 |
| *** dims has joined #openstack-infra | 10:00 | |
| *** dims has quit IRC | 10:00 | |
| *** dims has joined #openstack-infra | 10:00 | |
| openstackgerrit | Victor Sergeyev proposed openstack/requirements: Fix url regexp to proceed oslo-specific url's properly https://review.openstack.org/209432 | 10:01 |
| *** jasondotstar has quit IRC | 10:02 | |
| *** jasondotstar has joined #openstack-infra | 10:03 | |
| *** nadya has joined #openstack-infra | 10:04 | |
| *** ddieterly has joined #openstack-infra | 10:04 | |
| Zara | rcarriolocruz: thanks, is it okay to do that if we won't be there for the meeting? (hopefully we'll always be there and this week was a one-off, but it'd be good to know just in case). | 10:05 |
| *** yamamoto has joined #openstack-infra | 10:07 | |
| *** otter768 has joined #openstack-infra | 10:08 | |
| *** pc_m has joined #openstack-infra | 10:08 | |
| *** ddieterly has quit IRC | 10:09 | |
| *** otter768 has quit IRC | 10:12 | |
| *** fhubik is now known as fhubik_afk | 10:20 | |
| *** BobBall is now known as BobBall_AWOL | 10:20 | |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: remove neutron advanced services from grenade https://review.openstack.org/201279 | 10:25 |
| openstackgerrit | Mohankumar proposed openstack/requirements: Add networking-sfc to the list https://review.openstack.org/209446 | 10:25 |
| openstackgerrit | Mohankumar proposed openstack/requirements: Add networking-sfc to the list https://review.openstack.org/209446 | 10:26 |
| *** claudiub has joined #openstack-infra | 10:27 | |
| *** yamamoto has quit IRC | 10:29 | |
| *** e0ne has joined #openstack-infra | 10:30 | |
| *** pal has quit IRC | 10:33 | |
| *** coolsvap is now known as coolsvap|away | 10:33 | |
| *** salv-orlando has joined #openstack-infra | 10:34 | |
| *** gaurangt has joined #openstack-infra | 10:37 | |
| *** yfried_ is now known as yfried|afk | 10:37 | |
| *** gildub has joined #openstack-infra | 10:38 | |
| *** yfried|afk is now known as yfried_ | 10:39 | |
| *** bexelbie has joined #openstack-infra | 10:41 | |
| *** pal has joined #openstack-infra | 10:42 | |
| *** e0ne has quit IRC | 10:46 | |
| *** nadya has quit IRC | 10:47 | |
| *** bharath_ has quit IRC | 10:47 | |
| *** bharath_ has joined #openstack-infra | 10:50 | |
| *** nadya has joined #openstack-infra | 10:50 | |
| *** yfried_ has quit IRC | 10:51 | |
| *** yushiro has quit IRC | 10:52 | |
| *** yfried has joined #openstack-infra | 10:52 | |
| *** ftersin has joined #openstack-infra | 10:53 | |
| *** markus_z has quit IRC | 10:57 | |
| *** ihrachyshka has joined #openstack-infra | 11:01 | |
| openstackgerrit | Feodor Tersin proposed openstack-infra/project-config: Add VPNaaS to ec2api Neutron gating jobs https://review.openstack.org/194670 | 11:02 |
| *** aukhan has quit IRC | 11:02 | |
| *** baoli has joined #openstack-infra | 11:02 | |
| *** sobersabre has quit IRC | 11:03 | |
| *** sobersabre has joined #openstack-infra | 11:04 | |
| *** mwagner_lap has joined #openstack-infra | 11:04 | |
| *** ddieterly has joined #openstack-infra | 11:05 | |
| openstackgerrit | Anastasia Kuznetsova proposed openstack-infra/project-config: Add Mistral gate that run unit tests and use postgresql https://review.openstack.org/207961 | 11:06 |
| *** bharath_ has quit IRC | 11:06 | |
| *** baoli has quit IRC | 11:06 | |
| openstackgerrit | vikram.choudhary proposed openstack/requirements: Add networking-onos to the project list https://review.openstack.org/209461 | 11:06 |
| *** baoli has joined #openstack-infra | 11:06 | |
| *** vikram has joined #openstack-infra | 11:07 | |
| *** e0ne has joined #openstack-infra | 11:07 | |
| *** KasDev has quit IRC | 11:09 | |
| *** ddieterly has quit IRC | 11:10 | |
| *** sergmelikyan has joined #openstack-infra | 11:11 | |
| *** yamamoto has joined #openstack-infra | 11:11 | |
| *** yamamoto_ has joined #openstack-infra | 11:12 | |
| *** markus_z has joined #openstack-infra | 11:12 | |
| *** marzif__ has quit IRC | 11:15 | |
| *** yamamoto has quit IRC | 11:16 | |
| *** marzif__ has joined #openstack-infra | 11:16 | |
| *** ihrachyshka has quit IRC | 11:19 | |
| *** alexpilotti has joined #openstack-infra | 11:21 | |
| *** AJaeger has joined #openstack-infra | 11:21 | |
| pc_m | sdague: Hi! I'm still having problem with grenade plugin. | 11:24 |
| *** jaypipes has joined #openstack-infra | 11:24 | |
| *** dtantsur|brb is now known as dtantsur | 11:25 | |
| *** mrmartin has quit IRC | 11:25 | |
| *** fhubik_afk is now known as fhubik | 11:26 | |
| *** dims_ has joined #openstack-infra | 11:26 | |
| *** yfried is now known as yfried|afk | 11:28 | |
| pc_m | sdague: Fails at upgrade stage. See three things that seem odd. 1) When neutron q-svc starts, it tries to load VPN plugin, but is accessing it in /opt/stack/old. 2) The neutron-vpnaas project is scheduled for upgrade, but at the end of the process and not when Neutron is upgraded. 3) I don't see neutron-vpnaas plugin actions during neutron upgrade. | 11:28 |
| *** gordc has joined #openstack-infra | 11:28 | |
| *** dims has quit IRC | 11:28 | |
| sdague | right, so that's an expected problem. How would you expect deploys to address this? | 11:28 |
| *** mika has joined #openstack-infra | 11:29 | |
| *** mika has joined #openstack-infra | 11:29 | |
| sdague | how about you start from an upgrade path that's written down in english about the steps that people will need to take | 11:30 |
| pc_m | not sure | 11:30 |
| *** gaurangt has left #openstack-infra | 11:32 | |
| pc_m | seems like, when neutron is updated, after clone new code and migrate database, each of the enable services need to be upgraded (clone, migrate, run process), and then q-svc process run | 11:33 |
| *** pal has quit IRC | 11:33 | |
| *** buzzco has joined #openstack-infra | 11:33 | |
| *** e0ne has quit IRC | 11:34 | |
| *** yfried|afk is now known as yfried | 11:34 | |
| AJaeger | flaper87: are you around to discuss the doc review you commented on yesterday? | 11:34 |
| *** yamamoto_ has quit IRC | 11:35 | |
| flaper87 | AJaeger: yes | 11:36 |
| flaper87 | AJaeger: was it yday? | 11:36 |
| flaper87 | jesus, I don't even know where I am nor what era I'm living in | 11:36 |
| flaper87 | T_T | 11:36 |
| flaper87 | it's all morganfainberg's fault | 11:37 |
| morganfainberg | Bah | 11:37 |
| morganfainberg | No blaming me :P | 11:37 |
| AJaeger | flaper87, morganfainberg: Hi. I'm here to solve a puzzle not to blame you ;) | 11:38 |
| AJaeger | morganfainberg, flaper87 this is about https://review.openstack.org/#/c/195473/ | 11:38 |
| AJaeger | Basically, when you run "glance --os-image-api-version 2 help image-create" and have no creditionals setup, you get: | 11:38 |
| *** gildub has quit IRC | 11:38 | |
| AJaeger | "<unavailable> Please run with connection parameters set to retrieve the schema for generating help for this command" | 11:38 |
| flaper87 | AJaeger: just read your answer | 11:39 |
| pc_m | sdague: For old services, it happens during upgrade, as DevStack seems to do the clone, ini files, migrate. | 11:39 |
| flaper87 | that makes more sense now | 11:39 |
| morganfainberg | I am pretty clueless here. | 11:39 |
| flaper87 | morganfainberg: nah, I was just trolling you :P | 11:39 |
| morganfainberg | Very very out of the loop. | 11:39 |
| AJaeger | flaper87, morganfainberg : So how can we document image-create from its help in such a way that it's cloud independent? | 11:39 |
| * morganfainberg goes back to lurking. | 11:39 | |
| flaper87 | AJaeger: morganfainberg has nothing to do with this patch | 11:39 |
| flaper87 | :D | 11:39 |
| pc_m | sdague: doesn't seem to be doing similar for devstack plugins. Nor, do I see the process start piece. | 11:39 |
| AJaeger | morganfainberg: Sorry! | 11:39 |
| *** ldnunes has joined #openstack-infra | 11:39 | |
| * flaper87 wonders if morganfainberg even knows what glance is #trololo | 11:40 | |
| andrey-mp | AJaeger: please, could you review our efforts on ec2api jobs? first https://review.openstack.org/#/c/208535/ we need to move further with testing ec2 with swift3 addon. and second - https://review.openstack.org/#/c/194670/ we need to restore our gating at all (because q-vpn has removed from defaults but our tests use this feature) | 11:40 |
| *** hdd has joined #openstack-infra | 11:40 | |
| AJaeger | flaper87: we basically run "glance help COMMAND" and create a document from it. | 11:40 |
| flaper87 | AJaeger: so, I now understand the issue and I'm embarrassed to know that's happenning | 11:40 |
| AJaeger | So, my question is, what is the best way to document the command now? | 11:40 |
| AJaeger | flaper87: glad to hear... | 11:40 |
| flaper87 | AJaeger: thinking | 11:41 |
| openstackgerrit | Victor Ryzhenkin proposed openstack-infra/project-config: Add opportunity to install muranoclient from change https://review.openstack.org/204159 | 11:42 |
| flaper87 | AJaeger: would it be ok to document it with a running glance instance until we fix this ? | 11:42 |
| flaper87 | AJaeger: I can do it if you guide me through the process | 11:42 |
| AJaeger | sdague: do you have time to review https://review.openstack.org/#/c/194670/ to help andrey-mp ? I reviewed both patches... | 11:43 |
| andrey-mp | AJaeger: thank you very much! | 11:43 |
| pc_m | AJaeger: andrey-mp see my comments on 194670 | 11:43 |
| AJaeger | flaper87: Yes, we could do something there. | 11:43 |
| AJaeger | The source code is in openstack-doc-tools file os_doc_tools/commands.py. | 11:44 |
| *** coolsvap|away is now known as coolsvap | 11:44 | |
| *** dims_ has quit IRC | 11:44 | |
| AJaeger | pc_m: ah... let's see what sdague thinks | 11:45 |
| sdague | pc_m: ignore the devstack case for a second, what is the upgrade path expectations on the deployer. devstack / grenade are there to create approximations of that. I think the hack and slash approach here isn't going to get you anywhere. Step back and write out in english what a production neutron deploy needs to do to upgrade from kilo to liberty | 11:45 |
| andrey-mp | pc_m: thanks for comment. should we change PROJECT variable if we change q-vpn to plugin? | 11:45 |
| *** thorst has joined #openstack-infra | 11:45 | |
| flaper87 | AJaeger: ok | 11:45 |
| AJaeger | flaper87: or comment in that review and either of us will update it... Thanks! | 11:46 |
| *** afazekas_ has joined #openstack-infra | 11:46 | |
| *** pal has joined #openstack-infra | 11:46 | |
| *** deepakcs has quit IRC | 11:47 | |
| flaper87 | AJaeger: commenting | 11:47 |
| pc_m | sdague: ok. will think about it. I thought this was already worked out. | 11:48 |
| sdague | AJaeger: I agree with pc_m | 11:48 |
| sdague | pc_m: it's possible it's written down somewhere already, but I feel like the concept of the steps involved is missing here. | 11:48 |
| *** afazekas has quit IRC | 11:48 | |
| pc_m | andrey-mp: I think you need to add a line like L70 only with neutron-vpnaas. You can look at devstack-gate.yaml for example. | 11:49 |
| *** julim has joined #openstack-infra | 11:49 | |
| *** yamamoto has joined #openstack-infra | 11:50 | |
| *** gildub has joined #openstack-infra | 11:50 | |
| *** dims has joined #openstack-infra | 11:51 | |
| andrey-mp | sdague: is it needed modification of PROJECT variable? or just add export DEVSTACK_LOCAL_CONFIG+=$'\n'"enable_plugin neutron-vpnaas git://git.openstack.org/..." ? | 11:51 |
| *** bexelbie has quit IRC | 11:53 | |
| sdague | if it's already in the PROJECT list in devstack-gate, that's it | 11:55 |
| sdague | and I think this one is | 11:55 |
| openstackgerrit | Nadya Shakhat proposed openstack-infra/project-config: Make gate-ceilometer-dsvm-functional-mongodb voting https://review.openstack.org/209398 | 11:55 |
| *** jistr has quit IRC | 11:56 | |
| *** jcoufal has quit IRC | 11:56 | |
| *** yfried is now known as yfried|afk | 11:57 | |
| andrey-mp | sdague: thanks, we will change it | 11:57 |
| *** BharatK_ has quit IRC | 11:58 | |
| *** buzzco has quit IRC | 11:59 | |
| *** buzzco has joined #openstack-infra | 12:00 | |
| *** ftersin has quit IRC | 12:00 | |
| *** yfried|afk is now known as yfried | 12:00 | |
| pc_m | sdague: Here is my stab at the process for kilo to liberty upgrade for neutron (with only VPN service enabled) - http://paste.openstack.org/show/408265/ | 12:01 |
| *** olaph has quit IRC | 12:01 | |
| *** olaph has joined #openstack-infra | 12:02 | |
| openstackgerrit | vikram.choudhary proposed openstack-infra/project-config: Add check-requirements to networking-onos https://review.openstack.org/209482 | 12:02 |
| sdague | pc_m: ok, so that means that all these services are atomic and they aren't separate things | 12:02 |
| openstackgerrit | vikram.choudhary proposed openstack/requirements: Add networking-onos to the project list https://review.openstack.org/209461 | 12:04 |
| pc_m | sdague: So don't need a neutron-vpnaas "project" upgrade for grenade, right? | 12:04 |
| pc_m | sdague: Are we just missing neutron doing devstack plugin steps for enabled plugins? | 12:04 |
| sdague | I really think that's not a direction we want to go either. | 12:05 |
| *** ihrachyshka has joined #openstack-infra | 12:05 | |
| sdague | I think we need a conversation with mestery about what the long term direction here is, because this feels massively broken | 12:05 |
| sdague | that neutron advanced services are sometimes distinct things, but they can't exist distinctly | 12:06 |
| *** ddieterly has joined #openstack-infra | 12:06 | |
| *** rfolco has joined #openstack-infra | 12:06 | |
| *** e0ne has joined #openstack-infra | 12:06 | |
| *** olaph has quit IRC | 12:06 | |
| sdague | and adding another layer of complexity in devstack / grenade to deal with this approach isn't really appealing | 12:06 |
| *** olaph has joined #openstack-infra | 12:07 | |
| openstackgerrit | Feodor Tersin proposed openstack-infra/project-config: Add VPNaaS to ec2api Neutron gating jobs https://review.openstack.org/194670 | 12:07 |
| *** ddieterly has quit IRC | 12:07 | |
| *** ddieterly has joined #openstack-infra | 12:08 | |
| *** otter768 has joined #openstack-infra | 12:09 | |
| pc_m | mestery: Are you around yet? If so, ^^^ | 12:09 |
| *** dprince has joined #openstack-infra | 12:10 | |
| pc_m | sdague: Would like to have some short term solution, so that grenade testing of neutron-vpnaas can proceed (and commits doing migration can be tested). | 12:10 |
| *** fhubik is now known as fhubik_afk | 12:12 | |
| *** ddieterly has quit IRC | 12:12 | |
| *** ayoung has quit IRC | 12:13 | |
| *** otter768 has quit IRC | 12:13 | |
| *** yfried is now known as yfried|afk | 12:14 | |
| *** andrey-mp has quit IRC | 12:14 | |
| openstackgerrit | Feodor Tersin proposed openstack-infra/project-config: Add VPNaaS to ec2api Neutron gating jobs https://review.openstack.org/194670 | 12:14 |
| *** buzzco has quit IRC | 12:16 | |
| sdague | pc_m: I think the short term solution is to just remove it from testing. Because I don't see changing grenade to do support this 2 level upgrade path. Once the neutron advanced services are actually independent they can be tested in this model. | 12:17 |
| *** tiswanso has joined #openstack-infra | 12:17 | |
| *** jcoufal has joined #openstack-infra | 12:17 | |
| *** jistr has joined #openstack-infra | 12:18 | |
| *** jistr is now known as jistr|biab | 12:19 | |
| *** tiswanso has quit IRC | 12:19 | |
| *** cburroughs has joined #openstack-infra | 12:20 | |
| pc_m | sdague: Concerns with that are A) requires manual testing of dbase migration changes, B) eventually LB (and maybe FW) will switch to using devstack plugins, C) how long it will be for a completely independent adv svcs. | 12:20 |
| *** tiswanso has joined #openstack-infra | 12:20 | |
| sdague | pc_m: sure, so C should be the #1 priority of neutron across the board right now | 12:21 |
| sdague | you should be able to do reasonable intree testing of A), nova does, and has since grizzly | 12:22 |
| *** buzzco has joined #openstack-infra | 12:22 | |
| *** sobersabre has quit IRC | 12:23 | |
| *** dkranz has joined #openstack-infra | 12:23 | |
| *** sobersabre has joined #openstack-infra | 12:24 | |
| *** ddieterly has joined #openstack-infra | 12:24 | |
| *** bexelbie has joined #openstack-infra | 12:27 | |
| *** bexelbie has quit IRC | 12:28 | |
| *** bexelbie has joined #openstack-infra | 12:29 | |
| pc_m | sdague: Been doing A, but really would like automated solution. | 12:29 |
| sdague | no, I mean automated in tree testing | 12:30 |
| pc_m | sdague: Could we modify 50_neutron/upgrade.sh to do the calls to services devstack plugin install step for each plugin? | 12:30 |
| sdague | most projects have db migration testing in tree. I wrote the infrastructure for it circa grizzly timeframe, and we got it into cinder, glance | 12:31 |
| openstackgerrit | Nadya Shakhat proposed openstack-infra/project-config: Make gate-ceilometer-dsvm-functional-mongodb voting https://review.openstack.org/209398 | 12:32 |
| pc_m | sdague: Interesting... any pointers on how to do that testing? I've been stacking, creating database entries, running neutron-db-manage and then checking the tables. | 12:32 |
| openstackgerrit | Imre Farkas proposed openstack-infra/project-config: Add openstack/python-dracclient project https://review.openstack.org/203991 | 12:33 |
| sdague | pc_m: honestly, I'm not interested in building work arounds here without the full plan on when neutron is fixing the half done split out | 12:33 |
| sdague | git grep migration in the nova tree | 12:33 |
| sdague | the base infrastructure is all in oslo.db now | 12:33 |
| *** warpc__ has joined #openstack-infra | 12:34 | |
| *** yfried|afk is now known as yfried | 12:34 | |
| openstackgerrit | Mohankumar proposed openstack-infra/project-config: Add check-requirements to networking-sfc https://review.openstack.org/209491 | 12:34 |
| openstackgerrit | Mohankumar proposed openstack/requirements: Add networking-sfc repository to the project library list https://review.openstack.org/209446 | 12:37 |
| *** salv-orlando has quit IRC | 12:37 | |
| *** fawadkhaliq has quit IRC | 12:38 | |
| *** esker has quit IRC | 12:38 | |
| *** markus_z has quit IRC | 12:40 | |
| *** markus_z has joined #openstack-infra | 12:42 | |
| *** abregman is now known as abregman|brb | 12:42 | |
| bookwar | we have a lot of errors when cloning from git.openstack.org today. Is it a known issue? | 12:42 |
| openstackgerrit | will soula proposed openstack-infra/jenkins-job-builder: Add Ability to use the AWS Cloudformation Plugin https://review.openstack.org/202300 | 12:42 |
| *** aysyd has joined #openstack-infra | 12:42 | |
| *** yamamoto has quit IRC | 12:43 | |
| bookwar | "Error cloning remote repo 'gerrit'" - cloning from gerrit actually, not git.o.o | 12:43 |
| *** jkraj has joined #openstack-infra | 12:43 | |
| *** marzif__ has quit IRC | 12:43 | |
| *** hdd has quit IRC | 12:44 | |
| *** marzif__ has joined #openstack-infra | 12:44 | |
| *** yfried is now known as yfried|afk | 12:44 | |
| *** olaph has quit IRC | 12:44 | |
| *** salv-orlando has joined #openstack-infra | 12:45 | |
| *** olaph has joined #openstack-infra | 12:45 | |
| *** buzzco has quit IRC | 12:45 | |
| *** tlian has joined #openstack-infra | 12:46 | |
| *** marcusvrn has joined #openstack-infra | 12:49 | |
| nadya | jd__: influx failed again :( | 12:50 |
| *** bcourt has joined #openstack-infra | 12:50 | |
| *** yfried|afk is now known as yfried | 12:51 | |
| *** hdd has joined #openstack-infra | 12:52 | |
| *** prad has joined #openstack-infra | 12:52 | |
| *** bapalm has joined #openstack-infra | 12:52 | |
| *** pilgrimstack1 has joined #openstack-infra | 12:53 | |
| *** bapalm_ has joined #openstack-infra | 12:53 | |
| *** yamamoto has joined #openstack-infra | 12:54 | |
| *** pilgrimstack has quit IRC | 12:54 | |
| *** tiswanso has quit IRC | 12:56 | |
| *** dims_ has joined #openstack-infra | 12:57 | |
| *** bapalm has quit IRC | 12:57 | |
| *** mrmartin has joined #openstack-infra | 12:58 | |
| *** ftersin has joined #openstack-infra | 12:59 | |
| *** dims has quit IRC | 12:59 | |
| mescanef | hi, guys :-) somebody maybe knows how to create a new branch of a stackforge project using commandline instead of gerrit web UI? | 13:00 |
| sdague | fungi (et al): why is http://logs.openstack.org/25/193725/23/check/gate-nova-python34/caf352e/ console.html not being served to my browser inline? | 13:00 |
| sdague | I'm getting really weird freaking headers being returned | 13:01 |
| sdague | looks like a massive amount of swift leaking through poorly | 13:02 |
| *** abregman|brb is now known as abregman | 13:03 | |
| fungi | file says /srv/static/logs/25/193725/23/check/gate-nova-python34/caf352e/console.html: data | 13:04 |
| fungi | so that sounds likely | 13:04 |
| fungi | lots of non-html content | 13:04 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Throw an exception on a server without an IP https://review.openstack.org/208105 | 13:04 |
| *** yfried has quit IRC | 13:04 | |
| *** amitgandhinz has joined #openstack-infra | 13:04 | |
| *** zul has joined #openstack-infra | 13:05 | |
| *** jistr|biab is now known as jistr | 13:06 | |
| *** yushiro has joined #openstack-infra | 13:07 | |
| *** mrmartin has quit IRC | 13:07 | |
| fungi | sdague: jhesketh: i wonder if it's at all related to the change for os-loganalyze to support serving non-text/html file formats? | 13:07 |
| *** rlandy has joined #openstack-infra | 13:07 | |
| fungi | were we maybe previously coercing that into text/plain or html? | 13:07 |
| *** esker has joined #openstack-infra | 13:07 | |
| *** mriedem_away is now known as mriedem | 13:08 | |
| pleia2 | mescanef: I think these are the docs you're looking for https://review.openstack.org/Documentation/project-setup.html#_registering_additional_branches | 13:09 |
| *** afazekas_ has quit IRC | 13:10 | |
| sdague | fungi: don't know | 13:10 |
| *** dkranz has quit IRC | 13:10 | |
| *** yamamoto has quit IRC | 13:10 | |
| *** john-davidge has joined #openstack-infra | 13:10 | |
| sdague | fungi: was there a push to apache to change mime type detection | 13:10 |
| sdague | because I'd have thought it would do that correctly | 13:11 |
| jd__ | nadya: what? | 13:11 |
| *** esker has quit IRC | 13:12 | |
| *** miqui has joined #openstack-infra | 13:15 | |
| *** kgiusti has joined #openstack-infra | 13:16 | |
| *** ajmiller has joined #openstack-infra | 13:17 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Be consistent with accessing server dict https://review.openstack.org/209514 | 13:17 |
| *** wznoinsk1 is now known as wznoinsk | 13:18 | |
| fungi | sdague: not that i know of, but i'm looking through the git history on that module now | 13:18 |
| AJaeger | pleia2, fungi, clarkb : Transifex blacklists now glossary so I had to fix our scripts, could you review https://review.openstack.org/209418, please? Right now all openstack-manuals sync fail | 13:18 |
| fungi | AJaeger: i have a feeling you mean a different change, though i'm happy to review that change to the openstack-manuals repo if it will help | 13:20 |
| *** buzzco has joined #openstack-infra | 13:20 | |
| *** alexpilotti has quit IRC | 13:21 | |
| *** sdake has quit IRC | 13:21 | |
| openstackgerrit | Nadya Shakhat proposed openstack-infra/project-config: Make gate-ceilometer-dsvm-functional-mongodb voting https://review.openstack.org/209398 | 13:21 |
| mescanef | pleia2: thanks, but im trying to find specific example on the command(s) | 13:23 |
| *** dkranz has joined #openstack-infra | 13:23 | |
| *** alexpilotti has joined #openstack-infra | 13:23 | |
| *** dprince has quit IRC | 13:23 | |
| openstackgerrit | KATO Tomoyuki proposed openstack-infra/infra-specs: Translation check site spec https://review.openstack.org/184559 | 13:24 |
| *** dprince has joined #openstack-infra | 13:24 | |
| *** gildub has quit IRC | 13:25 | |
| *** btully has joined #openstack-infra | 13:25 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config: add gerrit groups for monasca-log https://review.openstack.org/208430 | 13:25 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Be consistent with accessing server dict https://review.openstack.org/209514 | 13:26 |
| fungi | mescanef: the syntax is `git push gerrit localbranchname:remotebranchname` | 13:26 |
| fungi | mescanef: though you'll need create permissions on refs/heads/* or maybe even refs/* to support doing it that way, i think | 13:26 |
| *** baoli has quit IRC | 13:28 | |
| *** bknudson has joined #openstack-infra | 13:28 | |
| openstackgerrit | Christian Berendt proposed openstack-infra/groups: Install ruby-bundler during bootstrapping https://review.openstack.org/209517 | 13:28 |
| fungi | well, `git push <remote> <localref>:<remotebranch>` technically | 13:28 |
| *** baoli has joined #openstack-infra | 13:28 | |
| mescanef | fungi: thanks, will checkout that :-) | 13:28 |
| AJaeger | fungi: AGreed, let's try https://review.openstack.org/209419 - same title ;/ | 13:29 |
| fungi | where <remote> is the name of your gerrit remote or the full remote url, <localref> is the refname (could be a branch or something else) and <remotebranch> is the name of the branch you want created from it | 13:29 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Be consistent with accessing server dict https://review.openstack.org/209514 | 13:29 |
| *** dkranz has quit IRC | 13:29 | |
| *** coolsvap is now known as coolsvap|away | 13:30 | |
| *** tiswanso has joined #openstack-infra | 13:30 | |
| AJaeger | pleia2, clarkb: https://review.openstack.org/209419 is the change I meant. | 13:31 |
| AJaeger | fungi, thanks for reviewing | 13:31 |
| *** jd__` has joined #openstack-infra | 13:34 | |
| *** jd__- has joined #openstack-infra | 13:35 | |
| *** olaph has quit IRC | 13:36 | |
| *** dims has joined #openstack-infra | 13:36 | |
| mescanef | fungi: btw, what is required (the access) for creating not a new branch but tag(s)? | 13:37 |
| *** dims_ has quit IRC | 13:38 | |
| *** ashuk has joined #openstack-infra | 13:39 | |
| mescanef | fungi: like refs/heads/*? | 13:39 |
| *** tvanderwiel has joined #openstack-infra | 13:40 | |
| *** tvanderwiel has left #openstack-infra | 13:40 | |
| *** ashuk has quit IRC | 13:40 | |
| fungi | mescanef: http://docs.openstack.org/infra/manual/creators.html#creation-of-tags | 13:41 |
| fungi | mescanef: though i recommend the entire document if you haven't read it yet | 13:42 |
| *** dims_ has joined #openstack-infra | 13:42 | |
| pabelanger | morningt | 13:42 |
| fungi | is it mornight already?!? | 13:42 |
| *** dkranz has joined #openstack-infra | 13:43 | |
| *** ajmiller has quit IRC | 13:43 | |
| AJaeger | fungi: afternoon here ;) | 13:44 |
| *** dims has quit IRC | 13:44 | |
| mescanef | fungi: yeah, cool, thanks :-) | 13:44 |
| *** jd__` has quit IRC | 13:45 | |
| *** jd__- has quit IRC | 13:45 | |
| *** jd__ has quit IRC | 13:45 | |
| pabelanger | haven't read backscroll, but is there an issue with puppet-zanata? | 13:45 |
| pabelanger | http://logs.openstack.org/47/209247/1/check/gate-infra-puppet-apply-bare-precise/10910b8/console.html#_2015-08-04_21_14_49_788 | 13:45 |
| *** jd__ has joined #openstack-infra | 13:45 | |
| *** jd__ has quit IRC | 13:46 | |
| *** sc68cal_ has joined #openstack-infra | 13:46 | |
| *** kushal has quit IRC | 13:46 | |
| *** jd__ has joined #openstack-infra | 13:46 | |
| *** soren has joined #openstack-infra | 13:47 | |
| *** esker has joined #openstack-infra | 13:47 | |
| StevenK | pabelanger: java_ks was added to system-config in 325c7d5, do you need to rebase? | 13:48 |
| *** tiswanso has quit IRC | 13:48 | |
| pabelanger | StevenK: zuul should have done done that for me. rechecking now | 13:49 |
| *** ayoung has joined #openstack-infra | 13:51 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Be consistent with accessing server dict https://review.openstack.org/209514 | 13:52 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Throw an exception on a server without an IP https://review.openstack.org/208105 | 13:52 |
| *** niska has quit IRC | 13:52 | |
| *** briancline has quit IRC | 13:52 | |
| *** sc68cal has quit IRC | 13:52 | |
| *** niska has joined #openstack-infra | 13:52 | |
| *** salv-orlando has quit IRC | 13:53 | |
| *** isaacb has joined #openstack-infra | 13:53 | |
| pabelanger | that is odd, https://review.openstack.org/#/c/206307/ had the proper depends-on headers but zuul the job still failed to work properly | 13:53 |
| pabelanger | will have to look into why | 13:53 |
| *** bexelbie has quit IRC | 13:54 | |
| *** briancline has joined #openstack-infra | 13:54 | |
| *** fhubik_afk is now known as fhubik | 13:54 | |
| *** dims has joined #openstack-infra | 13:55 | |
| *** olaph has joined #openstack-infra | 13:55 | |
| *** tiswanso has joined #openstack-infra | 13:56 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 13:56 | |
| *** dims_ has quit IRC | 13:57 | |
| *** isaacb has quit IRC | 13:57 | |
| openstackgerrit | David Lyle proposed openstack-infra/irc-meetings: Adding new meeting for Horizon Drivers https://review.openstack.org/209523 | 13:58 |
| *** ParsectiX has quit IRC | 14:00 | |
| *** rguillebert has quit IRC | 14:00 | |
| *** alivigni has joined #openstack-infra | 14:01 | |
| *** zz_dimtruck is now known as dimtruck | 14:02 | |
| *** pal has quit IRC | 14:02 | |
| *** esker has quit IRC | 14:04 | |
| *** zigo has quit IRC | 14:05 | |
| *** esker has joined #openstack-infra | 14:05 | |
| *** olaph has quit IRC | 14:06 | |
| *** zigo has joined #openstack-infra | 14:06 | |
| *** abregman is now known as abregman|brb | 14:07 | |
| *** ajmiller has joined #openstack-infra | 14:08 | |
| *** woodster_ has joined #openstack-infra | 14:08 | |
| *** otter768 has joined #openstack-infra | 14:10 | |
| *** jsavak has joined #openstack-infra | 14:10 | |
| *** yamamoto has joined #openstack-infra | 14:11 | |
| *** esker has quit IRC | 14:12 | |
| *** kushal has joined #openstack-infra | 14:12 | |
| *** afazekas_ has joined #openstack-infra | 14:12 | |
| *** vlaza has left #openstack-infra | 14:12 | |
| *** otter768 has quit IRC | 14:14 | |
| *** sobersabre has quit IRC | 14:14 | |
| *** esker has joined #openstack-infra | 14:14 | |
| *** asselin has joined #openstack-infra | 14:14 | |
| *** sobersabre has joined #openstack-infra | 14:15 | |
| *** rguillebert has joined #openstack-infra | 14:15 | |
| *** yamamoto has quit IRC | 14:16 | |
| *** EricGonczer_ has joined #openstack-infra | 14:16 | |
| openstackgerrit | Mateusz Matuszkowiak proposed openstack-infra/project-config: Added new SF repository for Fuel Ironic plugin integration https://review.openstack.org/209112 | 14:18 |
| *** geoffarnold has joined #openstack-infra | 14:19 | |
| openstackgerrit | Austin Anderson proposed openstack-infra/zuul: Jobs can now be configured to re-run if they fail https://review.openstack.org/209529 | 14:20 |
| *** soren has quit IRC | 14:21 | |
| *** geoffarnold has quit IRC | 14:22 | |
| *** fawadkhaliq has joined #openstack-infra | 14:22 | |
| *** jsavak has quit IRC | 14:23 | |
| *** geoffarnold has joined #openstack-infra | 14:23 | |
| *** olaph has joined #openstack-infra | 14:23 | |
| *** annegentle has joined #openstack-infra | 14:23 | |
| *** jsavak has joined #openstack-infra | 14:23 | |
| *** flepied has quit IRC | 14:24 | |
| dougwig | sdague pc_m mestery : there are reviews in flight where neutrons separate alembics register themselves on install, so I don't think hard coding it into grenade should happen. | 14:24 |
| *** mordred has quit IRC | 14:25 | |
| *** dims_ has joined #openstack-infra | 14:27 | |
| *** wenlock has joined #openstack-infra | 14:29 | |
| *** dims has quit IRC | 14:29 | |
| *** isaacb has joined #openstack-infra | 14:30 | |
| *** dkranz has quit IRC | 14:30 | |
| *** atuvenie_ has joined #openstack-infra | 14:31 | |
| *** dims has joined #openstack-infra | 14:31 | |
| openstackgerrit | Mateusz Matuszkowiak proposed openstack-infra/project-config: Added new SF repository for Fuel NSXV plugin integration https://review.openstack.org/209535 | 14:32 |
| *** atuvenie has quit IRC | 14:32 | |
| *** fhubik is now known as fhubik_afk | 14:33 | |
| *** dims_ has quit IRC | 14:33 | |
| *** MarkAtwood has joined #openstack-infra | 14:33 | |
| *** dims_ has joined #openstack-infra | 14:34 | |
| *** fhubik_afk is now known as fhubik | 14:35 | |
| *** tonytan4ever has joined #openstack-infra | 14:35 | |
| *** MarkAtwood_ has joined #openstack-infra | 14:35 | |
| flaper87 | AJaeger: https://review.openstack.org/#/c/209536/ | 14:36 |
| *** olaph has quit IRC | 14:36 | |
| AJaeger | thanks, flaper87. Will review later | 14:36 |
| *** dims has quit IRC | 14:37 | |
| *** olaph has joined #openstack-infra | 14:37 | |
| flaper87 | AJaeger: more a heads up but reviews are welcomed | 14:37 |
| * AJaeger likes the commit message already ;) | 14:38 | |
| *** dims has joined #openstack-infra | 14:38 | |
| *** vikram has quit IRC | 14:38 | |
| sileht | Hi, can someone can take a look to https://review.openstack.org/#/c/207298/ , it already have a +2 | 14:38 |
| *** AJaeger has quit IRC | 14:38 | |
| *** bapalm_ has quit IRC | 14:38 | |
| *** austin81 has joined #openstack-infra | 14:39 | |
| *** esker has quit IRC | 14:39 | |
| *** jcoufal has quit IRC | 14:39 | |
| *** esker has joined #openstack-infra | 14:40 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config: add gerrit groups for monasca-log https://review.openstack.org/208430 | 14:40 |
| *** dims_ has quit IRC | 14:40 | |
| *** jsavak has quit IRC | 14:42 | |
| *** afazekas_ has quit IRC | 14:43 | |
| *** dkranz has joined #openstack-infra | 14:43 | |
| *** afazekas_ has joined #openstack-infra | 14:44 | |
| *** jistr has quit IRC | 14:44 | |
| *** jistr has joined #openstack-infra | 14:44 | |
| *** sobersabre has quit IRC | 14:45 | |
| *** nadya has quit IRC | 14:45 | |
| *** stevemar has joined #openstack-infra | 14:46 | |
| *** bapalm_ has joined #openstack-infra | 14:46 | |
| *** abregman|brb is now known as abregman | 14:47 | |
| *** patrickeast has joined #openstack-infra | 14:47 | |
| *** jcoufal has joined #openstack-infra | 14:47 | |
| *** jcoufal has quit IRC | 14:48 | |
| *** jcoufal has joined #openstack-infra | 14:49 | |
| sileht | ianw, ^ | 14:49 |
| sileht | https://review.openstack.org/#/c/207298/ | 14:49 |
| *** jsavak has joined #openstack-infra | 14:49 | |
| akuznetsova | guys, hi, how can I re-run Jenkins gate jobs ? | 14:50 |
| *** sobersabre has joined #openstack-infra | 14:50 | |
| fungi | akuznetsova: the answer depends on the reason you need to rerun them. have an example? | 14:51 |
| *** jcoufal has quit IRC | 14:51 | |
| *** jcoufal has joined #openstack-infra | 14:52 | |
| *** jsavak has quit IRC | 14:53 | |
| akuznetsova | fungi, I got second +2 on my commit, one of Jenkins gate jobs failed because of one unit tests (I assume that it can be periodic fail) | 14:53 |
| *** afazekas_ has quit IRC | 14:54 | |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new kiloeyes repository for OpenStack monitoring service https://review.openstack.org/203259 | 14:54 |
| pabelanger | fungi: do you mind approving https://review.openstack.org/#/c/203759/ ? | 14:54 |
| *** witek has quit IRC | 14:55 | |
| fungi | akuznetsova: if there's a workflow +1 on it already, then just leave a review comment starting with the word "recheck" | 14:55 |
| *** maishsk has left #openstack-infra | 14:55 | |
| *** nadya has joined #openstack-infra | 14:55 | |
| fungi | akuznetsova: and it will enter the gate pipeline once the check pipeline succeeds | 14:55 |
| *** jsavak has joined #openstack-infra | 14:56 | |
| akuznetsova | fungi, thanks, I thought that it will run only Jenkins check jobs | 14:56 |
| *** thedodd has joined #openstack-infra | 14:56 | |
| SpamapS | Ng: thanks, that should do it. | 14:57 |
| fungi | akuznetsova: zuul sees a jenkins +1 comment as a potential trigger to enter the gate if other merge criteria are met (dependent changes already mergeable, change has at least one code review +2 and at least one workflow +1 and no workflow -1 or code review -2) | 14:57 |
| * SpamapS said 8 hours later. :) | 14:57 | |
| *** isaacb has quit IRC | 14:57 | |
| *** andrey-mp has joined #openstack-infra | 14:57 | |
| akuznetsova | fungi, thanks for an explanation | 14:58 |
| *** hdd has quit IRC | 14:58 | |
| andrey-mp | fungi: please, could you review our efforts on ec2api jobs? first https://review.openstack.org/#/c/208535/ (it has +2) we need to move further with testing ec2 with swift3 addon. and second - https://review.openstack.org/#/c/194670/ we need to restore our gating at all (because q-vpn has removed from defaults but our tests use this feature) | 14:59 |
| clarkb | StevenK: are you still awake? | 14:59 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Fix search for docs tox job in run-docs.sh https://review.openstack.org/209305 | 15:00 |
| *** jsavak has quit IRC | 15:00 | |
| andrey-mp | clarkb: maybe you could review our efforts? :) | 15:00 |
| *** mtanino has joined #openstack-infra | 15:00 | |
| mtreinish | fungi: ^^^ that should be a simple patch to review | 15:00 |
| clarkb | StevenK: if you happen to be around, curious about how you want to do the zanata upgrade. I should probably approve that change during hours that work for you and carlos? Let me know what you think | 15:00 |
| *** BrandonL08 has joined #openstack-infra | 15:00 | |
| *** esker has quit IRC | 15:00 | |
| *** hdd has joined #openstack-infra | 15:00 | |
| *** jsavak has joined #openstack-infra | 15:01 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Make gate-infra-puppet-apply-dsvm-centos7 voting https://review.openstack.org/209549 | 15:01 |
| clarkb | andrey-mp: whats with the unrelated changes on 208535? | 15:01 |
| *** hemna has joined #openstack-infra | 15:02 | |
| *** esker has joined #openstack-infra | 15:02 | |
| *** esker has quit IRC | 15:02 | |
| *** dangers_away is now known as dangers | 15:02 | |
| clarkb | pc_m: ^ so there are things that do use q-vpn, should they be using something else? | 15:02 |
| *** esker has joined #openstack-infra | 15:02 | |
| clarkb | andrey-mp: according to pc_m q-vpn support will be removed from devstack? I would check with pc_m what you should be doing there | 15:02 |
| andrey-mp | clarkb: Ian Wienandasked me to regroup new lines, so i regrouped them in other job also | 15:03 |
| fungi | ahh, yep they're not changed, simply reordered | 15:04 |
| andrey-mp | clarkb: q-vpn disabled here - https://review.openstack.org/#/c/208167/ | 15:04 |
| *** fdegir has quit IRC | 15:04 | |
| *** dtantsur has quit IRC | 15:04 | |
| clarkb | andrey-mp: yes I know I approved that change | 15:05 |
| clarkb | andrey-mp: but it was removed there so it could be removed from devstack | 15:05 |
| openstackgerrit | Merged openstack-infra/system-config: Add centos7 support for apply-test.sh https://review.openstack.org/203759 | 15:05 |
| clarkb | andrey-mp: which means you can't use it at all | 15:05 |
| clarkb | andrey-mp: so you should find out what the replacement is supposed to be | 15:05 |
| *** fdegir has joined #openstack-infra | 15:05 | |
| *** dtantsur has joined #openstack-infra | 15:06 | |
| *** yushiro has quit IRC | 15:06 | |
| andrey-mp | clarkb: q-vpn can be used as plugin (as we did) and it present in PROJECTS in devstack-gate`s definition | 15:06 |
| pabelanger | clarkb: fungi: Do you also mind pushing through https://review.openstack.org/#/c/209247/ today? I'm diving into nodepool logs and that would help greatly | 15:06 |
| pc_m | clarkb: Yes saw that they are using q-vpn in ec2-api. Suggested they use the devstack plugin, instead of enabling the service. | 15:06 |
| andrey-mp | as I understand devstack doesn't know about neutron-vpnaas plugin and we can enable it in our job definition | 15:07 |
| clarkb | ok so that enables neutron vpnaas not q-vpn | 15:08 |
| pc_m | clarkb: The neutron-vpnaas is the name of the devstack plugin for the neutron-vpnaas repo, which is the service for q-vpn. | 15:09 |
| andrey-mp | fungi: should I just rewrite commit message in 208535? | 15:09 |
| *** garyk has quit IRC | 15:09 | |
| fungi | andrey-mp: if you add a sentence to your commit message about the unrelated cleanup in the other job template that's probably sufficient | 15:09 |
| *** pal has joined #openstack-infra | 15:10 | |
| fungi | otherwise it looks like it might be there by mistake | 15:10 |
| *** flepied has joined #openstack-infra | 15:10 | |
| *** kurtmartin has joined #openstack-infra | 15:10 | |
| clarkb | pc_m: you don't name the service "neutron-vpn" or similar? seems like the perfect opportunity to finally drop the q | 15:10 |
| pc_m | clarkb: History, neutron had a q-vpn service. The adv services were split into their own repos - VPN to neutron-vpnaas. Recently, I made a devstack plugin for that repo, so that we could use the new plugin arch instead of hardcoding the service in devstack. | 15:10 |
| *** arxcruz has quit IRC | 15:10 | |
| clarkb | pc_m: ya I know, but I also know that we want to stop using the ancient 'q' is for quantum names or at least I thought we wanted to | 15:10 |
| *** dprince has quit IRC | 15:11 | |
| clarkb | (also I think devstack wanted to go verbose and move away from single letter service indicators) | 15:11 |
| *** yamahata has joined #openstack-infra | 15:11 | |
| pc_m | clarkb: Yeah, didn't get rid of that. | 15:11 |
| openstackgerrit | Andrey Pavlov proposed openstack-infra/project-config: Adding swift3 to list of projects for particular ec2 job https://review.openstack.org/208535 | 15:11 |
| pabelanger | Are we actually running thick_slave.pp these days? | 15:12 |
| clarkb | pabelanger: yes on all the bare nodes iirc | 15:12 |
| andrey-mp | fungi: i've made it. | 15:12 |
| fungi | pabelanger: that's what's used on bare-.* nodes, yes | 15:12 |
| pabelanger | okay, that explains it. We don't have bare-fedora21 | 15:12 |
| pc_m | clarkb: Was trying to keep in compatible for a while and modified devstack to accept either name. | 15:13 |
| fungi | pabelanger: i hope we can make the thick_slave class go away soon | 15:13 |
| pc_m | clarkb: I'm worried... as I see that heat, nodepool, and sahara yaml files in project config use the q-vpn name. Don't expect them to use the service, but I didn't expect ec2 to use it either. | 15:14 |
| *** radez is now known as radez_g0n3 | 15:15 | |
| clarkb | pc_m: this is what I was trying t ofigure out yesterday | 15:15 |
| *** marzif__ has quit IRC | 15:15 | |
| pc_m | clarkb: We may want to hold on 201119, until we are sure q-vpn is not used, or all uses have been switched (or removed). | 15:15 |
| *** afazekas_ has joined #openstack-infra | 15:16 | |
| *** nadya has quit IRC | 15:16 | |
| *** marzif__ has joined #openstack-infra | 15:16 | |
| *** armax has joined #openstack-infra | 15:16 | |
| clarkb | pabelanger: reviewed | 15:17 |
| pc_m | clarkb: I put 201119 into WIP state, until this is sorted out. | 15:17 |
| *** kurtmartin is now known as kmartin | 15:18 | |
| *** tonytan4ever has quit IRC | 15:21 | |
| *** pc_m is now known as pc_m_AWAY | 15:21 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add puppet-apply converage to both node types https://review.openstack.org/209558 | 15:21 |
| *** bswartz has joined #openstack-infra | 15:22 | |
| *** esker has quit IRC | 15:22 | |
| *** flepied has quit IRC | 15:22 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add puppet-apply converage to both node types https://review.openstack.org/209558 | 15:22 |
| *** esker has joined #openstack-infra | 15:23 | |
| *** esker has quit IRC | 15:23 | |
| *** sergmelikyan has quit IRC | 15:24 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add puppet-apply converage to both node types https://review.openstack.org/209558 | 15:24 |
| *** pal_ has joined #openstack-infra | 15:25 | |
| *** pilgrimstack1 has left #openstack-infra | 15:25 | |
| *** esker has joined #openstack-infra | 15:25 | |
| andrey-mp | fungi: thanks! | 15:25 |
| *** pal has quit IRC | 15:26 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add fedora support for apply-test.sh https://review.openstack.org/203756 | 15:26 |
| fungi | clarkb: if you get a sec, 207472 and 208937 are needed in preparation for the askbot trusty migration/redeploy | 15:28 |
| pabelanger | clarkb: thanks for review, checking it out now | 15:28 |
| *** bpokorny has joined #openstack-infra | 15:29 | |
| *** andrey-mp has quit IRC | 15:29 | |
| *** sobersabre has quit IRC | 15:29 | |
| *** sobersabre has joined #openstack-infra | 15:29 | |
| *** esker has quit IRC | 15:30 | |
| *** ashleighfarnham has joined #openstack-infra | 15:30 | |
| *** resker has joined #openstack-infra | 15:30 | |
| *** resker has quit IRC | 15:30 | |
| *** esker has joined #openstack-infra | 15:30 | |
| openstackgerrit | Merged openstack-infra/project-config: Add VPNaaS to ec2api Neutron gating jobs https://review.openstack.org/194670 | 15:31 |
| *** tsekiyama has joined #openstack-infra | 15:32 | |
| *** pal_ has quit IRC | 15:33 | |
| clarkb | fungi: 208937 has a confusing commit message | 15:33 |
| *** afazekas_ has quit IRC | 15:33 | |
| clarkb | did we disable these services to do the trusty upgrade or are we enabling services to do the trusty upgrade? | 15:33 |
| *** pal has joined #openstack-infra | 15:34 | |
| *** bapalm_ has quit IRC | 15:34 | |
| *** hdd has quit IRC | 15:36 | |
| *** gyee has joined #openstack-infra | 15:36 | |
| fungi | clarkb: the module needed a heavy refactor which was going to break production, so the commit being reverted commented out use of the module for the prod server only | 15:37 |
| fungi | then it was proven viable with the newer askbot release on the ask-staging server | 15:37 |
| fungi | running on trusty instead of precise | 15:37 |
| *** Sukhdev has joined #openstack-infra | 15:37 | |
| fungi | now that we're ready to switch to the newer askbot version and newer ubuntu release in production, it's being brought into sync with how ask-staging is puppeted | 15:38 |
| clarkb | fungi: so is it safe to approve or will this change merge need to be coordinated? | 15:39 |
| fungi | clarkb: safe to approve since, per my comment, puppet agent is disabled on the current ask.o.o and will only be enabled on the replacement ask.o.o i've built | 15:40 |
| *** esker has quit IRC | 15:40 | |
| clarkb | kk | 15:40 |
| *** dtantsur has quit IRC | 15:40 | |
| fungi | hoping to schedule an outage rsn to do the migration and then we can delete the old precise-based ask server | 15:40 |
| fungi | and turn google authentication back on for people who had been using it | 15:41 |
| *** dtantsur has joined #openstack-infra | 15:41 | |
| fungi | and work out the remaining details for the integrated akismet support | 15:41 |
| fungi | which seems to be their answer to combatting spam posts | 15:41 |
| fungi | er, combating | 15:42 |
| fungi | unfortunately the support in puppet-askbot for trusty and newer askbot versions couldn't sanely coexist. water under the bridge from a couple months ago now https://review.openstack.org/181873 | 15:44 |
| clarkb | fungi: and it is a revert because we jumped the gun at some point and had to undo that? | 15:44 |
| clarkb | I think "revert" is what is really tripping me up here | 15:45 |
| clarkb | anyways approved | 15:45 |
| fungi | basically we temporarily commented out use of the ::askbot puppet module in production, and this change reverts the commenting out | 15:45 |
| clarkb | apparently I have +2 on puppet-refstack not sure how that happened | 15:46 |
| openstackgerrit | Merged openstack-infra/system-config: Add solr dist-root to askbot-staging https://review.openstack.org/207472 | 15:46 |
| fungi | basically the askbot module was being changed in non-backward-compatible ways to cope with poor/nonexistent release process within askbot itself | 15:47 |
| *** olaph has quit IRC | 15:47 | |
| clarkb | fungi: I see and simplest way to deal with that was to stop puppeting it on the old srever | 15:47 |
| fungi | right | 15:48 |
| *** esker has joined #openstack-infra | 15:48 | |
| fungi | and the hope is that now that's settled, it can be continuously deployed/upgraded in a more seamless fashion | 15:48 |
| fungi | this would be a "2.0.0" release if we were already semver'ing it | 15:49 |
| *** tonytan4ever has joined #openstack-infra | 15:49 | |
| clarkb | pabelanger: you around to help debug 203729 if it breaks? I want to approve it now if so | 15:50 |
| *** sdake has joined #openstack-infra | 15:50 | |
| *** ftersin has left #openstack-infra | 15:50 | |
| pabelanger | clarkb: yup, here and ready | 15:50 |
| fungi | clarkb: why wouldn't you have +2 on puppet-refstack? it's an infra module | 15:50 |
| clarkb | fungi: its an infra module? | 15:51 |
| *** garyk has joined #openstack-infra | 15:51 | |
| fungi | http://git.openstack.org/cgit/openstack-infra/puppet-refstack/ | 15:51 |
| clarkb | I must've missed when all that happened | 15:51 |
| clarkb | ya but refstack is stackforge and random thing over there | 15:51 |
| *** AJaeger has joined #openstack-infra | 15:51 | |
| clarkb | we don't even control the domain name iirc | 15:51 |
| fungi | clarkb: we're working on deploying the service on an infra server | 15:52 |
| *** geoffarnold has quit IRC | 15:52 | |
| fungi | then refstack.org will end up redirecting to refstack.openstack.org once it's running | 15:52 |
| AJaeger | clarkb: good morning. Could you review https://review.openstack.org/#/c/209419/ , please? The recent transifex change did break something ;( | 15:52 |
| fungi | clarkb: http://specs.openstack.org/openstack-infra/infra-specs/specs/refstack_dot_org.html | 15:52 |
| fungi | probably happened while you were otherwise indisposed | 15:52 |
| GheRivero | fungi: you mean refstack.net? | 15:52 |
| fungi | GheRivero: whichever | 15:53 |
| clarkb | AJaeger: sure | 15:53 |
| fungi | GheRivero: i think refstack.net is a temporary testbed, but i haven't been following closely | 15:53 |
| GheRivero | fungi: ok. thanks | 15:53 |
| fungi | clarkb: https://review.openstack.org/198869 would add the server, though it still needs some work (i started the change but haven't had time to follow through with it) | 15:54 |
| openstackgerrit | Merged openstack-infra/system-config: Revert the ask.o.o resources disabled by trusty upgrade https://review.openstack.org/208937 | 15:54 |
| *** olaph has joined #openstack-infra | 15:55 | |
| *** yamamoto has joined #openstack-infra | 15:55 | |
| pabelanger | crinkle: do you mind reviewing: https://review.openstack.org/#/c/203164/8 | 15:56 |
| *** amitgandhinz has quit IRC | 15:56 | |
| clarkb | AJaeger: did they do the rename on the server side already? so this all maps properly? | 15:56 |
| openstackgerrit | Merged openstack-infra/puppet-refstack: Add config.json file generation https://review.openstack.org/203764 | 15:56 |
| *** EricGonczer_ has quit IRC | 15:56 | |
| AJaeger | clarkb: transifex did the rename for us - and I followed it in the script. | 15:56 |
| AJaeger | We could rename it to something different as well but they are in sync with the change | 15:57 |
| *** bapalm_ has joined #openstack-infra | 15:57 | |
| clarkb | AJaeger: sounds good /me approves | 15:57 |
| crinkle | pabelanger: +1 | 15:57 |
| AJaeger | thanks, clarkb ! | 15:57 |
| openstackgerrit | Christian Berendt proposed openstack-infra/groups: Add a basic Vagrantfile for a development environment https://review.openstack.org/209577 | 15:57 |
| *** Ala has quit IRC | 15:57 | |
| *** esker has quit IRC | 15:57 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Switch ask.openstack.org to Ubuntu Trusty https://review.openstack.org/209579 | 15:58 |
| fungi | clarkb: ^ i noticed that as forgotten | 15:58 |
| fungi | er, was | 15:58 |
| clarkb | fungi: +2 as soon as my internet connection allows me to | 15:58 |
| *** esker has joined #openstack-infra | 15:59 | |
| *** ifarkas has quit IRC | 15:59 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Partial migrate to puppet-httpd module https://review.openstack.org/198819 | 15:59 |
| fungi | not urgent since the staging server was already being tested for trusty anyway, but more accurate | 15:59 |
| *** dims_ has joined #openstack-infra | 16:00 | |
| *** maishsk has joined #openstack-infra | 16:00 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Remove puppet-apache now that we have migrated https://review.openstack.org/205696 | 16:00 |
| fungi | clarkb: we've got a preliminary migration plan going at https://etherpad.openstack.org/p/askbot-trusty-migration | 16:00 |
| pabelanger | crinkle: danke | 16:00 |
| clarkb | pabelanger: out of curiousity with ntp why only go to 3.2.1 and not 3.3.0 or 4.x? | 16:01 |
| *** ildikov has quit IRC | 16:01 | |
| *** flepied has joined #openstack-infra | 16:01 | |
| *** jschwarz has quit IRC | 16:01 | |
| *** ayoung is now known as ayoung-lunch | 16:01 | |
| pabelanger | clarkb: that looked to be the minimual bump needed to get fedora22 support | 16:01 |
| openstackgerrit | Merged openstack-infra/project-config: Fix search for docs tox job in run-docs.sh https://review.openstack.org/209305 | 16:01 |
| *** qb_ has joined #openstack-infra | 16:01 | |
| pabelanger | clarkb: I didn't want to refactor stuff right now to get the newer version | 16:01 |
| *** puranamr has joined #openstack-infra | 16:01 | |
| pabelanger | mind you, we just do include ::ntp, so it should be painless | 16:01 |
| *** dims__ has joined #openstack-infra | 16:02 | |
| clarkb | pabelanger: ok, looking at changelogs I don't think we have to refactor anything | 16:02 |
| clarkb | pabelanger: the reason for the semver major version bump is puppet 2.7 support was dropped | 16:02 |
| pabelanger | clarkb: Ya, that is what I seen | 16:02 |
| clarkb | in any case we should be 3.2.1 in aout an hour | 16:02 |
| pabelanger | ack | 16:02 |
| *** puranamr has quit IRC | 16:02 | |
| *** dims has quit IRC | 16:02 | |
| *** puranamr has joined #openstack-infra | 16:03 | |
| *** fhubik has quit IRC | 16:03 | |
| *** EricGonczer_ has joined #openstack-infra | 16:03 | |
| *** jlanoux has quit IRC | 16:03 | |
| *** sshnaidm has quit IRC | 16:04 | |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new kiloeyes repository for OpenStack monitoring service https://review.openstack.org/203259 | 16:04 |
| *** dims_ has quit IRC | 16:04 | |
| *** sharikapongubala has joined #openstack-infra | 16:05 | |
| qb_ | hi there, can anyone summarize the current recommendation as to how to go about setting up a complex third party CI? | 16:05 |
| *** sharikapongubala has quit IRC | 16:05 | |
| qb_ | last I checked, Ramy from HP was working on making the code more easily consumable. Is that still the route we should take? | 16:05 |
| *** abregman has quit IRC | 16:05 | |
| *** alexpilotti has quit IRC | 16:06 | |
| Clint | qb_: http://specs.openstack.org/openstack-infra/infra-specs/specs/openstackci.html | 16:06 |
| *** salv-orlando has joined #openstack-infra | 16:06 | |
| asselin | qb_, hi that solution is still work in progress | 16:06 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Make gate-infra-puppet-apply-dsvm-(centos7|f21) voting https://review.openstack.org/209549 | 16:06 |
| *** baoli has quit IRC | 16:06 | |
| openstackgerrit | Merged openstack-infra/project-config: Fix translation sync with Transifex https://review.openstack.org/209419 | 16:06 |
| qb_ | hi Ramy, now that we have some headcount assigned, would it make sense for us to pitch in and help you? | 16:06 |
| openstackgerrit | litong01 proposed openstack-infra/project-config: Add new kiloeyes repository for OpenStack monitoring service https://review.openstack.org/203259 | 16:06 |
| *** erlon has joined #openstack-infra | 16:07 | |
| asselin | qb_, sure! | 16:07 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/project-config: Run beaker jobs on puppet-openstack_extras https://review.openstack.org/209141 | 16:07 |
| *** buzzco has quit IRC | 16:07 | |
| asselin | qb_ give me a few mins | 16:07 |
| *** jsavak has quit IRC | 16:08 | |
| *** alexsyip has joined #openstack-infra | 16:08 | |
| *** jsavak has joined #openstack-infra | 16:08 | |
| openstackgerrit | Brandon Leonard proposed openstack-infra/jenkins-job-builder: Adds view functionality https://review.openstack.org/206178 | 16:08 |
| greghaynes | ianw: You may like https://review.openstack.org/#/c/209201/ | 16:08 |
| greghaynes | should make things a little better | 16:09 |
| *** ddieterly has quit IRC | 16:09 | |
| pabelanger | greghaynes: cool, I was about to start getting fedora22 dibs going again local | 16:09 |
| *** stevemar has quit IRC | 16:10 | |
| asselin | qb_, Clint we're tracking the stories here: https://storyboard.openstack.org/#!/story/2000101 | 16:10 |
| *** otter768 has joined #openstack-infra | 16:10 | |
| *** AJaeger has quit IRC | 16:10 | |
| openstackgerrit | Merged openstack-infra/system-config: Bump puppet-ntp to 3.2.1 https://review.openstack.org/203729 | 16:11 |
| *** dims has joined #openstack-infra | 16:11 | |
| *** stevemar has joined #openstack-infra | 16:11 | |
| *** radez_g0n3 is now known as radez | 16:12 | |
| asselin | qb_, Clint we have this etherpad to help track all the patches: https://etherpad.openstack.org/p/common-ci-sprint | 16:13 |
| *** dims__ has quit IRC | 16:13 | |
| *** tsg has joined #openstack-infra | 16:13 | |
| *** amitgandhinz has joined #openstack-infra | 16:13 | |
| asselin | qb_, Clint right now the focus is on reviewing the nodepool patches. | 16:13 |
| *** mrmartin has joined #openstack-infra | 16:14 | |
| asselin | qb_, it's more than a simple refactor. It also includes restructuring the nodepool.yaml file so that the private credentials are removed and put into a separate file. | 16:15 |
| *** otter768 has quit IRC | 16:15 | |
| asselin | qb_, Clint then there's a patch I'm working on to create a sample single-node deployment that 3rd party folks can use/customize. | 16:16 |
| pabelanger | nibalizer: do you think you could review https://review.openstack.org/#/c/203164/8 today? Like to get that merged to continue work on fedora22 dibs | 16:16 |
| pabelanger | nibalizer: and follow up patch: https://review.openstack.org/#/c/202884/ | 16:16 |
| asselin | Clint, sorry for not getting back to you yesterday. hopefully now's a good time ^^ :) | 16:17 |
| openstackgerrit | Merged openstack-infra/system-config: Add doc on contributing cloud resources https://review.openstack.org/206247 | 16:18 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool: Expand documentation on timeouts https://review.openstack.org/203727 | 16:19 |
| *** maishsk_ has joined #openstack-infra | 16:19 | |
| *** maishsk has quit IRC | 16:19 | |
| *** maishsk_ is now known as maishsk | 16:19 | |
| *** unicell has quit IRC | 16:19 | |
| *** pal has quit IRC | 16:19 | |
| *** john-davidge has left #openstack-infra | 16:22 | |
| *** dtantsur is now known as dtantsur|afk | 16:23 | |
| *** jistr has quit IRC | 16:23 | |
| *** esker has quit IRC | 16:23 | |
| *** patrickeast has quit IRC | 16:23 | |
| *** jistr has joined #openstack-infra | 16:23 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add grafana node into cacti.o.o https://review.openstack.org/209591 | 16:24 |
| *** jistr has quit IRC | 16:24 | |
| *** ub has quit IRC | 16:25 | |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-dashboard: This module is no longer supported https://review.openstack.org/207603 | 16:25 |
| *** patrickeast has joined #openstack-infra | 16:26 | |
| *** dannywilson has joined #openstack-infra | 16:26 | |
| *** asselin has quit IRC | 16:26 | |
| *** pcaruana has quit IRC | 16:26 | |
| pabelanger | clarkb: fungi: is there any reason why jenkins.o.o is not part of the ssl_cert_check domains? It seems to be a self-signed cert | 16:26 |
| *** asselin has joined #openstack-infra | 16:26 | |
| clarkb | pabelanger: because it is a self signed cert | 16:27 |
| *** esker has joined #openstack-infra | 16:27 | |
| clarkb | pabelanger: the primary interface to all that is gerrit -> logs.o.o we try to avoid exposing jenkins directly as we now have 8 of them | 16:27 |
| pabelanger | clarkb: Makes sense, I _thought_ we had a valid cert for it. Guess we don't any more | 16:28 |
| clarkb | we did in the past since historically we had a single jenkins and the cert was still good for it. but we let it expire and replcaed it | 16:28 |
| *** geoffarnold has joined #openstack-infra | 16:29 | |
| *** openstackgerrit_ has joined #openstack-infra | 16:30 | |
| clarkb | pabelanger: I am worried about the '' -> undef change here https://review.openstack.org/#/c/203164/8/manifests/jenkinsuser.pp | 16:31 |
| clarkb | they have completely different semantic meanings | 16:31 |
| clarkb | and the reason for using '' was to actually override defaults iirc | 16:31 |
| clarkb | undef does not do that | 16:31 |
| *** puranamr has quit IRC | 16:31 | |
| pabelanger | clarkb: my understanding is puppet4 changes the way '' string are handled. So, what if we removed the default and expected a user to pass something? | 16:33 |
| clarkb | pabelanger: puppet4 does but we are using puppet3 today | 16:34 |
| *** thorst has quit IRC | 16:34 | |
| *** BrandonL08 has quit IRC | 16:34 | |
| pabelanger | clarkb: well, if we bring a fedora22 node online, it will have puppet4. That is the reason for this change to python-jenkins | 16:34 |
| clarkb | pabelanger: sure but my point is this must be compatible with puppet3 | 16:35 |
| *** ddieterly has joined #openstack-infra | 16:35 | |
| pabelanger | clarkb: agreed | 16:35 |
| clarkb | and I am not ocnvinced that that change is puppet3 safe given the way the modules are written | 16:35 |
| pabelanger | clarkb: to be honest, crinkle can likely explain the changes better. Since most of that Gemfile is from other puppet modules I referenced for puppet-4 | 16:36 |
| clarkb | because now any defaults will be chosen over our overrides | 16:36 |
| clarkb | because undef will not override defaults | 16:36 |
| pabelanger | clarkb: so, why do we want jenkinsuser::ssk_key to be empty? | 16:36 |
| clarkb | pabelanger: we don't | 16:36 |
| zaro | morning | 16:36 |
| clarkb | pabelanger: we us ethat as a null/nil/none/undefined value that actually works | 16:36 |
| *** jsavak has quit IRC | 16:36 | |
| clarkb | undef does not work | 16:36 |
| pabelanger | clarkb: Hmm... do we actually leave it as ''? | 16:37 |
| pabelanger | checking scripts to see | 16:37 |
| clarkb | pabelanger: no I think there is a check to see if not '' then do thing | 16:37 |
| *** BharatK has joined #openstack-infra | 16:38 | |
| *** dprince has joined #openstack-infra | 16:38 | |
| *** olaph has quit IRC | 16:38 | |
| crinkle | clarkb: we look for those and correct them | 16:39 |
| zaro | anybody know how to override a python dependency to reference a local git checkout from the requirements.txt file? | 16:39 |
| *** olaph has joined #openstack-infra | 16:39 | |
| *** esker has quit IRC | 16:39 | |
| nibalizer | I mean we can run f22 on puppet 3 | 16:39 |
| nibalizer | until the fleet is ready to go puppet 4 thats what I expect we would do | 16:39 |
| nibalizer | I mean we need an apply test for puppet 4 before anything else really | 16:40 |
| nibalizer | pabelanger: reviewed | 16:40 |
| clarkb | nibalizer: yes the point is puppet3 must still work | 16:40 |
| jeblair | zaro: i just activate a venv, install the local dependency, then install the thing that uses it. the dep will already be satisfied so it won't try to install from pypi | 16:40 |
| clarkb | ignore puppet4 its not an issue | 16:40 |
| clarkb | the issue is "will this change break puppet3?" | 16:40 |
| crinkle | is there a case where you're not currently passing a value? because I would expect that to be broken before this change | 16:41 |
| *** esmute_ has joined #openstack-infra | 16:41 | |
| clarkb | crinkle: the cases I expect to break are where a module has a default value we went to unset to nil | 16:41 |
| clarkb | crinkle: because undef cannot do that in puppet3 | 16:42 |
| nibalizer | clarkb: yea there are a lot of places where infra module parameters default to '' which doesn't make sense, those should be required parameters | 16:42 |
| clarkb | nibalizer: yes I agree that we can clean those up | 16:42 |
| jesusaurus | and there are changes to clean a lot of those up | 16:42 |
| clarkb | nibalizer: but we aren't really doing that here, instead we are setting them to undef and not making them required | 16:43 |
| *** ildikov has joined #openstack-infra | 16:43 | |
| reed | Shrews, do you maintain a list of clouds that are regularly tested with shade? I'm addressing Tom's comments to my patch | 16:44 |
| jeblair | clarkb or fungi: can you review https://review.openstack.org/201662 ? (we may be about to change some of it for the stackforge thing, but i'd like to land it first) | 16:44 |
| openstackgerrit | Merged openstack-infra/infra-manual: Fix one typo on infrastructure manual https://review.openstack.org/206004 | 16:44 |
| Shrews | reed: I only have access to 3 cloud providers. mordred has access to much more (i think because he pays for them). | 16:45 |
| reed | Shrews, for you those are HP, RAX and DreamHost? | 16:46 |
| Shrews | reed: correct. we don't have any sort of automated testing against them | 16:46 |
| reed | Shrews, OVH and citycloud have free tier accounts, too, btw | 16:46 |
| Shrews | oh? | 16:47 |
| *** dmorita has joined #openstack-infra | 16:48 | |
| *** tonytan4ever has quit IRC | 16:48 | |
| *** sputnik13 has joined #openstack-infra | 16:48 | |
| *** ayoung-lunch is now known as ayoung | 16:49 | |
| openstackgerrit | Merged openstack-infra/infra-manual: Add documentation for project renames https://review.openstack.org/201662 | 16:49 |
| pabelanger | clarkb: so, checking our puppet modules, if we require ssh_key then nothing will break. We pass '' into it from ::simple_slave and ::slave. If that doesn't work, I'm open to other suggestions to get the lint-checks to pass | 16:50 |
| *** thorst has joined #openstack-infra | 16:51 | |
| *** unicell has joined #openstack-infra | 16:51 | |
| clarkb | pabelanger: requiring instead wfm | 16:52 |
| *** rguillebert has quit IRC | 16:52 | |
| pabelanger | clarkb: ack | 16:52 |
| clarkb | as catalog compiles fail early rater than late | 16:52 |
| crinkle | so what i've been doing is things like https://review.openstack.org/#/c/205323 before https://review.openstack.org/#/c/205323 to fix those things | 16:53 |
| *** maishsk_ has joined #openstack-infra | 16:53 | |
| jeblair | crinkle: those are the same link, right? | 16:53 |
| crinkle | er | 16:53 |
| openstackgerrit | Merged openstack-infra/infra-manual: Expand on directions for creating drivers team https://review.openstack.org/187212 | 16:53 |
| crinkle | https://review.openstack.org/#/c/205323 before https://review.openstack.org/#/c/205324/ is what i meant | 16:53 |
| *** maishsk has quit IRC | 16:54 | |
| *** maishsk_ is now known as maishsk | 16:54 | |
| *** puranamr has joined #openstack-infra | 16:54 | |
| *** scheuran has quit IRC | 16:55 | |
| *** sharikapongubala has joined #openstack-infra | 16:55 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-jenkins: Add Gemfile and puppet 4 checks https://review.openstack.org/203164 | 16:55 |
| openstackgerrit | Merged openstack-infra/infra-manual: Mention license in document footer https://review.openstack.org/167799 | 16:55 |
| *** ybathia has joined #openstack-infra | 16:55 | |
| *** sabeen1 has joined #openstack-infra | 16:55 | |
| *** pc_m_AWAY is now known as pc_m | 16:57 | |
| crinkle | clarkb: pabelanger if we're going to make $ssh_key required in that patch it seems sort of inconsistent to leave $username and $password undef https://review.openstack.org/#/c/203164/9/manifests/job_builder.pp | 16:57 |
| jeblair | clarkb, fungi: here's the order of events from the nodepool node-reuse we saw the other day: | 16:58 |
| *** sdake has quit IRC | 16:58 | |
| jeblair | periodic cleanup thread starts; node 1 is launched; node 1 is ready; periodic cleanup thread finds instance 1 in nova list, but is unable to find it in the local database; periodic cleanup thread deletes instance 1; node 1 is added to jenkins; periodic cleanup thread terminates; much later: node 2 is launched with the same ip, and added to jenkins and runs a job normally, but since there are two jenkins computers for that ip, it runs two jo | 16:58 |
| pabelanger | crinkle: ya, I was just about to look them over to see if they could be changed to required too | 16:58 |
| jeblair | i'm stumped: i can not find any reason that nodepool would not have been able to find the database record for that node id in the periodic cleanup thread. | 16:58 |
| clarkb | crinkle: we should do it across the boad where it makes sense | 16:59 |
| clarkb | the ssl certs are likely the only odd ones | 16:59 |
| mrmartin | fungi: re | 17:00 |
| crinkle | pabelanger: clarkb imo i would do it in a prerequisite patch | 17:00 |
| fungi | mrmartin: i've always wanted to ask, what does "re" mean in this context? | 17:00 |
| *** bapalm_ has quit IRC | 17:00 | |
| fungi | i see you say it all the time when you join the channel | 17:00 |
| mrmartin | fungi: http://www.ircbeginner.com/ircinfo/abbreviations.html | 17:01 |
| mrmartin | :) | 17:01 |
| jeblair | fungi: 'hi again' | 17:01 |
| Clint | asselin: ah, how 'bout now | 17:01 |
| *** sputnik13 has quit IRC | 17:01 | |
| *** hdd has joined #openstack-infra | 17:01 | |
| fungi | weird, i never knew that one. also the definition listed there seems to be recursive | 17:02 |
| mrmartin | anyway, the ask-staging solr smartcn fails after the solr-dist patch approval: http://puppetboard.openstack.org/report/ask-staging.openstack.org/f32b90f96aa7389dc7213cda7041d5132a68b864 | 17:02 |
| *** sputnik13 has joined #openstack-infra | 17:02 | |
| *** crc32 has joined #openstack-infra | 17:02 | |
| jeblair | i can confirm its use on irc back to the mid nineties at least :) | 17:02 |
| fungi | mrmartin: but it worked in your test environment? | 17:02 |
| fungi | jeblair: likely i just ignored it as people hitting random keys on the keyboard | 17:03 |
| mrmartin | and it seems to be we need to move away the WEB-INF directory of solr to retrigger the download / extract of solr packages to proper directory: https://github.com/vamsee/puppet-solr/blob/e2dd66d8e45dd44dfe2ba570822c0a681cb59ebb/manifests/config.pp | 17:03 |
| jeblair | fungi: i guess that could happen a lot dependon on the channel you're in | 17:03 |
| mrmartin | this line exactly: https://github.com/vamsee/puppet-solr/blob/e2dd66d8e45dd44dfe2ba570822c0a681cb59ebb/manifests/config.pp#L47 | 17:03 |
| fungi | mrmartin: is that going to be a recurrent issue do you think, or just a missing transition in the puppet module to support the new location? | 17:03 |
| mrmartin | fungi, don't you mind if I move away the WEB-INF on ask-staging and see what happens next? this one won't be a problem on a new deployment | 17:04 |
| * jeblair just invented the word 'dependon' whose definition should be self explanatory | 17:04 | |
| fungi | jeblair: is it a noun? | 17:04 |
| fungi | mrmartin: since the replacement server was deployed before that change merged, isn't it likely we'll run into it there too? | 17:05 |
| mrmartin | yeah, the upstream module not supports this type of transitions | 17:05 |
| *** openstackgerrit_ has quit IRC | 17:05 | |
| pabelanger | nibalizer: mind providing some code for the versioncmp logic? not sure I follow | 17:05 |
| *** abitha has joined #openstack-infra | 17:05 | |
| *** mquin has joined #openstack-infra | 17:05 | |
| jeblair | fungi: adjective phrase | 17:06 |
| mrmartin | fungi: that's a good a question, the upstream solr puppet not supports any change after a successful deployment. | 17:06 |
| fungi | mrmartin: okay, i've moved /usr/share/solr/WEB-INF out of the way on ask-staging. let's see how it goes | 17:07 |
| greghaynes | The dependons sounds like an alien race in a scifi film | 17:07 |
| SpamapS | greghaynes: so, lets try to deploy new bifrost nodes today. What do you think? | 17:07 |
| greghaynes | SpamapS: ! | 17:07 |
| greghaynes | SpamapS: yes | 17:07 |
| greghaynes | SpamapS: you mean replace bastion? | 17:07 |
| SpamapS | greghaynes: I'm kind of giving up on the hardware remediation.. so lets just get this thing rocking and we'll fight the noc later. | 17:08 |
| mrmartin | fungi: ok, waiting for the next of run of puppet agent there | 17:08 |
| *** pvaneck has joined #openstack-infra | 17:08 | |
| SpamapS | greghaynes: yeah, first question I have: was I just lucky in picking ic-006 to test ssh'ing to the ilo's from, or have they messed up and given the entire 10.10.16.128/26 access to the ilos? | 17:08 |
| greghaynes | SpamapS: I suspect the latter | 17:09 |
| *** esker has joined #openstack-infra | 17:09 | |
| SpamapS | me too | 17:09 |
| greghaynes | SpamapS: but unsure | 17:09 |
| *** sharikapongubala has quit IRC | 17:09 | |
| *** esker has quit IRC | 17:09 | |
| *** sharikapongubala has joined #openstack-infra | 17:09 | |
| openstackgerrit | Davide Agnello proposed openstack-infra/project-config: Adding RALLY_SCENARIO venv for Cue job https://review.openstack.org/201285 | 17:09 |
| SpamapS | greghaynes: so whatever happens, lets make a static inventory, pick two boxes that we know work, put them on addresses that are not in the general population with the others, and call those "the hardware cloud" | 17:10 |
| *** esker has joined #openstack-infra | 17:10 | |
| greghaynes | SGTM | 17:10 |
| SpamapS | greghaynes: also lets start backing up our inventory. ;) | 17:10 |
| *** nadya has joined #openstack-infra | 17:10 | |
| greghaynes | SpamapS: Yeeep. Where to do that do it is a wierd question | 17:11 |
| greghaynes | SpamapS: maybe to hp2? | 17:11 |
| SpamapS | http://docs.openstack.org/infra/system-config/sysadmin.html#backups | 17:11 |
| SpamapS | thats the permanent backup home | 17:11 |
| SpamapS | but we're not there yet | 17:11 |
| jeblair | SpamapS: the inventory has private info? | 17:11 |
| *** weshay_xchat has joined #openstack-infra | 17:11 | |
| greghaynes | yes | 17:11 |
| *** dims has quit IRC | 17:11 | |
| *** tongli has joined #openstack-infra | 17:12 | |
| *** dims has joined #openstack-infra | 17:12 | |
| clarkb | ansible inventory? | 17:12 |
| greghaynes | Its out ipmi creds, macs, etc inventory | 17:12 |
| greghaynes | s/out/our | 17:12 |
| zaro | jeblair: thanks. i usually do it like that as well but i was hoping that there would be a way to reference locally from requirements.txt so i can just run with tox and let it work its magic. | 17:12 |
| *** nadya has quit IRC | 17:13 | |
| *** e0ne has quit IRC | 17:13 | |
| jeblair | is it the nodes.json file we also have on the puppetmaster in the infra-cloud directory? | 17:13 |
| jeblair | [{"pm_password": ..., "mac": ["..."], .... ? | 17:13 |
| greghaynes | that looks like a form of it, yes | 17:13 |
| greghaynes | the source of truth for that right now is a directory of yaml files, we have to compile taht into bifrost format which is what that looks like | 17:14 |
| *** jamespage_ has joined #openstack-infra | 17:14 | |
| jeblair | i'd love it if the yaml files could go in system-config | 17:14 |
| jeblair | it's pretty much just the ipmi passwords we want to secret, right? | 17:15 |
| greghaynes | I'm not sure if were ok with the ipmi creds being public | 17:15 |
| greghaynes | yes | 17:15 |
| *** jaypipes has quit IRC | 17:15 | |
| greghaynes | Those are mostly homogenous... | 17:15 |
| jeblair | what are your thoughts about making them the same for all the hosts? | 17:15 |
| greghaynes | :) | 17:15 |
| SpamapS | jeblair: so, I think we can put all but the pm network info/user/password in system-config. We could very easily just have that stuff in hiera. | 17:15 |
| jeblair | then we would have a bunch of yaml that can be public, and one secret we can stick in hiera | 17:15 |
| greghaynes | I like the homogenize and keep the one secret separate idea | 17:16 |
| SpamapS | jeblair: that requires use of the proprietary ilo cli tools to automate setting. | 17:16 |
| *** fedora has joined #openstack-infra | 17:16 | |
| greghaynes | SpamapS: You cant change pass from ssh prompt? | 17:16 |
| SpamapS | nope | 17:16 |
| *** esker has quit IRC | 17:16 | |
| greghaynes | WHAT | 17:16 |
| greghaynes | :_( | 17:16 |
| *** monester has quit IRC | 17:17 | |
| *** dannywilson has quit IRC | 17:17 | |
| *** SumitNaiksatam has joined #openstack-infra | 17:17 | |
| *** weshay_xchat has quit IRC | 17:17 | |
| *** olaph has quit IRC | 17:17 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-jenkins: Require ssh_key, username and password https://review.openstack.org/209607 | 17:17 |
| SpamapS | hm | 17:18 |
| *** skylerberg has joined #openstack-infra | 17:18 | |
| greghaynes | SpamapS: We dont have a 10.x route on .2 anymore | 17:18 |
| greghaynes | Is that intentional? | 17:19 |
| jeblair | what's the current bastion? | 17:19 |
| SpamapS | greghaynes: hah I just found that | 17:19 |
| greghaynes | :) | 17:19 |
| SpamapS | greghaynes: it was the em3->eth0 problem | 17:19 |
| SpamapS | fixed | 17:19 |
| nibalizer | pabelanger: my thought is if you set the default on $version to 'latest' then you can if $version == 'latest' { ... | 17:19 |
| nibalizer | that way you avoid having to use version compare | 17:19 |
| greghaynes | jeblair: stack@15.184.52.2 | 17:19 |
| SpamapS | we haven't infra-puppetized it yet | 17:20 |
| SpamapS | as it is marked for death. ;) | 17:20 |
| *** esker has joined #openstack-infra | 17:20 | |
| jeblair | greghaynes: can you throw my key on it? https://launchpad.net/~corvus/+sshkeys | 17:20 |
| *** esker has quit IRC | 17:21 | |
| mrmartin | fungi: ask-staging looks better now: http://puppetboard.openstack.org/report/ask-staging.openstack.org/f6e7820ab841a85fcf90d4e56a7cb3b3135c90d1 | 17:21 |
| * SpamapS notes 'ssh-import-id corvus' will work | 17:21 | |
| jeblair | SpamapS: dude | 17:21 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-jenkins: Add Gemfile and puppet 4 checks https://review.openstack.org/203164 | 17:21 |
| SpamapS | jeblair: sweet? | 17:21 |
| jeblair | SpamapS: sweeet. | 17:22 |
| SpamapS | it works w/ github too | 17:22 |
| jeblair | bah | 17:22 |
| greghaynes | jeblair: added | 17:22 |
| SpamapS | it assumes lp, but 'ssh-import-id gh:xxxx' works too | 17:22 |
| jeblair | greghaynes: thx | 17:22 |
| nibalizer | wow TIL | 17:22 |
| jeblair | i am not corvus there so do not care | 17:22 |
| greghaynes | SpamapS: so, I found the property in the ilo ssh prompt.. | 17:22 |
| greghaynes | I kind of want to try writing to it | 17:22 |
| SpamapS | greghaynes: ? | 17:22 |
| greghaynes | for ilo pass | 17:23 |
| jeblair | greghaynes: you are ahead of me :) | 17:23 |
| SpamapS | oh the set thing | 17:23 |
| greghaynes | yes | 17:23 |
| SpamapS | so expect script then? | 17:23 |
| greghaynes | yes, which I already have written :) | 17:23 |
| greghaynes | because of NOC inventory madness | 17:23 |
| SpamapS | look at the big brain on Greg! | 17:23 |
| greghaynes | haha | 17:23 |
| jeblair | do you actually need expect? | 17:23 |
| greghaynes | yes :( | 17:23 |
| fungi | mrmartin: cool, and it looks like the replacement server never got far enough to create /usr/share/solr/WEB-INF anyway | 17:23 |
| greghaynes | its not real ssh | 17:23 |
| mrmartin | awesome | 17:24 |
| SpamapS | yeah it doesn't do bare commands, just terminal | 17:24 |
| * SpamapS is hoping somebody will disprove this one too | 17:24 | |
| *** mfmcdonagh has quit IRC | 17:24 | |
| *** openstackgerrit_ has joined #openstack-infra | 17:24 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Add networking-fujitsu project https://review.openstack.org/206019 | 17:24 |
| *** hdd has quit IRC | 17:25 | |
| *** baoli has joined #openstack-infra | 17:25 | |
| greghaynes | you know... | 17:25 |
| greghaynes | I could do something even more cool | 17:25 |
| greghaynes | and load ssh keys for the ilo users | 17:25 |
| *** achanda has joined #openstack-infra | 17:25 | |
| mrmartin | fungi: can I see the puppet-run report of new ask.o.o on puppetboard? | 17:25 |
| fungi | mrmartin: no, it won't be there until we switch dns | 17:26 |
| greghaynes | show map1/accounts1/Administrator password is the ILO command BTW | 17:26 |
| fungi | mrmartin: though if i run it manually it may report there | 17:26 |
| SpamapS | greghaynes: but we use IPMI ... ;) | 17:26 |
| fungi | mrmartin: i'll give that a try | 17:26 |
| mrmartin | oh ok, and do you see the status of the new node? | 17:26 |
| greghaynes | SpamapS: ah, right | 17:26 |
| pabelanger | nibalizer: okay, if people are fine with the change in functionality | 17:27 |
| greghaynes | jeblair: SpamapS: looks like it set | 17:27 |
| SpamapS | greghaynes: if we had gen8 servers, we could use an ilo driver btw | 17:27 |
| *** tiswanso has quit IRC | 17:28 | |
| *** ihrachyshka has quit IRC | 17:28 | |
| greghaynes | yep, totally works | 17:28 |
| *** hdd has joined #openstack-infra | 17:28 | |
| nibalizer | pabelanger: does that change functionality somehow? | 17:28 |
| greghaynes | SpamapS: jeblair so, ill try and get the expect script set up to homogenize these? | 17:28 |
| fungi | mrmartin: it's not getting far enough to report the error it has | 17:28 |
| jeblair | ssh Administrator@10.10.16.15 "set /map1/accounts1/Administrator password=..." | 17:28 |
| nibalizer | I guess 0 could be passed in via a manifest | 17:28 |
| jeblair | greghaynes: ^ that wfm | 17:28 |
| greghaynes | jeblair: oh nice! | 17:28 |
| fungi | mrmartin: Error: Failed to apply catalog: Could not find dependent Httpd::Vhost[askbot::site::ssl] for File[/etc/ssl/certs/ask.openstack.org.pem] at /etc/puppet/modules/askbot/manifests/site/ssl.pp:21 | 17:28 |
| *** jaypipes has joined #openstack-infra | 17:28 | |
| nibalizer | we should check for that | 17:28 |
| pabelanger | nibalizer: well, guess not. actually | 17:28 |
| pabelanger | nibalizer: ya, checking for that | 17:29 |
| pabelanger | don't see anything right now | 17:29 |
| greghaynes | oh! I know what I ran into before - you cant do command1; command2 | 17:29 |
| jeblair | greghaynes, SpamapS: so throw a "-p" in there in a script and we don't need expect | 17:29 |
| mrmartin | fungi: it is related to the certs somehow, the prod server have a different cert configuration then ask-staging | 17:29 |
| jeblair | greghaynes: aah | 17:29 |
| fungi | mrmartin: do we need to change the parameters we're passing in from site.pp? | 17:29 |
| mrmartin | fungi: I don't think so | 17:29 |
| greghaynes | jeblair: SpamapS so, ill make them all same ipmi pass as bastion? | 17:29 |
| mquin | can anyone point me towards whoever runs 'openstackgerrit'? | 17:30 |
| *** ddieterly has quit IRC | 17:30 | |
| *** tiswanso has joined #openstack-infra | 17:30 | |
| mrmartin | it is a dependency issue in puppet-askbot | 17:30 |
| fungi | mquin: the irc bot? the openstack community runs it | 17:30 |
| jeblair | greghaynes, SpamapS: this seems reasonable to me... fungi, clarkb: any reason we shouldn't homogenize the ipmi passwords? | 17:30 |
| fungi | mquin: is it creating a problem in one of your channels? | 17:31 |
| mquin | we're seeing an exceptionally high number of authentication failures to its account | 17:31 |
| crinkle | SpamapS: greghaynes i wrote https://review.openstack.org/#/c/208751/ to help with bifrost deployment, but it has no information about inventory | 17:31 |
| mquin | trying to find out if that's malicious or something misconfigured | 17:32 |
| jeblair | mquin: oh interesting; can you give us source ips? | 17:32 |
| fungi | mquin: is it coming from 104.130.159.134 or 2001:4800:7818:102:be76:4eff:fe05:9b12 | 17:32 |
| mquin | jeblair: openstackgerrit_!~openstack@217.25.5.178 | 17:32 |
| fungi | mquin: yep, sounds like someone decided to try to deploy a copy of our configuration but doesn't have our credentials | 17:32 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-jenkins: Gate on puppet-lint-version_comparison-check https://review.openstack.org/202884 | 17:33 |
| mrmartin | fungi: maybe this dependency is wrong, https://github.com/openstack-infra/puppet-askbot/blob/master/manifests/site/ssl.pp#L20 it was not tested before because we used previously the built-in self-signed certs of the | 17:33 |
| *** hdd has quit IRC | 17:33 | |
| *** maishsk has quit IRC | 17:33 | |
| greghaynes | jeblair: oh, the other issue - we dont have ssh keys so you have to somehow type in the pass to ssh | 17:33 |
| greghaynes | jeblair: which requires expect AFAIK | 17:33 |
| *** nadya has joined #openstack-infra | 17:33 | |
| fungi | mquin: feel free to block that ip address. it's not operated by anyone with control of the openstackgerrit account | 17:33 |
| *** marzif__ has quit IRC | 17:34 | |
| jeblair | greghaynes: or "sshpass" | 17:34 |
| *** openstackgerrit_ has quit IRC | 17:34 | |
| SpamapS | greghaynes: we may want to ask if the NOC wants us to not set those.. they likely have databases of IP<->PW | 17:34 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Migrate to puppet-httpd module https://review.openstack.org/198819 | 17:35 |
| SpamapS | crinkle: thats ok, we just want to deploy the software and get it running | 17:35 |
| greghaynes | SpamapS: hrmm | 17:35 |
| greghaynes | SpamapS: jeblair ok, I can ask, thatll probably be a bit so maybe just punt on backing up until they confirm whether or not we can set them | 17:35 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Remove puppet-apache now that we have migrated https://review.openstack.org/205696 | 17:36 |
| *** sharikapongubala has quit IRC | 17:36 | |
| greghaynes | We can also temporary backup to hp2 for now if we want.. | 17:36 |
| jeblair | greghaynes: sounds good | 17:36 |
| pabelanger | clarkb: green fedora21: https://review.openstack.org/#/c/203756/ that is better | 17:37 |
| pabelanger | jeblair: are you fine with the following? https://review.openstack.org/#/c/209558/3 | 17:37 |
| jeblair | greghaynes, SpamapS: and then we can add the yaml to system-config, and the answer from the noc will just determine whether we have 1 password in secret-hiera or many | 17:37 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for gate-neutron-dsvm-api vpnaas bug 1481862 https://review.openstack.org/209611 | 17:37 |
| openstack | bug 1481862 in neutron "test_vpnaas_extensions.VPNaaSTestJSON is broken in gate-neutron-dsvm-api on stable/kilo" [Undecided,New] https://launchpad.net/bugs/1481862 | 17:37 |
| *** Sukhdev has quit IRC | 17:37 | |
| nibalizer | ++ to the inventory being in system-config | 17:37 |
| clarkb | jeblair: not that I can think of, ironic is own the castle anyways | 17:37 |
| greghaynes | jeblair: SpamapS sounds good | 17:37 |
| nibalizer | 'the yaml' is the ansible inventory for infra-cloud right? | 17:38 |
| *** nadya has quit IRC | 17:38 | |
| *** baoli has quit IRC | 17:38 | |
| *** baoli has joined #openstack-infra | 17:39 | |
| jeblair | nibalizer: i think 'bifrost pre-input' in this case | 17:39 |
| nibalizer | ah right | 17:39 |
| jeblair | pabelanger: yes | 17:39 |
| nibalizer | there will also be an ansible inventory file thats used to kick of puppet apply runs ya? | 17:39 |
| jeblair | nibalizer: what went wrong here? http://logs.openstack.org/58/209558/3/check/gate-openstackci-beaker-trusty-dsvm/de3991b/console.html | 17:39 |
| nibalizer | jeblair: bug fixed in https://review.openstack.org/#/c/208618/ | 17:40 |
| *** sharikapongubala has joined #openstack-infra | 17:40 | |
| *** baoli_ has joined #openstack-infra | 17:42 | |
| greghaynes | SpamapS: ok, so gettin bifrosty with it... | 17:42 |
| pabelanger | jeblair: great, thanks | 17:43 |
| *** baoli_ has quit IRC | 17:44 | |
| *** baoli has quit IRC | 17:44 | |
| openstackgerrit | Marton Kiss proposed openstack-infra/puppet-askbot: Resolve SSL certificate dependency issue https://review.openstack.org/209614 | 17:44 |
| *** baoli has joined #openstack-infra | 17:45 | |
| *** fawadkhaliq has quit IRC | 17:45 | |
| *** markus_z has quit IRC | 17:46 | |
| mrmartin | fungi: the fix for ask.o.o ssl dependency issue: https://review.openstack.org/#/c/209614/ | 17:46 |
| *** nadya has joined #openstack-infra | 17:46 | |
| *** yfried has joined #openstack-infra | 17:47 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Migrate to puppet-httpd module https://review.openstack.org/198819 | 17:47 |
| openstackgerrit | will soula proposed openstack-infra/jenkins-job-builder: Add Ability to use the AWS Cloudformation Plugin https://review.openstack.org/202300 | 17:48 |
| SpamapS | greghaynes: yeah, do you see /home/clint/inventory.json ? Thats in the bifrost dynamic inventory format. | 17:48 |
| SpamapS | greghaynes: and I think you already found gh2bf.py ;) | 17:48 |
| *** hdd has joined #openstack-infra | 17:48 | |
| *** kdas__ has joined #openstack-infra | 17:48 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Remove puppet-apache now that we have migrated https://review.openstack.org/205696 | 17:48 |
| greghaynes | I always wanted a file format named after me | 17:49 |
| *** baoli has quit IRC | 17:49 | |
| openstackgerrit | Merged openstack-infra/system-config: Add puppet-apply converage to both node types https://review.openstack.org/209558 | 17:49 |
| fungi | mrmartin: ahh, yep i expect that was overlooked in the config vs site/ssl refactor | 17:49 |
| mrmartin | typical copy-paste error :) | 17:50 |
| fungi | clarkb: 209614 also needed for the askbot migration | 17:50 |
| *** baoli has joined #openstack-infra | 17:50 | |
| *** kushal has quit IRC | 17:51 | |
| openstackgerrit | Merged openstack-infra/puppet-openstackci: Enable apache modules before starting the server https://review.openstack.org/208618 | 17:51 |
| SpamapS | greghaynes: also did we ever figure out why we only have 22 nodes? | 17:52 |
| SpamapS | I mean, frankly, I'd be happy to get 22 nodes up in an openstack. | 17:53 |
| SpamapS | and just iterate forward from there | 17:53 |
| *** e0ne has joined #openstack-infra | 17:53 | |
| clarkb | +1 | 17:53 |
| *** bharath has joined #openstack-infra | 17:53 | |
| *** jamespage_ has quit IRC | 17:53 | |
| *** yamamoto has quit IRC | 17:53 | |
| greghaynes | SpamapS: Yes, the inventory when you bifrosted was just the nodes I was able to contact (the other nodes were in various states of dead). I updated the inventory to be all the nodes and more info if I could contact them and we never re-enrolled | 17:53 |
| greghaynes | in theory redoing it should try to enroll all the nodes though | 17:54 |
| SpamapS | greghaynes: ah ok, nice | 17:54 |
| clarkb | fungi: sitename is "askbot" but it is being used as a vhost name according to that change. typically we use fqdns there right? | 17:54 |
| SpamapS | greghaynes: so what we should do is go ahead and try to boot them all, but we should pick two and name them 'baremetal0' and 'baremetal1' instead of ic-xxx | 17:54 |
| clarkb | fungi: apparently we pass that through so thats correct | 17:55 |
| fungi | clarkb: yeah, that would be more consistent with our other modules, but seems to be arbitrary in this case | 17:55 |
| clarkb | I guess it is just a puppet resource name doesn't affect the actual vhost | 17:55 |
| fungi | the main issue is that it was missing | 17:56 |
| *** annegentle has quit IRC | 17:56 | |
| fungi | and we were passing the wrong variable into the templates | 17:56 |
| clarkb | pabelanger: ntp 3.2.1 is in place so any subsequent runs will use it | 17:57 |
| fungi | jenkins02 is in a bad way. i'm putting it into prepare for shutdown | 17:58 |
| *** baoli has quit IRC | 17:59 | |
| *** baoli has joined #openstack-infra | 18:00 | |
| fungi | jenkins01 and 07 look like they're probably getting unhappy as well. i'll do them later | 18:00 |
| *** spzala has joined #openstack-infra | 18:00 | |
| clarkb | the good news is we got independent confirmation of this bug | 18:00 |
| pabelanger | clarkb: great | 18:00 |
| *** bapalm_ has joined #openstack-infra | 18:00 | |
| clarkb | maybe upstream will start looking into it | 18:00 |
| *** nadya has quit IRC | 18:02 | |
| *** dannywilson has joined #openstack-infra | 18:02 | |
| *** sharikapongubala has quit IRC | 18:02 | |
| *** kdas__ has quit IRC | 18:02 | |
| *** kushal has joined #openstack-infra | 18:03 | |
| greghaynes | SpamapS: did you see the response from NOC? | 18:03 |
| greghaynes | SpamapS: They say they will change the ILO passwords for us, even :) | 18:03 |
| greghaynes | but they do want to know | 18:03 |
| greghaynes | SpamapS: We should probably wait until we do a deploy though - when we file a ticket we dont really know when theyll go around and change the passwords | 18:04 |
| *** tonytan4ever has joined #openstack-infra | 18:04 | |
| SpamapS | greghaynes: oh nice! | 18:05 |
| SpamapS | greghaynes: yeah, lets try to aim at a deploy we don't need to re-do en-masse and then file that ticket. | 18:05 |
| greghaynes | ++ | 18:06 |
| *** tsg has quit IRC | 18:06 | |
| *** bapalm_ has quit IRC | 18:06 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config: First stab at making things bifrosty https://review.openstack.org/208751 | 18:07 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config: Add openstack-modules.env https://review.openstack.org/209617 | 18:07 |
| greghaynes | SpamapS: Do you want to use .3 as baremetal0? | 18:07 |
| greghaynes | since that is clenish | 18:07 |
| SpamapS | greghaynes: we need to reinstall it | 18:07 |
| SpamapS | greghaynes: I resized it too far to test resizing/grub fail | 18:07 |
| greghaynes | ah | 18:07 |
| *** baoli has quit IRC | 18:08 | |
| SpamapS | greghaynes: unless you can get it into a rescue mode and downsize / ;) | 18:08 |
| crinkle | greghaynes: SpamapS is the inventory list something publicly available? | 18:08 |
| SpamapS | crinkle: not yet | 18:08 |
| pabelanger | took a few days, but both centos7 and fedora 21 now passing 100% :) | 18:08 |
| SpamapS | crinkle: but we were just talking about making it so | 18:08 |
| greghaynes | crinkle: nope, we were just discussing how to do that | 18:08 |
| SpamapS | I mean, honestly, it's mostly a puppet manifest at that point... | 18:08 |
| pabelanger | https://review.openstack.org/#/c/209549/ kicks them to voting | 18:08 |
| greghaynes | SpamapS: .3 is enrolled in the ironic on .2? | 18:09 |
| SpamapS | the yaml files are all "the way greg's ansible saw it at that point in time" .. so I'm not sure if we want to check those into system-config | 18:09 |
| SpamapS | greghaynes: correct | 18:09 |
| *** baoli has joined #openstack-infra | 18:09 | |
| *** sharikapongubala has joined #openstack-infra | 18:09 | |
| SpamapS | greghaynes: perhaps what we should do, is delete all the other nodes that we don't want to be baremetal0 and baremetal1 from .2, and then re-run deploy.yaml there. | 18:09 |
| SpamapS | greghaynes: note that we should document how we see this bootstrap going for other regions. :) | 18:10 |
| SpamapS | I started to and forgot about it.. some etherpad somewhere.. | 18:10 |
| * SpamapS hopes it's still in firefox history | 18:10 | |
| SpamapS | https://etherpad.openstack.org/p/infra-cloud-bootstrap | 18:10 |
| greghaynes | SpamapS: deploy.yaml will then re-enroll baremetal0 and baremetal1 AIUI | 18:11 |
| SpamapS | greghaynes: not so much.. everything is already enrolled | 18:11 |
| *** otter768 has joined #openstack-infra | 18:11 | |
| SpamapS | greghaynes: deploy.yaml will just tweak them into deploy mode. | 18:11 |
| SpamapS | if they're in available/enroll status. | 18:12 |
| greghaynes | ah, sweet | 18:12 |
| greghaynes | so alternative idea | 18:12 |
| *** tsg has joined #openstack-infra | 18:12 | |
| *** olaph has joined #openstack-infra | 18:12 | |
| greghaynes | redeploy .3 as a new node, do a whole new bifrost install (because id like to), enroll everything but current bastion and itself on there | 18:13 |
| *** esker has joined #openstack-infra | 18:13 | |
| krotscheck | Mornin' everyone. How're things broken today? | 18:13 |
| greghaynes | If you want to not kill that much time though its fine with me though | 18:14 |
| *** BrandonL08 has joined #openstack-infra | 18:14 | |
| SpamapS | greghaynes: thats fine. I was thinking that we'd build .3 and a .4 as baremetal0 and baremetal1, and otherwise the same. | 18:15 |
| greghaynes | ah, ok. Same idea just different order then | 18:16 |
| *** otter768 has quit IRC | 18:16 | |
| greghaynes | well, if we use .3 as the bm host we deploy from, we can then redeploy .2 when were happy we have everything off and .2 and .3 are then baremetal0 and baremetal1 | 18:16 |
| *** kushal has quit IRC | 18:17 | |
| greghaynes | and in theory starting from a new host is good for knowing what all info we have to pull off | 18:17 |
| *** patrickeast has quit IRC | 18:17 | |
| greghaynes | SpamapS: What did you use for an image? | 18:17 |
| greghaynes | deployment_image.qcow2? | 18:17 |
| SpamapS | greghaynes: yeah, you can also let install.yaml rebuild that. | 18:18 |
| SpamapS | greghaynes: and note local_bifrost_settings.yaml | 18:19 |
| greghaynes | having to either impersonate you or go as root | 18:19 |
| *** sergmelikyan has joined #openstack-infra | 18:20 | |
| *** baoli has quit IRC | 18:20 | |
| HenryG | mriedem: come back to the neutron channel? I was having lunch. | 18:20 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: OIDC - OpenId Connect Implementation https://review.openstack.org/198458 | 18:21 |
| *** moravec has joined #openstack-infra | 18:21 | |
| *** baoli has joined #openstack-infra | 18:21 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-jenkins: Gate on puppet-lint-version_comparison-check https://review.openstack.org/202884 | 18:21 |
| greghaynes | SpamapS: heh, the reboot I think was not good to our ironic | 18:21 |
| *** sergmelikyan has quit IRC | 18:22 | |
| *** ivar-lazzaro has joined #openstack-infra | 18:22 | |
| *** kmartin has quit IRC | 18:23 | |
| *** nadya has joined #openstack-infra | 18:23 | |
| *** sobersabre has quit IRC | 18:24 | |
| *** rlandy has quit IRC | 18:24 | |
| *** monester has joined #openstack-infra | 18:26 | |
| *** rlandy has joined #openstack-infra | 18:26 | |
| greghaynes | SpamapS: wow, nagios is on this box even | 18:26 |
| greghaynes | kill it with fire | 18:26 |
| SpamapS | greghaynes: yeah its AMAZING | 18:27 |
| SpamapS | it's the "Oh I need to run something" box :) | 18:27 |
| *** nadya has quit IRC | 18:28 | |
| *** AJaeger has joined #openstack-infra | 18:28 | |
| clarkb | fungi: jeblair https://review.openstack.org/#/c/207191/ can I get eyes on that? there are a few failures in nova that we are hitting and I think getting it running a bit more often will help debug that. I also have https://bugs.launchpad.net/nova/+bug/1445569 open for one of the big hitters | 18:29 |
| openstack | Launchpad bug 1445569 in OpenStack Compute (nova) "No dhcp lease after shelve unshelve" [High,Confirmed] | 18:29 |
| clarkb | and theory is it does hit single node jobs just less badly since we don't ssh there? | 18:29 |
| openstackgerrit | Merged openstack-infra/nodepool: Expand documentation on timeouts https://review.openstack.org/203727 | 18:32 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Fix glossary recognition https://review.openstack.org/209622 | 18:32 |
| *** sdake has joined #openstack-infra | 18:32 | |
| openstackgerrit | Merged openstack-infra/project-config: Add new kiloeyes repository for OpenStack monitoring service https://review.openstack.org/203259 | 18:33 |
| AJaeger | clarkb, fungi: My change was in the wrong branch of the if, could you review carefully again, please ^ | 18:33 |
| fungi | clarkb: lgtm, and i take sdague's +2 to also be speaking for qa on that one | 18:33 |
| greghaynes | SpamapS: oh! since ironic couldnt talk to the ilo's because the bad route it updated the state of all our nodes | 18:33 |
| clarkb | fungi: ya I think sdague is happy as long as its not voting/gating and someone is working the issues (I am trying to) | 18:33 |
| *** yamamoto has joined #openstack-infra | 18:34 | |
| *** annegentle has joined #openstack-infra | 18:35 | |
| fungi | AJaeger: got it, so you only need the glossary workaround for non-rst documents | 18:35 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for gate-neutron-dsvm-api vpnaas bug 1481862 https://review.openstack.org/209611 | 18:36 |
| openstack | bug 1481862 in neutron "test_vpnaas_extensions.VPNaaSTestJSON is broken in gate-neutron-dsvm-api on stable/kilo" [Undecided,New] https://launchpad.net/bugs/1481862 - Assigned to Paul Michali (pcm) | 18:36 |
| AJaeger | fungi: exactly. The glossary is in docbook XML | 18:38 |
| *** ayoung has quit IRC | 18:38 | |
| fungi | clarkb: looking at the nodepool image-list i think the ubuntu-trusty uploads succeeded yesterday? | 18:38 |
| fungi | at least i don't recall uploading them myself | 18:38 |
| greghaynes | SpamapS: TheJulia If I want to force ironic to resync the power state for nodes... | 18:38 |
| greghaynes | any protips? | 18:38 |
| *** jcoufal has quit IRC | 18:38 | |
| clarkb | fungi: yes looks like it updated across the board yesterday | 18:39 |
| fungi | wow. nova says we have 74 error nodes and 4 active nodes in iad | 18:40 |
| fungi | seeing if it'll let me manually delete them now | 18:41 |
| *** tiswanso has quit IRC | 18:41 | |
| fungi | yeah, doesn't seem like it | 18:42 |
| AJaeger | fungi, clarkb: Thanks for the reviews! | 18:42 |
| SpamapS | greghaynes: it will always force sync the power | 18:44 |
| SpamapS | greghaynes: if you turn a node off it thinks should be on.. just wait | 18:44 |
| greghaynes | SpamapS: I think if it cant talk to a node for long enough it puts it in maintenence mode automagically | 18:45 |
| openstackgerrit | Andrei V. Ostapenko proposed openstack-infra/project-config: Move from stackforge/magnetodb to openstack/magnetodb https://review.openstack.org/203524 | 18:45 |
| greghaynes | it also removes instance UUID... | 18:45 |
| openstackgerrit | Andrei V. Ostapenko proposed openstack-infra/project-config: Changes jobs description for MagnetoDB https://review.openstack.org/204478 | 18:45 |
| greghaynes | and AFAICT its no longer updating the one I turned maintenence mode off for | 18:45 |
| *** tiswanso has joined #openstack-infra | 18:45 | |
| SpamapS | greghaynes: yes thats true. | 18:45 |
| SpamapS | greghaynes: the point of maintenance mode is "hands off ironic" | 18:46 |
| fungi | i've updated the currently open rax ticket asking to have the instances which are stuck in a pending deletion state in dfw and iad cleared out, though no idea if they're going to come back and want specific uuid lists for them | 18:46 |
| *** openstackgerrit has quit IRC | 18:46 | |
| *** openstackgerrit has joined #openstack-infra | 18:47 | |
| SergK | Hi, who can help me (skulanov@mirantis.com) to become a member of fuel-mirror-core group, this stackforge project was recently created and the corresponding patch was successfully merged https://review.openstack.org/#/c/205963/ | 18:47 |
| pabelanger | fungi: do you mind reviewing: https://review.openstack.org/#/c/203756/ | 18:48 |
| clarkb | SergK: I can add you | 18:48 |
| SergK | clarkb, thank you very much | 18:48 |
| clarkb | SergK: done | 18:49 |
| openstackgerrit | Merged openstack-infra/project-config: Switch to using full multinode jobs everywhere https://review.openstack.org/207191 | 18:49 |
| *** monester has quit IRC | 18:49 | |
| *** puranamr has quit IRC | 18:50 | |
| SergK | clarkb, thank you, can i kindly ask you to have a look on some permissions updates regarding this (fuel-mirror) project https://review.openstack.org/#/c/209223/ | 18:50 |
| AJaeger | fungi: ianw has a change to remove puppet-apply jobs from project-config. Could you review the system-config part https://review.openstack.org/#/c/205873/ and the project-config one: https://review.openstack.org/#/c/206319/ , please? Clarkb already +2 and since this reduces running job, I wanted to point it out. But this is not urgent... | 18:51 |
| clarkb | SergK: done, you should glob te release string so we don't have to update every time | 18:51 |
| pabelanger | So looking at our check jobs for system-config, I see gate-openstackci-beaker-trusty-dsvm is failing. Is anybody looking at it to make it voting? | 18:52 |
| *** monester has joined #openstack-infra | 18:52 | |
| pabelanger | more in general, how is working on beaker jobs for system-config? | 18:52 |
| pabelanger | who* | 18:52 |
| clarkb | pabelanger: I think nibalizer was, there were issues with http mod installation order | 18:52 |
| clarkb | have to install mod rewrite before we can restart apache with rewrite config | 18:53 |
| pabelanger | ack | 18:53 |
| clarkb | which I think was a patch to the httpd module | 18:53 |
| *** puranamr has joined #openstack-infra | 18:54 | |
| SergK | clarkb, thanks will check | 18:54 |
| *** tsg has quit IRC | 18:54 | |
| *** tsg has joined #openstack-infra | 18:54 | |
| *** geoffarnold has quit IRC | 18:55 | |
| *** akraminsky__ has quit IRC | 18:59 | |
| *** armax has quit IRC | 19:00 | |
| nibalizer | clarkb: patch to openatackci | 19:01 |
| *** geoffarnold has joined #openstack-infra | 19:01 | |
| *** ub has joined #openstack-infra | 19:03 | |
| *** nadya has joined #openstack-infra | 19:03 | |
| openstackgerrit | Merged openstack-infra/project-config: Make job gate-manila-tempest-dsvm-neutron-scenario non-voting https://review.openstack.org/209067 | 19:04 |
| *** thedodd has quit IRC | 19:04 | |
| *** pblaho has quit IRC | 19:05 | |
| *** dtantsur|afk has quit IRC | 19:05 | |
| timrc | pabelanger, Remind me how dashboards are going to get loaded into Grafana with grafyaml? I remember you saying you were doing to do something similar to jjb? | 19:06 |
| *** dtantsur has joined #openstack-infra | 19:06 | |
| pabelanger | timrc: Yup. We'll take yaml, convert to json and post into grafana. | 19:06 |
| clarkb | and thats straight into the db or via an api? | 19:07 |
| pabelanger | timrc: however, work needs to be done on the caching dashboards. Right now, we can just force push over them. | 19:07 |
| pabelanger | clarkb: RestAPI | 19:07 |
| timrc | pabelanger, So is the work to pull dashboard yaml to the grafana server and then process it with grafyaml still in the works? | 19:07 |
| pabelanger | timrc: for the most part it is done. I am working on upgrading to grafana 2.1.0 to get some basic auth support. But, once that happens, we can get grafyaml running local to manage our first dashboard | 19:08 |
| pabelanger | here is the review to upgrade grafana: https://review.openstack.org/#/c/209218/ | 19:09 |
| pabelanger | I still need to get puppet-grafyaml going | 19:09 |
| pabelanger | have it local, just need to get a review up | 19:09 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config: Create playbook to puppet InfraCloud nodes https://review.openstack.org/209632 | 19:09 |
| *** sharikapongubala has quit IRC | 19:10 | |
| openstackgerrit | Ryan Moats proposed openstack-infra/project-config: Move multinode dvr job to neutron check list. https://review.openstack.org/209165 | 19:10 |
| *** pblaho has joined #openstack-infra | 19:10 | |
| openstackgerrit | Andrey Pavlov proposed openstack-infra/project-config: Adding swift3 to list of projects for particular ec2 job https://review.openstack.org/208535 | 19:12 |
| timrc | pabelanger, Awesome. | 19:12 |
| *** jkraj has quit IRC | 19:13 | |
| *** cdent has quit IRC | 19:13 | |
| *** angdraug has joined #openstack-infra | 19:13 | |
| pabelanger | timrc: one of the first dashboards I'm working on is rendering zuul like we do on status.o.o/zuul | 19:13 |
| timrc | pabelanger, Interesting that authentication was a motivation for your upgrade? Are people going to be logging into Grafana? | 19:14 |
| pabelanger | timrc: no, grafyaml is. | 19:14 |
| timrc | Personally I think building dashboards in Grafana is the way to go. I couldn't imagine doing it in yaml. | 19:14 |
| timrc | pabelanger, Isn't grafyaml already authenticating with a token? | 19:15 |
| fungi | (doing it in yaml means you can check it into config management) | 19:15 |
| pabelanger | timrc: it is, but there is no way to automate the token creation. Otherwise, we need to hack the database to inject it | 19:15 |
| timrc | fungi, Certainly _exporting_ and _importing_ your dashboards would be useful for that. | 19:15 |
| pabelanger | timrc: so, I had upstream grafana add basic auth support | 19:15 |
| *** yamamoto has quit IRC | 19:15 | |
| *** baoli has quit IRC | 19:16 | |
| timrc | pabelanger, Ah, I see. | 19:16 |
| timrc | Well the LDAP authentication support is going to be tremendously useful for us downstream. | 19:16 |
| pabelanger | timrc: you'll still be able to manually build dashboards, and export them in grafana.o.o, just not save them | 19:16 |
| *** baoli has joined #openstack-infra | 19:16 | |
| *** achanda has quit IRC | 19:16 | |
| timrc | pabelanger, Oh yeah, anonymously. | 19:16 |
| greghaynes | SpamapS: ok, im going to just unenroll all the things | 19:17 |
| greghaynes | +1? | 19:17 |
| greghaynes | not sure how to apply hammer to make nodes resync properly... not going to get all hung up on that | 19:17 |
| *** r-daneel has joined #openstack-infra | 19:20 | |
| pabelanger | fungi: clarkb: nibalizer: do you guys think we can land: https://review.openstack.org/#/c/198819/ today? Thats 1 of 3 patches to complete puppet-httpd migration | 19:22 |
| SpamapS | greghaynes: +1 | 19:22 |
| fungi | pabelanger: looking | 19:22 |
| SpamapS | greghaynes: if you delete all the things, they should all be turned off | 19:22 |
| *** baoli has quit IRC | 19:22 | |
| *** baoli has joined #openstack-infra | 19:23 | |
| fungi | pabelanger: not quite sure how to parse that first sentence in the commit message. is it fork but missing a k? | 19:23 |
| jeblair | clarkb: marked for after lunch | 19:23 |
| pabelanger | fungi: oh, ya. that is an old typo | 19:23 |
| openstackgerrit | Merged openstack-infra/system-config: Add fedora support for apply-test.sh https://review.openstack.org/203756 | 19:24 |
| pabelanger | fungi: thought I fixed it | 19:24 |
| jeblair | clarkb: context: https://review.openstack.org/#/c/207191/ | 19:24 |
| fungi | pabelanger: thanks, interpreting accordingly | 19:24 |
| *** elarson has quit IRC | 19:24 | |
| clarkb | jeblair: oh I think fungi got it | 19:24 |
| clarkb | jeblair: feel free to do post merge review though :) | 19:24 |
| harlowja | fungi clarkb jeblair one of u guys mind +A https://review.openstack.org/#/c/209186/ (would be much appreciated, hopefully the last one of those patches, ha) | 19:24 |
| fungi | yeah, i approved it already | 19:24 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 19:24 | |
| jeblair | clarkb, fungi: i win? :) | 19:24 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 19:24 | |
| *** ub has quit IRC | 19:25 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/project-config: Run beaker jobs on puppet-openstack_extras https://review.openstack.org/209141 | 19:25 |
| fungi | jeblair: you win several additional seconds, or you would have, except you squandered them discussing it | 19:25 |
| *** ub has joined #openstack-infra | 19:25 | |
| greghaynes | SpamapS: you mean I should turn them off, or they will end up turned off? | 19:25 |
| openstackgerrit | Terry Howe proposed openstack/os-testr: Convert file names to regular expressions https://review.openstack.org/203424 | 19:26 |
| openstackgerrit | Merged openstack-infra/system-config: Don't clone project-config for puppet-apply jobs https://review.openstack.org/205873 | 19:26 |
| *** MarkAtwood_ has quit IRC | 19:26 | |
| greghaynes | I assumed when you re-enroll ironic asserts power off | 19:26 |
| *** olaph has quit IRC | 19:26 | |
| harlowja | fungi thx :) | 19:26 |
| clarkb | I have to say the ::foo is really annoying since you can'tdo that for built in resources | 19:26 |
| clarkb | so we hvae ::http but then httpd_mod | 19:27 |
| *** HeOS has quit IRC | 19:27 | |
| asselin_ | nibalizer, I'm trying to test my sing-node ci solution, but puppet is returning without running anything. Any tips? http://paste.openstack.org/show/409384/ | 19:27 |
| fungi | i assume this is puppet^4 madness? | 19:27 |
| clarkb | fungi: yes, its apparnetly how we have to do it | 19:27 |
| fungi | mumblegrumble | 19:28 |
| crinkle | clarkb: I haven't found a satisfactory answer about whether we should be doing that for defined types | 19:28 |
| crinkle | clarkb: just for classes as far as I know | 19:28 |
| clarkb | one wonders if the engineering team at puppetlabs is actually using puppet | 19:28 |
| clarkb | crinkle: iirc you can't do it for proper types | 19:28 |
| clarkb | I asked nibalizer and he said its a parse fail | 19:28 |
| crinkle | clarkb: right, but it works for defined types e.g. httpd::vhost | 19:29 |
| crinkle | i just don't know whether we should care | 19:29 |
| clarkb | well I care in that supposed I have to do include ::httpd but also have to do httpd_mod instead of ::httpd_mod or include httpd | 19:29 |
| clarkb | it just doesn't read well | 19:29 |
| *** kmartin has joined #openstack-infra | 19:30 | |
| fungi | pabelanger: you plan to have eyeballs glued to puppetboard for a while after 198819 merges? | 19:30 |
| clarkb | and reminds me of perl and fighting $ @ % | 19:30 |
| fungi | clarkb: at least in perl it looks like you're swearing | 19:30 |
| pabelanger | fungi: yes sir | 19:30 |
| clarkb | is this a scalar array hash? why should I have to keep track | 19:31 |
| fungi | pabelanger: thanks! it's punted into the hopper, on its way through the sausage grinder now | 19:31 |
| pabelanger | om nom nom | 19:31 |
| clarkb | fungi: any progress on ubuntu-trusty package caching? | 19:32 |
| clarkb | I need to see if the latest image build passes the devstack jobs but its possible we can switch those over soonish | 19:32 |
| fungi | clarkb: i'm playing with an updated node now... it looks like we're successfully caching _some_ things, but not everything that the image build log claims was downloaded | 19:33 |
| fungi | so even more confusing | 19:33 |
| fungi | well, i _will_ be playing with an updated node when nodepool eventually gets around to booting one anyway | 19:33 |
| *** rockyg has joined #openstack-infra | 19:34 | |
| clarkb | I should note image uploads are very slow | 19:34 |
| clarkb | so while reliable don't happen quickly | 19:35 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/puppet-openstackci: WIP: Initial start at creating sample ci https://review.openstack.org/200330 | 19:35 |
| openstackgerrit | Merged openstack-infra/project-config: Don't run puppet-apply jobs for project-config https://review.openstack.org/206319 | 19:35 |
| *** MarkAtwood has quit IRC | 19:35 | |
| *** olaph has joined #openstack-infra | 19:35 | |
| fungi | yeah, it's taking somewhere in the neighborhood of 6-8 hours for our nodepool image updates to complete lately, if memory serves | 19:35 |
| clarkb | we can probably shift the image build cron kickoff a few hours earlier and still get new images after we are awake | 19:36 |
| openstackgerrit | Merged openstack-infra/project-config: telemetry: add integration tests job https://review.openstack.org/207298 | 19:37 |
| openstackgerrit | Merged openstack-infra/project-config: midonet: Enable tempest run_validation https://review.openstack.org/208861 | 19:37 |
| * greghaynes mentions that theres nodepool builders patches up for review | 19:37 | |
| greghaynes | which should help speed up builds a lot | 19:37 |
| clarkb | greghaynes: oh right and it doesn't need the db anymore/ | 19:38 |
| *** hdd has quit IRC | 19:38 | |
| *** e0ne has quit IRC | 19:38 | |
| clarkb | and does it pass the devstack job? | 19:38 |
| fungi | clarkb: i'm tempted to parameterize the image-update time and then pass ours in from out global site manifest (or even hiera) | 19:38 |
| fungi | s/out/our/ | 19:38 |
| greghaynes | with those patches it still does, but removing the need for the db would be next step | 19:38 |
| clarkb | gotcha | 19:38 |
| greghaynes | clarkb: good question | 19:38 |
| pabelanger | greghaynes: review? | 19:38 |
| *** hdd has joined #openstack-infra | 19:38 | |
| clarkb | greghaynes: I think I will -1 until it passes the dsvm test since a big part of thta is acutlly upload an image | 19:38 |
| clarkb | greghaynes: :P | 19:39 |
| SpamapS | greghaynes: if you change their status to 'deleted' in ironic, Ironic will try to turn them off. | 19:39 |
| greghaynes | pabelanger: https://review.openstack.org/#/q/status:open+topic:nodepool-workers,n,z | 19:39 |
| *** bswartz has quit IRC | 19:39 | |
| greghaynes | clarkb: yea, it definitely shouldnt merge if it doesnt pass that | 19:39 |
| *** monester has quit IRC | 19:40 | |
| clarkb | oh wait | 19:40 |
| clarkb | it doesnt dib (yet) | 19:40 |
| clarkb | so it really shoukd pass | 19:41 |
| greghaynes | clarkb: https://review.openstack.org/#/c/208780/ | 19:41 |
| greghaynes | hrm | 19:41 |
| greghaynes | and that failed | 19:41 |
| fungi | jeblair: do you still need those two nodepool nodes with the same ip address held, or should i recycle them now? | 19:41 |
| clarkb | greghaynes: check the nodepool log | 19:42 |
| pabelanger | greghaynes: thanks, will check that out\ | 19:42 |
| greghaynes | SpamapS: hrmmm, I apparently dont understand ironic as well as I though | 19:45 |
| greghaynes | SpamapS: cant delete when in provision state acrive | 19:45 |
| *** rguillebert has joined #openstack-infra | 19:45 | |
| greghaynes | active | 19:45 |
| clarkb | greghaynes: to make it dib we need to be able to build a super light dib image that supports sftp | 19:45 |
| greghaynes | and also, have to lunch meeting | 19:45 |
| clarkb | which ubuntu isn't really though it would work just be very slow | 19:45 |
| greghaynes | clarkb: why super light? | 19:45 |
| greghaynes | clarkb: you have to boot and sftp something in? | 19:45 |
| clarkb | greghaynes: so that build doesn't take long as we already need 42 ish minutes | 19:46 |
| *** baoli has quit IRC | 19:46 | |
| greghaynes | clarkb: builds take only a couple mins at most | 19:46 |
| clarkb | greghaynes: yes nodepool uses sftp to write files to disk after boot | 19:46 |
| greghaynes | or at least they should | 19:46 |
| clarkb | greghaynes: ah ok in that case maybe ubuntu is fine | 19:46 |
| *** puranamr has quit IRC | 19:46 | |
| pabelanger | clarkb: fungi: jeblair: last one for today! https://review.openstack.org/#/c/209549/ now that our centos7 and fedora21 jobs are green | 19:46 |
| *** olaph has quit IRC | 19:46 | |
| greghaynes | clarkb: yea, id just try ubuntu, if theres an issue we can maybe get funky with ubuntu-minimal | 19:46 |
| pabelanger | also, looking forward to Clint plan for project management :D And interested to see how we'll track it all | 19:46 |
| Clint | iteratively, i imagine ;) | 19:47 |
| *** baoli has joined #openstack-infra | 19:47 | |
| openstackgerrit | Merged openstack-infra/puppet-askbot: Resolve SSL certificate dependency issue https://review.openstack.org/209614 | 19:48 |
| fungi | mrmartin: ^ i'm rerunning puppet agent on the new ask.o.o replacement now | 19:48 |
| clarkb | fungi: dont forget to update the module on the master first | 19:49 |
| fungi | well, once the modules update on puppetmaster anyway | 19:49 |
| fungi | heh, jinx | 19:49 |
| *** BrandonL18 has joined #openstack-infra | 19:49 | |
| pabelanger | looking at puppetdb.o.o, seems that grafana.o.o has not checked-in in 4 days: http://puppetdb.openstack.org/node/grafana.openstack.org | 19:49 |
| pabelanger | any infra-root mind check? | 19:50 |
| *** olaph has joined #openstack-infra | 19:50 | |
| *** ayoung has joined #openstack-infra | 19:51 | |
| *** bswartz has joined #openstack-infra | 19:51 | |
| jroll | greghaynes: ironic node-set-provision-state uuid deleted, to make them inactive, then you can delete | 19:51 |
| fungi | pabelanger: The authenticity of host 'grafana.openstack.org (2001:4800:7815:105:be76:4eff:fe05:f5de)' can't be established. | 19:51 |
| fungi | pabelanger: i'm accepting the ssh host key on our puppetmaster now | 19:52 |
| *** thedodd has joined #openstack-infra | 19:52 | |
| *** BrandonL08 has quit IRC | 19:52 | |
| fungi | pabelanger: it should begin puppeting at the regular cadence with the next pulse | 19:52 |
| *** baoli has quit IRC | 19:53 | |
| fungi | okay, jenkins02 has finally quiesced. cleaning it up and restarting it now | 19:54 |
| *** esker has quit IRC | 19:55 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Remove query for bug 1476770 https://review.openstack.org/209650 | 19:55 |
| openstack | bug 1476770 in OpenStack-Gate "_translate_from_glance fails with "AttributeError: id" in grenade" [Undecided,Fix committed] https://launchpad.net/bugs/1476770 - Assigned to Matt Riedemann (mriedem) | 19:55 |
| *** rlandy has quit IRC | 19:56 | |
| *** baoli has joined #openstack-infra | 19:56 | |
| *** olaph has quit IRC | 19:56 | |
| *** sergmelikyan has joined #openstack-infra | 19:58 | |
| *** EricGonczer_ has quit IRC | 19:58 | |
| *** rm_work is now known as rm_work|away | 19:59 | |
| *** rlandy has joined #openstack-infra | 19:59 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config: Add openstack-infra/puppet-infra-cookiecutter project https://review.openstack.org/207109 | 20:01 |
| pabelanger | fungi: perfect. Thanks | 20:03 |
| *** baoli has quit IRC | 20:04 | |
| *** rm_work|away is now known as rm_work | 20:04 | |
| *** dkranz has quit IRC | 20:04 | |
| *** baoli has joined #openstack-infra | 20:05 | |
| *** puranamr has joined #openstack-infra | 20:06 | |
| fungi | clarkb: on the package cache front, note for example that http://nodepool.openstack.org/dib.ubuntu-trusty.log.2015-08-04_17 shows it downloading mysql-client-5.5 at 2015-08-04 18:01:00,196 but if you ssh into 23.253.67.58 there is no such deb in /var/cache/apt/archives | 20:06 |
| fungi | clarkb: i have a feeling our uploads may be failing and nodepool doesn't know | 20:07 |
| *** jasondot_ has joined #openstack-infra | 20:07 | |
| openstackgerrit | Catherine Diep proposed openstack-infra/project-config: Move Refstack project from stackforge to openstack-infra. https://review.openstack.org/205777 | 20:07 |
| *** jasondotstar has quit IRC | 20:08 | |
| greghaynes | jroll: ah, ty | 20:08 |
| *** monester has joined #openstack-infra | 20:09 | |
| pabelanger | fungi: did you check /var/cache/apt/archives on nodepool.o.o? | 20:09 |
| fungi | pabelanger: it won't be there. this happens in a chroot | 20:10 |
| pabelanger | possible they are getting saved outside path of diskimage builder? | 20:10 |
| *** tonytan4ever has quit IRC | 20:12 | |
| fungi | clarkb: pabelanger: yeah, i think the image is stale? that same log shows it's downloading libpython2.7-stdlib amd64 2.7.6-8ubuntu0.2 but in the archive on that node in find libpython2.7-stdlib_2.7.6-8_amd64.deb (-8 instead of -8ubuntu0.2). http://changelogs.ubuntu.com/changelogs/pool/main/p/python2.7/python2.7_2.7.6-8ubuntu0.2/changelog places the window between march 22 and june 22 | 20:12 |
| clarkb | huh | 20:12 |
| mrmartin | fungi: what's the status of new puppet run on ask.o.o ? | 20:12 |
| *** otter768 has joined #openstack-infra | 20:12 | |
| *** boris-42 has joined #openstack-infra | 20:12 | |
| *** sc68cal_ is now known as sc68cal | 20:13 | |
| AJaeger | do we have a problem with zuul? Looking at http://status.openstack.org/zuul/, the first entry is greyed out which looks wrong - and then there are a couple of jobs that are finished but zuul is not cleaning them up. | 20:15 |
| *** yamamoto has joined #openstack-infra | 20:16 | |
| *** akraminsky has joined #openstack-infra | 20:16 | |
| *** bapalm has joined #openstack-infra | 20:16 | |
| fungi | mrmartin: after the module updated on our puppetmaster now i applied on the replacement server and it's all green. puppetboard report did post for that at http://puppetboard.openstack.org/report/ask.openstack.org/f4057040214c784aca8e63260cfa13ee64d29b3a after i triggered it | 20:16 |
| *** baoli has quit IRC | 20:16 | |
| *** akraminsky_ has joined #openstack-infra | 20:16 | |
| *** patrickeast has joined #openstack-infra | 20:16 | |
| *** baoli has joined #openstack-infra | 20:16 | |
| *** dimtruck is now known as zz_dimtruck | 20:17 | |
| *** otter768 has quit IRC | 20:17 | |
| mrmartin | fungi: thnx. | 20:17 |
| *** ashleighfarnham has quit IRC | 20:17 | |
| pabelanger | fungi: looking at the DIB_APT_LOCAL_CACHE setting, seems to be we want to set it to 0? To disable the shared cache? | 20:17 |
| fungi | AJaeger: i have a feeling the windowing minimum is being applied across multiple queues | 20:17 |
| *** HeOS has joined #openstack-infra | 20:18 | |
| AJaeger | windowing minimum? In the check queue? | 20:18 |
| fungi | AJaeger: oh, in check. i was looking at the gate | 20:18 |
| AJaeger | fungi: Yes, check | 20:18 |
| AJaeger | just look at the first and third job there - and there are more further down with 0 minutes | 20:19 |
| fungi | AJaeger: looks like the status page failed to draw a line between that and the one after it | 20:19 |
| AJaeger | but what about the zaqar and manila ones? Shouldn't those get cleaned up? | 20:19 |
| *** che-arne has joined #openstack-infra | 20:19 | |
| AJaeger | we can wait another minute but I looked already for a few minutes on a change further down... | 20:20 |
| AJaeger | Or am I just not patient enough? | 20:20 |
| fungi | AJaeger: yeah, it seems to claim that there's a window being applied in the check pipeline too | 20:20 |
| *** BrandonL08 has joined #openstack-infra | 20:20 | |
| *** yamamoto has quit IRC | 20:21 | |
| fungi | AJaeger: OH! notice we also have named dependent queues in the check pipeline now? | 20:21 |
| fungi | jeblair: ^ definitely odd | 20:21 |
| AJaeger | fungi: named dependent queues? Where do you see that? | 20:21 |
| fungi | AJaeger: queues named "integrated" in the check pipeline, for one | 20:22 |
| AJaeger | fungi: I see it now as well - yes, that looks new | 20:22 |
| *** BrandonL18 has quit IRC | 20:23 | |
| fungi | checking to see whether something major changed in the most recent layout update commits | 20:23 |
| *** ashleighfarnham has joined #openstack-infra | 20:23 | |
| *** puranamr has quit IRC | 20:24 | |
| *** baoli has quit IRC | 20:25 | |
| jeblair | fungi, AJaeger: the grey appears to be the usual 'dependent change independently tested', and indeed 209581 --git-depends--> 192617 | 20:25 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-jenkins: Add Gemfile and puppet 4 checks https://review.openstack.org/203164 | 20:25 |
| SpamapS | greghaynes: right, you have to set provision state to deleted | 20:25 |
| fungi | jeblair: that one, yes, though there are some further down that seem to lack any obvious coupling | 20:26 |
| SpamapS | greghaynes: because that API was really meant for Nova. ;) | 20:26 |
| jeblair | fungi, AJaeger: so i don't think that's windowing; i'm not certain why there are no jobs, nor why there is no line. | 20:26 |
| SpamapS | not humans :) | 20:26 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-jenkins: Gate on puppet-lint-version_comparison-check https://review.openstack.org/202884 | 20:26 |
| jeblair | fungi: numbers? | 20:26 |
| fungi | jeblair: for example 209660,1 | 20:26 |
| SpamapS | greghaynes: for node in $(ironic node-list|awk '/active/ { print $2 }') ; do ironic node-set-provision-state $node deleted ; done | 20:26 |
| *** sergmelikyan has quit IRC | 20:26 | |
| *** baoli has joined #openstack-infra | 20:26 | |
| fungi | jeblair: 209240,2 as well | 20:26 |
| jeblair | fungi: ah, yes, those greys are 'waiting until closer to head of queue' so are windowing | 20:27 |
| AJaeger | jeblair: to me it looks like finished jobs do not get removed from the queue at all. | 20:27 |
| fungi | jeblair: but also, is the named "integrated" queue in the check pipeline a new thing? or have i just never noticed it before? | 20:27 |
| pabelanger | AJaeger: I was going to say that too | 20:28 |
| openstackgerrit | Sean M. Collins proposed openstack-infra/project-config: [WIP] Make the Linux Bridge CI voting https://review.openstack.org/205674 | 20:28 |
| jeblair | fungi: i'm not certain; i think its intended behavior is less defined in that context so have not really paid much attention | 20:28 |
| *** pal has joined #openstack-infra | 20:28 | |
| *** baoli has quit IRC | 20:29 | |
| fungi | other odd things going on there too | 20:29 |
| *** hdd has quit IRC | 20:29 | |
| fungi | for example 209538,1 seems to be passing and completed but has no jobs associated with it | 20:29 |
| *** baoli has joined #openstack-infra | 20:29 | |
| jeblair | pabelanger, AJaeger: i agree 195010,3 for example should have been removed | 20:29 |
| *** hdd has joined #openstack-infra | 20:29 | |
| *** hdd has quit IRC | 20:30 | |
| *** abdelwas has joined #openstack-infra | 20:30 | |
| AJaeger | jeblair: I was looking at 209622 - it's done since 10+ minutes | 20:30 |
| openstackgerrit | Tom Jose Kalapura proposed openstack-infra/project-config: Add python-multiclound client to openstack. https://review.openstack.org/209662 | 20:31 |
| *** monester has quit IRC | 20:31 | |
| fungi | zuul-server process seems to be eating an entire cpu at the moment | 20:31 |
| jeblair | "multiclound" typo, or not? | 20:31 |
| AJaeger | jeblair: typo | 20:31 |
| fungi | ohyeah, ERROR zuul.Scheduler: Exception in run handler: | 20:31 |
| jeblair | ah, i believe we're in a similar stuck situation as before | 20:32 |
| *** abregman has joined #openstack-infra | 20:32 | |
| *** krtaylor has quit IRC | 20:32 | |
| jeblair | 2015-08-05 19:50:00,737 ERROR zuul.Server: Reconfiguration failed: | 20:33 |
| jeblair | it will take a few minutes for me to get to that part of the log :) | 20:33 |
| *** monester has joined #openstack-infra | 20:34 | |
| *** amitgandhinz has quit IRC | 20:34 | |
| *** cburroughs has quit IRC | 20:34 | |
| *** tonytan4ever has joined #openstack-infra | 20:35 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 20:36 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 20:36 | |
| * AJaeger has to leave and waves good bye... | 20:36 | |
| openstackgerrit | Sean M. Collins proposed openstack-infra/project-config: [WIP] IPv6 only neutron experimental job https://review.openstack.org/209663 | 20:36 |
| *** AJaeger has quit IRC | 20:36 | |
| *** marzif__ has joined #openstack-infra | 20:38 | |
| *** mrunge has quit IRC | 20:39 | |
| *** changbl has quit IRC | 20:39 | |
| *** changbl has joined #openstack-infra | 20:40 | |
| jeblair | https://etherpad.openstack.org/p/NH0cYWvXq5 has notes as i compile them | 20:40 |
| jeblair | fungi: ^ | 20:40 |
| *** monester has quit IRC | 20:40 | |
| *** akraminsky_ has quit IRC | 20:41 | |
| *** wayneeseguin has quit IRC | 20:41 | |
| crinkle | SpamapS: are you still working from the patch series starting from 195276? anything i can do to help there at this point? | 20:42 |
| *** wayneeseguin has joined #openstack-infra | 20:43 | |
| openstackgerrit | Matthew Treinish proposed openstack/os-testr: Add basic bash completion to --no-discover arg https://review.openstack.org/209664 | 20:43 |
| *** miqui has quit IRC | 20:44 | |
| jeblair | it feels like a result of I2e24b300fe31e6eec5acaca48fba94cd5633f38f but that should be the bug that was fixed already... | 20:44 |
| fungi | we restarted on august 1 | 20:45 |
| jeblair | Date: Fri Jul 31 10:40:16 2015 +0000 | 20:45 |
| jeblair | Merge "Fix bug with removing a failed job" | 20:45 |
| *** puranamr has joined #openstack-infra | 20:45 | |
| fungi | yep | 20:45 |
| jeblair | yeah so that should be in there (and i'm pretty sure we restarted explicitly for that, plus restarted again later for the enqueue-ref thing) | 20:46 |
| fungi | pbr freeze says 91e3459 is installed | 20:46 |
| pabelanger | jeblair: so, project-config just landed: https://github.com/openstack-infra/project-config/commit/40bb9094a1e447ae9e4f9f3188d0e5d0d69c712e which removed infra-puppet-apply-jobs. I think this is the first job to run with out the check jobs | 20:46 |
| *** stevemar has quit IRC | 20:46 | |
| fungi | so it definitely looks like we restarted with those patches in place | 20:46 |
| *** sergmelikyan has joined #openstack-infra | 20:46 | |
| jeblair | oh.... | 20:47 |
| jeblair | i wonder if it's because it was removed from a shared queue? | 20:47 |
| fungi | that would certainly have been in a shared queue | 20:47 |
| jeblair | and i'm pretty sure it's not now | 20:48 |
| fungi | yeah, project-config was severed from the shared queue with the puppet modules by that change | 20:49 |
| jeblair | i believe i don't need any more running state to debug this... | 20:49 |
| jeblair | fungi: do you want to work on a restart while i work on a test case? | 20:49 |
| fungi | jeblair: yep, getting started on that now. should i expect to be able to sanely dump the current queue states? | 20:50 |
| openstackgerrit | Sean M. Collins proposed openstack-infra/project-config: [WIP] IPv6 only OpenStack experimental job https://review.openstack.org/209663 | 20:50 |
| jeblair | fungi: yes | 20:50 |
| fungi | dumped gate and check and they look like what i expect. have you been dumping/restoring any other pipelines when you do this lately? | 20:52 |
| fungi | yeah, looks like it still can't do e.g. post | 20:52 |
| fungi | okay, restarting zuul now | 20:52 |
| *** Sukhdev has joined #openstack-infra | 20:52 | |
| fungi | seems to have started up okay | 20:53 |
| fungi | jeblair: do i need to delay for gearman registrations before reenqueuing or should it be fine to restore immediately? | 20:54 |
| *** amitgandhinz has joined #openstack-infra | 20:54 | |
| *** sharikapongubala has joined #openstack-infra | 20:55 | |
| jeblair | fungi: i'd check with nodepool and see if it looks like there are ready nodes for everything -- i don't think the system went fully idle, so it may be good to wait a sec | 20:56 |
| clarkb | fungi: it should be fine as long as we have nodes attached to jenkins that will register the functions | 20:56 |
| *** rguillebert has quit IRC | 20:56 | |
| *** abregman has quit IRC | 20:56 | |
| *** thedodd has quit IRC | 20:58 | |
| fungi | nodepool list seems to indicate there are existing representatives of all the labels we care about anyway | 20:58 |
| *** gordc has quit IRC | 20:59 | |
| *** zz_dimtruck is now known as dimtruck | 20:59 | |
| fungi | #status notice Zuul has been restarted to resolve a reconfiguration failure: previously running jobs have been reenqueued but change events between 19:50-20:54 UTC have been lost and will need to be rechecked or their approvals reapplied to trigger testing. | 21:04 |
| openstackstatus | fungi: sending notice | 21:04 |
| jeblair | fungi: oh, i don't think any events were lost | 21:04 |
| jeblair | fungi: only ones during the downtime itself | 21:04 |
| zigo | Does anyone know bits of RPM packaging policy for libraries? Is it normal for a lib to tweak the rpath? | 21:04 |
| zigo | FYI, in Debian, it shouldn't be done at all. | 21:05 |
| -openstackstatus- NOTICE: Zuul has been restarted to resolve a reconfiguration failure: previously running jobs have been reenqueued but change events between 19:50-20:54 UTC have been lost and will need to be rechecked or their approvals reapplied to trigger testing. | 21:05 | |
| russellb | zigo: #rdo would be a good place to ask i thikn | 21:05 |
| fungi | jeblair: it kept adding changes even though the main scheduler loop was raising an exception? i'll double check in just a second | 21:06 |
| zigo | russellb: Thanks. | 21:06 |
| openstackstatus | fungi: finished sending notice | 21:06 |
| *** bapalm has quit IRC | 21:06 | |
| *** armax has joined #openstack-infra | 21:07 | |
| fungi | jeblair: you are indeed correct. the last entry i exported from the check pipeline was uploaded 3-4 minutes prior to the restart | 21:08 |
| openstackgerrit | Merged openstack-infra/project-config: Ensure missing rpm packages installed & useable for anvil-jobs https://review.openstack.org/209186 | 21:08 |
| fungi | i'll issue a retraction | 21:08 |
| fungi | #status notice Correction: change events between 20:50-20:54 UTC (during the restart only) have been lost and will need to be rechecked or their approvals reapplied to trigger testing. | 21:09 |
| openstackstatus | fungi: sending notice | 21:09 |
| *** abregman has joined #openstack-infra | 21:09 | |
| *** tiswanso has quit IRC | 21:09 | |
| -openstackstatus- NOTICE: Correction: change events between 20:50-20:54 UTC (during the restart only) have been lost and will need to be rechecked or their approvals reapplied to trigger testing. | 21:10 | |
| *** julim has quit IRC | 21:10 | |
| openstackgerrit | Merged openstack-infra/project-config: Make gate-infra-puppet-apply-dsvm-(centos7|f21) voting https://review.openstack.org/209549 | 21:10 |
| *** radez is now known as radez_g0n3 | 21:10 | |
| openstackgerrit | Merged openstack-infra/system-config: Migrate to puppet-httpd module https://review.openstack.org/198819 | 21:10 |
| *** mrmartin has quit IRC | 21:11 | |
| openstackstatus | fungi: finished sending notice | 21:11 |
| openstackgerrit | Merged openstack-infra/system-config: Migrate grafana.o.o to puppet-httpd https://review.openstack.org/201759 | 21:12 |
| *** yamamoto has joined #openstack-infra | 21:12 | |
| openstackgerrit | Davide Agnello proposed openstack-infra/project-config: Adding RALLY_SCENARIO venv for Cue job https://review.openstack.org/201285 | 21:12 |
| *** chlong has quit IRC | 21:16 | |
| fungi | https://github.com/stackforge/puppet-ceph/pull/11#issuecomment-128151002 is yet more fodder for 1. our cla does get in the way, and 2. projects which enforce it have people who are willing to bypass it making it basically pointless | 21:16 |
| *** yamamoto has quit IRC | 21:16 | |
| fungi | russellb: ^ fun times! | 21:17 |
| SpamapS | crinkle: no we're done with 195276 .. starting over .. but I learned a lot! :) | 21:17 |
| * russellb perks up | 21:17 | |
| *** ldnunes has quit IRC | 21:18 | |
| russellb | fungi: nice reference | 21:18 |
| jeblair | grr... somehow we still have the suggestion that you need to do something other than "git review" | 21:19 |
| russellb | fungi: it's a very good example against the weak argument about "foo % of contributors are already covered by CCLA" because it shows exactly that the people we miss aren't tracked because they just walk away | 21:19 |
| * jeblair rage-removes text from infra-manual | 21:19 | |
| openstackgerrit | Doug Wiegley proposed openstack-infra/project-config: Create openstack/kosmos project repo and jobs as experimental https://review.openstack.org/201658 | 21:20 |
| fungi | russellb: (or someone else just uploads the change for them, ignoring the fact that it's undermining whatever it is the cla was supposed to be protecting against) | 21:22 |
| *** krtaylor has joined #openstack-infra | 21:23 | |
| *** abdelwas has quit IRC | 21:23 | |
| *** pal has quit IRC | 21:24 | |
| jeblair | russellb, fungi: and those steps can/should be simply "sign in to gerrit" "install git-review" "run git review" | 21:24 |
| *** aysyd has quit IRC | 21:25 | |
| *** alivigni has quit IRC | 21:25 | |
| jeblair | which is, for the record _less_ difficult than using github | 21:25 |
| *** rguillebert has joined #openstack-infra | 21:25 | |
| *** nadya has quit IRC | 21:25 | |
| *** abregman has quit IRC | 21:26 | |
| fungi | yep | 21:26 |
| jeblair | ("sign into github", "create a local fork", "upload your change to your fork", "create a pull request from your fork") | 21:26 |
| *** baoli has quit IRC | 21:26 | |
| *** baoli_ has joined #openstack-infra | 21:26 | |
| *** b10n1k has quit IRC | 21:27 | |
| nibalizer | jeblair: i have 'git fork' and 'git pull-request' | 21:27 |
| fungi | the things done by git review -s are done by any initial invocation of git review, and if your username seems to be incorrect it will prompt you | 21:27 |
| jesusaurus | jeblair: there's also a 'git hub' tool that i have used but not in a while | 21:28 |
| nibalizer | my tooling wraps around that p-o-s | 21:29 |
| jesusaurus | heh | 21:29 |
| fungi | that said, now that git-review is carried in distros, there are some versions of it in the wild with known bugs (particularly around the credential dry-run failure detection) which get confused by cla/contact info errors and claim your username is wrong rather than passing through the actual error message | 21:29 |
| clarkb | I think github wants you to click ui buttona in their gui thing | 21:29 |
| jeblair | nibalizer: yeah, that gets it from 4 to 3 steps, nearly exactly analagous to gerrit | 21:29 |
| *** chlong has joined #openstack-infra | 21:30 | |
| dstufft | gerrit's UI is pretty ugly though | 21:31 |
| jeblair | dstufft: that is hurtful | 21:31 |
| russellb | meh, it's quite functional IMO | 21:33 |
| russellb | i don't get all the hate | 21:33 |
| fungi | gerrit's ui used to be a lot worse | 21:33 |
| fungi | then they borrowed our improvements | 21:33 |
| russellb | i was very disappointed in the tone change at the last board meeting around DCO | 21:33 |
| russellb | i hope it turns around | 21:33 |
| fungi | then they decided to make it worse again ;) | 21:33 |
| taron | ugly as it may be it's still more usable than github | 21:33 |
| openstackgerrit | Kirill Zaitsev proposed openstack-infra/project-config: make gate-murano-dashboard-npm-run-lint voting https://review.openstack.org/209676 | 21:33 |
| dstufft | russellb: maybe if you're used to it. I rarely have cause to use it and typically get frustrated when I do *shrug* | 21:34 |
| russellb | yeah, been using it a lot for the last 4 years or whatever | 21:34 |
| fungi | russellb: you mean "oh, there are some lawyers at member companies who haven't heard about this, we should make sure they're _all_ okay with it"? | 21:34 |
| russellb | hard to have the newcomer perspective | 21:34 |
| russellb | fungi: that's what was said, but i think it was a little worse than that, because not a single person in the room actually expressed that opinion | 21:35 |
| jeblair | dstufft: is there something that could be improved for you? | 21:35 |
| russellb | fungi: it was that some particular company's lawyers claimed that in the private lawyer meeting | 21:35 |
| *** patrickeast has quit IRC | 21:35 | |
| russellb | fungi: trying to shut the whole thing down it seems | 21:35 |
| russellb | really frustrating, because it seemed we had consensus | 21:35 |
| fungi | russellb: oh, right. also private lawyer meetups. i can't begin to imagine the punch and brownies they serve at those things | 21:35 |
| *** baoli_ has quit IRC | 21:36 | |
| Daviey | russellb: Forgive my ignorance, but DCO? | 21:36 |
| fungi | dstufft: have to say i much prefer gertty to any webui. i'd probably be mildly less allergic to github if it had a polished console-based interface (is there a good one?), though my biggest issue with it is still that it's not free software | 21:36 |
| russellb | Daviey: Developer Certificate of Origin | 21:37 |
| fungi | Daviey: http://docs.openstack.org/infra/manual/developers.html#using-signed-off-by | 21:37 |
| *** patrickeast has joined #openstack-infra | 21:38 | |
| *** kgiusti has left #openstack-infra | 21:38 | |
| fungi | dstufft: though i get that free software's not for everyone, and that there are probably plenty of people who think i'm a zealot for trying to use only free and open toolchains | 21:39 |
| dstufft | jeblair: I'm not particularly good at UX so I wouldn't feel comfortable trying to make concrete suggestions. It's not really one singular thing but just a bunch of small things. It's quite possible that it doesn't mesh well with my expectations and I'm just the weirdo :) I just know that I tend to get frustrated. | 21:39 |
| Daviey | fungi: Hah, funnily enough - i use that on all my commits by habbit. | 21:39 |
| *** marzif__ has quit IRC | 21:39 | |
| dstufft | (I also don't care if folks, or openstack or whatever uses github or gerrit or smoke sigals :V) | 21:39 |
| dstufft | fungi: I'm not aware of a console based interface for github, but I've never attempted to look for one | 21:39 |
| rockyg | dstufft, I get frstrated, too. | 21:39 |
| fungi | dstufft: i pushed for smoke signals, but the fire marshall overrode me | 21:40 |
| *** dkalleg has joined #openstack-infra | 21:40 | |
| rockyg | But, part of mine is that I'm on a windows machine. | 21:40 |
| clarkb | rockyg: that is easily fixed | 21:40 |
| rockyg | Yeah. Gotta get a "lab" machine to have linux. They don't let us hook it up to the corporate net. | 21:41 |
| * fungi is surprised that your corporate security is less afraid of windows. i'd be frightened to have one plugged into my networks | 21:41 | |
| clarkb | when I had to deal wiht that I just ran a full screened VM on windows | 21:41 |
| rockyg | But also, changed machines twice in the past nine months. IT copies over all my stuff, and gerritt vomits on the old configs | 21:41 |
| rockyg | clarkb, they don't want us to have that on corporate, either | 21:42 |
| *** MarkAtwood has joined #openstack-infra | 21:42 | |
| clarkb | oh I am sure they didn't want me to either but meh | 21:42 |
| clarkb | it worked better than the windows host for getting work done | 21:42 |
| clarkb | and was likely more secure | 21:42 |
| rockyg | still haven't gotten gerritt to work on the new machine. Same error message as that guy who's been posting to the dev list | 21:42 |
| dstufft | fungi: eh, there's different layers of zealotry I suppose :) My personal line is less "all free all the time", and more about picking whatever tool lets me more effectively do what I want regardless of if it's free/open or not (factoring in how hard it would be to migrate off said thing if I needed to) | 21:43 |
| clarkb | rockyg: can you link to the thread? I am not sure I have seen it | 21:43 |
| dkalleg | Hi, I'm working on a patch that will pass our local integration tests, but will fail when Jenkins runs the same tests. I've spent about a day doing trial and error, but what would really help is being able to look at the live setup that Jenkins creates before it is torn down. I'm told that this can be done on special occasions/request. Can anybody help me with this? | 21:43 |
| rockyg | clarkb, lemme see. I've got it flagged, just have to backtrack... | 21:43 |
| clarkb | dkalleg: can you point to the failure(s) first so we can see if we can debug it with the existing logs? | 21:44 |
| dkalleg | clarkb: you bet, 1 min | 21:44 |
| clarkb | dkalleg: we try to only hold nodes and hand them out in exceptional cases. most of the time the logs are sufficient to at least work on reproducing then debug from there | 21:44 |
| *** zul has quit IRC | 21:45 | |
| *** ajmiller has quit IRC | 21:45 | |
| dkalleg | clarkb: http://logs.openstack.org/28/206228/7/check/gate-cue-integration-dsvm-rabbitmq/0cf041f/ | 21:45 |
| *** ajmiller has joined #openstack-infra | 21:46 | |
| dkalleg | clarkb: I've been pulling as much info from the logs as I can. I'm with the messaging as a service team. The test that is failing will provision a cluster of nodes that are running RabbitMQ. As part of the setup, some setup commands must be ran against RabbitMQ (enabling the management plugin). The success/failure of this isn't logged. | 21:47 |
| *** tiswanso has joined #openstack-infra | 21:47 | |
| *** cburroughs has joined #openstack-infra | 21:47 | |
| *** mriedem is now known as mriedem_away | 21:47 | |
| clarkb | dkalleg: there are a bunch of urlerrors in cue.taskflow | 21:47 |
| clarkb | connections being refused | 21:48 |
| *** geoffarnold has quit IRC | 21:48 | |
| *** prad has quit IRC | 21:48 | |
| greghaynes | SpamapS: ok, back from lunch meeting, im also going to demo this to crinkle while I mess with nodes so she can get a bit of an idea of what were up to | 21:48 |
| clarkb | dkalleg: ok first step is to log those then | 21:48 |
| dkalleg | Yes. Taskflow is running a task that will loop for 30 mins until the url open succeeds. The errors show a connection refused. If the cloudinit scripts run properly, the management plugin will be enabled and we shouldn't see this. | 21:49 |
| clarkb | dkalleg: wait are you booting VMs and expecting them to do things? | 21:49 |
| dkalleg | clarkb: Do cloudinit scripts write log output anywhere that I can access? | 21:49 |
| dkalleg | clarkb: yes | 21:49 |
| *** monester has joined #openstack-infra | 21:49 | |
| clarkb | dkalleg: ok that likely won't work because everything is running under qeu and its super slow | 21:50 |
| clarkb | *qemu | 21:50 |
| *** geoffarnold has joined #openstack-infra | 21:50 | |
| openstackgerrit | Steven Dake proposed openstack-infra/project-config: Move Kolla to the openstack git namespace https://review.openstack.org/209343 | 21:50 |
| dkalleg | clarkb: well, we're not putting a load on it, we're running commands to make sure that a rabbitMQ cluster goes active | 21:50 |
| rockyg | clarkb, http://lists.openstack.org/pipermail/openstack-dev/2015-July/070551.html | 21:50 |
| *** ub has quit IRC | 21:50 | |
| clarkb | dkalleg: and yes you can ask nova for the hosts console log | 21:50 |
| *** patrickeast has quit IRC | 21:50 | |
| clarkb | dkalleg: yes even that may not be possible with qemu | 21:50 |
| rockyg | I think I have old cruft in some of my dev directories' config dirs/files | 21:51 |
| jeblair | fungi: should we file a CVE for the infra DoS vulnerability (mention "CLA" or "github" in channel and the infra team suspends processing for an indeterminate amount of time) | 21:51 |
| *** tlian has quit IRC | 21:51 | |
| *** tiswanso has quit IRC | 21:51 | |
| *** patrickeast has joined #openstack-infra | 21:51 | |
| clarkb | jeblair: ha, I am helping dkalleg :P | 21:51 |
| dkalleg | clarkb: The test that is failing has been doing this task for a while now. Before, to confirm that a RMQ cluster was successfully provisioned, we would telnet the RMQ port. My patch will instead use the RMQ rest api to check if the nodes are clustered. (So it seems that RMQ isn't init'ing properly since we are seeing url connection refused). | 21:52 |
| jeblair | clarkb has been patched | 21:52 |
| clarkb | dkalleg: OH! in that case you may need to open security groups | 21:52 |
| *** davideagnello has quit IRC | 21:52 | |
| *** davideagnello has joined #openstack-infra | 21:53 | |
| dkalleg | clarkb: Yes, I believe those are open as well, but could be wrong :) Our cloud init scripts do this properly when I run these tests locally. I'm not yet sure where the delta is between my local setup and the Jenkins environmen | 21:53 |
| fungi | jeblair: there probably already is a cve for that ;) | 21:53 |
| clarkb | dkalleg: so things I would check: grab console log from nova, make sure security groups in nova allow the traffic to pass that you are making. Also you have to communicate over a floating ip iirc as the 10 net private network is isolated from the tempest host when running devstack there | 21:53 |
| fungi | anyway, i'm popping out to find food. back later | 21:53 |
| clarkb | dkalleg: cloud init does not update security groups usually. cloud init only change sthings on the VM itself | 21:53 |
| *** olaph has joined #openstack-infra | 21:54 | |
| *** ajmiller has quit IRC | 21:54 | |
| clarkb | dkalleg: the localrc we use is published in the logs dir, you can grab it and run devstack again then run your test script | 21:54 |
| clarkb | dkalleg: that should get you a very similar test env | 21:54 |
| *** b10n1k has joined #openstack-infra | 21:54 | |
| *** ajmiller has joined #openstack-infra | 21:54 | |
| clarkb | dkalleg: just remove the ERROR_ON_CLONE setting | 21:55 |
| *** rguillebert has quit IRC | 21:55 | |
| dkalleg | clarkb: Ok i'll take some time double checking the nova console logs. How do I make sure sec groups are set properly in the Jenkins env? (thats one of my biggest suspicions, I just don't know how to prove it yet) Will nova console logs show that? I wasn't aware that I needed a floating IP, i'll look into that as well. Will check out the localrc as well | 21:58 |
| dkalleg | clarkb: may come back with new questions :) | 21:58 |
| clarkb | dkalleg: the devstack env does not add any rules iirc so you would hvae to explicitly add them | 21:59 |
| clarkb | dkalleg: in tempest the tests set up whatever rules they need | 21:59 |
| clarkb | dkalleg: and for the private IP I am pretty sure that that isn't expected to be routable from tempest | 22:01 |
| clarkb | as its supposed to be black box testing and not get into the internals so has to be treated as public access | 22:01 |
| dkalleg | clarkb: I found where my team is setting the reset of their secgroup rules and added my rule there. I'm told this should be getting copied to the Jenking devstack host and being applied. Its under our contrib/devstack/lib/ dir (if that means anything to you, not sure how much of this is standardized) | 22:02 |
| clarkb | I thinkdevstack plugins are supposed to be devstack/plugin.sh so if that includes from the lib dir that probably works | 22:03 |
| dkalleg | and yes, sorry, I had a brain freeze, our cloud init script should be taking care of enabling the RMQ management plugin (which is my 2nd biggest suspicion) | 22:03 |
| *** rlandy has quit IRC | 22:04 | |
| clarkb | I thought puppet-ceph wasn't CLA'd | 22:05 |
| *** hemna has quit IRC | 22:05 | |
| clarkb | that was one of their requirements for being gerrited /me checks | 22:05 |
| *** hemna has joined #openstack-infra | 22:06 | |
| clarkb | thats correct no CLA required | 22:06 |
| *** hemna has quit IRC | 22:06 | |
| *** abregman has joined #openstack-infra | 22:06 | |
| *** amitgandhinz has quit IRC | 22:08 | |
| greghaynes | SpamapS: huh, I cant ping our bastion from an hpcloud node I made | 22:09 |
| greghaynes | SpamapS: Im thinking theres some messed up routing going on :( | 22:09 |
| clarkb | greghaynes: are your nodes hosted in hpcloud undercloud land? | 22:10 |
| *** monester has quit IRC | 22:10 | |
| clarkb | thats normal if so | 22:10 |
| jeblair | clarkb, fungi, pabelanger: found/fixed zuul bug | 22:10 |
| jeblair | it's a subtle variant of the thing we thought we fixed, only tangentially related to shared queues | 22:10 |
| greghaynes | clarkb: hrm? I mean I just booted a node in hpcloud to dev on, ping bastion and no workey | 22:10 |
| clarkb | greghaynes: ya hpcloud is not routable from hp | 22:10 |
| greghaynes | hah | 22:11 |
| greghaynes | ok | 22:11 |
| crinkle | lolwut | 22:11 |
| clarkb | I think they made ports 80 and 443 work but everything else is bitbucketed | 22:11 |
| clarkb | (or was when I was there | 22:11 |
| greghaynes | wooow | 22:11 |
| clarkb | ya it made being on the VPN a real pita | 22:11 |
| *** BrandonL08 has quit IRC | 22:11 | |
| lifeless | greghaynes: are you pinging from hpcloud to the bastion? | 22:11 |
| lifeless | greghaynes: cause that should be fine | 22:12 |
| lifeless | hpcloud to corp is a problem, but our bastions were separate IIRC | 22:12 |
| *** pc_m has quit IRC | 22:12 | |
| rcarrillocruz | if it's a LR1 network, it's more than likely that only 80 and 443 is allowed to whichever bastion systems | 22:13 |
| rcarrillocruz | kind of 'standard' network setups within HP | 22:13 |
| *** otter768 has joined #openstack-infra | 22:13 | |
| greghaynes | lifeless: yes | 22:13 |
| greghaynes | lifeless: hrm | 22:13 |
| *** angdraug has quit IRC | 22:14 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Fix bug with shared jobs in reconfiguration https://review.openstack.org/209686 | 22:14 |
| jeblair | clarkb, fungi: ^ | 22:15 |
| clarkb | looking | 22:15 |
| *** armax has quit IRC | 22:15 | |
| *** rguillebert has joined #openstack-infra | 22:15 | |
| lifeless | rcarrillocruz: its a dedicated rack in one of the hpcloud public cloud dc's | 22:17 |
| lifeless | rcarrillocruz: e.g. not 'standard' in any sense of the word | 22:17 |
| *** yamamoto has joined #openstack-infra | 22:17 | |
| *** otter768 has quit IRC | 22:18 | |
| openstackgerrit | Merged openstack-infra/project-config: Fix glossary recognition https://review.openstack.org/209622 | 22:19 |
| clarkb | jeblair: I see it attached the job because the job existsed but the job no longer belonged to that project | 22:19 |
| openstackgerrit | Khai Do proposed stackforge/python-jenkins: Rework the examples https://review.openstack.org/207590 | 22:19 |
| jeblair | clarkb: yep; we had blinders on looking at the earlier change -- of course the application of the job to the project is just as important as whether it exists at all. :) | 22:20 |
| *** cburroughs has quit IRC | 22:20 | |
| openstackgerrit | Khai Do proposed stackforge/python-jenkins: Rework the examples https://review.openstack.org/207590 | 22:21 |
| openstackgerrit | Davide Agnello proposed openstack-infra/project-config: Adding RALLY_SCENARIO venv for Cue job https://review.openstack.org/201285 | 22:21 |
| *** yamamoto has quit IRC | 22:22 | |
| *** rockyg has quit IRC | 22:22 | |
| clarkb | jeblair: lgtm | 22:23 |
| *** tonytan4ever has quit IRC | 22:23 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:27 | |
| *** achanda has joined #openstack-infra | 22:28 | |
| *** tongli has quit IRC | 22:28 | |
| greghaynes | SpamapS: So, the stuff in your homedir is as your user... im torn between doing stuff as root or adding keys to your user | 22:28 |
| clarkb | TOOR! | 22:28 |
| greghaynes | I dont want to uber troll you with perm issues if you try and bifrost later | 22:28 |
| morganfainberg | harlowja: great intro to the DLM stuff | 22:29 |
| greghaynes | eh, just gonna do it as root, I want this box dead anyhow | 22:29 |
| harlowja | morganfainberg thx, i just updated commit message, so now its truely better | 22:29 |
| harlowja | truley super now | 22:29 |
| * harlowja will see how long that lasts (the commit message), ha | 22:29 | |
| *** hemna has joined #openstack-infra | 22:30 | |
| morganfainberg | harlowja: and I agree with the other thread (and SpamapS and fungi) re: pacemaker - but these are somewhat orthogonal concerns. Good conversations to have | 22:30 |
| harlowja | morganfainberg i wasn't kidding that i've never heard of a distributed system that needed it before openstack :-/ | 22:30 |
| harlowja | no other big distributed system at yahoo has such a requirement, so it confuzzled me and still does that openstack has it listed as the 'way to HA' | 22:31 |
| *** thorst has quit IRC | 22:32 | |
| *** dprince has quit IRC | 22:32 | |
| *** rguillebert has quit IRC | 22:32 | |
| *** angdraug has joined #openstack-infra | 22:33 | |
| openstackgerrit | Merged openstack/os-testr: update requirements https://review.openstack.org/207691 | 22:33 |
| *** bknudson has quit IRC | 22:33 | |
| openstackgerrit | Matthew Treinish proposed openstack/os-testr: Set min pbr version in setup_requires https://review.openstack.org/209691 | 22:34 |
| *** rguillebert has joined #openstack-infra | 22:34 | |
| *** minwang2 has joined #openstack-infra | 22:38 | |
| andreaf | can I get reviews on https://review.openstack.org/#/c/208481/? we're trying to add a non-voting job to tempest with server validation enabled to asses stability of ssh checks | 22:38 |
| *** prad has joined #openstack-infra | 22:38 | |
| greghaynes | SpamapS: reenrolling .3 as bastion01 on the bastion host... | 22:38 |
| *** dimtruck is now known as zz_dimtruck | 22:42 | |
| *** marcusvrn has quit IRC | 22:43 | |
| *** puranamr has quit IRC | 22:45 | |
| *** patrickeast has quit IRC | 22:46 | |
| *** patrickeast has joined #openstack-infra | 22:48 | |
| *** abregman has quit IRC | 22:49 | |
| *** tsg has quit IRC | 22:50 | |
| *** puranamr has joined #openstack-infra | 22:51 | |
| *** mordred has joined #openstack-infra | 22:52 | |
| *** Sukhdev has quit IRC | 22:52 | |
| *** patrickeast has quit IRC | 22:53 | |
| fungi | morganfainberg: harlowja: i was mostly just trolling said use of definite article | 22:54 |
| harlowja | to much english for me, lol | 22:54 |
| * fungi needed some amusement this morning, admittedly a selfish indulgence | 22:54 | |
| morganfainberg | fungi: and it was good trollllllllololololololololol | 22:54 |
| *** patrickeast has joined #openstack-infra | 22:55 | |
| fungi | harlowja: i sort of buy the argument that there are systems which, in their endeavor to create a high availability model, may badly reinvent existing high availability wrappers. that's not to say that some systems don't just use the same wrappers to paper over poor design | 22:56 |
| mordred | IRC bouncer died. did I miss anything good? | 22:57 |
| *** xnox has quit IRC | 22:57 | |
| harlowja | fungi sure i get that | 22:57 |
| *** austin81 has left #openstack-infra | 22:57 | |
| clarkb | mordred: zuul died again for similar reasons to before but jeblair has a fix | 22:57 |
| clarkb | mordred: nodepool dib builds appear t o be uplodaing to both cloud sreliably now | 22:57 |
| fungi | mordred: 209686 | 22:58 |
| fungi | (reviewing now) | 22:58 |
| harlowja | fungi the ability to distinguish between the 2 is the hard part (and is biased by your scale, your past experience, the shade of the moon...) | 22:58 |
| mordred | clarkb: a) yuck but good - b) AWESOME | 22:58 |
| fungi | harlowja: i prefer the dark side of the moon | 22:59 |
| harlowja | lol | 22:59 |
| harlowja | isn't that where the transformers hide? | 22:59 |
| fungi | http://www.theverge.com/2015/8/5/9100101/earth-moon-gif-nasa | 22:59 |
| *** xnox has joined #openstack-infra | 22:59 | |
| nibalizer | fungi: I prefer 'Wish You Were Here' | 23:00 |
| fungi | epic and pink floyd, two great tastes that go great together | 23:00 |
| fungi | nibalizer: careful with that axe, eugene | 23:00 |
| fungi | also have a cigar and welcome to the machine | 23:00 |
| fungi | those would have to be my top 3 | 23:01 |
| * fungi has a vast collection of live floyd bootlegs | 23:01 | |
| *** tsg has joined #openstack-infra | 23:01 | |
| clarkb | fungi: pretty sure those make up half of pandoras collection | 23:01 |
| fungi | heh | 23:01 |
| fungi | maybe pandora uses my collection ;) | 23:02 |
| jhesketh | Morning | 23:03 |
| clarkb | fungi: I think live recordings hve different ownership as its a performance or whatever and its easier for them to buy up the rights to them | 23:03 |
| clarkb | fungi: right now a live acoustic version of hotel california is playing | 23:03 |
| mordred | fungi: lgtm | 23:04 |
| *** achanda has quit IRC | 23:04 | |
| *** olaph has quit IRC | 23:06 | |
| *** esker has joined #openstack-infra | 23:06 | |
| fungi | and it's passing the py26 unit tests too, so approving | 23:06 |
| greghaynes | clarkb: nice! | 23:07 |
| greghaynes | clarkb: now we just need to fix resizing | 23:07 |
| greghaynes | ETOOMANYTHINGS | 23:07 |
| clarkb | greghaynes: yup and check that devstack-gate things work | 23:07 |
| clarkb | greghaynes: which they *should* with parted installed but who knows | 23:07 |
| *** notnownikki has quit IRC | 23:07 | |
| *** wenlock has quit IRC | 23:10 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config: First stab at making things bifrosty https://review.openstack.org/208751 | 23:12 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config: Create playbook to puppet InfraCloud nodes https://review.openstack.org/209632 | 23:12 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud https://review.openstack.org/209698 | 23:12 |
| harlowja | morganfainberg ok, i'll let https://review.openstack.org/#/c/209661/ sit for a while, let's see what people think so far, ha | 23:13 |
| greghaynes | hrmm, thi snode sure is hanging out at wait call-back for a while | 23:13 |
| *** olaph has joined #openstack-infra | 23:13 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Remove query for bug 1476770 https://review.openstack.org/209650 | 23:15 |
| openstack | bug 1476770 in OpenStack-Gate "_translate_from_glance fails with "AttributeError: id" in grenade" [Undecided,Fix committed] https://launchpad.net/bugs/1476770 - Assigned to Matt Riedemann (mriedem) | 23:15 |
| *** geoffarnold has quit IRC | 23:15 | |
| *** Sukhdev has joined #openstack-infra | 23:15 | |
| krotscheck | Anyone out there want to help me do javascript things? I'd need another +2, and someone willing to go register openstackci on npm | 23:20 |
| *** woodster_ has quit IRC | 23:20 | |
| *** pvaneck has quit IRC | 23:20 | |
| krotscheck | (Also, hiera) | 23:21 |
| *** esker has quit IRC | 23:22 | |
| *** esker has joined #openstack-infra | 23:23 | |
| asselin_ | clarkb, I saw your name on this. Do you know if it was ever implemented? https://wiki.openstack.org/wiki/Spec-provider-firewall | 23:24 |
| greghaynes | SpamapS: so, when trying to deploy to .3 it looks like its not successfully pxe'ing | 23:25 |
| greghaynes | SpamapS: textcons on the ilo shows attempting to boot from nic, then c:, then just hangs | 23:25 |
| greghaynes | im wondering if we derped up something else with routing where dhcp isnt getting back to ironic from main nic | 23:25 |
| openstackgerrit | Clark Boylan proposed openstack-infra/elastic-recheck: Track nova net dhcp failures https://review.openstack.org/209699 | 23:25 |
| clarkb | asselin_: I have no idea how my name would be attached to that :) don't know if it was implemented | 23:26 |
| greghaynes | SpamapS: oh, I take that back | 23:26 |
| clarkb | asselin_: that said rules like that are far more trivial to manage at the firewall betwnee your cloud nad the internet | 23:27 |
| asselin_ | clarkb, it was hidden in the history :) https://wiki.openstack.org/w/index.php?title=Spec-provider-firewall&action=history | 23:27 |
| greghaynes | I got mellanox'd | 23:27 |
| clarkb | greghaynes: ahahahahahahaha, sorry | 23:27 |
| greghaynes | clarkb: I think we should keep these cards purely for your entertainment | 23:28 |
| *** dangers is now known as dangers_away | 23:28 | |
| clarkb | greghaynes: you should | 23:28 |
| greghaynes | yeaaa, doesnt get a dhcp | 23:29 |
| greghaynes | awesome | 23:29 |
| clarkb | asselin_: that looks like I did some sort of formating cleanup probably when we switched Wiki implementations | 23:30 |
| *** puranamr has quit IRC | 23:31 | |
| openstackgerrit | Merged openstack-infra/project-config: Make gate-ceilometer-dsvm-functional-mongodb voting https://review.openstack.org/209398 | 23:33 |
| openstackgerrit | Merged openstack-infra/project-config: Add check-requirements to networking-onos https://review.openstack.org/209482 | 23:33 |
| openstackgerrit | Merged openstack-infra/project-config: Add check-requirements to networking-sfc https://review.openstack.org/209491 | 23:33 |
| *** olaph has quit IRC | 23:34 | |
| greghaynes | SpamapS: hrm, dnsmasq config looks wierd, seems like its setup to talk to a virbr0 rather than on brbm | 23:34 |
| *** olaph has joined #openstack-infra | 23:34 | |
| greghaynes | SpamapS: did I nuke a config you had in place maybe by running the install playbook? | 23:34 |
| *** dingyichen has joined #openstack-infra | 23:36 | |
| clarkb | I recommend running the dhcp sserver where the dhcpdiscovers will appear | 23:38 |
| greghaynes | I think I may have found teh problem | 23:39 |
| greghaynes | we will see after 2 mins of HP post though | 23:39 |
| *** esker has quit IRC | 23:39 | |
| clarkb | speaking of slow boots, I keep meaning to check if our dib build images kill the grub timeout | 23:40 |
| greghaynes | clarkb: they do | 23:40 |
| clarkb | yay | 23:40 |
| greghaynes | or at least, they did last I checked, im not sure I re-checked for ubuntu-minimal | 23:40 |
| greghaynes | I think the vm element does it though, so should | 23:40 |
| clarkb | perfect | 23:40 |
| harlowja | 'quick/lucrative remote consulting opportunity' oh man, temptation to troll, must resist | 23:40 |
| harlowja | * http://lists.openstack.org/pipermail/openstack-operators/2015-August/007893.html | 23:41 |
| *** esker has joined #openstack-infra | 23:41 | |
| harlowja | must not respond, muuust not, lol | 23:41 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config: Add controller class for Infra Cloud https://review.openstack.org/209698 | 23:42 |
| *** armax has joined #openstack-infra | 23:43 | |
| nibalizer | oooooooo | 23:44 |
| *** MarkAtwood has quit IRC | 23:44 | |
| *** MarkAtwood has joined #openstack-infra | 23:45 | |
| *** sergmelikyan has quit IRC | 23:46 | |
| *** esker has quit IRC | 23:46 | |
| *** oomichi has joined #openstack-infra | 23:47 | |
| *** tiswanso has joined #openstack-infra | 23:48 | |
| *** bswartz has quit IRC | 23:48 | |
| *** julim has joined #openstack-infra | 23:50 | |
| *** kmartin has quit IRC | 23:51 | |
| greghaynes | SpamapS: yeaaa, I remember you had some funkyness with getting dhcp to work here... | 23:51 |
| *** patrickeast has quit IRC | 23:52 | |
| *** tiswanso has quit IRC | 23:52 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/elastic-recheck: Track nova net dhcp failures https://review.openstack.org/209699 | 23:53 |
| *** hemna has quit IRC | 23:53 | |
| *** SumitNaiksatam has quit IRC | 23:56 | |
| *** sdake has quit IRC | 23:56 | |
| minwang2 | i got a problem when i setup upstream gate | 23:59 |
| minwang2 | https://jenkins02.openstack.org/job/gate-neutron-lbaasv1-dsvm-api/504/console | 23:59 |
| minwang2 | can someone help me to take a look? | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
