| lifeless | harlowja: k; let me know if you want such a bootstrap | 00:00 |
|---|---|---|
| harlowja | np | 00:00 |
| harlowja | van.pg looks pretty simple, so probably can figure it out | 00:00 |
| lifeless | I'm not suggesting you use testresources with it - thats a separate thing we can and should do | 00:01 |
| lifeless | just make a resource manager and call setup() or whatever the method is (I forget :/) on it and pull out the resource. done. | 00:01 |
| harlowja | probably start or something, https://github.com/jinty/van.pg/blob/master/van/pg/_cluster.py#L157 | 00:02 |
| harlowja | and initdb | 00:02 |
| harlowja | something like that | 00:02 |
| *** reed has quit IRC | 00:04 | |
| *** atiwari has quit IRC | 00:04 | |
| *** rcleere has quit IRC | 00:05 | |
| *** matsuhashi has joined #openstack-infra | 00:06 | |
| *** masayukig has joined #openstack-infra | 00:12 | |
| *** wchrisj|away is now known as chris_johnson | 00:13 | |
| openstackgerrit | Joshua Harlow proposed a change to openstack-infra/config: Extend unittest jenkins builder https://review.openstack.org/70511 | 00:14 |
| harlowja | another question, maybe for the larger audience, now that taskflow is under oslo, how does it start to connect into more of the gating and such | 00:14 |
| harlowja | *should it, whats the process if so, maybe nothing special | 00:15 |
| nibalizer | im now doing some testing around telling puppetdb to disable its firewall | 00:15 |
| *** kmartin has quit IRC | 00:15 | |
| *** thomasem has quit IRC | 00:15 | |
| nibalizer | well, telling the puppetdb module to not manage the firewall | 00:15 |
| *** changbl has quit IRC | 00:16 | |
| *** homeless has quit IRC | 00:16 | |
| *** homeless has joined #openstack-infra | 00:17 | |
| *** kmartin has joined #openstack-infra | 00:17 | |
| clarkb | fungi: jeblair: 16 is running latest logstash using the elasticsearch_http output, if you puppet agent --test --noop you should see the diff of the config. The reason I am using http output is that latest logstash wants a newer version of elasticsearch for its elasticsearch output | 00:17 |
| clarkb | the biggest problem with elasticsearch_http output is you have to specify an exact host to POST stuff to, the elasticsearch output joins the cluster so can talk to anyone in the cluster | 00:19 |
| *** masayukig has quit IRC | 00:20 | |
| *** matsuhashi has quit IRC | 00:21 | |
| *** flaper87 is now known as flaper87|afk | 00:21 | |
| *** matsuhashi has joined #openstack-infra | 00:21 | |
| *** matsuhashi has quit IRC | 00:25 | |
| *** matsuhas_ has joined #openstack-infra | 00:25 | |
| fungi | clarkb: i think part of that went over my head. what are we using as the transport for previous elasticsearch communication with logstash? | 00:26 |
| *** zhiwei has joined #openstack-infra | 00:26 | |
| fungi | ahh, i see now, "elasticsearch" is a separate protocol from "elasticsearch_http" | 00:28 |
| *** mriedem has joined #openstack-infra | 00:28 | |
| jeblair | clarkb: what are you passing as the host? | 00:29 |
| clarkb | jeblair: elasticsearch.o.o | 00:29 |
| clarkb | however, did someone run puppet agent --test without the noop? | 00:29 |
| clarkb | because the process just restarted and reverted its config :) | 00:30 |
| fungi | clarkb: oh! :( | 00:30 |
| fungi | i cut and pasted and was certain i had --noop in there | 00:30 |
| * fungi is so sorrt | 00:30 | |
| fungi | i can patch it back | 00:30 |
| clarkb | fungi: there are basically two modes, elasticsearch clustering which is over ports 93XX and elasticsearch http over ports 92XX | 00:30 |
| clarkb | fungi: I got it | 00:30 |
| fungi | i still have it in scrollback | 00:30 |
| clarkb | fungi: I know what it is | 00:30 |
| fungi | okay | 00:30 |
| * fungi feels like a complete doofus now | 00:31 | |
| jeblair | clarkb: i assume we don't want all the workers to push to es.o.o, right? | 00:31 |
| clarkb | jeblair: right, we would probably have them mod 6 in the config | 00:31 |
| jeblair | clarkb: what's the prospect for an elasticsearch upgrade? | 00:31 |
| clarkb | jeblair: fungi: or, I can upgrade the cluster | 00:31 |
| clarkb | jeblair: fungi: its actually pretty good imo | 00:31 |
| clarkb | last time it went smoothly, basically just shut everything down, start nodes on new code | 00:32 |
| clarkb | I am happy to do that as well and just use native logstash | 00:32 |
| jeblair | i like that | 00:32 |
| clarkb | wfm | 00:32 |
| clarkb | er native elasticsearch | 00:32 |
| *** medieval1_ has joined #openstack-infra | 00:32 | |
| fungi | sounds worth the impact | 00:32 |
| *** masayukig has joined #openstack-infra | 00:32 | |
| fungi | rather than dealing with workarounds we'd eventually just rip out anyway | 00:33 |
| clarkb | I will start composing puppet changes to mimic what I do there | 00:34 |
| *** thuc has joined #openstack-infra | 00:36 | |
| *** masayukig has quit IRC | 00:37 | |
| *** thuc has quit IRC | 00:37 | |
| *** wenlock has quit IRC | 00:37 | |
| * fungi is starting to run out of brain for the evening, so winding down | 00:37 | |
| *** thuc has joined #openstack-infra | 00:38 | |
| * clarkb made a slight mistake and started logstash 1.3.3 against es 0.90.3 using the elasticsearch output and it seems to be working >_> | 00:38 | |
| clarkb | might not need to do a full upgrade but doing one is probably a good idea anywaysw | 00:38 |
| clarkb | going to switch back to http for now so that it doesn't have a chance to go sidewyas later | 00:39 |
| *** dcramer_ has joined #openstack-infra | 00:40 | |
| fungi | oh, fair warning. they say we've got waves of freezing rain coming through tomorrow, possibly worst in more than a decade, so trees are likely to be coming down across power lines and i may disappear for prolonged periods due to lack of electricity | 00:44 |
| * fungi wishes he could wire a generator into this townhouse easily | 00:45 | |
| clarkb | fungi: gl | 00:48 |
| clarkb | portland went through that over the weekend, sea was spared | 00:48 |
| fungi | yeah, they're saying up to 0.75" ice accumulation on trees and lines | 00:48 |
| sdague | fungi: the portable ones just plug into an outlet, and you just throw the mains | 00:52 |
| fungi | that's tempting. punch it into a 3-phase outlet? | 00:52 |
| jhesketh__ | jeblair: Do you have a few minutes to discuss the swift log instructions patch for zuul? | 00:53 |
| sdague | I think so, I haven't done a ton of research on it. My folks have one of those | 00:53 |
| sdague | just simpler than full wiring it in | 00:53 |
| * fungi makes a note to research those | 00:53 | |
| fungi | but when i buy a house soonish i plan to just get one installed with an ats | 00:53 |
| fungi | the islands have much less stable power too | 00:54 |
| fungi | so i'd rather go with one which will cut in automatically with no more than a few seconds delay | 00:54 |
| *** melwitt has quit IRC | 00:58 | |
| *** CaptTofu has joined #openstack-infra | 01:00 | |
| jeblair | jhesketh__: sure | 01:01 |
| jeblair | jhesketh__: assuming you're still here.. | 01:01 |
| *** nosnos has joined #openstack-infra | 01:01 | |
| jhesketh__ | jeblair: so I like your feedback, but I'm not sure which way I prefer or which might be better (re using parameter-function vs 1st class citizen) | 01:04 |
| *** lttrl has quit IRC | 01:04 | |
| jhesketh__ | I'm leaning towards 1st class, but in the current structure it kinda makes more sense to be a parameter-function | 01:05 |
| jhesketh__ | do you have any opinions | 01:05 |
| *** MarkAtwood has joined #openstack-infra | 01:05 | |
| *** sarob has joined #openstack-infra | 01:05 | |
| *** david-lyle has quit IRC | 01:05 | |
| *** lttrl has joined #openstack-infra | 01:06 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Upgrade to logstash 1.3.3 and ElasticSearch 0.90.9 https://review.openstack.org/72805 | 01:07 |
| clarkb | going to WIP ^ as it won't be an automagic upgrade | 01:07 |
| jeblair | jhesketh__: i've grown fond of 1st class; i feel like it has the flexibility and generality we have elsewhere (eg pipelines themselves), but we're still making something complex easy to use | 01:07 |
| clarkb | but I think I can dive right into that tomorrow | 01:08 |
| jhesketh__ | jeblair: so what if we took the current openstack functions and made them 1st class too? so job parameters become a kind of plugin | 01:08 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Add support to run nova-api-metadata as separate binary https://review.openstack.org/68304 | 01:08 |
| *** wenlock has joined #openstack-infra | 01:09 | |
| *** ryanpetrello has joined #openstack-infra | 01:10 | |
| jeblair | jhesketh__: that sounds like a good idea. we only use it for 2 things right now; the swift change will obviate one of them, and the other one could be set with a yaml-parameter. though getting rid of jenkins will obviate the other one i expect. :) | 01:10 |
| *** sarob has quit IRC | 01:12 | |
| jeblair | jhesketh__: but yeah, we can certainly express "parameter: OFFLINE_NODE_WHEN_COMPLETE: 1" in the "name: ^.*dsvm.*$" job | 01:12 |
| jhesketh__ | right | 01:12 |
| jhesketh__ | okay, I like this plan but wonder how best to implement it... perhaps we should have more generic plugins that can hook into various parts | 01:13 |
| jhesketh__ | which could be quite complicated | 01:13 |
| jhesketh__ | otherwise maybe we just have a 'jobs' plugin that acts on the jobs part of the layout and parameter-function type items | 01:14 |
| jeblair | jhesketh__: what did you think of my final suggestion? that we make a changish have a descriptive path (eg "12345/1" or "abcdef" or "", and use that as the basis for "path: {change.path}/{pipeline.name}/{build.uuid}" in the swift section? | 01:15 |
| jhesketh__ | oh yeah, we should do that | 01:15 |
| clarkb | I am going to AFK now. jeblair fungi let me know what you think about doing a logstash and es upgrade tomorrow. It should be mostly painless. THe last upgrade was much scarier because of the lucene change beneath es and the es schema for logstash being simplified | 01:15 |
| clarkb | for this one I should be able to stop all logstash indexer and gearman workers, then kill the cluster, upgrade nodes bring cluster back, upgrade logstash then start all the workers again | 01:15 |
| jhesketh__ | jeblair: if zuul is going to give instructions on where to put logs it should also define the final destination if possible | 01:15 |
| jeblair | jhesketh__: i'm not opposed to genericizing this; but i would lean toward just building the swift stuff in to start with so we don't get too distracted with plugin architectures... | 01:16 |
| jhesketh__ | agreed | 01:16 |
| jeblair | jhesketh__: but designing the yaml and the code in a modular way to facilitate cleaning up the abstraction is good | 01:16 |
| clarkb | http://www.openstack.org/software/roadmap/ ceilometer link there is broken, where do we submit bugs for stuff like that? | 01:16 |
| clarkb | jeblair: fungi ^ is there a foundation website bug tracker? | 01:17 |
| jeblair | clarkb: no time like the present | 01:17 |
| jeblair | clarkb: launchpad.net/openstack-org i think | 01:17 |
| clarkb | thanks | 01:17 |
| jhesketh__ | jeblair: okay, I think I have enough to reroll that patch, thanks :-) | 01:18 |
| jeblair | jhesketh__: cool, i'm really excited. thanks! | 01:18 |
| jhesketh__ | no worries :-) | 01:19 |
| jhesketh__ | sorry this log stuff has been taking so long! | 01:19 |
| jeblair | jhesketh__: also, if you need something to multiplex, i think the os-loganalyze project will need corresponding updates to fetch from swift | 01:19 |
| *** MarkAtwood has quit IRC | 01:20 | |
| jhesketh__ | yes, that's on my todo list | 01:20 |
| jhesketh__ | wouldn't mind some feedback on my config script that actually pushes to swift | 01:20 |
| jhesketh__ | still not sure how to actually tie that in with Jenkins+console scripts yet | 01:20 |
| jhesketh__ | (possibly should mark it WIP) | 01:20 |
| jeblair | jhesketh__: will do; i just starred it for review tomorrow | 01:21 |
| *** dims has quit IRC | 01:21 | |
| jhesketh__ | cheers | 01:21 |
| jhesketh__ | I like all the merger stuff btw :-) | 01:21 |
| jeblair | jhesketh__: i'm trying to dig out from the review backlog and prioritize, but i haven't seen everything yet, so feel free to ping me on things where i'm blocking you | 01:21 |
| clarkb | jeblair: and filed https://bugs.launchpad.net/openstack-org/+bug/1279140 thanks | 01:21 |
| uvirtbot | Launchpad bug 1279140 in openstack-org "Broken ceilometer link" [Undecided,New] | 01:21 |
| jeblair | mattoliverau: that goes for you too ^ :) | 01:21 |
| jeblair | jhesketh__: cool! i'm hoping we can get it running before zuul falls over again; it did have a 140 load average earlier this week | 01:22 |
| clarkb | this ++ I have been swamped and trying to help with git-review reviews do ping me if I can help elsewhere | 01:22 |
| clarkb | there are changes up to add https git review support | 01:22 |
| jhesketh__ | jeblair: you're not blocking :-) | 01:22 |
| jhesketh__ | jeblair: I wonder if zuul-server should launch zuul-merger by default and have a parameter to tell it not to? | 01:23 |
| jhesketh__ | so that by default zuul bundles itself up into one box nicely | 01:23 |
| jeblair | jhesketh__: good idea; matches what we did with gearman for basically the same reason | 01:23 |
| jeblair | jhesketh__: we can add that on to the end of the series | 01:24 |
| jhesketh__ | works for me | 01:24 |
| *** MarkAtwood has joined #openstack-infra | 01:26 | |
| *** krtaylor has joined #openstack-infra | 01:28 | |
| jhesketh__ | jeblair: would you mind adding your thoughts to my comments on https://review.openstack.org/#/c/71628/ if you get a chance? | 01:28 |
| *** zhiwei has quit IRC | 01:30 | |
| clarkb | ok AFKing for real now | 01:32 |
| jeblair | jhesketh__: whoops, forgot to save draft comments. done. :) | 01:32 |
| jhesketh__ | thanks :-) | 01:33 |
| *** masayukig has joined #openstack-infra | 01:33 | |
| *** masayukig has quit IRC | 01:37 | |
| *** thuc has quit IRC | 01:38 | |
| *** thuc has joined #openstack-infra | 01:38 | |
| *** thuc has quit IRC | 01:43 | |
| *** ryanpetrello has quit IRC | 01:47 | |
| *** wenlock has quit IRC | 01:48 | |
| *** ryanpetrello has joined #openstack-infra | 01:48 | |
| *** mgagne has quit IRC | 01:49 | |
| *** banix has joined #openstack-infra | 01:49 | |
| *** zhiwei has joined #openstack-infra | 01:50 | |
| *** weshay has joined #openstack-infra | 01:50 | |
| *** chris_johnson is now known as wchrisj|away | 01:51 | |
| *** lttrl has quit IRC | 01:51 | |
| *** lttrl has joined #openstack-infra | 01:55 | |
| *** wenlock has joined #openstack-infra | 01:56 | |
| *** zhiwei has quit IRC | 01:58 | |
| *** ryanpetrello has quit IRC | 01:59 | |
| *** amcrn has quit IRC | 02:03 | |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/reviewday: Ensure non-WIP reviews have a score of 1 as a base https://review.openstack.org/72815 | 02:04 |
| *** dkranz has joined #openstack-infra | 02:06 | |
| *** wenlock has quit IRC | 02:08 | |
| *** sarob has joined #openstack-infra | 02:09 | |
| *** MarkAtwood has quit IRC | 02:10 | |
| *** sarob has quit IRC | 02:13 | |
| *** yaguang has joined #openstack-infra | 02:14 | |
| *** MarkAtwood has joined #openstack-infra | 02:24 | |
| *** ryanpetrello has joined #openstack-infra | 02:27 | |
| *** masayukig has joined #openstack-infra | 02:34 | |
| *** oubiwann_ has joined #openstack-infra | 02:35 | |
| *** masayukig has quit IRC | 02:38 | |
| *** banix has quit IRC | 02:42 | |
| *** mwagner_lap has joined #openstack-infra | 02:45 | |
| *** banix has joined #openstack-infra | 03:01 | |
| *** sarob has joined #openstack-infra | 03:09 | |
| *** gokrokve has joined #openstack-infra | 03:14 | |
| *** sarob has quit IRC | 03:16 | |
| *** rcleere has joined #openstack-infra | 03:17 | |
| *** sarob has joined #openstack-infra | 03:19 | |
| *** koofoss has joined #openstack-infra | 03:20 | |
| *** esker has joined #openstack-infra | 03:21 | |
| koofoss | Hi All, I just did a "git review -d" in a new environment and ran a "tox -epy27" and got an error (http://paste.openstack.org/show/64484/). (I was recommended to ask in #opestack-infra) | 03:22 |
| koofoss | can anybody point me in the right direction? | 03:22 |
| koofoss | PS: was working on glance | 03:23 |
| *** esker has quit IRC | 03:23 | |
| *** sarob has quit IRC | 03:24 | |
| *** esker has joined #openstack-infra | 03:24 | |
| clarkb | koofoss: is pbr in glance's requirements? pbr is the thing that installs python setup.py test | 03:25 |
| koofoss | yes it is ... | 03:26 |
| koofoss | pbr>=0.6,<1.0 | 03:26 |
| clarkb | and if so what version is installed by tox (you can `.tox/py27/bin/pip freeze` to get a listing of what was actually installed) | 03:26 |
| koofoss | [~/tmp/glance].tox/py27/bin/pip freeze | grep -i pbr | 03:26 |
| koofoss | pbr==0.6 | 03:26 |
| *** wchrisj|away is now known as chris_johnson | 03:26 | |
| *** chris_johnson has quit IRC | 03:27 | |
| clarkb | that should be plenty new enough | 03:27 |
| clarkb | koofoss: what happens if you `.tox/py27/bin/python setup.py test --slowest --testr-args='--concurrency 1' | 03:28 |
| clarkb | oh and what version of tox are you using? tox --version | 03:28 |
| koofoss | clarkb: same error: 'error: option --slowest not recognized' | 03:28 |
| koofoss | [~/tmp/glance]tox --version | 03:29 |
| koofoss | 1.6.1 imported from /usr/local/lib/python2.7/dist-packages/tox/__init__.pyc | 03:29 |
| clarkb | :/ that is the good version of tox, I don't know what is happening | 03:29 |
| clarkb | mordred: ^ any ideas? | 03:29 |
| clarkb | also why are we using lockutils with concurrency = 1? /me smells a cargo cult | 03:30 |
| koofoss | should I try rebuilding my tox environment again (ugh - it takes ages :()? | 03:30 |
| clarkb | I don't think you need to, you can certainly try | 03:30 |
| *** CaptTofu has quit IRC | 03:31 | |
| koofoss | regarding the concurrency thing ... I didn't put that in ... it came with the download | 03:31 |
| koofoss | so what should I be setting it to? | 03:31 |
| clarkb | concurrency is fine as 1, but if you are running one test at a time there is no need for the python -m lockutils thing | 03:32 |
| koofoss | I see ... I'll try rebuilding my tox. *fingers-crossed* | 03:33 |
| *** dcramer_ has quit IRC | 03:34 | |
| *** masayukig has joined #openstack-infra | 03:34 | |
| *** pcrews has quit IRC | 03:35 | |
| clarkb | koofoss: which change are you testing? | 03:35 |
| koofoss | https://review.openstack.org/#/c/63674/ | 03:36 |
| *** hdd_ has joined #openstack-infra | 03:36 | |
| koofoss | clarkb: It was a patch I uploaded and now I'm downloading it to incorporate review comments | 03:36 |
| koofoss | clarkb: I usually delete my branches after uploading, and then re-download the branches to work on the review comments | 03:37 |
| clarkb | that should be fine, when was the last time the tox env was built? | 03:37 |
| koofoss | clarkb: yesterday morning | 03:38 |
| koofoss | but one thing I note is that yesterday when I ran tests, it used nosetests | 03:38 |
| koofoss | this morning it's using testr! | 03:38 |
| clarkb | ah | 03:38 |
| clarkb | so python setup.py test is the way to testr | 03:38 |
| clarkb | its possible some corner case was missed | 03:38 |
| *** matsuhas_ has quit IRC | 03:39 | |
| clarkb | I just tried to reproduce but couldn't I am on a precise machine like the jenkins slaves | 03:39 |
| *** masayukig has quit IRC | 03:39 | |
| clarkb | but yeah, if it switched you will need to rebuild the tox env | 03:39 |
| clarkb | or update it by hand to include the new dependencies | 03:39 |
| koofoss | ok, will bite the bullet. Many thanks for your help :) | 03:40 |
| *** mgagne has joined #openstack-infra | 03:43 | |
| *** wenlock has joined #openstack-infra | 03:48 | |
| *** chandan_kumar has joined #openstack-infra | 03:51 | |
| *** mgagne1 has joined #openstack-infra | 03:53 | |
| *** mgagne has quit IRC | 03:56 | |
| *** markmcclain has joined #openstack-infra | 03:56 | |
| *** markmcclain has quit IRC | 03:57 | |
| *** yamahata has quit IRC | 03:59 | |
| *** dkehn__ has joined #openstack-infra | 04:00 | |
| *** mgagne has joined #openstack-infra | 04:01 | |
| *** dkehn has quit IRC | 04:03 | |
| *** mgagne1 has quit IRC | 04:03 | |
| *** dkehn_ has quit IRC | 04:03 | |
| *** arborism has joined #openstack-infra | 04:04 | |
| *** changbl has joined #openstack-infra | 04:04 | |
| openstackgerrit | Cyril Roelandt proposed a change to openstack-infra/config: python-keystoneclient: make the py33 gate voting https://review.openstack.org/72831 | 04:04 |
| *** koofoss has left #openstack-infra | 04:04 | |
| *** dkehn has joined #openstack-infra | 04:05 | |
| *** yamahata has joined #openstack-infra | 04:05 | |
| *** amcrn_ has joined #openstack-infra | 04:05 | |
| openstackgerrit | Cyril Roelandt proposed a change to openstack-infra/config: python-keystoneclient: make the py33 gate voting https://review.openstack.org/72831 | 04:06 |
| *** amcrn_ is now known as amcrn | 04:08 | |
| *** arborism has quit IRC | 04:09 | |
| *** dkehn__ is now known as dkehn_ | 04:12 | |
| *** fifieldt has joined #openstack-infra | 04:19 | |
| *** talluri has joined #openstack-infra | 04:31 | |
| wenlock | wondering is login for review working for anyone else? seems launchpad openid server might be down... but maybe it's me | 04:33 |
| StevenK | https://login.launchpad.net/ works for me | 04:33 |
| wenlock | Stevenk: and you were able to login? | 04:34 |
| StevenK | I already was | 04:34 |
| wenlock | ahh | 04:34 |
| wenlock | ok... will try again later. | 04:34 |
| StevenK | wenlock: I just logged out and back in, and it worked. | 04:34 |
| wenlock | stevenk: awesome, thanks for checking for me... it must be my comp then! | 04:35 |
| *** masayukig has joined #openstack-infra | 04:35 | |
| wenlock | time to flush that browser cache | 04:35 |
| *** lcheng_ has joined #openstack-infra | 04:38 | |
| *** masayukig has quit IRC | 04:40 | |
| *** harlowja is now known as harlowja_away | 04:44 | |
| *** matsuhashi has joined #openstack-infra | 04:52 | |
| SergeyLukjanov | morning/evening :) | 04:54 |
| *** markwash has joined #openstack-infra | 04:55 | |
| *** lcheng_ has quit IRC | 04:58 | |
| openstackgerrit | A change was merged to openstack-infra/config: Enable verbose output for welcome message https://review.openstack.org/72436 | 04:59 |
| fungi | SergeyLukjanov: i guess it's time to hand off the baton. my tuesday ends as your wednesday begins | 05:00 |
| fungi | i hope it's an uneventful one! | 05:01 |
| SergeyLukjanov | fungi :) | 05:01 |
| SergeyLukjanov | fungi, good night | 05:01 |
| openstackgerrit | A change was merged to openstack-infra/git-review: Attach Gerrit logs and known_hosts to failed tests https://review.openstack.org/70527 | 05:03 |
| *** zul has joined #openstack-infra | 05:04 | |
| *** zenoway has joined #openstack-infra | 05:05 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: Add jenkins job to build the gerrit plugins https://review.openstack.org/70014 | 05:08 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: Setup tooling and jobs for openstack Gerrit ver 2.8 and beyond https://review.openstack.org/60348 | 05:08 |
| *** zul has quit IRC | 05:08 | |
| *** gokrokve has quit IRC | 05:09 | |
| *** cadenzajon has joined #openstack-infra | 05:13 | |
| *** sarob has joined #openstack-infra | 05:13 | |
| *** lttrl has quit IRC | 05:14 | |
| *** banix has quit IRC | 05:15 | |
| *** lttrl has joined #openstack-infra | 05:16 | |
| *** gokrokve has joined #openstack-infra | 05:17 | |
| *** cadenzajon has quit IRC | 05:18 | |
| *** sarob has quit IRC | 05:19 | |
| openstackgerrit | Steve Kowalik proposed a change to openstack-infra/config: Add an overcloud test for tripleo https://review.openstack.org/72836 | 05:20 |
| *** sarob has joined #openstack-infra | 05:23 | |
| *** cody-somerville has quit IRC | 05:24 | |
| *** mriedem has quit IRC | 05:25 | |
| *** zhiwei has joined #openstack-infra | 05:26 | |
| *** sarob has quit IRC | 05:28 | |
| zhiwei | clarkb: hi, I saw the meeting log. You can rename the project this weekend? | 05:30 |
| clarkb | zhiwei: a little after that jeblair mentioned he wants to do it during the next openstack downtime | 05:33 |
| clarkb | which we don't have a date for | 05:33 |
| clarkb | but savanna rename should catch that in a few weeks if nothing does prior | 05:33 |
| *** talluri has quit IRC | 05:34 | |
| *** mgagne has quit IRC | 05:35 | |
| zhiwei | oh, we are branching stable/havana in our repo, and want to use the new cookbook name in stable/havana branch. | 05:35 |
| clarkb | the branch name should be independent of the project name | 05:35 |
| zhiwei | yes, I know. but the name in the cookbook code. | 05:36 |
| *** masayukig has joined #openstack-infra | 05:36 | |
| *** ryanpetrello has quit IRC | 05:37 | |
| zhiwei | the other havana cookbook depends on metering cookbook. so, if the stable/havana is branched, they can't use this branch because the cookbook name is changed to another. | 05:38 |
| *** talluri has joined #openstack-infra | 05:38 | |
| clarkb | you would just update the other dependencies right? | 05:39 |
| zhiwei | yes | 05:39 |
| zhiwei | I don't want to update the stable branch with new cookbook name, this is a big change. | 05:39 |
| *** masayukig has quit IRC | 05:40 | |
| *** mgagne has joined #openstack-infra | 05:41 | |
| *** nicedice has quit IRC | 05:43 | |
| *** nosnos_ has joined #openstack-infra | 05:43 | |
| *** nosnos_ has quit IRC | 05:44 | |
| *** nosnos_ has joined #openstack-infra | 05:44 | |
| *** DinaBelova_ is now known as DinaBelova | 05:45 | |
| *** nosnos has quit IRC | 05:46 | |
| openstackgerrit | A change was merged to openstack-infra/git-review: Remove tox locale overrides https://review.openstack.org/72217 | 05:47 |
| *** zhiyan has joined #openstack-infra | 05:52 | |
| *** talluri has quit IRC | 05:53 | |
| *** jamielennox is now known as jamielennox|away | 05:55 | |
| *** lttrl has quit IRC | 05:55 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: replace jeepyb's trivial rebase gerrit hook with copyAllScoresOnTrivialRebase https://review.openstack.org/69800 | 05:56 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: use gerrit's built-in rebase detector instead of the jeepyb trivial rebase hook. https://review.openstack.org/69800 | 05:59 |
| *** saju_m has joined #openstack-infra | 06:03 | |
| *** bhuvan has joined #openstack-infra | 06:03 | |
| *** nosnos_ has quit IRC | 06:04 | |
| *** nosnos has joined #openstack-infra | 06:04 | |
| *** gokrokve has quit IRC | 06:04 | |
| *** sarob has joined #openstack-infra | 06:07 | |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Also copy tempest.config for old with grenade https://review.openstack.org/72736 | 06:07 |
| *** e0ne has joined #openstack-infra | 06:08 | |
| *** e0ne has quit IRC | 06:09 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: update comment filter for gerrit 2.8 https://review.openstack.org/70864 | 06:11 |
| *** morganfainberg is now known as morganfainberg_Z | 06:12 | |
| *** jaypipes has quit IRC | 06:12 | |
| *** MarkAtwood has quit IRC | 06:13 | |
| *** dkehn__ has joined #openstack-infra | 06:14 | |
| *** jaypipes has joined #openstack-infra | 06:14 | |
| *** dkehn__ has quit IRC | 06:16 | |
| *** dkehn__ has joined #openstack-infra | 06:17 | |
| *** dkehn has quit IRC | 06:17 | |
| *** dkehn_ has quit IRC | 06:17 | |
| *** lttrl has joined #openstack-infra | 06:17 | |
| *** dkehn has joined #openstack-infra | 06:19 | |
| *** wenlock has quit IRC | 06:21 | |
| *** lttrl has quit IRC | 06:23 | |
| *** yolanda has joined #openstack-infra | 06:32 | |
| *** rlandy has joined #openstack-infra | 06:32 | |
| *** lifeless has quit IRC | 06:34 | |
| *** gokrokve has joined #openstack-infra | 06:35 | |
| *** denis_makogon has joined #openstack-infra | 06:35 | |
| *** dkehn_ has joined #openstack-infra | 06:36 | |
| *** masayukig has joined #openstack-infra | 06:37 | |
| *** gokrokve_ has joined #openstack-infra | 06:37 | |
| *** dkehn has quit IRC | 06:38 | |
| *** dkehn__ has quit IRC | 06:39 | |
| *** lttrl has joined #openstack-infra | 06:39 | |
| *** pblaho has joined #openstack-infra | 06:40 | |
| *** gokrokve has quit IRC | 06:40 | |
| *** dkehn has joined #openstack-infra | 06:40 | |
| *** max_lobur_afk has quit IRC | 06:40 | |
| *** max_lobur_afk has joined #openstack-infra | 06:41 | |
| *** MarkAtwood has joined #openstack-infra | 06:41 | |
| *** masayukig has quit IRC | 06:41 | |
| *** gokrokve_ has quit IRC | 06:42 | |
| *** markwash has quit IRC | 06:42 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add an overcloud test for tripleo https://review.openstack.org/72836 | 06:43 |
| *** max_lobur_afk has quit IRC | 06:44 | |
| *** bogdando has quit IRC | 06:44 | |
| *** bogdando has joined #openstack-infra | 06:44 | |
| *** max_lobur_afk has joined #openstack-infra | 06:44 | |
| *** max_lobur_afk has quit IRC | 06:45 | |
| *** esker has quit IRC | 06:45 | |
| *** max_lobur_afk has joined #openstack-infra | 06:45 | |
| *** talluri_ has joined #openstack-infra | 06:45 | |
| *** sarob has quit IRC | 06:47 | |
| *** sarob has joined #openstack-infra | 06:48 | |
| *** pblaho has quit IRC | 06:49 | |
| *** pblaho has joined #openstack-infra | 06:49 | |
| *** sarob has quit IRC | 06:53 | |
| *** esker has joined #openstack-infra | 06:54 | |
| *** hdd_ has quit IRC | 06:59 | |
| *** gokrokve has joined #openstack-infra | 07:00 | |
| *** lttrl has quit IRC | 07:00 | |
| *** dkehn__ has joined #openstack-infra | 07:01 | |
| *** lttrl has joined #openstack-infra | 07:02 | |
| *** dkehn has quit IRC | 07:04 | |
| *** dkehn_ has quit IRC | 07:04 | |
| *** zenoway has quit IRC | 07:05 | |
| *** gokrokve has quit IRC | 07:05 | |
| *** dkehn has joined #openstack-infra | 07:06 | |
| *** MarkAtwood has quit IRC | 07:17 | |
| *** zhiyan has quit IRC | 07:17 | |
| *** sarob has joined #openstack-infra | 07:18 | |
| openstackgerrit | A change was merged to openstack-infra/gitdm: Fix misspellings in gitdm https://review.openstack.org/71760 | 07:18 |
| *** coolsvap has joined #openstack-infra | 07:21 | |
| *** sarob has quit IRC | 07:24 | |
| *** zhiyan has joined #openstack-infra | 07:25 | |
| *** coolsvap1 has joined #openstack-infra | 07:26 | |
| *** zhiyan has quit IRC | 07:27 | |
| *** coolsvap has quit IRC | 07:28 | |
| *** MarkAtwood has joined #openstack-infra | 07:31 | |
| *** jhesketh__ has quit IRC | 07:34 | |
| *** jhesketh__ has joined #openstack-infra | 07:34 | |
| *** AlexF has joined #openstack-infra | 07:36 | |
| *** coolsvap1 has quit IRC | 07:36 | |
| *** coolsvap has joined #openstack-infra | 07:36 | |
| *** masayukig has joined #openstack-infra | 07:37 | |
| *** masayukig has quit IRC | 07:42 | |
| openstackgerrit | Noorul Islam K M proposed a change to openstack-dev/cookiecutter: Update copyright year https://review.openstack.org/72862 | 07:42 |
| *** flaper87|afk is now known as flaper87 | 07:47 | |
| *** AaronGr is now known as AaronGr_Zzz | 07:51 | |
| *** e0ne has joined #openstack-infra | 07:53 | |
| *** bhuvan has quit IRC | 07:57 | |
| *** gokrokve has joined #openstack-infra | 08:00 | |
| *** jooools has quit IRC | 08:03 | |
| *** gokrokve has quit IRC | 08:05 | |
| esker | quick gerrit question: will it reject a new submission if the associated bp is newly filed and not yet approved? | 08:05 |
| *** lttrl has quit IRC | 08:05 | |
| *** amotoki has joined #openstack-infra | 08:06 | |
| *** AlexF has quit IRC | 08:10 | |
| *** jcoufal has joined #openstack-infra | 08:13 | |
| *** bhuvan has joined #openstack-infra | 08:14 | |
| *** mrmartin has joined #openstack-infra | 08:15 | |
| *** lttrl has joined #openstack-infra | 08:17 | |
| openstackgerrit | Robert Esker proposed a change to openstack-infra/config: Implements: blueprint add-manila-to-gerritbot https://review.openstack.org/72872 | 08:18 |
| *** MarkAtwood has quit IRC | 08:18 | |
| *** sarob has joined #openstack-infra | 08:19 | |
| *** dkehn_ has joined #openstack-infra | 08:24 | |
| *** sarob has quit IRC | 08:24 | |
| openstackgerrit | Robert Esker proposed a change to openstack-infra/config: Removes extraneous carriage return https://review.openstack.org/72875 | 08:25 |
| *** dkehn_ has quit IRC | 08:26 | |
| *** dkehn_ has joined #openstack-infra | 08:26 | |
| *** dkehn has quit IRC | 08:26 | |
| *** dkehn__ has quit IRC | 08:27 | |
| *** dkehn has joined #openstack-infra | 08:28 | |
| *** salv-orlando has joined #openstack-infra | 08:29 | |
| *** boris-42_ has joined #openstack-infra | 08:30 | |
| *** hashar has joined #openstack-infra | 08:31 | |
| *** bhuvan has quit IRC | 08:32 | |
| *** masayukig has joined #openstack-infra | 08:38 | |
| *** lttrl has quit IRC | 08:38 | |
| *** masayukig has quit IRC | 08:42 | |
| *** zhiwei has left #openstack-infra | 08:44 | |
| *** lttrl has joined #openstack-infra | 08:45 | |
| openstackgerrit | Robert Esker proposed a change to openstack-infra/config: Implements: blueprint add-manila-to-gerritbot https://review.openstack.org/72872 | 08:49 |
| *** hashar has quit IRC | 08:50 | |
| *** lttrl has quit IRC | 08:51 | |
| openstackgerrit | Robert Esker proposed a change to openstack-infra/config: Implements: blueprint add-manila-to-gerritbot https://review.openstack.org/72872 | 08:51 |
| *** hashar has joined #openstack-infra | 08:51 | |
| *** coolsvap1 has joined #openstack-infra | 08:59 | |
| *** gokrokve has joined #openstack-infra | 09:00 | |
| openstackgerrit | Robert Esker proposed a change to openstack-infra/config: Implements: blueprint add-manila-to-gerritbot https://review.openstack.org/72872 | 09:01 |
| *** coolsvap has quit IRC | 09:01 | |
| *** Clabbe has quit IRC | 09:03 | |
| openstackgerrit | Robert Esker proposed a change to openstack-infra/config: Implements: blueprint add-manila-to-gerritbot https://review.openstack.org/72872 | 09:05 |
| *** gokrokve has quit IRC | 09:05 | |
| *** jpich has joined #openstack-infra | 09:06 | |
| *** afazekas has joined #openstack-infra | 09:07 | |
| *** afazekas is now known as afazekas|sick | 09:08 | |
| *** sarob has joined #openstack-infra | 09:08 | |
| *** fifieldt has quit IRC | 09:08 | |
| openstackgerrit | Yongli He proposed a change to openstack-infra/storyboard: Use six.moves.urllib.parse instead of urlparse https://review.openstack.org/72887 | 09:09 |
| *** Clabbe has joined #openstack-infra | 09:11 | |
| *** lttrl has joined #openstack-infra | 09:12 | |
| *** sarob has quit IRC | 09:13 | |
| *** mrmartin has quit IRC | 09:13 | |
| *** hashar_ has joined #openstack-infra | 09:13 | |
| *** hashar has quit IRC | 09:17 | |
| *** yassine has joined #openstack-infra | 09:18 | |
| *** sarob has joined #openstack-infra | 09:19 | |
| *** e0ne has quit IRC | 09:20 | |
| *** e0ne has joined #openstack-infra | 09:21 | |
| *** sarob has quit IRC | 09:24 | |
| openstackgerrit | Andreas Rehn proposed a change to openstack-infra/jenkins-job-builder: Added support for Delivery Pipeline Plugin https://review.openstack.org/71658 | 09:25 |
| *** locke105 has quit IRC | 09:26 | |
| *** jcoufal has quit IRC | 09:30 | |
| *** andreaf has quit IRC | 09:30 | |
| *** jcoufal has joined #openstack-infra | 09:30 | |
| *** andreaf has joined #openstack-infra | 09:31 | |
| *** lttrl has quit IRC | 09:33 | |
| *** fbo_away is now known as fbo | 09:36 | |
| *** jooools has joined #openstack-infra | 09:37 | |
| *** masayukig has joined #openstack-infra | 09:39 | |
| *** DinaBelova is now known as DinaBelova_ | 09:40 | |
| *** lttrl has joined #openstack-infra | 09:40 | |
| *** yaguang has quit IRC | 09:40 | |
| *** masayukig has quit IRC | 09:44 | |
| *** jp_at_hp has joined #openstack-infra | 09:44 | |
| *** lttrl has quit IRC | 09:45 | |
| *** resker has joined #openstack-infra | 09:46 | |
| *** lttrl has joined #openstack-infra | 09:47 | |
| *** esker has quit IRC | 09:50 | |
| *** talluri has joined #openstack-infra | 09:52 | |
| *** talluri_ has quit IRC | 09:53 | |
| *** denis_makogon has quit IRC | 09:57 | |
| *** talluri has quit IRC | 09:59 | |
| *** gokrokve has joined #openstack-infra | 10:00 | |
| *** Steap_ has quit IRC | 10:03 | |
| *** Steap_ has joined #openstack-infra | 10:04 | |
| *** gokrokve has quit IRC | 10:05 | |
| hughsaunders | hey, a gate job failed when attempting to fetch a gem from rubygems.org, should I report that as bug and reverify against that? Its not really a bug in the openstack infrastructure though.. http://logs.openstack.org/46/68446/17/gate/gate-cookbook-openstack-dashboard-chef-lint/24db2c3/console.html | 10:05 |
| *** talluri has joined #openstack-infra | 10:06 | |
| openstackgerrit | Yuriy Taraday proposed a change to openstack-infra/git-review: Don't try to attach known_hosts if it's not there https://review.openstack.org/72910 | 10:08 |
| *** markmc has joined #openstack-infra | 10:10 | |
| *** rossella_s has joined #openstack-infra | 10:12 | |
| *** dizquierdo has joined #openstack-infra | 10:14 | |
| *** hashar_ is now known as hashar | 10:16 | |
| openstackgerrit | Lin Tan proposed a change to openstack-infra/zuul: Use six.moves.urllib.parse instead of urlparse https://review.openstack.org/72824 | 10:16 |
| openstackgerrit | Lin Tan proposed a change to openstack-infra/storyboard: Use six.moves.urllib.parse instead of urlparse https://review.openstack.org/72826 | 10:19 |
| *** CaptTofu has joined #openstack-infra | 10:19 | |
| *** sarob has joined #openstack-infra | 10:19 | |
| *** oanufriev has quit IRC | 10:24 | |
| *** sarob has quit IRC | 10:24 | |
| *** oanufriev has joined #openstack-infra | 10:24 | |
| *** masayukig has joined #openstack-infra | 10:40 | |
| *** CaptTofu has quit IRC | 10:40 | |
| openstackgerrit | Dirk Mueller proposed a change to openstack/requirements: Use >=, not >, for the lower bound of a requirement https://review.openstack.org/72918 | 10:41 |
| *** matsuhashi has quit IRC | 10:41 | |
| *** markmc has quit IRC | 10:43 | |
| *** masayukig has quit IRC | 10:45 | |
| *** apevec has joined #openstack-infra | 10:45 | |
| *** markmc has joined #openstack-infra | 10:48 | |
| *** markmc has quit IRC | 10:53 | |
| *** markmc has joined #openstack-infra | 10:57 | |
| *** saju_m has quit IRC | 10:57 | |
| *** masayukig has joined #openstack-infra | 10:58 | |
| *** gokrokve has joined #openstack-infra | 11:00 | |
| *** max_lobur_afk is now known as max_lobur | 11:03 | |
| *** CaptTofu has joined #openstack-infra | 11:04 | |
| *** gokrokve has quit IRC | 11:05 | |
| *** markmc has quit IRC | 11:06 | |
| *** lifeless has joined #openstack-infra | 11:07 | |
| openstackgerrit | Marton Kiss proposed a change to openstack-infra/config: Update groups gating scripts https://review.openstack.org/72921 | 11:07 |
| *** amcrn has quit IRC | 11:08 | |
| *** markmc has joined #openstack-infra | 11:12 | |
| *** dkehn__ has joined #openstack-infra | 11:18 | |
| *** sarob has joined #openstack-infra | 11:19 | |
| *** dkehn__ has quit IRC | 11:20 | |
| *** dkehn__ has joined #openstack-infra | 11:20 | |
| *** dkehn has quit IRC | 11:20 | |
| *** dkehn_ has quit IRC | 11:21 | |
| *** dkehn has joined #openstack-infra | 11:22 | |
| *** bhuvan has joined #openstack-infra | 11:23 | |
| *** ArxCruz has joined #openstack-infra | 11:23 | |
| *** markmc has quit IRC | 11:23 | |
| *** sarob has quit IRC | 11:24 | |
| *** kmartin has quit IRC | 11:26 | |
| *** branen has quit IRC | 11:26 | |
| *** markmc has joined #openstack-infra | 11:26 | |
| *** hashar is now known as hasharEat | 11:29 | |
| *** ilyashakhat has joined #openstack-infra | 11:30 | |
| *** bhuvan has quit IRC | 11:34 | |
| *** markmc has quit IRC | 11:36 | |
| *** lttrl has quit IRC | 11:38 | |
| *** saju_m has joined #openstack-infra | 11:40 | |
| *** lttrl has joined #openstack-infra | 11:40 | |
| *** hasharEat has quit IRC | 11:42 | |
| *** markmc has joined #openstack-infra | 11:42 | |
| *** matsuhashi has joined #openstack-infra | 11:44 | |
| *** garyk has joined #openstack-infra | 11:44 | |
| garyk | mordred: ping | 11:44 |
| *** vbala has joined #openstack-infra | 11:44 | |
| garyk | SergeyLukjanov: ping | 11:45 |
| *** oanufriev has quit IRC | 11:46 | |
| garyk | fungi: ping | 11:47 |
| *** mrfatstrat has joined #openstack-infra | 11:48 | |
| *** saju_m has quit IRC | 11:49 | |
| *** markmc has quit IRC | 11:50 | |
| *** b3nt_pin has joined #openstack-infra | 11:53 | |
| *** lcostantino has joined #openstack-infra | 11:53 | |
| *** bhuvan has joined #openstack-infra | 11:54 | |
| *** markmc has joined #openstack-infra | 11:55 | |
| *** CaptTofu has quit IRC | 11:55 | |
| *** gokrokve has joined #openstack-infra | 12:00 | |
| *** lcostantino has quit IRC | 12:00 | |
| *** lcostantino has joined #openstack-infra | 12:00 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack/requirements: Add marconi to projects.txt https://review.openstack.org/72925 | 12:01 |
| *** DinaBelova_ is now known as DinaBelova | 12:04 | |
| *** gokrokve has quit IRC | 12:05 | |
| *** rfolco has joined #openstack-infra | 12:08 | |
| *** jhesketh__ has quit IRC | 12:12 | |
| *** coolsvap_ has joined #openstack-infra | 12:14 | |
| *** xBsd has joined #openstack-infra | 12:15 | |
| *** coolsvap1 has quit IRC | 12:17 | |
| *** b3nt_pin has quit IRC | 12:17 | |
| *** dkehn_ has joined #openstack-infra | 12:18 | |
| *** oubiwann_ has quit IRC | 12:19 | |
| *** sarob has joined #openstack-infra | 12:19 | |
| *** dkehn has quit IRC | 12:20 | |
| *** dkehn__ has quit IRC | 12:21 | |
| *** dkehn has joined #openstack-infra | 12:22 | |
| *** sarob has quit IRC | 12:24 | |
| openstackgerrit | Alexander Jones proposed a change to openstack-infra/git-review: Fix parsing of SCP-style URLs, as these are valid in Git itself https://review.openstack.org/72751 | 12:29 |
| openstackgerrit | Alexander Jones proposed a change to openstack-infra/git-review: Properly work around the fact that ssh: isn't a real URI scheme https://review.openstack.org/72752 | 12:29 |
| openstackgerrit | Alexander Jones proposed a change to openstack-infra/git-review: Properly work around the fact that ssh: isn't a real URI scheme https://review.openstack.org/72752 | 12:35 |
| *** eharney has quit IRC | 12:36 | |
| openstackgerrit | David Caro proposed a change to openstack-infra/jenkins-job-builder: Added matrix trigger option to email-ext publisher https://review.openstack.org/40214 | 12:36 |
| *** ociuhandu has joined #openstack-infra | 12:37 | |
| *** hashar has joined #openstack-infra | 12:37 | |
| *** ociuhandu has quit IRC | 12:38 | |
| *** oanufriev has joined #openstack-infra | 12:41 | |
| SergeyLukjanov | garyk, pong | 12:44 |
| garyk | SergeyLukjanov: hi, maybe you can help here | 12:46 |
| SergeyLukjanov | garyk, hi, what's up? | 12:46 |
| garyk | any idea what is required to get https://review.openstack.org/#/c/70761/ approved? | 12:46 |
| *** bhuvan has quit IRC | 12:47 | |
| garyk | i am not exactly sure what is missing. we would like to get the code posted soon so that we can get in in for I | 12:47 |
| SergeyLukjanov | garyk, there are some problems with manage-projects script | 12:48 |
| SergeyLukjanov | garyk, so, new projects couldn't be created right now | 12:49 |
| *** talluri has quit IRC | 12:49 | |
| SergeyLukjanov | garyk, I hope that we'll fix it soon | 12:49 |
| SergeyLukjanov | let me find the bug on luanchpad | 12:49 |
| *** dims has joined #openstack-infra | 12:50 | |
| garyk | Thanks! Finally we understand | 12:50 |
| *** smarcet has joined #openstack-infra | 12:50 | |
| SergeyLukjanov | garyk, here it is https://bugs.launchpad.net/openstack-ci/+bug/1242569 | 12:50 |
| *** talluri has joined #openstack-infra | 12:50 | |
| uvirtbot | SergeyLukjanov: Error: Could not parse data returned by Launchpad: The read operation timed out | 12:50 |
| *** freyes has joined #openstack-infra | 12:50 | |
| SergeyLukjanov | heh, launchpad sucks sometimes | 12:50 |
| garyk | :) | 12:51 |
| openstackgerrit | David Caro proposed a change to openstack-infra/jenkins-job-builder: Added matrix trigger option to email-ext publisher https://review.openstack.org/40214 | 12:53 |
| *** b3nt_pin has joined #openstack-infra | 12:55 | |
| openstackgerrit | David Caro proposed a change to openstack-infra/jenkins-job-builder: Added matrix trigger option to email-ext publisher https://review.openstack.org/40214 | 12:55 |
| *** b3nt_pin is now known as beagles | 12:55 | |
| *** bhuvan has joined #openstack-infra | 12:58 | |
| *** senthil has joined #openstack-infra | 13:00 | |
| *** gokrokve has joined #openstack-infra | 13:00 | |
| *** gokrokve has quit IRC | 13:05 | |
| *** CaptTofu has joined #openstack-infra | 13:05 | |
| *** saju_m has joined #openstack-infra | 13:05 | |
| *** jgallard has joined #openstack-infra | 13:05 | |
| *** saju_m has quit IRC | 13:06 | |
| *** saju_m has joined #openstack-infra | 13:07 | |
| *** mwagner_lap has quit IRC | 13:11 | |
| *** pdmars has joined #openstack-infra | 13:13 | |
| *** dkranz has quit IRC | 13:16 | |
| *** hashar has quit IRC | 13:18 | |
| *** heyongli has joined #openstack-infra | 13:18 | |
| *** sarob has joined #openstack-infra | 13:19 | |
| *** yamahata has quit IRC | 13:19 | |
| *** dstanek has joined #openstack-infra | 13:20 | |
| *** resker has quit IRC | 13:20 | |
| *** ociuhandu has joined #openstack-infra | 13:21 | |
| *** eharney has joined #openstack-infra | 13:22 | |
| *** sarob has quit IRC | 13:24 | |
| *** smarcet has left #openstack-infra | 13:25 | |
| *** yamahata has joined #openstack-infra | 13:25 | |
| *** oubiwann_ has joined #openstack-infra | 13:27 | |
| *** prad_ has joined #openstack-infra | 13:29 | |
| *** vbala has quit IRC | 13:29 | |
| *** jhesketh_ has quit IRC | 13:30 | |
| *** thomasem has joined #openstack-infra | 13:31 | |
| *** thomasem has left #openstack-infra | 13:31 | |
| *** CaptTofu has quit IRC | 13:32 | |
| *** dprince has joined #openstack-infra | 13:32 | |
| *** thomasem_ has joined #openstack-infra | 13:32 | |
| *** lcostantino has quit IRC | 13:34 | |
| *** lcostantino has joined #openstack-infra | 13:35 | |
| anteaya | SergeyLukjanov: we can still create new repos | 13:35 |
| SergeyLukjanov | anteaya, hi | 13:35 |
| anteaya | manage projects is logging right now, sometimes it falls over | 13:35 |
| SergeyLukjanov | you mean manually? | 13:35 |
| anteaya | last time it didn't | 13:35 |
| anteaya | well my patch was merged on Monday and I think we are back to automatically | 13:36 |
| anteaya | with logging | 13:36 |
| anteaya | keeping in mind it is fragile and might fall over | 13:36 |
| anteaya | that is my current understanding | 13:36 |
| SergeyLukjanov | anteaya, we can try to merge one more project creation CR, but it requires babysitting | 13:36 |
| *** pafuent has joined #openstack-infra | 13:37 | |
| SergeyLukjanov | anteaya, agreed, I'm planning to look on the script today | 13:37 |
| SergeyLukjanov | to probably generate some crazy ideas | 13:37 |
| anteaya | hughsaunders: yeah filing a bug against ci about a dependency not being available for download... doesn't sound right for me | 13:38 |
| anteaya | SergeyLukjanov: oh okay | 13:39 |
| hughsaunders | anteaya: so is it still ok to use "reverify no bug" in that situation? | 13:39 |
| *** xBsd has quit IRC | 13:39 | |
| SergeyLukjanov | hughsaunders, there is no more such command | 13:39 |
| anteaya | hughsaunders: except reverify no bug has been removed | 13:39 |
| SergeyLukjanov | hughsaunders, you can only 'reverify bug NNN' | 13:39 |
| hughsaunders | ah, so what should I use to reverify if the problem is external? | 13:39 |
| anteaya | I'm more inclined to make a comment on the patch to say that rubygems was having an issue | 13:40 |
| anteaya | and then get a core to reapprove | 13:40 |
| *** hashar has joined #openstack-infra | 13:40 | |
| hughsaunders | so if its reapproved it will get reverified? | 13:40 |
| anteaya | after ensuring you can download that gem by hand before sending it off to the gate again | 13:40 |
| *** CaptTofu has joined #openstack-infra | 13:40 | |
| anteaya | after it is reapproved it will go back into the gate queue | 13:41 |
| anteaya | and gate tests will run and it will be merged if it passes, yes | 13:41 |
| anteaya | SergeyLukjanov: I have been working on getting a mock gerrit set up to run manage projects and evaluate it, for instance to see what is making so many calls to the github api | 13:42 |
| *** senthil has quit IRC | 13:42 | |
| anteaya | but you are faster than I am, so don't let me hold you back | 13:42 |
| anteaya | I am still bogged down with the puppet trying to get the mock gerrit set up | 13:42 |
| SergeyLukjanov | hughsaunders, which change? | 13:43 |
| hughsaunders | SergeyLukjanov: https://review.openstack.org/#/c/68446/17 | 13:44 |
| SergeyLukjanov | anteaya, unfortunately I'm quite overloaded now and I'm not sure that I'll be able to debug it soon enough | 13:44 |
| SergeyLukjanov | anteaya, heh, I need to send summit talks proposals to make marketing team happy | 13:44 |
| sdague | anteaya: no, registering a bug on openstack-ci is right for that | 13:45 |
| anteaya | SergeyLukjanov: how ever you want to spend your time, I can certainly understand being overloaded | 13:46 |
| anteaya | sdague: okay great | 13:46 |
| *** yaguang has joined #openstack-infra | 13:46 | |
| anteaya | hughsaunders: I'm wrong, you were correct in the first place | 13:46 |
| *** talluri has quit IRC | 13:46 | |
| sdague | oh, wait, this is outside our mirror | 13:46 |
| anteaya | yes, rubygems | 13:46 |
| hughsaunders | sdague: yep, totally external | 13:46 |
| *** mfer has joined #openstack-infra | 13:46 | |
| SergeyLukjanov | sdague, I disagree, it doesn't looks like our issue | 13:46 |
| anteaya | I'm quite confident we will not be setting up a mirror for rubygems | 13:47 |
| hughsaunders | :) | 13:47 |
| sdague | anteaya: well, I'd actually suggest that is the right solution, as we should isolate against the network | 13:47 |
| sdague | every time we don't, we fail | 13:47 |
| sdague | not that it's a near term priority | 13:48 |
| sdague | but that's the bug I would register | 13:48 |
| hughsaunders | ok will register wishlist: should mirror rubygems | 13:48 |
| sdague | because then we could keep track of how often it happens, then figure out when it's a big enough deal to actually deal with | 13:48 |
| sdague | hughsaunders: yeh, and explain why in it, and the failures it causes not having | 13:49 |
| anteaya | okay, you have convinced me that filing the bug is in order | 13:49 |
| sdague | hughsaunders: though realistically, if you are core on this, if you just toggle +A it will resubmit | 13:49 |
| hughsaunders | sdague: I am not, but I can someone else to do that | 13:50 |
| sdague | has anyone looked into why gerrit has gotten so slow? | 13:51 |
| anteaya | I have not, what is your metric for the slow? | 13:52 |
| hughsaunders | that worked.. | 13:52 |
| sdague | anteaya: time to run git review | 13:53 |
| sdague | i'll actually time it on my next go | 13:53 |
| hashar | gerrit show-queue often gives clue | 13:53 |
| hashar | I mean the Gerrit cli command over ssh ( ssh -p 29418 <user@gerrit host> gerrit show-queue | 13:54 |
| ArxCruz | ALLL I'm getting this erros in all my jobs 13:07:45 FATAL: Null value not allowed as an environment variable: ZUUL_COMMIT | 13:54 |
| ArxCruz | does anyone knows what can be ? | 13:55 |
| *** lcostantino has quit IRC | 13:55 | |
| anteaya | ArxCruz: jobs running on our infra or yours? | 13:56 |
| anteaya | sdague: k | 13:56 |
| ArxCruz | anteaya: mine | 13:56 |
| *** krtaylor has quit IRC | 13:56 | |
| anteaya | ArxCruz: I am not personally familiar with that error, no | 13:56 |
| *** lcostantino has joined #openstack-infra | 13:56 | |
| *** krtaylor has joined #openstack-infra | 13:57 | |
| ruhe | ArxCruz: looks like a misconfiguration | 13:57 |
| anteaya | ArxCruz: when did it begin happening? | 13:57 |
| ArxCruz | anteaya: yesterday | 13:57 |
| ArxCruz | ruhe: misconfiguration where? | 13:57 |
| ruhe | in your zuul settings or related apache settings | 13:58 |
| *** hdd_ has joined #openstack-infra | 13:58 | |
| *** mrodden has quit IRC | 13:59 | |
| anteaya | sdague: I just looked at the cacti graphs for the review.o.o server and I am not seeing anything extraordinary, or that I recognize as being extraordinary | 13:59 |
| *** locke105 has joined #openstack-infra | 14:00 | |
| *** gokrokve has joined #openstack-infra | 14:00 | |
| ArxCruz | ruhe: anteaya http://paste.openstack.org/show/64642/ | 14:00 |
| ArxCruz | this is my layout.yaml | 14:00 |
| ArxCruz | not sure what's wrong :/ | 14:00 |
| *** heyongli has quit IRC | 14:00 | |
| *** krtaylor has quit IRC | 14:01 | |
| mattymo | anyone else noticing docs.openstack.org is awfully slow right now? | 14:01 |
| *** krtaylor has joined #openstack-infra | 14:02 | |
| *** coolsvap_ has quit IRC | 14:02 | |
| anteaya | mattymo: slow in what way? I can navigate to docs.o.o and click links and pages render for me in a reasonable amount of time | 14:02 |
| anteaya | reasonable being subjective here | 14:03 |
| *** dkliban_afk is now known as dkliban | 14:03 | |
| *** mrfatstrat has quit IRC | 14:04 | |
| anteaya | ArxCruz: beyond confirming that appears to be yaml, I am not much help here, never having configured zuul myself | 14:04 |
| ArxCruz | :/ | 14:04 |
| *** gokrokve has quit IRC | 14:05 | |
| *** yamahata has quit IRC | 14:05 | |
| *** pblaho has quit IRC | 14:07 | |
| *** pblaho has joined #openstack-infra | 14:07 | |
| *** dizquierdo has quit IRC | 14:07 | |
| mattymo | anteaya, must have been intermittent. sorry | 14:07 |
| *** CaptTofu has quit IRC | 14:08 | |
| sdague | anteaya: link to cacti graphs? | 14:09 |
| anteaya | np, perhaps others have noticed the same | 14:09 |
| anteaya | sdague: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=9&page=1 | 14:09 |
| anteaya | mattymo: was it the rendering of pages in your browser that felt slow for you? | 14:09 |
| sdague | anteaya: it's been a while | 14:09 |
| anteaya | np | 14:09 |
| *** CaptTofu has joined #openstack-infra | 14:10 | |
| *** yamahata has joined #openstack-infra | 14:10 | |
| *** thuc has joined #openstack-infra | 14:10 | |
| anteaya | since the cacti url requires the .php? thing I can't get that format in my brain, always have to look it up | 14:10 |
| anteaya | mattoliverau: did you ever put together a patch for the cacti url cleanup? | 14:11 |
| *** julim has joined #openstack-infra | 14:11 | |
| *** sld has joined #openstack-infra | 14:11 | |
| sdague | I expect this is probably the root cause - http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=30&rra_id=all | 14:12 |
| sdague | if you look at the year trend | 14:12 |
| anteaya | wow | 14:12 |
| *** mrodden has joined #openstack-infra | 14:12 | |
| anteaya | look at that ff spike | 14:13 |
| anteaya | but I think you are drawing attention to the slow, constant growth in traffic | 14:13 |
| anteaya | and look at that ff spike | 14:14 |
| anteaya | 5x the traffic compared to a year ago | 14:14 |
| anteaya | the ff is 3x current traffic levels | 14:15 |
| *** mrfatstrat has joined #openstack-infra | 14:16 | |
| *** talluri has joined #openstack-infra | 14:16 | |
| matrohon | anteaya : hi | 14:16 |
| anteaya | hello matrohon | 14:16 |
| anteaya | what's on your mind | 14:16 |
| matrohon | anteaya : i'd like to speak about multi-node setup. do you know the status from a infra point-of-view? | 14:17 |
| *** dkranz has joined #openstack-infra | 14:17 | |
| anteaya | the status from an infra point of view is that mestery said he would get more data and get back to us | 14:18 |
| anteaya | it was an agenda item at one meeting | 14:18 |
| matrohon | the last one? | 14:18 |
| anteaya | the consensus was that we could see this would be a need and that we needed more information before we could draft a plan | 14:18 |
| anteaya | no, before christmas | 14:18 |
| mestery | anteaya: Yes, and I apologize for dropping the ball there. I've been spending most of my free cycles trying to make OpenDaylight a production quality open source Neutron ML2 MechanismDriver. | 14:18 |
| anteaya | mestery: you didn't drop the ball | 14:19 |
| *** yamahata has quit IRC | 14:19 | |
| anteaya | there was an email post about it and I encouraged the querant to talk to you and to put another agenda item on the -infra meeting agenda | 14:19 |
| mestery | anteaya: Thanks. :) | 14:19 |
| *** sarob has joined #openstack-infra | 14:19 | |
| matrohon | do you think we should fill a BP on openstack-infra? | 14:19 |
| anteaya | it is possible that matrohon was the querant | 14:19 |
| anteaya | matrohon: I think you should confer with mestery | 14:20 |
| anteaya | then you should put an agenda item on the -infra agenda | 14:20 |
| anteaya | when it comes up, convey your findings | 14:20 |
| mestery | matrohon: I like that idea my friend. Is this from an L2 Pop perspective? | 14:20 |
| anteaya | then next steps can be discussed | 14:20 |
| matrohon | anteaya : ok this will be on the agenda of the next ML2 meeting | 14:20 |
| matrohon | mestery : yes, this is expected to test l2-pop in the gate without 3rd party testing | 14:21 |
| anteaya | matrohon: fine with me, but -infra will have no knowledge of it until it is on the -infra meeting agenda and gets discussed | 14:21 |
| *** sld has quit IRC | 14:21 | |
| *** dcramer_ has joined #openstack-infra | 14:21 | |
| mestery | matrohon: Very cool. | 14:21 |
| *** saju_m has quit IRC | 14:21 | |
| *** rossella_s has quit IRC | 14:22 | |
| anteaya | also keep in mind, once there is agreement that this is something infra needs to address, we need to put together a plan and a timeline | 14:22 |
| *** mrmartin has joined #openstack-infra | 14:22 | |
| matrohon | anteaya : fine | 14:22 |
| anteaya | great, thanks | 14:23 |
| anteaya | I look forward to the discussion about it | 14:23 |
| *** yamahata has joined #openstack-infra | 14:23 | |
| openstackgerrit | Marton Kiss proposed a change to openstack-infra/config: Update groups gating scripts https://review.openstack.org/72921 | 14:24 |
| matrohon | we will discuss this on the ML2 meeting and get back to infra if needed | 14:24 |
| *** sarob has quit IRC | 14:24 | |
| anteaya | okay, let us know | 14:24 |
| *** lcostantino has quit IRC | 14:24 | |
| sdague | anteaya: 33s to do a git review | 14:26 |
| sdague | that's the slowdown I was talking about | 14:26 |
| anteaya | sdague: you have me convinced | 14:26 |
| anteaya | that is slow | 14:26 |
| sdague | that's been pretty consistent | 14:26 |
| sdague | anyway, probably a post release thing to address | 14:27 |
| anteaya | so right now, we are operating on the theory this is due to a slow constant increase in tcp traffic, correct? | 14:27 |
| sdague | wonder if it is easy to count requests by id and figure out if bots are acting out of control | 14:27 |
| anteaya | well, if the prior ff spike is anything to go by would could run into problems this ff | 14:27 |
| anteaya | that's an idea | 14:28 |
| sdague | git review doesn't fail, it's just slow | 14:28 |
| *** gokrokve has joined #openstack-infra | 14:28 | |
| sdague | so something to note to address later, but I don't think it really impacts ff | 14:28 |
| anteaya | slow tools == frustrated devs that show up in -infra | 14:28 |
| anteaya | okay, well the backscroll will be read so others will weigh in if they have a mind to | 14:29 |
| *** thuc has quit IRC | 14:30 | |
| *** esker has joined #openstack-infra | 14:30 | |
| *** thuc has joined #openstack-infra | 14:30 | |
| *** senthil has joined #openstack-infra | 14:32 | |
| mattymo | ttx, are you available? | 14:33 |
| *** thuc has quit IRC | 14:35 | |
| mattymo | ^ or fungi? re: distribution security notices | 14:36 |
| *** zehicle_at_dell has quit IRC | 14:36 | |
| *** mriedem has joined #openstack-infra | 14:37 | |
| *** saju_m has joined #openstack-infra | 14:38 | |
| *** talluri has quit IRC | 14:39 | |
| sdague | SergeyLukjanov: can I hit you up for this review - https://review.openstack.org/#/c/72786/, 1 line, super fast | 14:40 |
| *** rossella_s has joined #openstack-infra | 14:41 | |
| *** ryanpetrello has joined #openstack-infra | 14:41 | |
| fungi | esker: gerrit doesn't accept or reject changes based on any blueprint status--also we're not terribly dilligent about reviewing (or even noticing) blueprints for openstack-ci since we don't really use them | 14:42 |
| fungi | sdague: i'll check into any resource contention on review.o.o | 14:42 |
| *** mgagne has quit IRC | 14:42 | |
| fungi | mattymo: i'm here--ready for me to test the e-mail address? | 14:42 |
| SergeyLukjanov | sdague, I'm on it | 14:43 |
| SergeyLukjanov | sdague, will be the results collected somehow? | 14:43 |
| mattymo | fungi, osci-security@mirantis.com | 14:43 |
| sdague | SergeyLukjanov: it makes a screen console | 14:43 |
| sdague | so that will get picked up | 14:43 |
| *** MarkAtwood has joined #openstack-infra | 14:43 | |
| *** mgagne has joined #openstack-infra | 14:44 | |
| SergeyLukjanov | sdague, ok, +2 | 14:44 |
| SergeyLukjanov | sdague, is the logs large enough? | 14:44 |
| *** mgagne has quit IRC | 14:44 | |
| sdague | large enough? | 14:44 |
| *** oubiwann_ has quit IRC | 14:45 | |
| SergeyLukjanov | sdague, oops, will it generate very large output? | 14:45 |
| sdague | no | 14:45 |
| sdague | smaller than pidstat | 14:45 |
| sdague | about the same as the sysstat console log | 14:46 |
| *** MarkAtwood has quit IRC | 14:46 | |
| *** oubiwann_ has joined #openstack-infra | 14:49 | |
| SergeyLukjanov | sdague, great | 14:49 |
| *** bhuvan has quit IRC | 14:49 | |
| *** lcostantino has joined #openstack-infra | 14:49 | |
| anteaya | was going to spend time this weekend catching up the the expense reports, the maintenance gods say thou shalt not | 14:50 |
| anteaya | sigh | 14:50 |
| *** mriedem has quit IRC | 14:51 | |
| *** oubiwann_ has quit IRC | 14:54 | |
| *** rcleere has quit IRC | 14:54 | |
| *** oubiwann_ has joined #openstack-infra | 14:54 | |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack/requirements: Oslo program has adopted cliff, pycadf, stevedore, and taskflow https://review.openstack.org/72330 | 14:55 |
| fungi | anteaya: well, you could always scan your receipts and make notes so you can just copy/paste/upload into the expense reporting system once they're done breaking it | 14:55 |
| *** yamahata has quit IRC | 14:55 | |
| anteaya | yes, it is the time to upload that it is tedieous | 14:56 |
| mattymo | I miss my old employer where you could create your report before you go on a trip empty, then get the ID, then you can photograph and email receipts as you get them and they get stored until later | 14:56 |
| anteaya | I don't know what hp feeds the gerbils but they need to switch to a higher octane kibil | 14:56 |
| mattymo | then you go back in and review each receipt image and type it out later | 14:57 |
| mattymo | instead of fiddling through messy receipts when you get back | 14:57 |
| anteaya | mattymo: that sounds like a system that might actually work | 14:57 |
| *** dims has quit IRC | 14:57 | |
| mattymo | my current work is small enough where accountants do all the work, but they will threaten you with bodily harm if you forgot to save a boarding pass | 14:58 |
| anteaya | normally this is not a problem, just back to back trips and a foggy brain that is running on low gear | 14:58 |
| anteaya | not a great combination | 14:58 |
| anteaya | boarding passes I have no problem | 14:58 |
| anteaya | they can have all my boarding passes | 14:58 |
| anteaya | saved every one | 14:58 |
| anteaya | mattymo: send over your accounts | 14:58 |
| mattymo | hahaha ok | 14:59 |
| anteaya | I feel better already, thanks | 14:59 |
| mattymo | they don't speak any english. I once accused them of trying to fool me, but confused some letters and said I thought she was trying to hug me | 14:59 |
| sdague | fungi: easy +A? - https://review.openstack.org/#/c/72786/ | 15:00 |
| anteaya | ha ha ha | 15:00 |
| *** andreaf has quit IRC | 15:00 | |
| anteaya | mattymo: as long as they can add | 15:00 |
| *** dims has joined #openstack-infra | 15:00 | |
| *** banix has joined #openstack-infra | 15:01 | |
| *** cody-somerville has joined #openstack-infra | 15:01 | |
| *** yamahata has joined #openstack-infra | 15:01 | |
| fungi | sdague: i'm looking at job logs for 72786 and not spotting the screen log for dstat... | 15:02 |
| sdague | fungi: because there is a devstack change in flight as well | 15:03 |
| fungi | sdague: aha. that was not mentioned in the changelog | 15:03 |
| * fungi was confused | 15:03 | |
| sdague | oh, sorry. It wasn't really clear to me if there was an order that was prefered | 15:04 |
| sdague | because nothing works until they are both in | 15:04 |
| sdague | I'll push the devstack one to the gate | 15:04 |
| fungi | it is kinda chicken-and-egg, agreed. would just be nice to have at least one of them in and rechecked to make sure the other is right | 15:05 |
| *** jnoller has joined #openstack-infra | 15:05 | |
| *** ryanpetrello_ has joined #openstack-infra | 15:05 | |
| sdague | fungi: sure | 15:05 |
| fungi | especially with te gate so shallow right now | 15:05 |
| sdague | i'll push the devstack one now | 15:05 |
| fungi | sounds great. we can recheck 72786 once that merges so that at least one of them is self-testing | 15:06 |
| sdague | sure | 15:06 |
| sdague | yeh, I was kind of thinking about self testing in the other direction, honestly, as the odds that the devstack bit has to change is higher than the odds the d-g side has to change | 15:06 |
| *** mgagne has joined #openstack-infra | 15:06 | |
| sdague | I also think it's really low risk, as I tested d-g change in local tempest env | 15:07 |
| sdague | and the worst that happens is a screen log isn't created | 15:07 |
| *** ryanpetrello has quit IRC | 15:07 | |
| mtreinish | fungi: quick gerrit question. Everytime I click on a Change-Id link in gerrit I get this error: http://paste.openstack.org/show/64662/ has something changed? | 15:08 |
| SergeyLukjanov | mtreinish, change-id linking was broken by one of the commits that changes links parsing | 15:09 |
| SergeyLukjanov | mtreinish, and it was applied while the latest gerrit restart | 15:09 |
| *** ryanpetrello_ has quit IRC | 15:10 | |
| mtreinish | SergeyLukjanov: ok, I guess that means it will take another restart to fix it then. Oh well | 15:10 |
| *** nicedice has joined #openstack-infra | 15:11 | |
| SergeyLukjanov | mtreinish, when we'll understand the reason for this breakage ;) | 15:11 |
| *** bhuvan has joined #openstack-infra | 15:13 | |
| fungi | sdague: does devstack ignore entries in the enabled services list which it doesn't yet know about? | 15:13 |
| *** max_lobur is now known as max_lobur_afk | 15:14 | |
| sdague | fungi: yeh, it's only possitive tests | 15:14 |
| sdague | is_server_enabled foo | 15:15 |
| sdague | is_service_enabled | 15:15 |
| sdague | which is why d-g change passed | 15:15 |
| sdague | it turned on a service that doesn't exist | 15:15 |
| fungi | got it | 15:15 |
| sdague | but that's fine | 15:15 |
| fungi | i assumed it must since the d-g change didn't fail all its jobs, but wanted to be sure that's intended behavior | 15:16 |
| sdague | yep | 15:16 |
| *** CaptTofu has quit IRC | 15:17 | |
| *** ryanpetrello has joined #openstack-infra | 15:17 | |
| *** sarob has joined #openstack-infra | 15:19 | |
| *** lcostantino has quit IRC | 15:19 | |
| *** lcostantino has joined #openstack-infra | 15:20 | |
| *** sarob has quit IRC | 15:24 | |
| *** rcleere has joined #openstack-infra | 15:24 | |
| *** wenlock has joined #openstack-infra | 15:31 | |
| *** andre__ has quit IRC | 15:34 | |
| *** dkehn__ has joined #openstack-infra | 15:37 | |
| *** coolsvap has joined #openstack-infra | 15:37 | |
| fungi | sdague: checking back over system resource utilization for review.o.o (cacti graphs and spot-checks with diagnostic tools locally on the system) nothing's jumping out at me as necessarily broken/choked. network performance also seems fine (no packet loss and reasonable rtt from my house) | 15:37 |
| sdague | dhellmann_: would it be totally insane to set PYTHONDONTWRITEBYTECODE in the tox.ini ? | 15:38 |
| fungi | sdague: though rackspace notes some disruption today (that vm is a next gen in dfw): https://status.rackspace.com/index/viewincidents?group=21&start=1392181200 | 15:38 |
| sdague | you get really bizarre issues running tests once things move to oslo | 15:38 |
| *** dkehn has quit IRC | 15:38 | |
| sdague | fungi: gerrit slowness has been long standing for me | 15:38 |
| *** dkehn_ has quit IRC | 15:38 | |
| sdague | a git review upload is ~30s | 15:38 |
| sdague | and has been for at least a month | 15:39 |
| *** atiwari has joined #openstack-infra | 15:39 | |
| *** andre__ has joined #openstack-infra | 15:39 | |
| *** chandan_kumar has quit IRC | 15:39 | |
| *** dkehn has joined #openstack-infra | 15:40 | |
| fungi | sdague: in that case, if it's resource constraints that's likely within the jvm. i'll see if i can replicate the behavior--it takes that long for me, but i assumed that's because i'm running git-review within a busy 256mb paravirtualized server using an nfs root filesystem | 15:40 |
| *** chandan_kumar has joined #openstack-infra | 15:40 | |
| sdague | yeh, no, this is everywhere | 15:40 |
| *** olaph has quit IRC | 15:40 | |
| fungi | so my performance should be worst-case regardless | 15:40 |
| sdague | yep | 15:40 |
| *** AaronGr_Zzz is now known as AaronGr | 15:40 | |
| sdague | I expect it's actually constrained on the tcp connect side, given the cacti graphs | 15:41 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: Merger: log non Git exceptions on merge https://review.openstack.org/70977 | 15:41 |
| jaypipes | SergeyLukjanov: hi :) do you have a sec to help debug an issue with Jenkins? | 15:42 |
| *** olaph has joined #openstack-infra | 15:42 | |
| jeblair | fungi, SergeyLukjanov: good morning; anything i can help with? | 15:42 |
| fungi | sdague: maybe... though cacti says it's hovering around 100 open tcp connections with ~8 events a second | 15:42 |
| sdague | fungi: yeh, though look at the year trend | 15:43 |
| fungi | jeblair: no emergencies this morning. just looking at slowness from git-review | 15:43 |
| jeblair | that doesn't sound terrible | 15:43 |
| fungi | presumably gerrit | 15:43 |
| jeblair | (the tcp level/rate) | 15:43 |
| SergeyLukjanov | jaypipes, hi, we can try :) it's about your attempt to install own infra? | 15:43 |
| sdague | fungi: honestly, the git-review slowness isn't a high priority thing | 15:43 |
| *** dkehn_ has joined #openstack-infra | 15:43 | |
| sdague | it was mostly a "hey, this takes 30s now, and it used to take 5s" | 15:43 |
| jeblair | keep in mind gerrit has its own internal limit/tuning parameters | 15:43 |
| SergeyLukjanov | jeblair, morning, gate is working now and I'm happy about it ;) | 15:43 |
| sdague | jeblair: ok, so we might be hitting internal rate limits? | 15:43 |
| jeblair | it's possible to connect to gerrit and wait for a thread | 15:44 |
| sdague | is there any way to measure that? | 15:44 |
| jeblair | sdague: it's an (unverified) possibility | 15:44 |
| sdague | because from cacti, it seems like the box has plenty of juice | 15:44 |
| jeblair | there's javamelody monitoring and some gerrit ssh cli tools, both of which are admin only :( | 15:44 |
| sdague | lots of idle, lots of extra mem | 15:44 |
| *** dkehn has quit IRC | 15:45 | |
| jeblair | sdague: honestly, it's a bit of a black art; afaict the most complete documentation of it is contained in our puppet manifests | 15:45 |
| jaypipes | SergeyLukjanov: yes :) I have everything installed and running. Jenkins Job Builder has built my job, and Jenkins shows it in its job list. Zuul is running properly now, and is triggering the jenkins job when receiving events on openstack-dev/sandbox project. However, I am getting LOST builds and there are messages in the debug log about the job not being registered with Gearman. | 15:45 |
| jeblair | sdague: i collected every bit of arcane knowledge i could and stuck them in there. sometimes while in airport lounges. :( | 15:45 |
| ArxCruz | Does anyone know's who's the Hyper-V CI responsible? I would like to talk with him/her :) | 15:46 |
| sdague | jeblair: yeh, no worries | 15:46 |
| sdague | like I said, this is kind of a low priority thing, but wanted to make sure folks knew it was there | 15:46 |
| fungi | ArxCruz: there is a (presumed working) contact address for that account. should be able to see it if you hover your cursor over the account name | 15:46 |
| *** dkehn__ has quit IRC | 15:47 | |
| *** dkehn__ has joined #openstack-infra | 15:47 | |
| ArxCruz | fungi: thanks | 15:47 |
| *** pdmars_ has joined #openstack-infra | 15:47 | |
| fungi | they just provided us that e-mail address last week, so it should be fresh | 15:47 |
| SergeyLukjanov | jaypipes, hm, can't remember that I had such errors | 15:48 |
| SergeyLukjanov | ruhe, ping | 15:48 |
| *** CaptTofu has joined #openstack-infra | 15:48 | |
| *** dkehn has joined #openstack-infra | 15:48 | |
| SergeyLukjanov | jaypipes, probably, ruhe saw them | 15:48 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: Log components starts in Zuul.Server https://review.openstack.org/66939 | 15:48 |
| *** sarob has joined #openstack-infra | 15:48 | |
| *** dstanek_afk has joined #openstack-infra | 15:48 | |
| ruhe | hey hey | 15:48 |
| *** prad_ has quit IRC | 15:49 | |
| SergeyLukjanov | jaypipes, how is the jenkins job logs? | 15:49 |
| SergeyLukjanov | jaypipes, any errors? | 15:49 |
| ruhe | yes, i've seen it | 15:49 |
| jeblair | sdague, fungi: it looks like we have yet another kind of poorly-configured ci system... | 15:49 |
| openstackgerrit | A change was merged to openstack-infra/git-review: Don't try to attach known_hosts if it's not there https://review.openstack.org/72910 | 15:49 |
| *** davidlenwell_ has joined #openstack-infra | 15:49 | |
| fungi | jeblair: some gerrit account spamming connections to the api? | 15:49 |
| krtaylor | ArxCruz, https://review.openstack.org/#/admin/groups/91,members | 15:49 |
| jeblair | sdague, fungi: last night the stable-maint list got spammed by a zuul that was configured with openstack's layout.yaml | 15:50 |
| *** jooools1 has joined #openstack-infra | 15:50 | |
| fungi | ohhh | 15:50 |
| fungi | ouch | 15:50 |
| jeblair | sdague, fungi: so it tried to run stable bitrot jobs and sent email as jenkins@ | 15:50 |
| sdague | jeblair: nice.... | 15:50 |
| *** talluri has joined #openstack-infra | 15:50 | |
| fungi | and the cargo cult gets one more member | 15:50 |
| ruhe | SergeyLukjanov: jaypipes: that happens, when you don't have jenkins slaves | 15:50 |
| sdague | yeh, I think there is this ton of lessons learned from 3rd party CI, like the fact that we need more defensive systems | 15:50 |
| jeblair | fungi, sdague: later today i will write some exim rules that will prevent that. | 15:51 |
| sdague | there is a lot of presumed trust in the whole process | 15:51 |
| ruhe | jaypipes: jobs will be registered only when a slave is attached to jenkins | 15:51 |
| *** oanufriev has quit IRC | 15:51 | |
| *** SpamapS_ has joined #openstack-infra | 15:51 | |
| ruhe | SergeyLukjanov: jaypipes: the only thing i'm not sure about is - should the slave be built with nodepool or not | 15:52 |
| jeblair | sdague, fungi: the ipv6 block it came from is rackspace HK | 15:52 |
| *** jp_at_hp1 has joined #openstack-infra | 15:52 | |
| sdague | interesting... | 15:53 |
| *** atiwari_ has joined #openstack-infra | 15:53 | |
| *** sarob has quit IRC | 15:53 | |
| *** yaguang has quit IRC | 15:53 | |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/reviewday: Make reviewday.json world readable. https://review.openstack.org/72682 | 15:54 |
| jeblair | to them i would like to say, congrats on using ipv6, now fix your zuul. :) | 15:54 |
| *** talluri has quit IRC | 15:54 | |
| fungi | well, presumably their use of ipv6 is entirely courtesy of it being enabled by default in rackspace hk | 15:55 |
| *** thedodd has joined #openstack-infra | 15:55 | |
| fungi | so there's every chance they don't even know | 15:55 |
| *** prad_ has joined #openstack-infra | 15:56 | |
| *** dkehn_ has quit IRC | 15:56 | |
| *** atiwari has quit IRC | 15:56 | |
| *** dkranz has quit IRC | 15:56 | |
| *** dstanek has quit IRC | 15:56 | |
| *** pdmars has quit IRC | 15:56 | |
| *** markmc has quit IRC | 15:56 | |
| *** apevec has quit IRC | 15:56 | |
| *** jp_at_hp has quit IRC | 15:56 | |
| *** jooools has quit IRC | 15:56 | |
| *** jaypipes has quit IRC | 15:56 | |
| *** SpamapS has quit IRC | 15:56 | |
| *** ogelbukh has quit IRC | 15:56 | |
| *** davidlenwell has quit IRC | 15:56 | |
| *** mordred has quit IRC | 15:56 | |
| *** zul has joined #openstack-infra | 15:56 | |
| *** markwash has joined #openstack-infra | 15:57 | |
| anteaya | looks like the start of another fun freenode day | 15:57 |
| krtaylor | hehheh | 15:57 |
| SergeyLukjanov | anteaya, I hope not today ;) | 15:57 |
| anteaya | I hope not too | 15:57 |
| *** ryanpetrello_ has joined #openstack-infra | 15:58 | |
| *** chandankumar_ has joined #openstack-infra | 15:59 | |
| *** chandan_kumar has quit IRC | 15:59 | |
| *** ryanpetrello has quit IRC | 15:59 | |
| *** ryanpetrello_ is now known as ryanpetrello | 15:59 | |
| jeblair | mrmartin: openstackid can use precise php after all? https://review.openstack.org/#/c/72973/1 | 15:59 |
| *** dims has quit IRC | 15:59 | |
| *** atiwari_ has quit IRC | 16:00 | |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/reviewday: Use a with block to create index.html. https://review.openstack.org/72975 | 16:00 |
| mrmartin | jeblair: yeap, I just made a test, and it seems to be that tarball packaging part will work, and we need to test the overall project | 16:00 |
| *** atiwari has joined #openstack-infra | 16:00 | |
| *** markwash_ has joined #openstack-infra | 16:00 | |
| fungi | i'll be amused if so. smarcet seemed certain it needed features only introduced in 5.4, though what exactly i'm not certain | 16:01 |
| mrmartin | jeblair: we did a quick grep, and not found php 5.4 specific short array usage, but more testing required here | 16:01 |
| *** coolsvap1 has joined #openstack-infra | 16:01 | |
| *** markwash has quit IRC | 16:01 | |
| *** markwash_ is now known as markwash | 16:01 | |
| *** amotoki_ has joined #openstack-infra | 16:02 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Allow bare-precise nodes to sudo https://review.openstack.org/72780 | 16:02 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Use a sudoers file for jenkins sudo rights https://review.openstack.org/72777 | 16:02 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Revoke sudo permissions before infra jobs https://review.openstack.org/72778 | 16:02 |
| *** jaypipes has joined #openstack-infra | 16:02 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Revoke sudo from most jobs https://review.openstack.org/72779 | 16:02 |
| *** lcostantino has quit IRC | 16:02 | |
| *** ogelbukh has joined #openstack-infra | 16:02 | |
| *** markmc has joined #openstack-infra | 16:02 | |
| *** apevec has joined #openstack-infra | 16:02 | |
| *** apevec has joined #openstack-infra | 16:02 | |
| *** lcostantino has joined #openstack-infra | 16:02 | |
| jeblair | mrmartin: cool; that will be much easier to work with! | 16:02 |
| jaypipes | SergeyLukjanov: apologies... I lost Internet connectivity :( | 16:03 |
| *** sarob has joined #openstack-infra | 16:03 | |
| *** dkranz has joined #openstack-infra | 16:03 | |
| mrmartin | jeblair: I also added basic php package support to jenkins slave.pp, as a part of this patch: https://review.openstack.org/#/c/72921/ | 16:03 |
| *** coolsvap has quit IRC | 16:03 | |
| SergeyLukjanov | jaypipes, have some issues too - 3g isn't stable enough here ;) | 16:03 |
| *** mordred has joined #openstack-infra | 16:04 | |
| jaypipes | SergeyLukjanov: did you get these messages from me? http://paste.openstack.org/show/64679/ | 16:04 |
| *** dstanek_afk is now known as dstanek | 16:04 | |
| jeblair | mrmartin: why did you want to move the build scripts? | 16:05 |
| fungi | sdague: do you remember what was done to fix gate-tempest-dsvm-neutron-pg-isolated recently? it's failing fairly consistently for havana so i assume we're missing a backport of something | 16:05 |
| mrmartin | jeblair: because I cannot update them now :) catch of 22, they are sitting inside groups/ project and I cannot update it because gating fails. | 16:06 |
| ruhe | jaypipes: do you have jenkins slave nodes? are they built with nodepool? | 16:06 |
| sdague | fungi: no, mtreinish might remember | 16:06 |
| jaypipes | ruhe: no, I do not. just the one master jenkins node. | 16:06 |
| fungi | well, failing for neutron changes on havana anyway | 16:06 |
| ruhe | jaypipes: then jobs will not register in gearman. you need at least one jenkins slave node | 16:07 |
| jeblair | mrmartin: oh, heh, okay. we can also make jobs non-voting in cases like that, but this might be better anyway. | 16:07 |
| SergeyLukjanov | jaypipes, reading | 16:07 |
| mrmartin | jeblair: I checked js-build, and we can make something similar with drush/drupal and make a generic build script version later. | 16:07 |
| jeblair | mrmartin: great | 16:07 |
| SergeyLukjanov | jaypipes, do you have slaves generated by nodepool? | 16:07 |
| mtreinish | fungi: the isolated jobs oh that was salv-orlando's work there were changes on the neutron side | 16:07 |
| mtreinish | I think they're too much to backport though | 16:08 |
| *** sarob has quit IRC | 16:08 | |
| fungi | mtreinish: okay cool. salv-orlando: should we consider disabling isolated jobs for neutron on havana? | 16:08 |
| jaypipes | ruhe: I suspected that... so I guess I need to include jenkins::slave in the main mainfest... | 16:08 |
| jaypipes | SergeyLukjanov: no, I only have the one jenkins master node | 16:08 |
| salv-orlando | fungi: moved away from nbd mounting. We started using metadata based injection | 16:08 |
| mrmartin | jeblair: push this through quickly, and I will deliver the openstackid related gating tomorrow, it also depends on php, and we can enhance it later, but currently I cannot deliver community portal, and the openid project is a must go also | 16:08 |
| jaypipes | SergeyLukjanov: I was hoping to get this up and running without nodepool at first... | 16:08 |
| SergeyLukjanov | jaypipes, probably you could add slave manually with label specified for job | 16:09 |
| ruhe | SergeyLukjanov: it should work with manually built and attached slaves. nodepool is not required | 16:09 |
| jeblair | mrmartin: yep, i will review those changes today | 16:09 |
| salv-orlando | fungi: There were a few patches which went in into nova, devstack, and openstack-infra/config | 16:09 |
| salv-orlando | \ | 16:09 |
| openstackgerrit | Alexander Jones proposed a change to openstack-infra/git-review: Fix parsing of SCP-style URLs, as these are valid in Git itself https://review.openstack.org/72751 | 16:09 |
| fungi | jaypipes: if you're running jobs which can be run safely on your jenkins master, i think just changing the node label on your jobs to master will work | 16:09 |
| mrmartin | jeblair: thank you, I need to leave soon, but will check when I arrive home. | 16:09 |
| salv-orlando | fungi: If you can wait a bit I can dig all of them so you can see if they can be backported | 16:09 |
| fungi | jaypipes: but anything which makes system-wide changes/uses sudo is likely to trash your jenkins master | 16:10 |
| jaypipes | interesting.... ok, lemme play around with that and see if gearman picks it up | 16:10 |
| jaypipes | fungi: like devstack? ;) | 16:10 |
| jaypipes | fungi: yeah, I know ... was just trying to get the full communication cycle working at first. | 16:10 |
| fungi | jaypipes: yes, you'll definitely want something like nodepool-created single-use slaves for running devstack | 16:10 |
| *** mrda_away is now known as mrda | 16:11 | |
| fungi | jaypipes: you could try gate-noop jobs with their node label changed to master | 16:11 |
| jaypipes | fungi: yeah, that's what I'm doing right now... | 16:11 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: restrict all queries to the relevant file https://review.openstack.org/72793 | 16:11 |
| jaypipes | just to see if all the pieces are communicating correctly... | 16:11 |
| *** gyee has quit IRC | 16:11 | |
| jaypipes | Hmm, what would Brian Boitano do?... | 16:12 |
| *** branen has joined #openstack-infra | 16:12 | |
| *** nosnos has quit IRC | 16:12 | |
| fungi | jaypipes: you just brought back a flood of memories of the original southpark short film | 16:12 |
| jaypipes | :) | 16:13 |
| *** virmitio has joined #openstack-infra | 16:13 | |
| fungi | salv-orlando: there's no rush. i was just noticing that neutron backports for 2013.2.2 are likely to be a real pain for apevec if we're constantly failing isolated jobs on them | 16:13 |
| *** dims has joined #openstack-infra | 16:14 | |
| *** markmcclain has joined #openstack-infra | 16:14 | |
| *** david-lyle has joined #openstack-infra | 16:15 | |
| sdague | fungi: realistically it's actually bad for all havana backports as well as the main gate | 16:17 |
| dhellmann_ | sdague: what problem would PYTHONDONTWRITEBYTECODE fix? | 16:17 |
| *** dhellmann_ is now known as dhellmann | 16:17 | |
| fungi | dhellmann_: any idea why pycadf installation into a virtualenv would have tried to write to /etc? http://logs.openstack.org/07/72207/1/gate/gate-requirements-integration-dsvm/2b6e7ae/console.html | 16:17 |
| fungi | dhellmann: ^ | 16:17 |
| dhellmann | fungi: looking | 16:17 |
| *** AlexF has joined #openstack-infra | 16:18 | |
| *** mriedem has joined #openstack-infra | 16:18 | |
| sdague | dhellmann_: the fact that when you migrate to an external oslo library, the pyc in your local test tree doesn't get removed from openstack/common | 16:18 |
| sdague | and then yuo explode on duplicate conf option error | 16:18 |
| SergeyLukjanov | jeblair, ttx, do we like to have a separated repo for release testing or just make corresponding jobs for existing sandbox? | 16:18 |
| fungi | sdague: i think it's probably not impacting other havana backports if we're only running those jobs against neutron changes for that branch, but i agree it does make gating slow due to constant resets | 16:18 |
| dhellmann | sdague: oh, I have a git checkout hook that removes all my .pyc files every time I change branches | 16:18 |
| SergeyLukjanov | jeblair, ttx, I'm 50-50 on both options :) | 16:19 |
| *** mrmartin has quit IRC | 16:19 | |
| jeblair | SergeyLukjanov: i think we can do it on the existing sandbox | 16:19 |
| ttx | Question: do we hate that milestone-proposed branches are deleted and recreated, creating stale branches in some repositories ? Do we hate it enough to reeducate people away from m-p and towards proposed/* model ? | 16:19 |
| sdague | dhellmann: so if that's the case, why not just suppress the pyc in tox? | 16:19 |
| *** gyee has joined #openstack-infra | 16:19 | |
| dhellmann | sdague: is this just a local dev issue, or a gate issue? | 16:19 |
| sdague | local dev | 16:19 |
| sdague | the gate is fresh all the time | 16:19 |
| mordred | yeah - isn't there an env var taht can be set to do taht? | 16:19 |
| sdague | mordred: yes | 16:20 |
| dhellmann | mordred: yeah, that's what he's suggesting | 16:20 |
| fungi | ttx: i think it's a filthy habit, but it doesn't really impact me in any actual way, so i don't suppose i care too strongly | 16:20 |
| jeblair | ttx: i'm okay with either, but i like the idea of changing when juno opens if we're going to change | 16:20 |
| ttx | jeblair: yes, that would be more consistent. | 16:20 |
| dhellmann | sdague: I guess I don't have an issue with it, but it feels a little weird | 16:20 |
| SergeyLukjanov | ttx, IMO proposed/* is much more consistent, but it looks quite scare to re-educate all contributors ;) | 16:20 |
| dhellmann | sdague: like there ought to be a "cleaner" way to solve it | 16:20 |
| sdague | <sdague> dhellmann_: would it be totally insane to set PYTHONDONTWRITEBYTECODE in the tox.ini ? | 16:20 |
| sdague | for mordred | 16:20 |
| mordred | sdague: I beliee we should set that | 16:21 |
| sdague | dhellmann: well, that's why I asked | 16:21 |
| sdague | mordred: ok | 16:21 |
| dhellmann | fungi: it looks like pycadf is installing a data file into /etc. I'll talk to gordc about removing that | 16:22 |
| *** lttrl has quit IRC | 16:22 | |
| dhellmann | mordred: I'm uneasy about it, can you explain more about why you think it's a good idea? | 16:22 |
| fungi | dhellmann: or make it such that it's relative to the virtualenv when it's being installed into one | 16:22 |
| dhellmann | fungi: the setup.cfg just lists data files, I'm not sure how to be more specific about the destination | 16:23 |
| dhellmann | fungi: http://git.openstack.org/cgit/openstack/pycadf/tree/setup.cfg#n24 | 16:23 |
| ttx | jeblair: let's WIP it until Juno summit and try to assess if it's useful by then | 16:24 |
| ttx | jeblair: agreed? | 16:24 |
| fungi | dhellmann: in that case, we might be dealing with another iteration of the system site packages broken we were seeing before | 16:24 |
| *** thomasem_ is now known as thomasem | 16:24 | |
| *** olaph has quit IRC | 16:25 | |
| *** beagles is now known as beagles_brb | 16:25 | |
| *** olaph has joined #openstack-infra | 16:26 | |
| *** prad__ has joined #openstack-infra | 16:26 | |
| *** lttrl has joined #openstack-infra | 16:27 | |
| *** thuc has joined #openstack-infra | 16:27 | |
| kiall | Hey guys - Is it just me (or @hp.com..) or is the Wiki failing to send emails? | 16:28 |
| *** prad_ has quit IRC | 16:28 | |
| kiall | Wanted to get notified of changes to watched wiki pages, and updated my email.. the confirmation doesn't seem to be arriving. :( | 16:28 |
| dhellmann | fungi: https://bugs.launchpad.net/pycadf/+bug/1279405 | 16:29 |
| uvirtbot | Launchpad bug 1279405 in pycadf "pycadf installs files to global /etc directory, breaking OpenStack gate jobs" [Undecided,New] | 16:29 |
| fungi | kiall: i'll have a look at its mail queue | 16:30 |
| kiall | thanks fungi .. | 16:30 |
| jeblair | ttx: agreed | 16:30 |
| fungi | dhellmann: thanks! it's entirely possible this isn't pycadf's doing since it's been a week from the last release and this is the first time i've spotted a failure on that job, but i guess we can figure that out | 16:31 |
| mordred | dhellmann: because we're doing virtualenvs, and because pyc files are pretty useless these days | 16:31 |
| dhellmann | fungi: sure | 16:31 |
| fungi | kiall: two messages pending to you, one for 12 minutes and teh other for 5 minutes | 16:31 |
| dhellmann | mordred: useless? | 16:32 |
| mordred | yeah | 16:32 |
| mordred | in a dev tree | 16:32 |
| fungi | kiall: hp may just be greylisting the wiki server. i don't see any other queued messages on it for anyone | 16:32 |
| *** thuc has quit IRC | 16:32 | |
| *** talluri has joined #openstack-infra | 16:32 | |
| dhellmann | mordred: ok, it still feels like something that's easy enough to clean up but I don't see any negative impact | 16:33 |
| fungi | kiall: 2014-02-12 16:24:36 SMTP timeout while connected to smtp.hp.com [15.201.208.58] after initial connection: Connection timed out | 16:33 |
| kiall | fungi: humm, ok .. That's possible all right. I know I tried this in the past, a few months back, and never got any email then either :/ Will wait and se | 16:33 |
| *** chadlung has joined #openstack-infra | 16:33 | |
| *** markmcclain has quit IRC | 16:33 | |
| kiall | Ahh .. Gotta love timeouts -_- | 16:33 |
| dhellmann | mordred: I suggest changing the cookiecutter repo and then updating the other projects -- I found the cookiecutter template to be a little out of date when I forked it for oslo-cookiecutter | 16:33 |
| *** markmcclain has joined #openstack-infra | 16:34 | |
| fungi | kiall: almost certainlg greylisting. i forced exim to retry delivery on those early and they went through this time | 16:34 |
| kiall | fungi: just arrived - I would have though Greylisting would respond with a 450 Temp Rejection, rather than a TCP timeout.. Oh well, It worked. Thanks :) | 16:36 |
| *** talluri has quit IRC | 16:36 | |
| fungi | kiall: spam mitigation techniques have become progressively more unfriendly over time as the spam problem continues to fester. nobody really seems to care about ietf rfc correctness when implementing mtas these days, at least where commercial/proprietary solutions are concerned | 16:37 |
| *** lcostantino has quit IRC | 16:38 | |
| *** talluri has joined #openstack-infra | 16:38 | |
| *** lcostantino has joined #openstack-infra | 16:39 | |
| kiall | Yep - That why I've deferred every MTA I've looked after to .. Google Apps ;) | 16:39 |
| kiall | (Well - There's still 1 MS Exchange left.. It needs to be shot ASAP though) | 16:39 |
| kiall | (Never, ever, ever agree to even think about running MS Exchange ... In order for it to just boot, it's going to consume 5GB of RAM.. To actually handle things like Search, it's going to want 8GB or so.. What a waste.) | 16:41 |
| *** CaptTofu has quit IRC | 16:41 | |
| fungi | kiall: for me it's spamd on openbsd providing teergrubing for flagrantly bad mtas, and then sa-exim (exim4 with spamassassin as a plugin filter) doing graduated classification and autolearning | 16:41 |
| *** pblaho has quit IRC | 16:41 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/zuul: Log components starts in Zuul.Server https://review.openstack.org/66939 | 16:42 |
| fungi | and then i've added a feedback loop where moving misclassified messages into specially-named maildirs updates the classification further | 16:42 |
| *** coolsvap1 has quit IRC | 16:44 | |
| *** sandywalsh has joined #openstack-infra | 16:44 | |
| *** lcostantino has quit IRC | 16:45 | |
| *** lcostantino has joined #openstack-infra | 16:46 | |
| kiall | fungi: I remember "playing" with most of those kinda tools years ago.. Never managed to get great classification of spam .. and gave up ;) | 16:47 |
| kiall | Google Apps was just so much easier and had a great UI.. One of these days, I'll probably re-visit just to move away from Google ;) | 16:47 |
| fungi | kiall: i did it for a living at an isp/datacenter hosting company for more than a decade, so i sort of had to figure it out | 16:47 |
| fungi | also, i don't trust google any further than i can kick them ;) | 16:48 |
| kiall | Yea - I don't trust them as much as I used to.. But.. I don't distrust them enough (yet) to take it all on myself! | 16:49 |
| *** lttrl has quit IRC | 16:51 | |
| *** olaph has quit IRC | 16:51 | |
| openstackgerrit | A change was merged to openstack-dev/hacking: HACKING.rst: Fix odd indentation in an example code https://review.openstack.org/72255 | 16:52 |
| *** CaptTofu has joined #openstack-infra | 16:54 | |
| *** lcostantino has quit IRC | 16:54 | |
| *** lcostantino has joined #openstack-infra | 16:54 | |
| *** lttrl has joined #openstack-infra | 16:55 | |
| mordred | dhellmann: ++ | 16:56 |
| *** julim has quit IRC | 16:57 | |
| openstackgerrit | Mate Lakat proposed a change to openstack-infra/nodepool: Support install phase with nodepool https://review.openstack.org/61463 | 16:58 |
| *** coolsvap has joined #openstack-infra | 17:00 | |
| *** dstufft|laptop has joined #openstack-infra | 17:01 | |
| *** rlandy has quit IRC | 17:01 | |
| *** dstufft|laptop has quit IRC | 17:01 | |
| *** yamamoto has joined #openstack-infra | 17:02 | |
| *** thomasem_ has joined #openstack-infra | 17:03 | |
| openstackgerrit | Mate Lakat proposed a change to openstack-infra/nodepool: Allow useage of server IDs as well as names. https://review.openstack.org/69424 | 17:03 |
| openstackgerrit | Mate Lakat proposed a change to openstack-infra/nodepool: Support install phase with nodepool https://review.openstack.org/61463 | 17:04 |
| *** thomasem_ has quit IRC | 17:04 | |
| *** ryanpetrello_ has joined #openstack-infra | 17:05 | |
| *** ryanpetrello has quit IRC | 17:05 | |
| *** ryanpetrello_ is now known as ryanpetrello | 17:05 | |
| yamamoto | who's maintaining ical feed for meetings? | 17:05 |
| *** e0ne has quit IRC | 17:05 | |
| *** thomasem has quit IRC | 17:05 | |
| jeblair | yamamoto: ttx; just change the wiki page and he will update | 17:06 |
| * ttx sobs | 17:07 | |
| ttx | i'm the missing link in meeting automation evolution | 17:07 |
| jeblair | yamamoto: he is an automatic ical updating machine. he's not happy unless you feed him a constant stream of meeting updates. | 17:07 |
| jeblair | ttx: do you do free/busy scheduling? :) | 17:08 |
| yamamoto | ml2 subteam meeting seems wrong in ical feed | 17:08 |
| ttx | it changes so many times I lost count | 17:08 |
| * ttx checks | 17:08 | |
| *** dkehn__ is now known as dkehn_ | 17:08 | |
| *** jgallard has quit IRC | 17:09 | |
| ttx | yamamoto: fixed | 17:09 |
| yamamoto | thanks! | 17:09 |
| *** markmc has quit IRC | 17:11 | |
| mordred | dhellmann: is there something other than adding the bytecode thing to cookiecutter you think needs to be added/changed? (off the top of your head) | 17:12 |
| *** hdd_ has quit IRC | 17:14 | |
| *** beagles_brb is now known as beagles | 17:15 | |
| *** kmartin has joined #openstack-infra | 17:15 | |
| *** zul has quit IRC | 17:17 | |
| *** AlexF has quit IRC | 17:17 | |
| *** zul has joined #openstack-infra | 17:18 | |
| *** zul has quit IRC | 17:18 | |
| *** zul has joined #openstack-infra | 17:18 | |
| *** mrfatstrat has quit IRC | 17:23 | |
| *** mrfatstrat has joined #openstack-infra | 17:24 | |
| *** garyk has quit IRC | 17:28 | |
| *** jdurgin has quit IRC | 17:28 | |
| *** mrfatstrat has quit IRC | 17:29 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add the climate-nova project https://review.openstack.org/71614 | 17:30 |
| *** gokrokve has quit IRC | 17:32 | |
| *** gokrokve has joined #openstack-infra | 17:32 | |
| * fungi watches review.o.o to see if that ^ works as intended | 17:32 | |
| fungi | oh, nm. that was a job config change not a new project addition | 17:32 |
| *** gokrokve has quit IRC | 17:37 | |
| *** smarcet has joined #openstack-infra | 17:39 | |
| *** rossella_s has quit IRC | 17:39 | |
| *** jdurgin has joined #openstack-infra | 17:40 | |
| *** beagles has quit IRC | 17:40 | |
| mriedem | anyone seeing routing fails in the check-tempest-dsvm-postgres-full job today? | 17:40 |
| *** b3nt_pin has joined #openstack-infra | 17:41 | |
| *** b3nt_pin is now known as beagles | 17:41 | |
| mriedem | http://logs.openstack.org/46/69046/5/check/check-tempest-dsvm-postgres-full/715bd3c/console.html | 17:41 |
| *** senthil has quit IRC | 17:41 | |
| *** lcostantino has quit IRC | 17:45 | |
| *** lcostantino has joined #openstack-infra | 17:45 | |
| *** matsuhashi has quit IRC | 17:48 | |
| fungi | mordred: if you're around, i've got a requirements integration failure on a tox.ini change for stable/havana exhibiting strange pbr/setuptools hoodoo... https://review.openstack.org/62209 | 17:49 |
| fungi | seems to have started happening around two weeks ago | 17:49 |
| *** eharney has quit IRC | 17:49 | |
| *** dprince has quit IRC | 17:50 | |
| *** saju_m has quit IRC | 17:50 | |
| mriedem | fungi: looks like what i'm seeing might only be a timeout on rax nodes: https://bugs.launchpad.net/openstack-ci/+bug/1279443 | 17:51 |
| uvirtbot | Launchpad bug 1279443 in openstack-ci "check-tempest-dsvm-postgres-full fails in routing setup - rax only?" [Undecided,New] | 17:51 |
| *** saju_m has joined #openstack-infra | 17:51 | |
| *** matsuhashi has joined #openstack-infra | 17:51 | |
| fungi | mriedem: https://status.rackspace.com/index/viewincidents?group=21&start=1392181200 | 17:52 |
| fungi | mriedem: looks like the issue was first reported in rax iad around 02:30 utc and corrected around 10:40 utc, so your test failure falls in that window and matches the symptoms i'd expect | 17:53 |
| *** hashar has quit IRC | 17:53 | |
| mriedem | fungi: ah, ok | 17:53 |
| mriedem | was about to yell at all the rackspace people at the nova meetup :) | 17:54 |
| clarkb | morning | 17:54 |
| *** matsuhashi has quit IRC | 17:55 | |
| clarkb | I am going to check on a couple things first, but would like to get the logstash/ES upgrades rolling. jeblair fungi anyone interested in doing a play by play via screen? | 17:55 |
| *** pdmars_ is now known as pdmars | 17:55 | |
| clarkb | its not terribly complicated, mostly just coordinating stops and starts of services and checking logs as we go | 17:56 |
| fungi | clarkb: i'm happy to screen -x into a shared session and follow along | 17:56 |
| *** saju_m has quit IRC | 17:58 | |
| mordred | fungi: I'm confused about taht | 17:59 |
| mordred | OH | 17:59 |
| *** saju_m has joined #openstack-infra | 17:59 | |
| mordred | crappit | 17:59 |
| *** AlexF has joined #openstack-infra | 18:01 | |
| openstackgerrit | Mate Lakat proposed a change to openstack-infra/nodepool: Support install phase with nodepool https://review.openstack.org/61463 | 18:01 |
| clarkb | fungi: great, I will start a root screen on elasticsearch.o.o | 18:01 |
| *** pleia2 has quit IRC | 18:02 | |
| *** morganfainberg_Z has quit IRC | 18:02 | |
| *** chandankumar_ has quit IRC | 18:03 | |
| *** alexpilotti has joined #openstack-infra | 18:03 | |
| *** pleia2 has joined #openstack-infra | 18:03 | |
| *** svarnau has joined #openstack-infra | 18:04 | |
| *** morganfainberg_Z has joined #openstack-infra | 18:06 | |
| *** AlexF has quit IRC | 18:06 | |
| *** morganfainberg_Z is now known as morganfainberg | 18:06 | |
| *** morganfainberg has quit IRC | 18:07 | |
| *** morganfainberg has joined #openstack-infra | 18:07 | |
| *** AlexF has joined #openstack-infra | 18:10 | |
| *** cadenzajon has joined #openstack-infra | 18:16 | |
| *** julim has joined #openstack-infra | 18:17 | |
| *** gokrokve has joined #openstack-infra | 18:19 | |
| *** ryanpetrello has quit IRC | 18:21 | |
| *** chuck_ has joined #openstack-infra | 18:22 | |
| *** zul has quit IRC | 18:22 | |
| sdake | re oslo.config 1.3.0 in the gate - are there plans to make it available? | 18:24 |
| sdake | it makes oslo.messaging porting work properly, atleast in heat :) | 18:24 |
| zaro | mordred: hey. i totally forgot what we wanted to do with gerrit submodules? | 18:26 |
| *** dprince has joined #openstack-infra | 18:26 | |
| zaro | mordred: they don't work in our forked gerrit openstack/2.8 branch | 18:27 |
| *** jpich has quit IRC | 18:27 | |
| *** morganfainberg is now known as morganfainberg_Z | 18:27 | |
| *** harlowja_away is now known as harlowja | 18:28 | |
| zaro | fungi: there was a gerrit 2.8.1 release 2 days ago so i'm thinking it would be good to synce openstack/2.8 branch with latest gerrit stable-2.8 branch? | 18:30 |
| *** morganfainberg_Z is now known as morganfainberg | 18:32 | |
| fungi | zaro: i'll have to check whether the upstream updates on review.o.o are back to working again | 18:32 |
| *** dkehn__ has joined #openstack-infra | 18:32 | |
| *** davidlenwell_ is now known as davidlenwell | 18:32 | |
| fungi | they were tied in with the manage-project script | 18:32 |
| zaro | fungi: do you remember anything about my qustion to mordred ? about the submodules? | 18:33 |
| jog0 | I just noticed the tp of the gate is waiting for a python26 job | 18:34 |
| openstackgerrit | Diane Fleming proposed a change to openstack-dev/hacking: Add style guide for CLI help strings https://review.openstack.org/73015 | 18:34 |
| zaro | fungi: ok. after you check you can just replace current branch with contents from upstream stabl-2.8. nothing of ours has merged to openstack/2.8 branch yet. | 18:34 |
| *** dkehn_ has quit IRC | 18:35 | |
| *** dkehn has quit IRC | 18:35 | |
| *** rossella_s has joined #openstack-infra | 18:35 | |
| fungi | zaro: okay, and no i don't remember right off the top of my head | 18:36 |
| *** pcrews has joined #openstack-infra | 18:36 | |
| wenlock | hi, im looking for someone that's familliar with zuul and a patch related to a problem im getting that looks like the one here: https://review.openstack.org/#/c/19928/9 basically zuul is not creating the tags on my local repo for zuul to do a proper fetch in the jenkins job. Any idea on how to troubleshoot zuul not creating tags? | 18:37 |
| *** dkehn has joined #openstack-infra | 18:37 | |
| *** dkehn__ is now known as dkehn_ | 18:37 | |
| fungi | wenlock: zuul doesn't create tags. are you talking about the ref objects it creates? | 18:38 |
| wenlock | yeah | 18:38 |
| *** sandywalsh has quit IRC | 18:38 | |
| wenlock | git fetch .... refs/zuul/master/blahblah | 18:38 |
| wenlock | fungi, locally it returns status code 128 | 18:39 |
| wenlock | which is understandable because i don't see anything in the tags folder | 18:39 |
| fungi | wenlock: zuul will only create those refs on the projects which are targeted by any dependent pipeline changes. devstack-gate, for example, tries and then falls back to the branch tip if there is no zuul ref for a particular project | 18:40 |
| fungi | wenlock: also, these are not tags | 18:40 |
| fungi | so they wont be in refs/tags | 18:40 |
| fungi | they're just normal git objects | 18:41 |
| *** lcostantino has quit IRC | 18:41 | |
| *** lcostantino has joined #openstack-infra | 18:42 | |
| *** apevec has quit IRC | 18:42 | |
| *** chadlung has quit IRC | 18:42 | |
| *** rossella_s has quit IRC | 18:42 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: Add jenkins job to build gerrit plugins https://review.openstack.org/70014 | 18:43 |
| *** svarnau has left #openstack-infra | 18:45 | |
| wenlock | fungi, so if i specify no ZUUL_REF in my yaml for scm section whre i git the project... it wouldn't try to use a ref/zuul tag??? just use the tip? | 18:45 |
| fungi | wenlock: there is no tag involved in this | 18:46 |
| fungi | just a git ref | 18:46 |
| fungi | and yes, zuul should create refs in the projects which have changes, but for the others you should just use the branches which that change is targeting if it's not for the same project | 18:47 |
| *** AlexF has quit IRC | 18:47 | |
| fungi | and if the other project has a relevant branch (devstack-gate tries the zuul ref, then the common branch for the change being tested, then the master branch in that order i believe) | 18:47 |
| *** oanufriev has joined #openstack-infra | 18:48 | |
| wenlock | ic | 18:48 |
| fungi | wenlock: if you look at the scripts in the openstack-infra/devstack-gate project you should see the logic we're using there | 18:49 |
| wenlock | fungi, thanks for the pointer... im going to take a good look at that | 18:49 |
| fungi | wenlock: right now devstack-gate does it on a "better to ask forgiveness than permission" model, where it just tries to fetch the ref, then does a fallback if it gets an error | 18:50 |
| *** saju_m has quit IRC | 18:51 | |
| openstackgerrit | A change was merged to openstack-infra/config: Spin up a bare-centos6 slave https://review.openstack.org/72784 | 18:51 |
| fungi | jog0: ^ | 18:52 |
| * fungi will keep tabs on nodepool to make sure we get an image and hopefully a node | 18:52 | |
| openstackgerrit | A change was merged to openstack-infra/reviewday: Make reviewday.json world readable. https://review.openstack.org/72682 | 18:54 |
| *** markmcclain has quit IRC | 18:55 | |
| *** oanufriev has quit IRC | 18:56 | |
| *** markmcclain has joined #openstack-infra | 18:57 | |
| *** chadlung has joined #openstack-infra | 18:58 | |
| *** yolanda has quit IRC | 18:58 | |
| SergeyLukjanov | fungi, sdake http://logs.openstack.org/86/72786/1/check/check-tempest-dsvm-full/21fbccd/logs/screen-dstat.txt.gz | 19:00 |
| SergeyLukjanov | fungi, sdake, so https://review.openstack.org/#/c/72786/ could be merged | 19:00 |
| SergeyLukjanov | oops, sdague | 19:00 |
| *** saju_m has joined #openstack-infra | 19:00 | |
| openstackgerrit | A change was merged to openstack/requirements: Fix happybase version https://review.openstack.org/68435 | 19:01 |
| *** chuck_ is now known as zul | 19:04 | |
| *** zul has quit IRC | 19:04 | |
| *** zul has joined #openstack-infra | 19:04 | |
| *** e0ne has joined #openstack-infra | 19:06 | |
| *** sarob has joined #openstack-infra | 19:07 | |
| openstackgerrit | A change was merged to openstack-infra/reviewday: Use a with block to create index.html. https://review.openstack.org/72975 | 19:09 |
| clarkb | fungi: since we are ewaiting for the cluster to recover (yay this is the slowest most boring part), I was thinking we should switch to pvhvm images for rax | 19:09 |
| pleia2 | reviewdayful morning | 19:09 |
| clarkb | iirc sdague's test time graphs indicated it was an improvement | 19:09 |
| fungi | clarkb: perhaps that could help, agreed | 19:10 |
| clarkb | fungi: not today for the elasticsearch stuff, but in the future (one problem at a time :) ) | 19:10 |
| clarkb | but we can definitely plow ahead on the slave front | 19:11 |
| *** zul has quit IRC | 19:13 | |
| *** sarob has quit IRC | 19:13 | |
| *** zul has joined #openstack-infra | 19:13 | |
| clarkb | fungi: also, can you tl;dr the git-review discussion from -dev early today? was that basically I need to review git-review changes? | 19:13 |
| fungi | clarkb: eh, we moved ahead with the change to collect additional log details and the change to add a timeout fixture | 19:14 |
| clarkb | ++ | 19:14 |
| fungi | i'm looking to see which ones have merged yet | 19:15 |
| pleia2 | dprince: huzzah! http://status.openstack.org/reviews/reviewday.json | 19:15 |
| fungi | right, still trying to get the timeout change merged | 19:15 |
| fungi | oh there's a new one... "java.lang.OutOfMemoryError: unable to create new native thread" | 19:16 |
| fungi | clarkb: http://logs.openstack.org/23/71223/1/gate/gate-git-review-python33/fe77367/console.html | 19:16 |
| clarkb | fungi: I think our test slaves are too small for many concurrent gerrits | 19:17 |
| clarkb | :( | 19:17 |
| fungi | heh | 19:17 |
| fungi | i suppose we could tune that downward some | 19:18 |
| dprince | pleia2: ah so did the permission fix do the trick? | 19:18 |
| clarkb | fungi: concurrency set to 2 maybe? | 19:18 |
| *** lcostantino has quit IRC | 19:18 | |
| clarkb | should default to 4 | 19:18 |
| dprince | pleia2: glad it is working now :) | 19:18 |
| jog0 | fungi: cool | 19:19 |
| fungi | clarkb: perhaps. i want to get this change landed and then do some more rechecks of the loopback network change | 19:19 |
| fungi | jog0: image has been building for 6 minutes so far | 19:19 |
| pleia2 | dprince: yeah, permissions fix | 19:19 |
| jog0 | fungi: neato thanks, you don't have alerting for that? | 19:19 |
| clarkb | fungi: ++ | 19:19 |
| fungi | jog0: alerting for what? | 19:20 |
| *** lcostantino has joined #openstack-infra | 19:20 | |
| dprince | pleia2: cool, I thought that looked suspicious. Thanks for looking into it all... | 19:20 |
| *** olaph has joined #openstack-infra | 19:20 | |
| jog0 | fungi: for when a job in the top patch in the gate has been in queued state for x amount of time | 19:21 |
| boris-42_ | sdague so the question is next Pecan is in globalrequirments | 19:21 |
| boris-42_ | sdague so I am able to add it in my project | 19:21 |
| fungi | jog0: well, we have you ;) | 19:21 |
| boris-42_ | sdague and pecan has own requrimetns https://github.com/stackforge/pecan/blob/master/requirements.txt#L2 | 19:21 |
| jog0 | fungi: I am not scalable | 19:21 |
| fungi | jog0: ultimately, we don't want alerts about these things. we want automation to make whatever we're missing | 19:22 |
| boris-42_ | sdague and if I try to add only Mako to my requirements => I'm getting error | 19:22 |
| jog0 | fungi: fair enough | 19:22 |
| fungi | next change will be to have nodepool create more jog0 template images | 19:22 |
| boris-42_ | sdague e.g. not such requirement in global requirements | 19:22 |
| jog0 | ^_^ | 19:22 |
| sdague | boris-42_: right, so that should probably be added to global requirements | 19:22 |
| boris-42_ | sdague so it is OK if I make patch? | 19:22 |
| pafuent | Hi. I noticed that a lot of review are waiting for python26, someone knows why? | 19:22 |
| sdague | boris-42_: yes, explain the rational in it | 19:23 |
| fungi | pafuent: because 14 centos6 slaves is not enough for our current py26 unit test volume | 19:23 |
| sdague | especially when py26 is 40% slower than py27 | 19:23 |
| fungi | pafuent: we're in the process of bringing changes in which will automatically make more centos6 slaves when we need them rather than reusing the same ones over and over | 19:23 |
| pafuent | fungi: Ahhh | 19:24 |
| clarkb | sdague: is it that pronounced on average? I know it hurts the nuetron tests quite a bit | 19:24 |
| boris-42_ | sdague thanks I will | 19:24 |
| pafuent | fungi: Thanks for the quick response. | 19:24 |
| sdague | clarkb: that's been my experience looking at the unit tests | 19:24 |
| sdague | gate-nova-python26 SUCCESS in 17m 11s | 19:25 |
| sdague | gate-nova-python27 SUCCESS in 12m 19s | 19:25 |
| *** thomasem has joined #openstack-infra | 19:27 | |
| *** yolanda has joined #openstack-infra | 19:28 | |
| *** lcostantino has quit IRC | 19:28 | |
| openstackgerrit | Boris Pavlovic proposed a change to openstack/requirements: Adding Mako>=0.4.0 to global requirments https://review.openstack.org/73025 | 19:29 |
| *** lcostantino has joined #openstack-infra | 19:29 | |
| *** dims has quit IRC | 19:30 | |
| *** VijayT has joined #openstack-infra | 19:30 | |
| *** oanufriev has joined #openstack-infra | 19:33 | |
| *** dims has joined #openstack-infra | 19:34 | |
| jeblair | clarkb, fungi: sorry, back now from appt | 19:35 |
| *** ociuhandu has quit IRC | 19:35 | |
| fungi | jeblair: in short, the logstash/elasticsearch upgrade is nearing completion, and we have a centos6 nodepool image in the process of building | 19:35 |
| clarkb | jeblair: es cluster is recovering replica indexes | 19:36 |
| clarkb | once that is done I will update and start the workers | 19:36 |
| clarkb | recovery is going slower than I hoped :( | 19:37 |
| jeblair | clarkb, fungi: great. my eyes are dialated and i'm using a giant terminal right now to try to read this. | 19:37 |
| fungi | jeblair: you didn't spring for the bionic optical implants yet, i guess | 19:38 |
| * clarkb files that statement away in the "Evidence people from NC are really robots) | 19:38 | |
| *** krtaylor has quit IRC | 19:39 | |
| clarkb | s/\)/"/ folder | 19:39 |
| jeblair | fungi: i'm planning on 3d printing my next eyes. you know they're doing that sort of thing at wake forest. | 19:39 |
| jeblair | clarkb: which is in north carolina | 19:39 |
| clarkb | more evidence! | 19:39 |
| fungi | craziness | 19:39 |
| *** _david_ has joined #openstack-infra | 19:40 | |
| *** oanufriev has quit IRC | 19:42 | |
| clarkb | almost up to 190k in the gearman queue, which isn't terrible considering it was at 130k before we started | 19:44 |
| clarkb | and fingers crossed we catch up relatively quickly | 19:45 |
| fungi | that is one of the goals of this exercise after all | 19:45 |
| fungi | and the backlog should do a good job of putting it through its paces | 19:45 |
| clarkb | indeed | 19:46 |
| *** ociuhandu has joined #openstack-infra | 19:47 | |
| *** Ajaeger1 has joined #openstack-infra | 19:48 | |
| SergeyLukjanov | my highlighter bumps me for the work craz* | 19:48 |
| *** saju_m has quit IRC | 19:48 | |
| jeblair | SergeyLukjanov: this channel must come up a lot then | 19:49 |
| SergeyLukjanov | jeblair, exactly :) | 19:49 |
| *** lcostantino has quit IRC | 19:52 | |
| *** lcostantino has joined #openstack-infra | 19:52 | |
| *** moted has quit IRC | 19:55 | |
| *** smarcet has left #openstack-infra | 19:56 | |
| *** rfolco has quit IRC | 19:57 | |
| *** senthil has joined #openstack-infra | 19:58 | |
| *** markmc has joined #openstack-infra | 19:59 | |
| *** julim has quit IRC | 19:59 | |
| *** zul has quit IRC | 20:03 | |
| *** zul has joined #openstack-infra | 20:03 | |
| *** zul has quit IRC | 20:04 | |
| Ajaeger1 | SergeyLukjanov, jeblair, infra team - here's some work that I love to see in ;) https://review.openstack.org/#/c/70594/ and https://review.openstack.org/#/c/71056/ | 20:08 |
| Ajaeger1 | jeblair: you already +2'ed 71056 and +2'ed an earlier version of 70594... | 20:08 |
| * Ajaeger1 is ready on the tooling site to remove maven for openstack manuals and api sites and have some templates ;) | 20:09 | |
| SergeyLukjanov | Ajaeger1, 71056 approved, looking on the second one | 20:10 |
| Ajaeger1 | SergeyLukjanov: Thanks a lot! | 20:10 |
| *** yolanda has quit IRC | 20:13 | |
| *** hashar has joined #openstack-infra | 20:15 | |
| SergeyLukjanov | Ajaeger1, about 70594 - technically, it looks correct, but I'd like to see +1 from Anne | 20:15 |
| Ajaeger1 | annegentle: could you give a +1 for 70594 again, please? | 20:15 |
| _david_ | jeblair, fungi, clarkb: As of upcoming Gerrit 2.9, WIP plugin can be used without patching Gerrit core. | 20:15 |
| SergeyLukjanov | Ajaeger1, because I don't know how docs building, so, it'll be better to have more eyes on it ;) | 20:15 |
| Ajaeger1 | SergeyLukjanov: sure ;) She reviewed patch 9 and spotted a naming inconsistency, so let's see whether annegentle is happy now | 20:16 |
| Ajaeger1 | SergeyLukjanov: I should know how the docs are building since I reworked it over the last 6 months, so if you have questions, just ask ;) | 20:18 |
| _david_ | the only prerequisite is: to disable DRAFT workflow, what we want anyway. Then DRAFT enum state is unused. Tha way we can "overload" gerrit's DRARFT workflow owth WIP workflow (ab-)using free DRAFT enum state. | 20:18 |
| SergeyLukjanov | Ajaeger1, I think I'll need your help if savanna will graduate from incubation :) | 20:18 |
| _david_ | jeblair, clarkb, fungi: https://gerrit-review.googlesource.com/54583 | 20:19 |
| openstackgerrit | A change was merged to openstack-infra/config: Use openstack-doc-tools for openstack-operations-guide-{lang} https://review.openstack.org/71056 | 20:19 |
| fungi | _david_: neat! | 20:20 |
| fungi | zaro: i've checked and the upstream/stable-2.8 branch is getting updated, so i've synchronized the openstack/2.8 branch to it | 20:21 |
| _david_ | fungi, the target is still to get WIP workflow be part of core. The both changes: overloading of DRAFT workflow or make it in addition to DRAFT are still pending, ... | 20:21 |
| fungi | looks to include the 2.8.1 point release tag | 20:21 |
| Ajaeger1 | SergeyLukjanov: if there's docu to build, just ask and I can help... | 20:21 |
| SergeyLukjanov | Ajaeger1, thanks! | 20:22 |
| jog0 | fungi: same thing again with py26 in gate | 20:24 |
| jog0 | oh my zuul page jsut updated | 20:25 |
| fungi | jog0: yep, it's going to keep happening until we either test less stuff or add more centos6 slaves | 20:25 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Import oslo.test repository https://review.openstack.org/73036 | 20:26 |
| jog0 | fungi: sigh | 20:27 |
| Ajaeger1 | SergeyLukjanov: annegentle gave her +1 for https://review.openstack.org/#/c/70594 | 20:28 |
| fungi | (or stop testing python 2.6) | 20:28 |
| jeblair | jog0: there are changes in process to dynamically spin up centos6 slaves | 20:28 |
| jog0 | fungi: \o/ | 20:28 |
| jog0 | jeblair: neato | 20:28 |
| fungi | jog0: jeblair: 1.25 hours so far into the image buil;d | 20:28 |
| jeblair | yeesh, that's not fast. | 20:28 |
| fungi | though the last entry from it in the log is 40 minutes old... | 20:29 |
| fungi | 2014-02-12 19:50:06,911 INFO nodepool.image.build.rax-iad.bare-centos6: Running: ['git', 'clone', 'https://git.openstack.org/openstack/neutron', 'openstack/neutron'] | 20:29 |
| fungi | i'll see if i can ssh in and look around | 20:29 |
| jeblair | fungi: i don't recall what the buffering is like on that | 20:30 |
| jeblair | fungi: i don't _think_ it's buffered... | 20:30 |
| jeblair | we could also set the gate to high priority | 20:31 |
| fungi | root 16730 0.0 0.1 92784 9680 pts/0 S+ 19:50 0:00 git-remote-https https://git.openstack.org/openstack/neutron https://git.openstack.org/openstack/neutron | 20:32 |
| * fungi sighs | 20:32 | |
| *** eharney has joined #openstack-infra | 20:32 | |
| jeblair | hrm | 20:33 |
| fungi | the most recent child of that is a git fetch-pack started at 19:55 | 20:34 |
| jeblair | fungi: oh, is it because we're cloning via https on centos6? | 20:34 |
| fungi | jeblair: could be. i'm stracing it now to see if it's doing anything at all. seems deadlocked | 20:35 |
| jeblair | oh; deadlock doesn't so much fit with that | 20:35 |
| openstackgerrit | A change was merged to openstack-infra/reviewday: Ensure non-WIP reviews have a score of 1 as a base https://review.openstack.org/72815 | 20:35 |
| fungi | yeah, i'd expect to see it trickling, but no syscalls for at least the past minute | 20:35 |
| fungi | looking on the git.o.o haproxy to see if it's still open for this host | 20:36 |
| pafuent | Is there a way to know if a screen ran correctly if no logs for those screens were uploaded in the job results? | 20:37 |
| mika | jeblair: hi! we've met a fosdem regarding my jjb issue, as promised I sent a mail to the infra mailing list -> http://lists.openstack.org/pipermail/openstack-infra/2014-February/000799.html - if you have any idea i'd very much appreciate it :) | 20:42 |
| boris-42_ | sdague done, seems look like it works https://review.openstack.org/#/c/73025/ =) | 20:42 |
| *** krtaylor has joined #openstack-infra | 20:43 | |
| *** markmcclain has quit IRC | 20:43 | |
| esker | Is there a protocol for requesting a review for a minor change in openstack-infra? Or should one patiently await? | 20:43 |
| esker | https://review.openstack.org/#/c/72872/ | 20:43 |
| fungi | jeblair: haproxy.log says that last request was balanced to git03, but doesn't look like there's an open socket for it nor anything in the access/error logs | 20:43 |
| fungi | jeblair: i'm going to delete the image and let it retry | 20:43 |
| hashar | fungi: I had some issues when cloning huge git repositories per http. I think it was a bug somewhere in git-http-backend, I never got it fixed. | 20:44 |
| hashar | fungi: switched to git-daemon and git:// urls | 20:44 |
| fungi | hashar: yeah, but with clients on centos6 (it uses an old git) i believe we're stuck doing that for now | 20:44 |
| boris-42_ | sdague thank you | 20:44 |
| fungi | hashar: or maybe it was the other way around. i could be misremembering | 20:44 |
| hashar | fungi: if you only care about a specific ref, you could use git archive , might speed things up | 20:44 |
| fungi | jeblair: clarkb: were we surmising git:// might work better on centos6's old git client rev? | 20:45 |
| *** smarcet has joined #openstack-infra | 20:45 | |
| fungi | hashar: in this case we're caching all our projects onto template server images | 20:45 |
| fungi | hashar: and we want whole repositories for each | 20:45 |
| hashar | so you can checkout different branches if need be | 20:46 |
| hashar | make sense | 20:46 |
| jeblair | fungi: stupid internet. yeah, i think we should change the nodepool scripts to use git:// | 20:46 |
| jeblair | hashar: this is also so that we don't have to clone whole repos at the start of jobs, and there's very little to fetch | 20:47 |
| fungi | jeblair: patch coming then | 20:47 |
| *** DinaBelova is now known as DinaBelova_ | 20:47 | |
| jeblair | hashar: basically jobs should only need to fetch whatever has happened to the repo in the last day, plus the change they want to test. | 20:47 |
| dhellmann | I have a quick question about how openstackci works with pypi | 20:47 |
| hashar | my slaves are bare metal, so I got Gerrit to replicate to a directory which is on the same disk as the workspace. I then use git clone git://zuul/<project> --reference /localdisk/<project> | 20:48 |
| dhellmann | When a new project is added, and starts publishing releases to pypi, do we need to manually register it first, or will it be registered the first time anything is published? | 20:48 |
| *** harlowja is now known as harlowja_away | 20:48 | |
| hashar | jeblair: yeah that is ideal | 20:48 |
| jeblair | hashar: that'll work. :) | 20:48 |
| hashar | jeblair: until the Jenkins Git Plugin manage to break --reference, yeah that works for me :-D | 20:48 |
| fungi | dhellmann: manually register it on pypi | 20:49 |
| dhellmann | fungi: ok | 20:49 |
| fungi | dhellmann: well, not register register | 20:49 |
| dhellmann | hmm? | 20:49 |
| jeblair | dhellmann: hrm, i always register first to squat the name; i technically don't know the answer | 20:49 |
| dhellmann | jeblair: I've been doing that, but wasn't sure I needed to | 20:49 |
| dhellmann | jeblair: and since I'm trying to write directions for others, who may not have pypi credentials, I wanted to skip that step if I could | 20:49 |
| fungi | dhellmann: i just create the project with openstackci as an index maintainer/owner and set the version to 0, leaving all the optional fields blank | 20:49 |
| dhellmann | ok | 20:50 |
| jeblair | twine upload -r pypi $FILENAME | 20:50 |
| fungi | dhellmann: the upload tooling takes care of teh rest | 20:50 |
| jeblair | that's what happens, apparently | 20:50 |
| dhellmann | how do you feel about me leaving oslo.test unregistered to see if that works? | 20:50 |
| * dhellmann doesn't know twine | 20:50 | |
| jeblair | me neither. that's not what that script did the last time i looked. :) | 20:50 |
| fungi | dhellmann: twine is a static pypi uploader dstufft wrote a few months ago after talking to us | 20:50 |
| dhellmann | ah | 20:50 |
| dhellmann | makes sense to have it, I just hadn't seen it before | 20:51 |
| jeblair | dstufft: will twine register a project on first upload? | 20:51 |
| dstufft | jeblair: no because I'm a bad person and forgot to do it | 20:51 |
| dhellmann | ok, I'll include the steps to manually register it | 20:51 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1277790 https://review.openstack.org/73055 | 20:51 |
| uvirtbot | Launchpad bug 1277790 in nova/havana "boto 2.25 causing unit tests to fail" [Critical,Fix committed] https://launchpad.net/bugs/1277790 | 20:51 |
| jeblair | dstufft: not too late for redemption. ;) thanks. | 20:52 |
| *** julim has joined #openstack-infra | 20:52 | |
| dhellmann | jeblair, fungi, dstufft : thanks! | 20:52 |
| *** _david_ has quit IRC | 20:53 | |
| *** arborism has joined #openstack-infra | 20:54 | |
| *** boris-42_ has quit IRC | 20:55 | |
| *** dripton has quit IRC | 20:55 | |
| *** arborism is now known as amcnr | 20:57 | |
| *** amcnr is now known as amcrn | 20:57 | |
| dhellmann | jeblair, fungi, dstufft : if you could double-check https://wiki.openstack.org/wiki/Oslo/CreatingANewLibrary#Give_OpenStack_Permission_to_Publish_Releases I would really appreciate it | 20:58 |
| *** jamielennox|away is now known as jamielennox | 20:58 | |
| *** dripton has joined #openstack-infra | 20:58 | |
| *** SpamapS_ is now known as SpamapS | 20:58 | |
| dstufft | dhellmann: I recommend using the web ui to register not the setup.py interface | 20:58 |
| *** SpamapS has quit IRC | 20:58 | |
| *** SpamapS has joined #openstack-infra | 20:58 | |
| dhellmann | dstufft: any particular reason? | 20:58 |
| dstufft | dhellmann: setup.py register|upload doesn't validate TLS | 20:59 |
| dstufft | so do it at OpenStack conference and anyone on the same WIFI as you could have just stolen your pasword :[ | 20:59 |
| dhellmann | dstufft : ok, I'll update that after my next meeting, thanks | 20:59 |
| *** boris-42_ has joined #openstack-infra | 20:59 | |
| *** zul has joined #openstack-infra | 20:59 | |
| *** hashar has quit IRC | 21:00 | |
| *** thomasem has quit IRC | 21:01 | |
| fungi | similar reasons led to the existence of twine for doing uploads | 21:01 |
| dstufft | yea | 21:02 |
| dstufft | I should really just add register to twine | 21:02 |
| dstufft | maybe i'll do that today | 21:02 |
| fungi | we would definitely make use of that feature | 21:03 |
| *** hashar has joined #openstack-infra | 21:03 | |
| dhellmann | dstufft: I would love to be able to use that, I didn't realize setuptools wasn't safe | 21:04 |
| dhellmann | Can anyone shed light on this error? I have no idea what it means? http://logs.openstack.org/36/73036/1/check/gate-config-layout/ebd58b5/console.html | 21:04 |
| dstufft | dhellmann: someday I'm going to break distutils/setuptools so that setup.py register|upload doesn't work | 21:05 |
| dhellmann | trying to add an oslo.test repo using instructions SergeyLukjanov and I worked out the other day, and I don't know if I have a typo or misunderstood, or what | 21:05 |
| dstufft | and I can just force everyone to use twine :V | 21:05 |
| dstufft | maybe called pip upload at that point | 21:05 |
| dhellmann | dstufft: let me know so I can make popcorn | 21:05 |
| dstufft | haven't decided on that yet | 21:05 |
| dstufft | dhellmann: yea I'm gonna get a lot of hate for that I'm sure :( | 21:05 |
| dhellmann | well, I support the idea | 21:05 |
| *** jp_at_hp1 has quit IRC | 21:06 | |
| jeblair | dhellmann: you need a 'doc-publisher-site' parameter; it should be 'docs.openstack.org'. | 21:06 |
| jeblair | dhellmann: (in projects.yaml) | 21:06 |
| dhellmann | dstufft: maybe you could update setuptools to start warning and promoting the use of twine, without breaking | 21:06 |
| dhellmann | jeblair: thanks! | 21:06 |
| SergeyLukjanov | dhellmann, excuse me, missed that on wiki :( | 21:06 |
| jaypipes | fungi, jeblair: so I'm making some progress getting jenkins slave up in LXC, but ran into an issue with cgroups-bin failing to install (because the LXC container is used a shared kernel, I presume). Do you think it would be safe for me to *not* include the jenkins::cgroups module from openstack-infra/config? | 21:07 |
| *** lcostantino has quit IRC | 21:07 | |
| dhellmann | SergeyLukjanov: I just looked and I see it, did you add it? :-) | 21:07 |
| jeblair | jaypipes: yes | 21:07 |
| SergeyLukjanov | dhellmann, nope, that's not me | 21:07 |
| *** lcostantino has joined #openstack-infra | 21:07 | |
| jaypipes | jeblair: cool, thx :) | 21:08 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Import oslo.test repository https://review.openstack.org/73036 | 21:08 |
| SergeyLukjanov | dhellmann, probably it was done by jeblair or I've added it with the jobs :) | 21:08 |
| jeblair | not i | 21:08 |
| fungi | jaypipes: we mainly use it to enforce memory limits on jobs | 21:09 |
| dhellmann | SergeyLukjanov: ok, it may have been a copy-paste error on my part | 21:09 |
| *** denis_makogon_ has joined #openstack-infra | 21:09 | |
| jaypipes | fungi: right, understood. I just wanted to double-check that there wasn't something really nasty I would run into :) | 21:09 |
| fungi | jaypipes: runaway jobs which use all the ram on the machine ;) | 21:09 |
| SergeyLukjanov | dhellmann, you've added doc-publisher-site to oslo.rootwrap, not to the oslo.test ;) | 21:10 |
| clarkb | back from lunch catching up on sb | 21:10 |
| dhellmann | SergeyLukjanov: bah | 21:10 |
| *** oubiwann_ has quit IRC | 21:10 | |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Import oslo.test repository https://review.openstack.org/73036 | 21:11 |
| clarkb | twine++ is a great tool | 21:13 |
| *** pafuent has left #openstack-infra | 21:13 | |
| *** derekh has joined #openstack-infra | 21:13 | |
| *** tomhe has joined #openstack-infra | 21:14 | |
| fungi | and about to maybe get even better | 21:15 |
| SergeyLukjanov | dhellmann, hm, having oslo.test-tarball in post means that you'll have a separated tarball (with a uniquq generated by pbr version) for each merged commit | 21:16 |
| chadlung | I'm working on the gate job required for Barbican's entry as an incubator project as per the TC meeting yesterday. The DevStack work is complete and can be seen here: https://review.openstack.org/#/c/70512/ | 21:17 |
| chadlung | The work I believe I need to do to accomplish this can be seen here: https://gist.github.com/chadlung/8964037 I haven't done this before so any feedback on whether these are the correct assumptions/steps is appreciated. | 21:17 |
| SergeyLukjanov | dhellmann, I'm not sure that you want iy | 21:17 |
| dhellmann | SergeyLukjanov: no, probably not | 21:17 |
| dhellmann | SergeyLukjanov: I think I copied that from another lib | 21:17 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Only allow mail from jenkins from zuul https://review.openstack.org/73067 | 21:17 |
| jeblair | that should take care of the recent forgeries ^ | 21:18 |
| SergeyLukjanov | dhellmann, hm, it means tons of tarballs at tarballs.o.o | 21:18 |
| dhellmann | SergeyLukjanov: yeah, I'll remove it | 21:18 |
| dhellmann | I think I copied oslo.rootwrap, it has oslo.rootwrap-branch-tarball but I remembered you said not to use that job | 21:18 |
| dhellmann | so I don't want any tarball jobs at all in post? | 21:18 |
| SergeyLukjanov | dhellmann, the python-jobs template already contain oslo.rootwrap-branch-tarball in post | 21:19 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Import oslo.test repository https://review.openstack.org/73036 | 21:19 |
| dhellmann | SergeyLukjanov: aha, ok | 21:19 |
| jeblair | -branch-tarball should be in post; -tarball should be in the release-related pipelines | 21:19 |
| *** masayukig has quit IRC | 21:20 | |
| dhellmann | jeblair: like this? https://wiki.openstack.org/wiki/Oslo/CreatingANewLibrary#Configure_Zuul_to_Run_Jobs | 21:20 |
| SergeyLukjanov | dhellmann, + python-jobs template adds coverage to post too | 21:20 |
| SergeyLukjanov | dhellmann, btw, here is a template definition - https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/zuul/layout.yaml#L152 | 21:21 |
| *** hashar has quit IRC | 21:21 | |
| dhellmann | SergeyLukjanov: that's helpful, I wasn't sure where to find that before | 21:21 |
| jeblair | that's going to keep getting smaller too as we template more stuff | 21:21 |
| *** masayukig has joined #openstack-infra | 21:21 | |
| *** senthil has quit IRC | 21:21 | |
| dhellmann | the last time I did this, I don't think there were templates in the zuul layout | 21:21 |
| dhellmann | does a project have to do anything special to make that coverage job work? | 21:22 |
| dhellmann | do we need anything in our test-requirements.txt or tox.ini for example? | 21:22 |
| SergeyLukjanov | dhellmann, it should have tox -e cover | 21:22 |
| *** denis_makogon_ is now known as denis_makogon | 21:22 | |
| dhellmann | SergeyLukjanov: ok | 21:23 |
| dhellmann | that's in the cookiecutter template, so we should be ok | 21:23 |
| SergeyLukjanov | dhellmann, could be useful - https://github.com/openstack/savanna/blob/master/.coveragerc https://github.com/openstack/savanna/blob/master/test-requirements.txt#L4 https://github.com/openstack/savanna/blob/master/tox.ini#L24 | 21:23 |
| SergeyLukjanov | dhellmann, great | 21:23 |
| SergeyLukjanov | dhellmann, so, only oslo.test-tarball in post, everything else looks ok | 21:24 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack-infra/reviewstats: Update the Ironic core team list https://review.openstack.org/73068 | 21:24 |
| *** mriedem has quit IRC | 21:24 | |
| dhellmann | SergeyLukjanov: the latest patch removes the post job | 21:24 |
| SergeyLukjanov | dhellmann, wiki page is uptodate too | 21:25 |
| dhellmann | SergeyLukjanov: patch 4 | 21:25 |
| dhellmann | SergeyLukjanov: yep, I think this was another case of me copying from the wrong place and not catching a difference | 21:25 |
| *** markwash has quit IRC | 21:25 | |
| SergeyLukjanov | dhellmann, patch 4 is lgtm | 21:25 |
| *** dprince has quit IRC | 21:26 | |
| * dhellmann waits eagerly for jenkins' loving praise | 21:26 | |
| *** masayukig has quit IRC | 21:26 | |
| SergeyLukjanov | dhellmann, it's just a test change, am I right? :) | 21:26 |
| jeblair | fungi: were you going to propose either the git or high-priority changes? | 21:27 |
| *** oubiwann_ has joined #openstack-infra | 21:27 | |
| jeblair | fungi: also, what's the centos6 image status? | 21:27 |
| dhellmann | SergeyLukjanov: we're moving some test base classes and fixture code out of the incubator into a new library | 21:27 |
| fungi | jeblair: working on the git change now. image rebuild is going, in the middle of cloning repos right now | 21:27 |
| SergeyLukjanov | dhellmann, oops ;) | 21:28 |
| jeblair | fungi: cool, i'll propose the priority change | 21:28 |
| clarkb | fungi: that is one of the last things it does iirc | 21:28 |
| jeblair | clarkb: but it got stuck there last run | 21:28 |
| clarkb | ah :/ | 21:28 |
| *** markmc has quit IRC | 21:29 | |
| jeblair | clarkb: what's the e-s status? (gearman queue is increasing) | 21:29 |
| *** oubiwan__ has joined #openstack-infra | 21:29 | |
| clarkb | jeblair: recovering slowly (more slowly than I had hoped, last time was much quicker, could be due to index sizes), we have recovered 99 shards out of 140 | 21:30 |
| clarkb | 12 are in flight concurrently with 29 idle | 21:30 |
| clarkb | jeblair: I am watching the gearman queue and memory use there, plan to kill puppet daemon to free up memory if necessary | 21:30 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Change gate queue to normal precedence https://review.openstack.org/73071 | 21:31 |
| *** oubiwann_ has quit IRC | 21:31 | |
| jeblair | ack | 21:32 |
| SergeyLukjanov | dhellmann, it works ;) | 21:32 |
| clarkb | now up to 100 active shards :) so it is moving | 21:32 |
| jeblair | fungi, clarkb: https://review.openstack.org/73071 should reduce the py26 problem in the gate | 21:32 |
| dhellmann | SergeyLukjanov: woo! | 21:32 |
| SergeyLukjanov | jeblair, agreed | 21:33 |
| clarkb | jeblair: has my +2 | 21:33 |
| *** lcostantino has quit IRC | 21:33 | |
| SergeyLukjanov | jeblair, fwiw my +2 too :) | 21:33 |
| jeblair | well, it has my +A :) | 21:34 |
| *** e0ne has quit IRC | 21:34 | |
| SergeyLukjanov | jeblair (y) | 21:35 |
| fungi | jeblair: i think we don't need the git clones in prepare_devstack.sh any longer now that we're cloning everything anyway, right? | 21:35 |
| fungi | those were the ones going into workspace-cache | 21:35 |
| *** Ajaeger1 has quit IRC | 21:36 | |
| jeblair | fungi: i am not certain of the current status of that move | 21:36 |
| fungi | okay, leaving those untouched for now | 21:36 |
| SergeyLukjanov | fungi, IIRC devstack-gate clones everything that it needs | 21:36 |
| jeblair | SergeyLukjanov: actually we want devstack-gate to never have to clone anything, unless we are adding a new project | 21:37 |
| jeblair | SergeyLukjanov: so the nodepool scripts which fungi is working on clone repos when nodepool builds the images that will get used to run the jobs | 21:37 |
| jeblair | SergeyLukjanov: that way when nodepool spins up a host, the git repos are already there | 21:38 |
| jeblair | SergeyLukjanov: if we cloned everything every time, we would need many, many more git servers. :) | 21:38 |
| SergeyLukjanov | jeblair, got it | 21:38 |
| SergeyLukjanov | heh, it's 1:40am for me, so, time to generate some crazy idea | 21:39 |
| openstackgerrit | A change was merged to openstack-infra/config: Change gate queue to normal precedence https://review.openstack.org/73071 | 21:39 |
| SergeyLukjanov | maybe clone all openstack repos | 21:39 |
| SergeyLukjanov | nope, it's too crazy | 21:39 |
| fungi | SergeyLukjanov: we do | 21:39 |
| fungi | because we're crazy | 21:40 |
| SergeyLukjanov | I mean fetch the list of repos automatically and clone them :) | 21:40 |
| SergeyLukjanov | and stackforge | 21:40 |
| fungi | yep, that's what the nodepool prep scripts do | 21:40 |
| jeblair | SergeyLukjanov: that's actually what the replacement does... | 21:40 |
| SergeyLukjanov | and github.com/Frostman too ;) | 21:40 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Clone git repos for images via git protocol https://review.openstack.org/73073 | 21:40 |
| fungi | i think that's ^ all we need to adjust the git clone protocol for nodepool image creation | 21:40 |
| SergeyLukjanov | oops :) | 21:40 |
| SergeyLukjanov | it's crazy enough to help us | 21:41 |
| fungi | prepare_devstack.sh is still cloning from gerrit rather than git.o.o, so i didn't touch it since i think that's going away anyway | 21:42 |
| jeblair | fungi: we should probably change it since it's still cloning... | 21:42 |
| jeblair | fungi: or, let me check and see if we can get rid of it | 21:42 |
| SergeyLukjanov | fungi, heh, that's really funny for me to generate an idea that's already used :) | 21:42 |
| jeblair | SergeyLukjanov: it makes me happy | 21:43 |
| *** andre__ has quit IRC | 21:43 | |
| fungi | also the current bare-centos6 image build (second try) is still going. a little over 50 minutes in now | 21:43 |
| fungi | still cloning repos | 21:43 |
| fungi | but it's doing things, not hung like the last one, at least | 21:44 |
| jeblair | fungi: it is safe to remove workspace-cache from the nodepool scripts now | 21:44 |
| fungi | jeblair: i'll add that as a second change | 21:44 |
| fungi | coming now | 21:44 |
| clarkb | jeblair: but maybe not for the tripleo scripts | 21:44 |
| clarkb | not sure what they are doing | 21:44 |
| jeblair | clarkb: we're not touching the tripleo scripts | 21:45 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/config: Add the kite (KDS) project and kiteclient project to gerrit/jenkins https://review.openstack.org/73074 | 21:45 |
| *** dkehn__ has joined #openstack-infra | 21:46 | |
| *** andre__ has joined #openstack-infra | 21:46 | |
| *** dkehn__ has quit IRC | 21:48 | |
| jaypipes | *sigh* I give up trying to configure a jenkins slave manually... :( guess I'll install nodepool and have it manage this stuff. | 21:48 |
| *** dkehn__ has joined #openstack-infra | 21:48 | |
| *** dkehn has quit IRC | 21:48 | |
| clarkb | jaypipes: running the nodepool script by hand didn't do it for you? | 21:48 |
| jaypipes | clarkb: ? sorry, which script? | 21:49 |
| *** dkehn_ has quit IRC | 21:49 | |
| jeblair | well, there's install_jenkins_slave.sh | 21:49 |
| clarkb | either that one ^ which gives you something useable but missing things like ntp management and so on, or the scripts in modules/openstack_project/files/nodepool/scripts | 21:50 |
| *** dkehn has joined #openstack-infra | 21:50 | |
| *** jhesketh_ has joined #openstack-infra | 21:52 | |
| *** jhesketh__ has joined #openstack-infra | 21:52 | |
| jaypipes | jeblair, clarkb: no, that didn't do it. I had copied the relevant pieces of that script to here: https://github.com/jaypipes/os-ext-testing/blob/master/puppet/install_slave.sh | 21:52 |
| jhesketh__ | Morning | 21:52 |
| jeblair | jhesketh__: good morning; i'm reading your uploader change now. or trying to, since my eyes are dialated after visiting the optometrist. | 21:52 |
| jhesketh__ | ah cool, cheers | 21:53 |
| jaypipes | clarkb: and then I just manually added a new node in Jenkins web UI, selected the "Launch slave agents on Unix via SSH" option, entered my slave LXC container's IP address and added a new credentials with the private SSH key file from the slave. | 21:55 |
| jaypipes | clarkb: but unfortunately, the master fails to communicate with the slave node, with this in the logs: http://paste.openstack.org/show/64787/ | 21:57 |
| jaypipes | clarkb: which I'm confused about since puppet apply on the slave node clearly says that the jenkins key was added to authorized_keys... | 21:57 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Stop separately caching repos for devstack images https://review.openstack.org/73076 | 21:57 |
| fungi | oh, that ^ was too aggressive | 22:00 |
| fungi | fixing | 22:00 |
| jeblair | fungi: yep :) | 22:00 |
| jeblair | ~/cache is still used | 22:00 |
| fungi | well, i just need to rip the git bits out of cache_devstack.py rather than rm the entire script | 22:00 |
| jeblair | yep. | 22:00 |
| jeblair | fungi: it looks like it does some operations in ~/workspace-cache/devstack; that will probably need to be changed to /opt/whatever/devstack | 22:01 |
| *** pleia2 has quit IRC | 22:01 | |
| fungi | oh, we still need to do things to the git repos there too? hrm... | 22:02 |
| jeblair | that script inspects the contents of various branches to decide what packages and images to cache | 22:02 |
| fungi | except we aren't using workspace-cache any longer, so whatever it was doing in there isn't currently being used anyway | 22:02 |
| fungi | ahh, got iy | 22:02 |
| fungi | it | 22:02 |
| fungi | read-only operations | 22:03 |
| jeblair | fungi: yep | 22:03 |
| openstackgerrit | Dolph Mathews proposed a change to openstack-infra/elastic-recheck: eliminate false positives for bug 1253482 https://review.openstack.org/69997 | 22:03 |
| uvirtbot | Launchpad bug 1253482 in devstack "Keystone's IANA-assigned default port in linux local ephemeral port range" [Undecided,In progress] https://launchpad.net/bugs/1253482 | 22:03 |
| jeblair | well, in principle at least. it probably does leave it checked out on some random branch. | 22:03 |
| jeblair | but that's not the purpose | 22:03 |
| fungi | right | 22:03 |
| *** pdmars has quit IRC | 22:04 | |
| *** morganfainberg has quit IRC | 22:05 | |
| *** pleia2 has joined #openstack-infra | 22:06 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Stop separately caching repos for devstack images https://review.openstack.org/73076 | 22:07 |
| *** morganfainberg has joined #openstack-infra | 22:07 | |
| *** gokrokve has quit IRC | 22:07 | |
| *** ryanpetrello has joined #openstack-infra | 22:07 | |
| jeblair | jhesketh__, clarkb: do you think that the swift uploader should gzip files, or should we instead try to get all of the jobs gzipping before we invoke it? | 22:07 |
| *** gokrokve has joined #openstack-infra | 22:08 | |
| clarkb | jeblair: imo, gzip by job. uploader should just move bits not transform them | 22:08 |
| jeblair | (i kind of think the latter, so that it doesn't have to worry too much about content specifics, but i'm also kind of on the fence) | 22:08 |
| *** gokrokve_ has joined #openstack-infra | 22:08 | |
| jhesketh__ | I agree | 22:08 |
| jeblair | cool | 22:08 |
| jhesketh__ | it's really just another builder step to package up whatever it needs | 22:08 |
| jhesketh__ | and that lets the job define how it wants its assets pushed | 22:09 |
| jeblair | for that matter, we can also do, erm, whatever it is we decide to do about console logs (wget $SOMEURL is still my proposal) right before invoking the uploader as well. | 22:09 |
| zaro | fungi: thanks for updating gerrit branch. | 22:10 |
| * fungi goes semi-afk to cook dinner while there is still electricity to be had | 22:10 | |
| jeblair | fungi: enjoy | 22:10 |
| *** gokrokve has quit IRC | 22:12 | |
| *** mfer has quit IRC | 22:13 | |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/config: Add the kite (KDS) project and kiteclient project to gerrit/jenkins https://review.openstack.org/73074 | 22:14 |
| *** markmcclain has joined #openstack-infra | 22:14 | |
| zaro | fungi: when you get back could you please help me get the gerrit submodules working? | 22:23 |
| openstackgerrit | A change was merged to openstack-infra/config: Publish draft docs on draft-docs https://review.openstack.org/70594 | 22:23 |
| *** ArxCruz has quit IRC | 22:24 | |
| fungi | zaro: i don't know what you mean by the gerrit submodules. do you mean submodule tracking we do for the openstack/openstack project, or do you mean submodules the gerrit project uses for their plugin repositories? | 22:24 |
| *** ryanpetrello has quit IRC | 22:24 | |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add link to Third-Party CI group in Gerrit docs https://review.openstack.org/73084 | 22:25 |
| fungi | if it's the latter, the problem is that gerrit has hardcoded the paths in .gitmodules to be relative to the current repository rather than absolute, so we probably need to work upstream to get them to fix that | 22:25 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add link to Third-Party CI group in Gerrit docs https://review.openstack.org/73084 | 22:26 |
| pleia2 | tsk, trailing spaces | 22:26 |
| *** zul has quit IRC | 22:26 | |
| jeblair | pleia2: don't want to waste spaces | 22:26 |
| *** smarcet has left #openstack-infra | 22:28 | |
| jeblair | mattoliverau: does the apache vhost in https://review.openstack.org/#/c/71190/9/modules/nodepool/manifests/init.pp need a custom template, or can it use the puppet module default? | 22:28 |
| zaro | fungi: the latter, can we make that change in our fork? | 22:29 |
| *** jnoller has quit IRC | 22:31 | |
| fungi | zaro: i expect so. just patch that file in the main repository to refer to the global urls to those submodules and i expect we'll be able to clone/update them fine | 22:31 |
| zaro | fungi: cool. i do up that change. | 22:31 |
| chadlung | Does anyone know where the latest docs are on bringing a project (Barbican) into infra? I have the DevStack work already done: https://review.openstack.org/#/c/70512/ I just need to get the required steps for infra now. This is what I have so far, but not sure that this is correct: https://gist.github.com/chadlung/8964037 | 22:32 |
| chadlung | ...and Barbican does already have active gate jobs | 22:32 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Add link to Third-Party CI group in Gerrit docs https://review.openstack.org/73084 | 22:34 |
| chadlung | I was looking at Marconi's PR https://git.openstack.org/cgit/openstack-infra/config/commit/modules/openstack_project/files/zuul/layout.yaml?id=f5a77e4b6fdee094f312e3ab20b4bdf982a7111d but it looks like Solum is doing it in a more preferred way... | 22:34 |
| jeblair | chadlung: will it run tempest? | 22:40 |
| jeblair | chadlung: (what you wrote will; i mean to ask do you want it to?) | 22:40 |
| chadlung | jeblair: We don't have Tempest tests yet, but those will be added by our QA. The tempest stuff would be a placeholder for now. I could leave it out if it causes issues. | 22:41 |
| jeblair | oh, actually i see that the GATE_WHATEVER_TEMPEST var is not set so it shouldn't run it | 22:42 |
| jeblair | chadlung: lgtm | 22:43 |
| *** shardy is now known as shardy_afk | 22:43 | |
| clarkb | up to 111 shards recovered, 12 in process, and 17 idle | 22:44 |
| chadlung | jeblair: so those two files in the GIST, and then the modifications to the Barbican repo to add functionaltests like solum | 22:44 |
| *** dcramer_ has quit IRC | 22:44 | |
| jeblair | chadlung: yep. then remove the non-voting bit once it's working. | 22:45 |
| *** jooools1 has quit IRC | 22:45 | |
| chadlung | jeblair: great, thanks | 22:45 |
| *** e0ne has joined #openstack-infra | 22:46 | |
| *** e0ne has quit IRC | 22:48 | |
| *** bhuvan has quit IRC | 22:49 | |
| *** mfer has joined #openstack-infra | 22:49 | |
| *** bhuvan has joined #openstack-infra | 22:50 | |
| morganfainberg | pleia2, woot json for reviewday a nice scoring for WIP! thanks for accepting those! :) | 22:51 |
| openstackgerrit | A change was merged to openstack-infra/config: Only allow mail from jenkins from zuul https://review.openstack.org/73067 | 22:51 |
| pleia2 | morganfainberg: sure thing, happy to see it working now :) if we're happy with how the json format ended up next step can be adding a link for it on the index.html so it's discoverable | 22:51 |
| morganfainberg | pleia2, i'm going to look over the json shortly also going to add to dolphm's next-review to allow using it instead of gerrit :) | 22:52 |
| *** CaptTofu has quit IRC | 22:52 | |
| pleia2 | morganfainberg: awesome, thank you | 22:52 |
| morganfainberg | pleia2 sure thing! :) | 22:52 |
| *** banix has quit IRC | 22:53 | |
| *** harlowja_away is now known as harlowja | 22:54 | |
| *** oubiwan__ has quit IRC | 22:55 | |
| *** VijayT has quit IRC | 22:56 | |
| *** dims has quit IRC | 22:57 | |
| *** mfer has quit IRC | 22:59 | |
| clarkb | hrm, I am starting to get a bit worried that the replica shard recovery may be too slow | 23:00 |
| clarkb | jeblair: fungi: what do you think about turning on workers now? if leaves us open to data loss today, but we are going to do that anyways if logstash.o.o OOMs | 23:01 |
| fungi | clarkb: it's almost done though, right? | 23:02 |
| jeblair | clarkb: logstash.o.o still has 200k of ram and has barely dipped into swap | 23:02 |
| *** ryanpetrello has joined #openstack-infra | 23:02 | |
| jeblair | 200m even | 23:03 |
| clarkb | fungi: ish, 112 out of 140 done | 23:03 |
| clarkb | these recent indexes are fairly large so it is taking time | 23:03 |
| fungi | oh, it's sharded/processed chronologically? | 23:04 |
| clarkb | sort of, it does chrono best it can but sometimes skips ahead to meet replica constraints | 23:05 |
| clarkb | right now it will only recover two shards per node at a time and not all shards can go on all nodes | 23:05 |
| *** rwsu has quit IRC | 23:06 | |
| clarkb | well I will let it be for now, keeping a watch on it though | 23:06 |
| *** rwsu has joined #openstack-infra | 23:08 | |
| fungi | btw, we've had a ready bare-centos6 node in rax-iad since shortly after i headed into the kitchen | 23:08 |
| jeblair | fungi: oh yay! | 23:08 |
| *** bhuvan has quit IRC | 23:08 | |
| * fungi does a quick jjb config patch to make these optional | 23:08 | |
| jeblair | i will exercise it manually a bit to sanity check it while you do that fungi | 23:09 |
| fungi | jeblair: all the better | 23:09 |
| clarkb | woot | 23:10 |
| *** dims has joined #openstack-infra | 23:10 | |
| zaro | fungi: can you please help me with the submodules? | 23:10 |
| zaro | fungi: i can't get it to work. i changed url references to "url = https://gerrit.googlesource.com" in .gitmodule | 23:11 |
| *** yamahata has quit IRC | 23:11 | |
| jesusaurus | clarkb: if you want to bump the node_concurrent_recoveries live: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html | 23:11 |
| clarkb | jesusaurus: thanks | 23:11 |
| zaro | fungi: when i go checkout i get "error: pathspec 'stable-2.8' did not match any file(s) known to git." | 23:11 |
| fungi | zaro: the branches in review.o.o are "upstream/stable-2.8" and "openstack/2.8" | 23:13 |
| fungi | ooh, i wonder whether it's expecting you to have a local branch named "stable-2.8"? | 23:13 |
| * fungi doesn't really know much about how git submodules work | 23:13 | |
| fungi | clarkb: ^ ideas there? | 23:14 |
| jeblair | Discussion of OpenStack Project Infrastructure | Docs http://ci.openstack. | 23:14 |
| jeblair | 23:15 < fungi> zaro: the branches in review.o.o are "upstream/stable-2.8" | 23:14 |
| jeblair | and "openstack/2.8" | 23:14 |
| jeblair | gah | 23:14 |
| jeblair | 2014-02-12 23:13:18.108 | ls: cannot access /var/log/secure-*: No such file or directory | 23:14 |
| zaro | fungi: all core plugins have a 'stable-2.8' branch on the gerrit repo | 23:15 |
| *** yamahata has joined #openstack-infra | 23:15 | |
| clarkb | fungi: no clue :/ | 23:15 |
| *** flaper87 is now known as flaper87|afk | 23:16 | |
| jeblair | oh, that may be okay | 23:17 |
| jeblair | yeah, i think it is | 23:17 |
| jeblair | it's just checking for old log files for sudo, and of course there aren't any | 23:17 |
| jeblair | fungi: https://jenkins01.openstack.org/job/dev-nova-python26/1/console | 23:18 |
| jeblair | clarkb: http://www.elasticsearch.org/blog/1-0-0-released/ | 23:19 |
| *** krtaylor has quit IRC | 23:19 | |
| clarkb | jeblair: ya :( oh well that is a breaking change so will need more time to upgrade to | 23:19 |
| clarkb | 0.90.9 is what latest logstash wants and is a non breaking change | 23:19 |
| zaro | fungi, clarkb : i have no clue either. so my workaround.. clone each core plugin into the plugins plugins folder to build gerrit. good thing there are only 4 core plugins right now. | 23:20 |
| fungi | zaro: oh, the few times i've used submodules there's a separate step you need for them to get cloned | 23:21 |
| *** dstanek has quit IRC | 23:21 | |
| fungi | i can reread the submodule documentation in a little bit once i'm freed up | 23:21 |
| clarkb | you can do it with the right flag to submodule init iirc | 23:21 |
| clarkb | or with a flag to clone, but by default it is multistep | 23:22 |
| zaro | fungi: i followed these steps.. http://stackoverflow.com/a/5828396/3294191 | 23:22 |
| *** markmcclain has quit IRC | 23:25 | |
| *** mriedem has joined #openstack-infra | 23:27 | |
| *** dstanek has joined #openstack-infra | 23:27 | |
| *** hdd is now known as hdd_ | 23:28 | |
| fungi | jeblair: poking and repeatedly rethinking this compared to the bare-precise transition, we may need to go all-in on this one? we can't really leverage per-project node label defaults like we did there... | 23:29 |
| fungi | i think it's got to be a (mostly) all-or-nothing transition unless we start making duplicate job definitions or add new parameters to define 2.6 node labels on a per-project basis. preference there? | 23:31 |
| jeblair | fungi: i think you're right. but nova looks good: https://jenkins01.openstack.org/job/dev-nova-python26/1/console | 23:31 |
| *** thedodd has quit IRC | 23:31 | |
| fungi | yep, saw that | 23:31 |
| jeblair | fungi: so i think we can merge a change to bump the min ready in nodepool, then switch a bunch of jobs | 23:31 |
| fungi | i'll do the quick-n-dirty cutover patch and we can revert and try something more subtle if it ends up being an issue | 23:32 |
| fungi | both patches coming now | 23:32 |
| jeblair | fungi: sounds good, thx | 23:32 |
| clarkb | I bumped the number of concurrent recoveries to 5 based on fuzzy math fron jesusaurus :) | 23:34 |
| clarkb | that is 5 per node | 23:34 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Add more bare-centos6 nodes https://review.openstack.org/73102 | 23:38 |
| *** sarob has joined #openstack-infra | 23:41 | |
| *** yassine has quit IRC | 23:41 | |
| *** yamahata has quit IRC | 23:42 | |
| *** david-lyle has quit IRC | 23:43 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add more bare-centos6 nodes https://review.openstack.org/73102 | 23:43 |
| *** derekh has quit IRC | 23:44 | |
| *** jcoufal has quit IRC | 23:46 | |
| *** esker has quit IRC | 23:49 | |
| *** esker has joined #openstack-infra | 23:50 | |
| ociuhandu | Hi all, we seem to have a lot of issues in the last few hours with the devstack instances on the hyper-v ci, due to nova-api failing to start. | 23:53 |
| ociuhandu | while checking, we found this bug opened: https://bugs.launchpad.net/devstack/+bug/1276341 | 23:53 |
| uvirtbot | Launchpad bug 1276341 in devstack "Restacking Causes Issues" [Undecided,New] | 23:53 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Switch to bare-centos6 single-use nodes https://review.openstack.org/73107 | 23:54 |
| fungi | ociuhandu: restacking is not really supported by devstack, which is why we don't reuse devstack servers (we rebuild them from scratch for every job) | 23:54 |
| fungi | ociuhandu: you might be able to get more detail from the devstack developers in #openstack-qa | 23:55 |
| *** esker has quit IRC | 23:55 | |
| ociuhandu | fungi: in u=our case even the first stack is failing with same error on nova-manage db sync | 23:55 |
| ociuhandu | here is a sample: http://paste.openstack.org/show/64817/ | 23:55 |
| *** CaptTofu has joined #openstack-infra | 23:56 | |
| fungi | ociuhandu: that looks like oslo.config is finding more than one rpc_backend set | 23:57 |
| ociuhandu | yes, and we're trying now to find out the reason, as this started to be an issue in the last few hours only | 23:57 |
| *** gaelL has quit IRC | 23:58 | |
| fungi | ociuhandu: i don't think we've seen that crop up, but you might have more luck checking with the nova, oslo.config or devstack developers | 23:58 |
| ociuhandu | fungi: thanks | 23:58 |
| *** denis_makogon has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!