| openstackgerrit | A change was merged to openstack-infra/config: Include stable/icehouse in gerritbot for manuals https://review.openstack.org/93125 | 00:00 |
|---|---|---|
| *** wenlock_ has quit IRC | 00:00 | |
| sdague | jeblair: +1, as long as we agree on fast revert if it acts up. Because it now puts a test load in devstack that I can't fix. | 00:01 |
| *** chuckC has joined #openstack-infra | 00:01 | |
| *** mriedem has joined #openstack-infra | 00:02 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add neutron-large-ops-testing job https://review.openstack.org/92901 | 00:02 |
| notmyname | has anything changed in gerrit that would prevent me from +2/+1 to a patch? | 00:02 |
| *** wenlock has quit IRC | 00:02 | |
| clarkb | notmyname: shouldn't have | 00:02 |
| clarkb | notmyname: have an example chnage? | 00:02 |
| notmyname | clarkb: to my own patch? | 00:03 |
| notmyname | https://review.openstack.org/#/c/94669/ | 00:03 |
| clarkb | notmyname: looks like you +2/+A'd it | 00:03 |
| *** doug-fish has left #openstack-infra | 00:03 | |
| openstackgerrit | A change was merged to openstack-infra/config: Drop large-ops-testing number to 100 from 175 https://review.openstack.org/93127 | 00:04 |
| notmyname | clarkb: yikes. horrible browser caching | 00:04 |
| notmyname | clarkb: thanks | 00:04 |
| jeblair | i don't actually see the messages related to that though | 00:04 |
| jeblair | i see the grid and alex's vote, but i don't see a message for john's | 00:04 |
| notmyname | jeblair: hmm..I had to empty caches and then reload and I saw it | 00:05 |
| *** markmcclain has quit IRC | 00:05 | |
| jeblair | i don't know why that would be cached for me :/ | 00:05 |
| notmyname | jeblair: but ya, I don't see any further jenkins messages there either | 00:06 |
| clarkb | oh hey same here | 00:06 |
| clarkb | I didn't check in comments just teh table | 00:06 |
| openstackgerrit | A change was merged to openstack-infra/config: Publish translated localized API documents https://review.openstack.org/91132 | 00:06 |
| *** W00die has joined #openstack-infra | 00:06 | |
| openstackgerrit | A change was merged to openstack-infra/config: Increasing the number of repos displayed on git.openstack.org to remove the need to paginate. https://review.openstack.org/94698 | 00:06 |
| jeblair | i still don't see it after a full browser restart | 00:06 |
| *** aconrad has quit IRC | 00:06 | |
| jeblair | i do see it in the ssh query | 00:07 |
| jeblair | gertty does not see it | 00:07 |
| clarkb | is the OSSG a blessed thing for openstack/ ? looking at 93747 | 00:07 |
| openstackgerrit | A change was merged to openstack-infra/config: Add selenium job for tuskar-ui https://review.openstack.org/92568 | 00:09 |
| openstackgerrit | A change was merged to openstack-infra/config: Give check-tripleo-overcloud-f20 back its voting rights https://review.openstack.org/92999 | 00:11 |
| jeblair | clarkb: i read annegentle_'s comments in 93747 as indicating it's in the doc program | 00:11 |
| * clarkb rereads | 00:11 | |
| clarkb | there it is, sorry for the noise | 00:12 |
| *** amcrn has joined #openstack-infra | 00:13 | |
| jesusaurus | clarkb: i responded to your kibana proxy comment | 00:13 |
| *** ramashri has quit IRC | 00:14 | |
| *** michchap_ has quit IRC | 00:14 | |
| *** chuckC has quit IRC | 00:14 | |
| clarkb | jesusaurus: thanks looking | 00:14 |
| clarkb | ok that confirms what I thought so my -1 stands | 00:15 |
| openstackgerrit | A change was merged to openstack-infra/config: Add nodepool documentation link https://review.openstack.org/93361 | 00:16 |
| clarkb | nuetron quota bump hasn't happeend yet | 00:16 |
| jeblair | clarkb: i just prioritized all the specs repo creation changes | 00:16 |
| clarkb | jeblair: ok, want me to do the same and flush those out? you don't happen to have a nice query for them do you? | 00:16 |
| jeblair | i think people are pretty eager for them; if you're reviewing you might want to do the same | 00:17 |
| jeblair | clarkb: no :( | 00:17 |
| jeblair | (need topic change support for gertty) | 00:17 |
| jeblair | clarkb: they all have good commit subjects though | 00:17 |
| clarkb | jeblair: quick question about not openstack sepcs | 00:18 |
| clarkb | are we going to be picky about names there? | 00:18 |
| clarkb | programs aren't really a thing in stackforge | 00:18 |
| jesusaurus | clarkb: oh i see what you were saying there. yeah agreed | 00:18 |
| jeblair | clarkb: i don't think so. and if they incubate, well, we're renaming anyway. | 00:18 |
| clarkb | ++ wfm | 00:18 |
| openstackgerrit | A change was merged to openstack-infra/config: Add object-specs repository https://review.openstack.org/94140 | 00:19 |
| openstackgerrit | A change was merged to openstack-infra/config: Add acl for creating dev branch for core group of compass https://review.openstack.org/93201 | 00:19 |
| jeblair | clarkb: i also don't care if they have noop jobs :) | 00:19 |
| *** openstackgerrit has quit IRC | 00:19 | |
| *** openstackgerrit has joined #openstack-infra | 00:21 | |
| *** lcheng_ has quit IRC | 00:22 | |
| *** sarob has joined #openstack-infra | 00:22 | |
| clarkb | jeblair: status:open project:openstack-infra/config message:specs seems to work | 00:23 |
| jeblair | cool i guess '-' acts as a token break | 00:23 |
| sdague | clarkb: have you figured out how to make message: take a proper string, not just a token? | 00:23 |
| notmyname | jeblair: clarkb: anything I need to do on that patch (https://review.openstack.org/#/c/94669/) before I step away for a while? | 00:24 |
| clarkb | sdague: I have not | 00:24 |
| *** msabramo1 has quit IRC | 00:24 | |
| jeblair | notmyname: i don't see it in zuul; i think you should re-approve it | 00:24 |
| jeblair | notmyname: i'm guessing the same thing that caused the messages not to be recorded may have caused the event not to be sent | 00:25 |
| notmyname | re re re re approved | 00:25 |
| sdague | clarkb: ok, yeh, I had to do some silly things to find Depends-On: in gerrit for that reason - https://review.openstack.org/#/c/94365/5/zuul/lib/gerrit.py | 00:25 |
| notmyname | jeblair: see it? | 00:25 |
| jeblair | notmyname: nope. try voting workflow 0 then voting +1 again | 00:26 |
| *** _nadya__ has quit IRC | 00:26 | |
| jeblair | notmyname: i do at least see the comment you left, which is promising. | 00:26 |
| notmyname | that got it | 00:26 |
| notmyname | jenkins seems to have got it this time | 00:26 |
| jeblair | cool. though the fact that this happened is very strange. we'll have to keep an eye out for other instances | 00:26 |
| *** sarob has quit IRC | 00:27 | |
| notmyname | jeblair: FWIW the first time I tried this (before pinging you in here) I got a 5xx server error. kept trying and got it. but maybe there's something in the logs to look for | 00:27 |
| jeblair | notmyname: cool, yeah, there might actually be logs then. | 00:28 |
| notmyname | jeblair: thanks for your help | 00:28 |
| jeblair | np | 00:28 |
| *** chuckC has joined #openstack-infra | 00:28 | |
| *** matsuhashi has joined #openstack-infra | 00:29 | |
| clarkb | jeblair: I think I got all of them | 00:29 |
| clarkb | see any I missed? | 00:29 |
| *** msabramo has joined #openstack-infra | 00:31 | |
| openstackgerrit | A change was merged to openstack-infra/config: update gerrit app on review-dev.o.o to same version as on review.o.o https://review.openstack.org/94413 | 00:32 |
| jeblair | clarkb: nope | 00:33 |
| ianw | speaking of comment tracking; is workflow statements not causing problems for anyone else? https://review.openstack.org/#/c/93862/ | 00:34 |
| *** openstackgerrit has quit IRC | 00:34 | |
| ianw | maybe issuing "check experimental" while putting a change into workflow-1 isn't common | 00:34 |
| jeblair | ianw: i guess not enough that we thought about testing it, but it does seem like the sort of thing that would be good to support | 00:35 |
| *** openstackgerrit has joined #openstack-infra | 00:35 | |
| openstackgerrit | A change was merged to openstack-infra/config: Adds stackforge/puppet-openstack-specs project https://review.openstack.org/94743 | 00:36 |
| clarkb | agreed | 00:36 |
| clarkb | we might even glob everything prior to the comment itself | 00:37 |
| *** marun has quit IRC | 00:37 | |
| ianw | jeblair: i didn't see anywhere to test those regex's apart from running them manually ... did i miss it? | 00:39 |
| openstackgerrit | A change was merged to openstack-infra/config: Create telemetry-specs git repo https://review.openstack.org/94044 | 00:40 |
| clarkb | ianw: if you do ssh -p29418 review.openstack.org gerrit stream-events you can get the actual json events that zuul receives | 00:40 |
| *** msabramo has quit IRC | 00:40 | |
| jeblair | ianw: no, you didn't; i meant in our pre-upgrade testing. | 00:40 |
| clarkb | then pass that data to your regex in a python interpreter | 00:40 |
| *** jhesketh has quit IRC | 00:40 | |
| clarkb | oh yeah we tested it a bit but not with the different categories | 00:40 |
| *** UtahDave has joined #openstack-infra | 00:41 | |
| ianw | clarkb: yep, that's what i did (but using the json reply you get from https://review.openstack.org/changes) | 00:41 |
| openstackgerrit | A change was merged to openstack-infra/config: Create baremetal-specs git repo https://review.openstack.org/94113 | 00:41 |
| *** wenlock has joined #openstack-infra | 00:41 | |
| openstackgerrit | A change was merged to openstack-infra/config: Enable pypi upload for stackalytics https://review.openstack.org/93693 | 00:41 |
| *** msabramo has joined #openstack-infra | 00:42 | |
| *** gokrokve has joined #openstack-infra | 00:43 | |
| *** jerryz_ has quit IRC | 00:43 | |
| openstackgerrit | A change was merged to openstack-infra/config: Create identity-specs git repo https://review.openstack.org/94119 | 00:43 |
| openstackgerrit | A change was merged to openstack-infra/config: New fuel-specs repo for Fuel designs and specs https://review.openstack.org/93560 | 00:43 |
| openstackgerrit | A change was merged to openstack-infra/config: Added cloudkitty to stackforge projects https://review.openstack.org/93743 | 00:44 |
| openstackgerrit | A change was merged to openstack-infra/config: Add a git repo for security documentation https://review.openstack.org/93747 | 00:44 |
| openstackgerrit | A change was merged to openstack-infra/config: Make Tempest job voting for Marconi https://review.openstack.org/93568 | 00:44 |
| openstackgerrit | A change was merged to openstack-infra/config: Create new job for testing of translated manual https://review.openstack.org/93809 | 00:47 |
| openstackgerrit | A change was merged to openstack-infra/config: Ensure zuul's status images are available https://review.openstack.org/93812 | 00:47 |
| sdague | jeblair: so did I hear that auto abandon is no longer a thing? | 00:47 |
| openstackgerrit | Adam Gandelman proposed a change to openstack-infra/elastic-recheck: Add a fingerprint for bug 1321494 https://review.openstack.org/94746 | 00:48 |
| uvirtbot | Launchpad bug 1321494 in ironic "NodeLocked causing random test failures" [High,Triaged] https://launchpad.net/bugs/1321494 | 00:48 |
| *** gokrokve has quit IRC | 00:48 | |
| jeblair | sdague: yeah; core groups have abandon perms now and dashboards make prioritizing things easier | 00:48 |
| clarkb | did we properly disable auto abandon or did it just break after upgrade? | 00:49 |
| jeblair | clarkb: i think it's broken; but i think we want to disable it rather than fix | 00:49 |
| clarkb | ++ | 00:49 |
| *** openstackgerrit has quit IRC | 00:49 | |
| *** nati_ueno has quit IRC | 00:50 | |
| *** yaguang has joined #openstack-infra | 00:50 | |
| *** openstackgerrit has joined #openstack-infra | 00:51 | |
| sdague | oh, I actually like auto abandon as it incents people to actually respond to feedback instead of leaving dead code out there | 00:51 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Re-add fingerprint for 1285833 https://review.openstack.org/94744 | 00:51 |
| SpamapS | sdague: +1 | 00:52 |
| harlowja_ | qq, i am seeing the following when trying to use a new requirements module | 00:52 |
| harlowja_ | 'wrapt' is not a global requirement but it should be,something went wrong (this causes grenade to break) | 00:52 |
| harlowja_ | and wrapt seem to be on pypi.openstack.org | 00:52 |
| *** _nadya_ has joined #openstack-infra | 00:52 | |
| clarkb | harlowja_: can you link to the change | 00:52 |
| harlowja_ | http://logs.openstack.org/55/87055/17/check/check-grenade-dsvm/ee69b13/console.html | 00:52 |
| harlowja_ | https://review.openstack.org/#/c/87055/ | 00:53 |
| clarkb | huh I have no idea | 00:56 |
| clarkb | that message doesn't show up in openstack/requirements/update.py | 00:57 |
| *** zns has joined #openstack-infra | 00:57 | |
| harlowja_ | weird :( | 00:57 |
| clarkb | and greping for it in devstack doesn't give me anything either | 00:57 |
| clarkb | oh wait grenade | 00:57 |
| * clarkb looks more | 00:57 | |
| harlowja_ | weird, i can't find 'something went wrong' in grenade | 00:58 |
| harlowja_ | magic messages, lol | 00:58 |
| *** lakshmiS has quit IRC | 00:58 | |
| harlowja_ | although i'm unsure what is going on there | 00:59 |
| harlowja_ | cd /opt/stack/old/requirements | 00:59 |
| harlowja_ | and then syncing old requirements or something, weird | 00:59 |
| clarkb | its using the old requirements which I think may actually be the problem | 00:59 |
| *** nati_ueno has joined #openstack-infra | 00:59 | |
| clarkb | wrapt isn't an icehouse requirement | 00:59 |
| clarkb | its in master | 01:00 |
| clarkb | which breaks your upgrade path | 01:00 |
| harlowja_ | so how does one add new dependencies :-P | 01:00 |
| clarkb | harlowja_: well most projects have stable versions | 01:00 |
| clarkb | they won't have dependencies added to stable versions | 01:00 |
| clarkb | taskflow apparently doesn't do this | 01:00 |
| clarkb | and thus must be backward compatible to way back when | 01:00 |
| harlowja_ | hmmm, taskflow has releases though | 01:01 |
| *** nati_ueno has quit IRC | 01:01 | |
| *** UtahDave has quit IRC | 01:01 | |
| clarkb | yup there the error message is in openstack/requirements stable/icehouse | 01:01 |
| clarkb | harlowja_: yes but not stable releases | 01:01 |
| * harlowja_ confused | 01:02 | |
| clarkb | harlowja_: taskflow does not have a stable/icehouse branch right? | 01:02 |
| clarkb | harlowja_: that means taskflow master is used against stable/icehouse other things | 01:02 |
| harlowja_ | hmmm, that shouldn't be the case, i'd expect the version released in icehouse would be used there | 01:03 |
| clarkb | harlowja_: in the gate we always do stable installs | 01:03 |
| clarkb | er s/stable/source/ | 01:03 |
| *** homeless has quit IRC | 01:03 | |
| clarkb | and there is no way of knowing what that version is | 01:03 |
| *** zul has quit IRC | 01:03 | |
| *** zul has joined #openstack-infra | 01:03 | |
| harlowja_ | odd so clients can't add new dependencies either? | 01:04 |
| harlowja_ | since afaik they don't create stable/icehouse branch either | 01:04 |
| clarkb | harlowja_: so you can add new dependencies you just have to be much more careful about it | 01:04 |
| clarkb | but yes the clients are in the same situation | 01:05 |
| *** crc32 has quit IRC | 01:05 | |
| harlowja_ | so what would be the recommended way to add a dependency, create a new branch? | 01:05 |
| harlowja_ | create a stable/icehouse branch? | 01:05 |
| jogo | jeblair: for the specs rename | 01:05 |
| clarkb | harlowja_: I think backport the dependency | 01:05 |
| jogo | jeblair: I didn't see nova-specs on the list, was that intentional | 01:05 |
| *** zns has quit IRC | 01:07 | |
| *** zns_ has joined #openstack-infra | 01:07 | |
| SpamapS | hm | 01:07 |
| *** michchap has joined #openstack-infra | 01:07 | |
| SpamapS | I see lots of slaves on our cloud that have no floating ip assigned, but are ACTIVE | 01:07 |
| harlowja_ | clarkb and the future way would be to have a stable/* branch? | 01:08 |
| harlowja_ | *more desirable way going forward? | 01:08 |
| clarkb | harlowja_: no I don't think we intend on having stable branches on a lot of things | 01:09 |
| clarkb | harlowja_: the clients for example won't have that | 01:09 |
| harlowja_ | k, so u have to basically prepare for dependencies additions a release before | 01:09 |
| clarkb | harlowja_: mostly this is making us thing about things a bit more. This si a good thing. eg is the use of wrapt worth doing knowing that distros may need to suddenly package it to fix a taskflow security bug | 01:10 |
| clarkb | not saying that is the situation here but it is the sort of thing this is intended to catch | 01:10 |
| clarkb | if you look at the global requirements list you will see that openstack hasn't been very good about this in the past | 01:10 |
| harlowja_ | nah, its mostly so that i can depreciate thigns :) | 01:10 |
| harlowja_ | *things | 01:10 |
| harlowja_ | just feels odd that it'd be expected to have knowledge of future dependencies before u desire to use them (space-time warping not applicable) | 01:11 |
| *** alugovoi has quit IRC | 01:12 | |
| harlowja_ | and without stable/* branches i guess u need to do this kind of predication | 01:12 |
| harlowja_ | *prediction | 01:12 |
| clarkb | harlowja_: you aren't expected to do that | 01:12 |
| clarkb | I think it is ok to backport things | 01:12 |
| clarkb | but this forces you to acknowledge what you are doing when backports hapen | 01:12 |
| clarkb | if there is a security issue and you fix it after backport happens you are forcing anyone deploying your code to package that new dependency | 01:13 |
| harlowja_ | sure, i can understand that | 01:14 |
| harlowja_ | just feels odd, idk, maybe i'll get used to it :) | 01:14 |
| clarkb | its definitely odd if you only ever consider the next release | 01:15 |
| clarkb | but apaprently people like running old code ;) | 01:15 |
| harlowja_ | whats up with that, ha | 01:16 |
| *** marcoemorais has quit IRC | 01:16 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/devstack-gate: Only copy symlink name of logs https://review.openstack.org/94753 | 01:19 |
| harlowja_ | clarkb https://review.openstack.org/94754 seem ok? | 01:19 |
| clarkb | yup | 01:20 |
| harlowja_ | cool | 01:20 |
| *** wenlock_ has joined #openstack-infra | 01:21 | |
| *** moted has quit IRC | 01:21 | |
| harlowja_ | clarkb thx for the explanation, sorta makes sense i guess :-P | 01:21 |
| harlowja_ | wrapt is a pretty small utiltiy library so hopefully no objections | 01:22 |
| *** jhesketh has joined #openstack-infra | 01:24 | |
| *** yaguang has quit IRC | 01:25 | |
| *** yaguang has joined #openstack-infra | 01:27 | |
| *** lttrl has left #openstack-infra | 01:27 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add a fingerprint for bug 1321494 https://review.openstack.org/94746 | 01:29 |
| uvirtbot | Launchpad bug 1321494 in ironic "NodeLocked causing random test failures" [High,Triaged] https://launchpad.net/bugs/1321494 | 01:29 |
| SlickNik | Can I get some eyeballs on https://review.openstack.org/#/c/92755 when you guys have a chance? Thanks! | 01:30 |
| SlickNik | (Trying to get the trove-guest image building patches in place.) | 01:31 |
| *** wenlock_ has quit IRC | 01:32 | |
| mordred | sdague, SpamapS: it does not incent that with me - with me it means I have to go look through abandoned patches all the time to find the things I'm working on | 01:32 |
| *** gyee has quit IRC | 01:32 | |
| * mordred says this as the person who most strongly thought the idea was good originally | 01:33 | |
| *** lcheng_ has joined #openstack-infra | 01:34 | |
| SpamapS | mordred: you should have emails saying you got timed out | 01:35 |
| *** marun has joined #openstack-infra | 01:35 | |
| *** msabramo has quit IRC | 01:35 | |
| mordred | SpamapS: oh. HAHAHAHAHAHAHA | 01:36 |
| mordred | SpamapS: you're so funny | 01:36 |
| mordred | you think I can deal with the email volume with anything other than a filter straight to /dev/null | 01:36 |
| *** hipster has joined #openstack-infra | 01:36 | |
| clarkb | SlickNik: done. see comment as it points out another potential breakage | 01:37 |
| SpamapS | mordred: procmail does it for me quite nicely. review emails are all tucked in a nice folder and tagged in my sup-mail for easy searching | 01:38 |
| clarkb | I /dev/null gerrit mail | 01:38 |
| clarkb | its much easier to use a tool like gertty, get a list of all the thigns and start working through it | 01:39 |
| clarkb | sdague: and re incentivising communication gertty knows how to kill things now | 01:40 |
| clarkb | or you can mark it as reviewed and wait for the next patchset | 01:40 |
| *** gokrokve has joined #openstack-infra | 01:43 | |
| *** gokrokve_ has joined #openstack-infra | 01:44 | |
| *** gokrokve_ has joined #openstack-infra | 01:44 | |
| *** thomasbiege1 has joined #openstack-infra | 01:47 | |
| *** gokrokve has quit IRC | 01:48 | |
| *** lcheng_ has quit IRC | 01:49 | |
| *** _nadya_ has quit IRC | 01:50 | |
| *** thomasbiege has quit IRC | 01:50 | |
| *** gokrokve_ has quit IRC | 01:52 | |
| *** michchap_ has joined #openstack-infra | 01:54 | |
| *** zul has quit IRC | 01:54 | |
| *** michchap has quit IRC | 01:56 | |
| *** zul has joined #openstack-infra | 01:56 | |
| *** msabramo has joined #openstack-infra | 01:56 | |
| *** amcrn has quit IRC | 01:57 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Some tweaks to get closer to Python 3 compat https://review.openstack.org/87810 | 01:57 |
| *** saschpe has quit IRC | 02:00 | |
| *** saschpe has joined #openstack-infra | 02:01 | |
| *** wenlock_ has joined #openstack-infra | 02:05 | |
| *** mrodden has joined #openstack-infra | 02:05 | |
| *** mengxd has joined #openstack-infra | 02:07 | |
| *** bhuvan has quit IRC | 02:11 | |
| *** gokrokve has joined #openstack-infra | 02:14 | |
| *** _nadya_ has joined #openstack-infra | 02:17 | |
| *** lcheng_ has joined #openstack-infra | 02:18 | |
| *** gokrokve has quit IRC | 02:18 | |
| *** gokrokve has joined #openstack-infra | 02:20 | |
| *** zul has quit IRC | 02:22 | |
| *** hipster has quit IRC | 02:25 | |
| *** Clabbe has quit IRC | 02:26 | |
| *** Clabbe has joined #openstack-infra | 02:27 | |
| *** xdmeng has joined #openstack-infra | 02:35 | |
| *** rfolco has quit IRC | 02:36 | |
| *** mengxd has quit IRC | 02:36 | |
| *** xchu has joined #openstack-infra | 02:39 | |
| jeblair | jogo: i haven't heard from mikal about nova-specs, and at the mtg yesterday he indicated he might prefer to wait a bit | 02:42 |
| *** morganfainberg is now known as morganfainberg_Z | 02:43 | |
| jogo | jeblair: ack | 02:47 |
| *** mengxd has joined #openstack-infra | 02:48 | |
| *** xdmeng has quit IRC | 02:50 | |
| *** thomasbiege has joined #openstack-infra | 02:54 | |
| *** matsuhas_ has joined #openstack-infra | 02:55 | |
| *** nithyag_ has joined #openstack-infra | 02:56 | |
| *** kmartin has joined #openstack-infra | 02:56 | |
| *** fbo_away has joined #openstack-infra | 02:56 | |
| *** nosnos_ has joined #openstack-infra | 02:57 | |
| *** GheRiver1 has joined #openstack-infra | 02:57 | |
| *** mkerrin1 has joined #openstack-infra | 02:58 | |
| *** ilyashakhat_ has joined #openstack-infra | 02:58 | |
| *** gokrokve_ has joined #openstack-infra | 03:00 | |
| *** Kiall_ has joined #openstack-infra | 03:02 | |
| *** lxsli0 has joined #openstack-infra | 03:02 | |
| *** mriedem has quit IRC | 03:03 | |
| *** fbo has quit IRC | 03:03 | |
| *** blamar has quit IRC | 03:03 | |
| *** thomasbiege1 has quit IRC | 03:03 | |
| *** matsuhashi has quit IRC | 03:03 | |
| *** jeblair has quit IRC | 03:03 | |
| *** GheRivero has quit IRC | 03:03 | |
| *** fbo_away is now known as fbo | 03:03 | |
| *** nosnos has quit IRC | 03:03 | |
| *** anteaya has quit IRC | 03:03 | |
| *** zehicle_at_dell has quit IRC | 03:03 | |
| *** mkerrin has quit IRC | 03:03 | |
| *** sdake has quit IRC | 03:03 | |
| *** russell_h has quit IRC | 03:03 | |
| *** lxsli has quit IRC | 03:03 | |
| *** dansmith has quit IRC | 03:03 | |
| *** Kiall has quit IRC | 03:03 | |
| *** NithyaG_AFK has quit IRC | 03:03 | |
| *** ilyashakhat has quit IRC | 03:03 | |
| *** jgriffith has quit IRC | 03:03 | |
| *** zehicle_at_dell has joined #openstack-infra | 03:03 | |
| *** russell_h has joined #openstack-infra | 03:03 | |
| *** russell_h has quit IRC | 03:03 | |
| *** zehicle_at_dell has quit IRC | 03:03 | |
| *** dansmith has joined #openstack-infra | 03:03 | |
| *** russell_h has joined #openstack-infra | 03:04 | |
| *** zehicle_at_dell has joined #openstack-infra | 03:04 | |
| *** anteaya has joined #openstack-infra | 03:04 | |
| *** saschpe- has joined #openstack-infra | 03:05 | |
| *** otherwiseguy has joined #openstack-infra | 03:05 | |
| *** russell_h has quit IRC | 03:05 | |
| *** russell_h has joined #openstack-infra | 03:05 | |
| *** sdake has joined #openstack-infra | 03:06 | |
| *** yamahata__ has joined #openstack-infra | 03:07 | |
| *** yamahata_ has quit IRC | 03:07 | |
| *** mrodden has quit IRC | 03:08 | |
| *** serverascode has quit IRC | 03:08 | |
| *** gokrokve has quit IRC | 03:09 | |
| *** saschpe has quit IRC | 03:09 | |
| *** KurtMartin has quit IRC | 03:09 | |
| *** mrodden has joined #openstack-infra | 03:09 | |
| *** serverascode has joined #openstack-infra | 03:09 | |
| *** jeblair has joined #openstack-infra | 03:09 | |
| *** hipster has joined #openstack-infra | 03:10 | |
| *** jgriffith has joined #openstack-infra | 03:13 | |
| *** blamar has joined #openstack-infra | 03:14 | |
| *** malini_afk is now known as malini | 03:15 | |
| *** isviridov has quit IRC | 03:15 | |
| *** eharney has quit IRC | 03:15 | |
| *** gokrokve_ has quit IRC | 03:21 | |
| *** eharney has joined #openstack-infra | 03:22 | |
| *** boris-42 has quit IRC | 03:23 | |
| *** alugovoi has joined #openstack-infra | 03:28 | |
| *** dstanek is now known as dstanek_zzz | 03:29 | |
| *** dstanek_zzz is now known as dstanek | 03:29 | |
| *** boris-42 has joined #openstack-infra | 03:29 | |
| jgriffith | clarkb: jeblair You folks here any grumblings from folks about insconsitent test results from folks trying to do 3'rd party CI? | 03:30 |
| jgriffith | I run 5 times and get different results every time | 03:30 |
| jgriffith | Typically 2 or 3 failures, all different areas | 03:31 |
| jgriffith | Running manually in my "other" devstack env, seeing the same sort of thing | 03:31 |
| jgriffith | s/here/hear/ | 03:31 |
| *** lcheng_ has quit IRC | 03:36 | |
| *** pcrews__ has quit IRC | 03:38 | |
| jhesketh | lifeless: ping | 03:38 |
| *** malini is now known as malini_afk | 03:39 | |
| *** marun has quit IRC | 03:39 | |
| *** mrodden has quit IRC | 03:40 | |
| *** praneshp has quit IRC | 03:41 | |
| *** prad_ has quit IRC | 03:50 | |
| *** prad has joined #openstack-infra | 03:52 | |
| *** nosnos_ has quit IRC | 03:52 | |
| *** hipster has quit IRC | 03:55 | |
| *** isviridov has joined #openstack-infra | 03:57 | |
| *** mayu_ has joined #openstack-infra | 04:09 | |
| mayu_ | zuul receive comment-add events from openstack ci , but can not trigger jenkins job, anybody help ? | 04:10 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: move python coverage jobs to check pipeline https://review.openstack.org/94769 | 04:12 |
| zaro | mayu_: is this from review.o.o? | 04:13 |
| mayu_ | yes | 04:14 |
| zaro | mayu_: do you have a change reference? | 04:15 |
| *** yaguang has quit IRC | 04:16 | |
| *** gokrokve has joined #openstack-infra | 04:20 | |
| *** marcoemorais has joined #openstack-infra | 04:21 | |
| *** fanhe has joined #openstack-infra | 04:31 | |
| *** marcoemorais1 has joined #openstack-infra | 04:32 | |
| *** marcoemorais has quit IRC | 04:34 | |
| *** ramashri has joined #openstack-infra | 04:35 | |
| *** nosnos has joined #openstack-infra | 04:35 | |
| *** gokrokve has quit IRC | 04:38 | |
| *** krtaylor has joined #openstack-infra | 04:41 | |
| *** gokrokve has joined #openstack-infra | 04:44 | |
| *** zns_ has quit IRC | 04:45 | |
| *** gokrokve has quit IRC | 04:47 | |
| *** arnaud has joined #openstack-infra | 04:52 | |
| *** ramashri has quit IRC | 04:52 | |
| *** terryw has joined #openstack-infra | 04:53 | |
| *** reed has quit IRC | 04:54 | |
| *** otherwiseguy has quit IRC | 04:56 | |
| *** igor_ has joined #openstack-infra | 04:59 | |
| *** alugovoi has quit IRC | 05:00 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Some tweaks to get closer to Python 3 compat https://review.openstack.org/87810 | 05:01 |
| *** arnaud has quit IRC | 05:01 | |
| *** igor__ has quit IRC | 05:02 | |
| msabramo | zaro: answered your question on https://review.openstack.org/90114 | 05:02 |
| msabramo | JJB working well with python-jenkins python3 changes | 05:03 |
| *** isviridov has quit IRC | 05:05 | |
| *** alugovoi has joined #openstack-infra | 05:05 | |
| *** marun has joined #openstack-infra | 05:06 | |
| *** marun has quit IRC | 05:07 | |
| *** marun has joined #openstack-infra | 05:08 | |
| *** arnaud has joined #openstack-infra | 05:10 | |
| *** wenlock has quit IRC | 05:11 | |
| zaro | msabramo: cool! thanks. | 05:11 |
| zaro | msabramo: well see how this goes.. https://review.openstack.org/94769 | 05:12 |
| msabramo | oh interesting | 05:13 |
| msabramo | that seems like it would be nice to see coverage when reviewing | 05:13 |
| msabramo | we have our CI at work rigged up to measure coverage in PR and target branch and if it goes down it can optionally fail the test | 05:14 |
| *** gokrokve has joined #openstack-infra | 05:14 | |
| zaro | msabramo: we are probably not ready to gate on coverage, but would be nice to do manual check on reviews. | 05:17 |
| *** harlowja_ is now known as harlowja_away | 05:17 | |
| msabramo | zaro: yeah | 05:17 |
| msabramo | zaro: we need a new pbr to use pbr for coverage with JJB | 05:18 |
| msabramo | zaro: I think with python-jenkins, I didn't use pbr for coverage; did it manually with coverage.py directly | 05:18 |
| zaro | msabramo: why not use pbr? | 05:19 |
| *** yaguang has joined #openstack-infra | 05:19 | |
| zaro | msabramo: ohh right, need a new one. | 05:19 |
| msabramo | zaro: yeah, because I added a new option to pbr | 05:19 |
| msabramo | merged but not released yet | 05:20 |
| *** wenlock has joined #openstack-infra | 05:20 | |
| zaro | msabramo: will that mean that python coverage job in the template will not work? | 05:20 |
| msabramo | zaro: For JJB, I think it will fail until we get that pbr; the python-jenkins one I think will work; let me find the change set for that | 05:21 |
| *** Longgeek has joined #openstack-infra | 05:22 | |
| msabramo | zaro: https://review.openstack.org/94649 (which you already approved) | 05:22 |
| msabramo | zaro: this one should work since it doesn't use pbr for the coverage | 05:23 |
| *** dstanek is now known as dstanek_zzz | 05:23 | |
| msabramo | hmmm forgot to post sample output | 05:24 |
| *** wenlock_ has quit IRC | 05:25 | |
| zaro | msabramo: cool. | 05:25 |
| zaro | msabramo: does my comments make sense on this one? https://review.openstack.org/#/c/94531/ | 05:26 |
| msabramo | zaro: lemme see | 05:27 |
| msabramo | zaro: ah ok so we can take it out of check and post because it's in the template | 05:28 |
| msabramo | zaro: Oh yeah, I hadn't even noticed before that it was already testing coverage — http://logs.openstack.org/49/94649/1/check/python-jenkins-coverage/a9dc119/console.html | 05:29 |
| *** woody has joined #openstack-infra | 05:31 | |
| zaro | msabramo: ohh, it's also already set to non-voting. so your change isn't required at all? | 05:31 |
| *** woody is now known as Guest14939 | 05:32 | |
| *** vkdrao has joined #openstack-infra | 05:32 | |
| msabramo | zaro: oh yeah, maybe | 05:32 |
| zaro | msabramo: something must already be setting the job to non-voting. | 05:32 |
| *** Longgeek has quit IRC | 05:32 | |
| *** Longgeek has joined #openstack-infra | 05:33 | |
| *** talluri has joined #openstack-infra | 05:33 | |
| *** Longgeek has quit IRC | 05:33 | |
| zaro | msabramo: ahh i think it's line 525 in layout.yaml | 05:34 |
| *** yfried has joined #openstack-infra | 05:34 | |
| *** Longgeek has joined #openstack-infra | 05:35 | |
| *** unicell has quit IRC | 05:36 | |
| *** xchu has quit IRC | 05:37 | |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: 100% test coverage https://review.openstack.org/90500 | 05:37 |
| msabramo | zaro: Rebased https://review.openstack.org/90500 to include the tox coverage target | 05:39 |
| zaro | msabramo: actually that's doesn't make sense. can't tell what's setting it to non-voting. | 05:39 |
| msabramo | zaro: woo hoo! 99% coverage! http://logs.openstack.org/00/90500/10/check/python-jenkins-coverage/a8e9616/console.html | 05:40 |
| clarkb | zaro: there is a job config in the zuul layout.yaml that makes python-jenkins-coverage nonvoting | 05:41 |
| clarkb | just search for python-jenkins-coverage in that file | 05:42 |
| zaro | clarkb: ohh, that is line 525 in layout.yaml | 05:44 |
| zaro | clarkb: but doesn't make sense to me because the job that msabramo wants non-voting is is 'jenkins-job-builder-coverage' | 05:45 |
| clarkb | you will have to deal with them separately | 05:46 |
| zaro | msabramo: ok, igore my last comment then. clarkb says you will need to set jenkins-job-builder-coverage to non-voting. you will need change 94531 | 05:49 |
| *** _nadya_ has quit IRC | 05:51 | |
| *** Longgeek has quit IRC | 05:51 | |
| *** mkoderer has joined #openstack-infra | 05:54 | |
| *** praneshp has joined #openstack-infra | 05:55 | |
| *** mengxd has quit IRC | 05:57 | |
| *** thomasbiege has quit IRC | 06:01 | |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: Make it work on Python 3 https://review.openstack.org/90114 | 06:02 |
| *** _nadya_ has joined #openstack-infra | 06:03 | |
| *** thomasbiege has joined #openstack-infra | 06:06 | |
| *** terryw has quit IRC | 06:06 | |
| *** arnaud has quit IRC | 06:08 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/config: Add non-voting jenkins-job-builder-coverage https://review.openstack.org/94531 | 06:12 |
| *** wenlock_ has joined #openstack-infra | 06:13 | |
| *** yaguang has quit IRC | 06:13 | |
| *** hogepodge has quit IRC | 06:13 | |
| *** dstanek_zzz is now known as dstanek | 06:14 | |
| *** yaguang has joined #openstack-infra | 06:14 | |
| *** rcarrill` has joined #openstack-infra | 06:15 | |
| msabramo | zaro: Thanks updated it to just have the non-voting line and that's it | 06:16 |
| mayu_ | hi zaro | 06:17 |
| *** rcarrillocruz has quit IRC | 06:18 | |
| mayu_ | zaro: hi ,there ? | 06:18 |
| msabramo | zaro: Gah rebasing it to include coverage nuked your +1 — https://review.openstack.org/90114 | 06:18 |
| *** Longgeek has joined #openstack-infra | 06:21 | |
| *** wenlock_ has quit IRC | 06:22 | |
| *** thomasbiege has left #openstack-infra | 06:23 | |
| *** wenlock has quit IRC | 06:23 | |
| *** dstanek is now known as dstanek_zzz | 06:24 | |
| mayu_ | zaro: zuul receive comment-add events from openstack ci , but can not trigger jenkins job, can you help？ | 06:25 |
| *** Longgeek has quit IRC | 06:26 | |
| *** eglynn has joined #openstack-infra | 06:28 | |
| *** gokrokve has quit IRC | 06:31 | |
| *** yamahata has joined #openstack-infra | 06:36 | |
| *** yaguang has quit IRC | 06:38 | |
| *** sandywalsh has quit IRC | 06:42 | |
| *** Hal_ has joined #openstack-infra | 06:42 | |
| *** _nadya_ has quit IRC | 06:42 | |
| *** sandywalsh has joined #openstack-infra | 06:44 | |
| *** dkranz has joined #openstack-infra | 06:45 | |
| *** locke105 has quit IRC | 06:46 | |
| *** yamahata has quit IRC | 06:48 | |
| *** locke105 has joined #openstack-infra | 06:48 | |
| *** bnemec has quit IRC | 06:48 | |
| *** bnemec has joined #openstack-infra | 06:49 | |
| *** che-arne_afk has quit IRC | 06:50 | |
| *** locke105 has quit IRC | 06:51 | |
| *** vtapia has joined #openstack-infra | 06:52 | |
| mattoliverau | I'm off, have a great night all. | 06:52 |
| *** jcoufal has joined #openstack-infra | 06:53 | |
| *** _nadya_ has joined #openstack-infra | 06:53 | |
| *** Clabbe has quit IRC | 06:54 | |
| *** Clabbe has joined #openstack-infra | 06:56 | |
| *** sandywalsh has quit IRC | 06:56 | |
| *** sandywalsh has joined #openstack-infra | 06:58 | |
| *** yaguang has joined #openstack-infra | 06:59 | |
| *** pblaho has joined #openstack-infra | 06:59 | |
| *** rdopieralski has joined #openstack-infra | 07:00 | |
| *** rdopieralski has quit IRC | 07:00 | |
| *** rdopieralski has joined #openstack-infra | 07:00 | |
| *** doude has joined #openstack-infra | 07:01 | |
| *** rdopieralski has quit IRC | 07:01 | |
| *** rdopieralski has joined #openstack-infra | 07:01 | |
| *** bnemec has quit IRC | 07:02 | |
| *** Clabbe has quit IRC | 07:02 | |
| *** bnemec has joined #openstack-infra | 07:02 | |
| openstackgerrit | A change was merged to openstack/requirements: Netifaces: bump to 0.10.4 https://review.openstack.org/94358 | 07:03 |
| *** rdopieralski has quit IRC | 07:05 | |
| *** yfried_ has joined #openstack-infra | 07:05 | |
| *** yfried has quit IRC | 07:06 | |
| *** locke105 has joined #openstack-infra | 07:07 | |
| *** yaguang has quit IRC | 07:08 | |
| *** dstanek_zzz is now known as dstanek | 07:15 | |
| rcarrill` | morning infra | 07:19 |
| *** rcarrill` is now known as rcarrillocruz | 07:19 | |
| rcarrillocruz | jeblair: thx for the +2 on https://review.openstack.org/#/c/93687/ | 07:19 |
| *** vkdrao has quit IRC | 07:23 | |
| *** praneshp has quit IRC | 07:23 | |
| *** yfried_ has quit IRC | 07:24 | |
| *** dstanek is now known as dstanek_zzz | 07:25 | |
| *** yfried_ has joined #openstack-infra | 07:25 | |
| *** alugovoi has quit IRC | 07:28 | |
| *** skolekonov has joined #openstack-infra | 07:31 | |
| *** unicell has joined #openstack-infra | 07:31 | |
| *** jgallard has joined #openstack-infra | 07:41 | |
| *** Clabbe has joined #openstack-infra | 07:41 | |
| *** ominakov has joined #openstack-infra | 07:41 | |
| *** eglynn has quit IRC | 07:45 | |
| *** isviridov has joined #openstack-infra | 07:45 | |
| *** jgallard has quit IRC | 07:45 | |
| *** Guest12451 is now known as BobBall | 07:46 | |
| *** jgallard has joined #openstack-infra | 07:46 | |
| *** unicell has quit IRC | 07:47 | |
| *** rlandy has joined #openstack-infra | 07:48 | |
| SpamapS | hm | 07:49 |
| *** yaguang has joined #openstack-infra | 07:50 | |
| *** dstanek_zzz is now known as dstanek | 07:54 | |
| *** shardy has joined #openstack-infra | 07:56 | |
| *** hashar has joined #openstack-infra | 07:56 | |
| *** vkdrao has joined #openstack-infra | 07:58 | |
| *** vkdrao has quit IRC | 08:03 | |
| *** dstanek is now known as dstanek_zzz | 08:04 | |
| *** vkdrao has joined #openstack-infra | 08:04 | |
| *** ihrachyshka has joined #openstack-infra | 08:05 | |
| *** yaguang has quit IRC | 08:06 | |
| *** dstanek_zzz is now known as dstanek | 08:07 | |
| *** yaguang has joined #openstack-infra | 08:09 | |
| *** ihrachyshka has quit IRC | 08:13 | |
| *** ihrachyshka has joined #openstack-infra | 08:13 | |
| *** pelix has joined #openstack-infra | 08:15 | |
| *** derekh_ has joined #openstack-infra | 08:15 | |
| *** dizquierdo has joined #openstack-infra | 08:15 | |
| *** dstanek is now known as dstanek_zzz | 08:17 | |
| *** jp_at_hp has joined #openstack-infra | 08:17 | |
| *** eglynn has joined #openstack-infra | 08:18 | |
| *** Hal_ has quit IRC | 08:20 | |
| *** Hal_ has joined #openstack-infra | 08:22 | |
| *** Rikkol has joined #openstack-infra | 08:22 | |
| *** afazekas_wfp has joined #openstack-infra | 08:22 | |
| *** Longgeek has joined #openstack-infra | 08:23 | |
| *** jooools has joined #openstack-infra | 08:25 | |
| *** Rikkol has left #openstack-infra | 08:27 | |
| *** Longgeek has quit IRC | 08:28 | |
| *** GheRiver1 is now known as GheRivero | 08:34 | |
| *** che-arne has joined #openstack-infra | 08:36 | |
| *** jamielennox is now known as jamielennox|away | 08:37 | |
| *** xchu has joined #openstack-infra | 08:40 | |
| *** nithyag_ is now known as NithyaG | 08:40 | |
| *** jpich has joined #openstack-infra | 08:48 | |
| *** yfried__ has joined #openstack-infra | 08:50 | |
| *** julienvey has joined #openstack-infra | 08:51 | |
| openstackgerrit | A change was merged to stackforge/python-jenkins: Add support for measuring coverage https://review.openstack.org/94649 | 08:53 |
| openstackgerrit | Julien Danjou proposed a change to openstack-infra/config: Add gnocchi project on StackForge https://review.openstack.org/93465 | 08:53 |
| *** jlibosva has joined #openstack-infra | 08:53 | |
| *** yfried_ has quit IRC | 08:54 | |
| *** andreaf has joined #openstack-infra | 08:54 | |
| openstackgerrit | A change was merged to stackforge/python-jenkins: 100% test coverage https://review.openstack.org/90500 | 08:55 |
| *** Longgeek has joined #openstack-infra | 08:57 | |
| *** YorikSar has quit IRC | 08:57 | |
| openstackgerrit | Nikhil Manchanda proposed a change to openstack-infra/config: Add new gate-trove-buildimage-mysql job https://review.openstack.org/92755 | 08:58 |
| *** ihrachyshka has quit IRC | 09:02 | |
| *** YorikSar has joined #openstack-infra | 09:03 | |
| *** marcoemorais1 has quit IRC | 09:03 | |
| *** unicell has joined #openstack-infra | 09:04 | |
| *** yaguang has quit IRC | 09:07 | |
| *** dstanek_zzz is now known as dstanek | 09:08 | |
| *** fanhe has quit IRC | 09:10 | |
| *** xchu has quit IRC | 09:10 | |
| *** afazekas_wfp is now known as afazekas | 09:11 | |
| *** Longgeek has quit IRC | 09:13 | |
| *** Longgeek has joined #openstack-infra | 09:13 | |
| *** dstanek is now known as dstanek_zzz | 09:18 | |
| derekh_ | Is anybody around who can promote https://review.openstack.org/#/c/94799/ in the tripleo-check queue | 09:20 |
| *** _nadya_ has quit IRC | 09:27 | |
| *** _nadya_ has joined #openstack-infra | 09:33 | |
| SergeyLukjanov | morning folks | 09:34 |
| * SergeyLukjanov in UTC+4 tz again | 09:34 | |
| SergeyLukjanov | recovering from summit | 09:34 |
| *** vkdrao has quit IRC | 09:39 | |
| jpich | SergeyLukjanov: Hello! And welcome back from Summit :) Do you have the power to kick a user from a channel or know the process to do so? User ciscor on #openstack is sending spam in PMs, to people after they speak I think | 09:41 |
| jpich | Infitialis confirmed he was getting the same spam messages from them | 09:41 |
| SergeyLukjanov | jpich, hey, yup, I have an op permissioins | 09:41 |
| jpich | SergeyLukjanov: I got the spam after 2 sentences fwiw :) | 09:42 |
| openstackgerrit | Nikhil Manchanda proposed a change to openstack-infra/config: Add new gate-trove-buildimage-mysql job https://review.openstack.org/92755 | 09:42 |
| *** Hal_ has quit IRC | 09:43 | |
| jpich | SergeyLukjanov: Thanks! | 09:43 |
| StevenK | SergeyLukjanov: They automatically rejoined, so you might need a +b | 09:44 |
| *** e0ne has joined #openstack-infra | 09:45 | |
| *** julienvey is now known as zz_julien | 09:45 | |
| SergeyLukjanov | jpich, StevenK, I've banned him by nickname for any host and kicked out of the channel, I hope it'll be enough | 09:48 |
| jpich | SergeyLukjanov: Thanks, hope so too | 09:50 |
| *** Longgeek has quit IRC | 09:56 | |
| *** Longgeek has joined #openstack-infra | 09:56 | |
| *** Longgeek has quit IRC | 09:59 | |
| *** jgallard has quit IRC | 10:00 | |
| *** Longgeek has joined #openstack-infra | 10:01 | |
| *** bookwar has left #openstack-infra | 10:02 | |
| *** maxbit has joined #openstack-infra | 10:05 | |
| *** mburned_out is now known as mburned | 10:05 | |
| *** lxsli0 has quit IRC | 10:06 | |
| *** BobBall has quit IRC | 10:07 | |
| *** dstanek_zzz is now known as dstanek | 10:08 | |
| *** e0ne_ has joined #openstack-infra | 10:16 | |
| *** dstanek is now known as dstanek_zzz | 10:18 | |
| *** e0ne has quit IRC | 10:19 | |
| *** giulivo has joined #openstack-infra | 10:21 | |
| giulivo | guys, has anyone ever investigated the possibility of sending the meeting reports via email to some aliases when a meeting is ended? | 10:22 |
| giulivo | is that possible at all? | 10:22 |
| *** _nadya_ has quit IRC | 10:23 | |
| *** _nadya_ has joined #openstack-infra | 10:23 | |
| *** talluri has quit IRC | 10:32 | |
| *** talluri has joined #openstack-infra | 10:32 | |
| *** talluri has quit IRC | 10:33 | |
| sdague | jhesketh: you happen to be up? | 10:33 |
| *** BobBall has joined #openstack-infra | 10:35 | |
| *** talluri has joined #openstack-infra | 10:41 | |
| *** e0ne_ has quit IRC | 10:54 | |
| *** Hal_ has joined #openstack-infra | 10:54 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Add slukjanov to users https://review.openstack.org/91012 | 10:56 |
| jhesketh | sdague: roger | 11:00 |
| sdague | jeblair suggested that you might have thoughts on why the depends on zuul patch doesn't quite work in the no-op case - https://review.openstack.org/#/c/94365/ | 11:01 |
| jhesketh | sure, I'll take a look in a bit | 11:01 |
| openstackgerrit | Lukas Bednar proposed a change to openstack-infra/jenkins-job-builder: scms: MultiSCM is generated for scms hidden in macro https://review.openstack.org/69827 | 11:02 |
| *** dstanek_zzz is now known as dstanek | 11:04 | |
| *** blamar has quit IRC | 11:04 | |
| *** flaper87|afk is now known as flaper87 | 11:05 | |
| derekh_ | Is anybody around who can promote https://review.openstack.org/#/c/94799/ in the tripleo-check queue | 11:11 |
| *** e0ne has joined #openstack-infra | 11:13 | |
| sdague | derekh_: honestly, I'd suggest just manually sniping | 11:17 |
| sdague | derekh_: sorry, early coffee, check doesn't have priorities | 11:17 |
| *** zul has joined #openstack-infra | 11:18 | |
| derekh_ | sdague: ahh so it can't be done, how about just clearing the queue and I can recheck it | 11:19 |
| *** miqui has quit IRC | 11:20 | |
| *** andreykurilin_ has joined #openstack-infra | 11:20 | |
| *** vkdrao has joined #openstack-infra | 11:21 | |
| sdague | derekh_: I don't know (also don't have the power), but it's not something we do for the regular queues, so I don't think it would be policy to do it on tripleo | 11:21 |
| *** vkdrao has quit IRC | 11:21 | |
| sdague | probably worth telling folks to stop pushing patches until your backlog can be chewed through | 11:21 |
| *** ihrachyshka has joined #openstack-infra | 11:22 | |
| sdague | have you figured out why the queues there are largely stalled? | 11:23 |
| derekh_ | sdague: fair enough if you don't have the power, i'm pretty sure its something lifeless has done in the past, the problem if all of the backlog will have to be recheck anyways untill that patch is merged | 11:23 |
| derekh_ | sdague: as we have a breaking job | 11:23 |
| derekh_ | sdague: one of our regions is not operating and pretty much waiting on a redeploy (which is waiting on us to find/fix a bug in trusty) | 11:24 |
| *** matsuhas_ has quit IRC | 11:27 | |
| *** flaper87 is now known as flaper87|afk | 11:27 | |
| *** prad has quit IRC | 11:28 | |
| *** _nadya_ has quit IRC | 11:34 | |
| *** ihrachyshka has quit IRC | 11:36 | |
| *** ihrachyshka has joined #openstack-infra | 11:36 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: add tempest jobs to dib and os-* tools https://review.openstack.org/94827 | 11:37 |
| *** dims has joined #openstack-infra | 11:53 | |
| *** e0ne has quit IRC | 11:54 | |
| *** mwagner_lap has quit IRC | 11:54 | |
| *** e0ne has joined #openstack-infra | 11:56 | |
| *** cdub has quit IRC | 11:58 | |
| *** zehicle_at_dell has quit IRC | 12:02 | |
| *** zehicle_at_dell has joined #openstack-infra | 12:02 | |
| *** jgallard has joined #openstack-infra | 12:03 | |
| *** dhellman_ has joined #openstack-infra | 12:03 | |
| *** dhellman_ has quit IRC | 12:04 | |
| *** dhellman_ has joined #openstack-infra | 12:04 | |
| *** eharney has quit IRC | 12:06 | |
| *** locke105 has quit IRC | 12:06 | |
| *** bnemec has quit IRC | 12:09 | |
| *** locke105 has joined #openstack-infra | 12:09 | |
| *** bnemec has joined #openstack-infra | 12:09 | |
| jhesketh | sdague: what makes you think it's the noop that's failing? | 12:10 |
| *** mbacchi has joined #openstack-infra | 12:11 | |
| pbelanyi | how can I check what packages are installed on the machines running gate tests? | 12:13 |
| pbelanyi | specifically I would like to know if node.js and npm are generally available | 12:13 |
| *** dhellman_ has quit IRC | 12:13 | |
| *** jgriffith has quit IRC | 12:16 | |
| *** weshay has joined #openstack-infra | 12:17 | |
| *** jgriffith has joined #openstack-infra | 12:17 | |
| sdague | jhesketh: so it's not that no-op is failing, it's that the change as written should degrade to the noop case, which should mean no change in the existing tests. However that's clearly not true. | 12:17 |
| *** flaper87|afk is now known as flaper87 | 12:18 | |
| *** michchap_ has quit IRC | 12:18 | |
| sdague | pbelanyi: they are built by devstack, so the end of a devstack run will tell you. node.js is not there as far as I know | 12:19 |
| jhesketh | right, but the test that is failing is test_client_promote_dependent | 12:19 |
| sdague | jhesketh: correct | 12:19 |
| sdague | which is where I'm bashing my head, because I really thought I had degraded this back to old behavior correctly | 12:19 |
| jhesketh | so there's something odd happening on re-enqueuing | 12:20 |
| pbelanyi | sdague: thanks for the pointer, I'll check | 12:20 |
| sdague | pbelanyi: what's the use case for nodejs in the gate? | 12:20 |
| *** pdmars has joined #openstack-infra | 12:20 | |
| sdague | jhesketh: yep, definitely | 12:20 |
| pbelanyi | sdague: we would like to use jshint to check js code in horizon and tuskar-ui | 12:21 |
| pbelanyi | sdague: unfortunatelly jshint can not be included in the projects due to licensing issues, but it's possible to install and use it with npm | 12:21 |
| *** aysyd has joined #openstack-infra | 12:21 | |
| *** pblaho has quit IRC | 12:22 | |
| jpich | pbelanyi: I think rdopieralski was looking into it a couple of days ago, he found the js testing env was setup via tox IIRC | 12:22 |
| sdague | jhesketh: the tests for that seem to do some non standard things in the way they specify dependencies as well, I'm not sure if that's vestigial or not | 12:22 |
| sdague | jhesketh: https://github.com/openstack-infra/zuul/blob/master/tests/test_scheduler.py#L109 is a thing I'm a little concerned on | 12:23 |
| *** _nadya_ has joined #openstack-infra | 12:23 | |
| sdague | because as far as I can tell that's not in the actual model any more | 12:23 |
| jhesketh | right | 12:23 |
| jhesketh | let me publish my comments that highlight that in the model | 12:24 |
| pbelanyi | jpich: well, he sent me this https://github.com/openstack-infra/storyboard-webclient/blob/master/tox.ini | 12:24 |
| sdague | ok, cool | 12:24 |
| pbelanyi | jpich: hmm, that depends on nodeenv... I should check, maybe that installs node.js and npm | 12:25 |
| jpich | pbelanyi: Ah, fair enough! Then you probably already know more than I do :) | 12:25 |
| jhesketh | sdague: https://review.openstack.org/#/c/94365/ They are mostly nits as I haven't dug towards the bug yet | 12:25 |
| jhesketh | (it's late here) | 12:25 |
| sdague | jhesketh: no worries, appreciate the extra eyes | 12:25 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/gerritlib: Enhance README.rst file https://review.openstack.org/94844 | 12:26 |
| *** malini_afk is now known as malini | 12:26 | |
| *** dhellman_ has joined #openstack-infra | 12:26 | |
| jhesketh | sdague: so this probably needs to be Dependencies() and appended to the list https://github.com/openstack-infra/zuul/blob/master/tests/test_scheduler.py#L313 | 12:26 |
| sdague | jhesketh: possibly, my concern is this - https://github.com/openstack-infra/zuul/blob/master/tests/test_scheduler.py#L335 | 12:27 |
| *** _nadya_ has quit IRC | 12:27 | |
| sdague | which is an interface this object definitely doesn't implement | 12:27 |
| *** pcm__ has joined #openstack-infra | 12:28 | |
| *** cgoncalv1s is now known as cgoncalves | 12:28 | |
| *** cgoncalves has joined #openstack-infra | 12:28 | |
| jhesketh | actually this is a fake change, so it should only depend on one item like a gerrit change would | 12:28 |
| sdague | yep, in the current iteration | 12:28 |
| *** michchap has joined #openstack-infra | 12:29 | |
| *** dstanek is now known as dstanek_zzz | 12:30 | |
| sdague | maybe I just need to add the needs_change in addition? Though I'm not sure how that would matter as that attribute doesn't exist in the FakeChange anyway | 12:30 |
| jhesketh | I'm not sure if the depends_on_change is a red herring | 12:31 |
| *** nosnos has quit IRC | 12:31 | |
| *** dhellman_ has quit IRC | 12:32 | |
| jhesketh | it seems like _doPromoteEvent in scheduler doesn't care for dependencies | 12:32 |
| *** markmcclain has joined #openstack-infra | 12:32 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-dev/cookiecutter: Add .gitreview generation https://review.openstack.org/94848 | 12:34 |
| *** rdopieralski has joined #openstack-infra | 12:34 | |
| *** _nadya_ has joined #openstack-infra | 12:36 | |
| *** _nadya_ has quit IRC | 12:37 | |
| *** dprince has joined #openstack-infra | 12:41 | |
| *** blamar has joined #openstack-infra | 12:42 | |
| *** dstanek_zzz is now known as dstanek | 12:45 | |
| *** hdd_ has joined #openstack-infra | 12:45 | |
| *** jgrimm has joined #openstack-infra | 12:46 | |
| *** rfolco has joined #openstack-infra | 12:46 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/gerritlib: Apply cookiecutter defaults https://review.openstack.org/94844 | 12:47 |
| sdague | jhesketh: yeh, I wonder how it worked before | 12:48 |
| jhesketh | yeah I was thinking the same thing.. it seems like you've uncovered a bug | 12:48 |
| jhesketh | that already existed that is | 12:49 |
| sdague | ok, interesting. | 12:50 |
| sdague | I'm definitely struggling a little with the fact that all the zuul tests are integration tests, so it's hard to grok if I correctly replicated an existing interface | 12:50 |
| jhesketh | sdague: ah, line 1119 of scheduler.py | 12:50 |
| jhesketh | I think you need to switch that to areAllChangesAlreadyInQueue | 12:51 |
| sdague | oh, good catch | 12:51 |
| jhesketh | actually I'm not so sure... | 12:52 |
| * jhesketh looks closer | 12:52 | |
| sdague | that moves the failure to a different test | 12:53 |
| *** julim has joined #openstack-infra | 12:54 | |
| jhesketh | so the addChange method is just checking if itself is in the queue... it should probably also check that its dependencies are too | 12:54 |
| openstackgerrit | Thierry Carrez proposed a change to openstack-infra/release-tools: Workaround LP timeouts in consolidate_page.py https://review.openstack.org/94852 | 12:54 |
| sdague | yeh, this is good though, at least I'm not fully stuck and have something to poke at. | 12:55 |
| sdague | ok, I need to make an emergency coffee run, as we ran out in the house back on in a bit | 12:55 |
| sdague | thanks for the help | 12:55 |
| jhesketh | heh, fair enough | 12:55 |
| jhesketh | happy to :-) | 12:55 |
| jhesketh | so I'm still not sure how your changes stopped it from working though | 12:55 |
| jhesketh | but it's 11pm here so I might return to it in the Morning | 12:56 |
| *** miqui has joined #openstack-infra | 12:57 | |
| anteaya | jhesketh: I'm surprised to see you still up | 12:57 |
| anteaya | you must have coffee at your house | 12:57 |
| jhesketh | oh I'm definitely a night person | 12:57 |
| jhesketh | heh | 12:57 |
| sdague | :) | 12:57 |
| jhesketh | but I'm still a bit jet lagged so I haven't adjusted my productive period back to midnight yet | 12:57 |
| jhesketh | plus distracted by TV | 12:58 |
| anteaya | ah | 12:58 |
| anteaya | the benefits of TV | 12:58 |
| *** thomasbiege has joined #openstack-infra | 12:58 | |
| *** lcostantino has joined #openstack-infra | 12:58 | |
| anteaya | when I was in Utah they had on Leave it to Beaver, I haven't seen that show in years | 12:59 |
| *** chuck__ has joined #openstack-infra | 12:59 | |
| anteaya | so I am looking at the status page, I haven't read all scrollback so sorry if this is a repeat | 12:59 |
| anteaya | we seem to be doing a _lot_ of deleted of nodes | 12:59 |
| anteaya | and check tripleo doesn't appear terribly happy | 12:59 |
| *** _dguerri has quit IRC | 12:59 | |
| *** afazekas is now known as afazekas_mtg | 12:59 | |
| anteaya | s/deleted/deleting | 13:00 |
| *** dguerri has joined #openstack-infra | 13:01 | |
| *** aveiga has quit IRC | 13:03 | |
| *** timrc is now known as timrc-afk | 13:04 | |
| *** afazekas_mtg has quit IRC | 13:04 | |
| *** radez has joined #openstack-infra | 13:05 | |
| radez | anyone have any information on the trystack website in rackspace's dc? I need to get a website update pushed out but not sure the process to get it updated | 13:05 |
| fungi | radez: the trystack.org website? | 13:07 |
| radez | fungi: yup | 13:07 |
| *** skolekonov_ has joined #openstack-infra | 13:08 | |
| *** skolekonov_ has quit IRC | 13:08 | |
| *** skolekonov has quit IRC | 13:08 | |
| radez | fungi: annegentle made an update in the gitrepo and we'd like it to land on the server | 13:08 |
| *** skolekonov has joined #openstack-infra | 13:08 | |
| *** zz_gondoi is now known as gondoi | 13:08 | |
| *** thedodd has joined #openstack-infra | 13:10 | |
| anteaya | sdague: I'm reading scrollback, I fought with message taking a string and lost, the best I have is using +'s between words: https://review.openstack.org/#/q/status:open+project:openstack-infra/config+message:specs%252Bhello%252Bgood%252Bbye,n,z | 13:10 |
| *** thomasbiege has quit IRC | 13:10 | |
| anteaya | sdague: also the use of hyphen's gets interpreted as whitespace | 13:11 |
| *** changbl has quit IRC | 13:11 | |
| fungi | radez: my guess is that you need to get in touch with fifieldt, reed, toddmorey or jbryce for it. seems to be a rackspace cloudsites site... not sure who they've delegated ftp access to though | 13:12 |
| radez | fungi: thx for the info | 13:12 |
| fungi | radez: i see someone added a "trystack" ftp account to that site, so presumably they handed that login info to someone who was taking care of the content | 13:13 |
| radez | perfect, I'll ask around and see if I can figure out who has it | 13:13 |
| fungi | radez: you might try contacting trystack@openstack.org (so sayeth the trystack webpage) | 13:14 |
| radez | fungi: haha, I don't thikn that email is active | 13:14 |
| fungi | nice | 13:14 |
| radez | annegentle_: that's another update we shoudl make ^^^ take off that email if it doesn't go anywhere | 13:14 |
| annegentle_ | fungi: oh maybe I have that | 13:15 |
| annegentle_ | huh | 13:15 |
| fungi | radez: but yeah, openstack infra doesn't have any automation i can see for publishing to that site | 13:15 |
| annegentle_ | fungi: I have the docs passwords, maybe I should have that one too | 13:15 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Remove rename call in log cleanup https://review.openstack.org/92981 | 13:15 |
| *** eharney has joined #openstack-infra | 13:15 | |
| *** signed8bit has joined #openstack-infra | 13:16 | |
| annegentle_ | fungi: yes there's no automation | 13:16 |
| *** bknudson has left #openstack-infra | 13:16 | |
| annegentle_ | radez: I'm sure that email goes somewhere | 13:16 |
| *** mriedem has joined #openstack-infra | 13:17 | |
| radez | annegentle_: :) somone said they tried to email it recently... I'm gonna give it a while | 13:17 |
| radez | *whirl | 13:17 |
| *** smarcet has joined #openstack-infra | 13:17 | |
| annegentle_ | radez: if I have that password it's on another computer | 13:18 |
| annegentle_ | radez: I can look at home tonight | 13:18 |
| radez | annegentle_: okey doke | 13:18 |
| radez | The error that the other server returned was: | 13:18 |
| radez | 550 5.7.1 <trystack@openstack.org>: Relay access denied. | 13:18 |
| radez | annegentle_: ^ | 13:18 |
| annegentle_ | radez: ok | 13:18 |
| radez | jsut sent an email to trystack@ and it died :( | 13:18 |
| *** afazekas_mtg has joined #openstack-infra | 13:19 | |
| annegentle_ | radez: back in Nov last year I still had the ability to update the site so I'll see if my old computer has the password | 13:19 |
| *** thedodd has quit IRC | 13:19 | |
| radez | annegentle_: haha, has it been that long since we made an update? | 13:19 |
| annegentle_ | radez: yes | 13:19 |
| *** MarkAtwood has joined #openstack-infra | 13:21 | |
| *** gondoi is now known as zz_gondoi | 13:22 | |
| *** zehicle_at_dell has quit IRC | 13:22 | |
| *** ArxCruz has quit IRC | 13:24 | |
| *** wenlock has joined #openstack-infra | 13:25 | |
| *** mfer has joined #openstack-infra | 13:25 | |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/gerritlib: Doc generation support https://review.openstack.org/94861 | 13:26 |
| *** ArxCruz has joined #openstack-infra | 13:27 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Configure direct download of files to browser https://review.openstack.org/93108 | 13:27 |
| *** wenlock has quit IRC | 13:29 | |
| anteaya | fungi: any thoughts on what all the deleting is about? | 13:30 |
| anteaya | in the node graph | 13:30 |
| phschwartz | Anyone have the url for the irc logs handy | 13:32 |
| *** sandywalsh_ has joined #openstack-infra | 13:32 | |
| *** talluri has quit IRC | 13:32 | |
| phschwartz | I missed something in scroll back, but my browser lost it from its history | 13:32 |
| anteaya | http://eavesdrop.openstack.org/ | 13:32 |
| *** talluri has joined #openstack-infra | 13:32 | |
| *** markmcclain1 has joined #openstack-infra | 13:32 | |
| phschwartz | ah, there is my issue. I miss spelled it | 13:32 |
| anteaya | that would do it | 13:33 |
| *** markmcclain2 has joined #openstack-infra | 13:33 | |
| *** markmcclain2 has quit IRC | 13:33 | |
| *** markmcclain has quit IRC | 13:33 | |
| fungi | anteaya: http://paste.openstack.org/show/81149/ (hpcloud-region-b nodes are all in delete state) | 13:33 |
| *** markmcclain has joined #openstack-infra | 13:33 | |
| anteaya | wooooo | 13:34 |
| anteaya | region-b might be our new az-2 | 13:35 |
| anteaya | in terms of needing attention | 13:35 |
| *** bknudson has joined #openstack-infra | 13:37 | |
| *** talluri has quit IRC | 13:37 | |
| fungi | they're all marked for deletion for 18-21 hours, so it's possible nodepool is stuck communication with that region | 13:37 |
| *** markmcclain1 has quit IRC | 13:38 | |
| fungi | communicating | 13:38 |
| *** r-daneel has joined #openstack-infra | 13:38 | |
| anteaya | well that would be something we could do something about | 13:38 |
| anteaya | why just one region though, I wonder? | 13:38 |
| fungi | could have been network disruption or any of a number of other causes. i might know more once i start digging into the thread dump | 13:39 |
| *** mbacchi has quit IRC | 13:41 | |
| anteaya | kk | 13:41 |
| * anteaya hands fungi a flashlight and rope | 13:42 | |
| openstackgerrit | A change was merged to openstack-infra/config: Configure direct download of files to browser https://review.openstack.org/93108 | 13:44 |
| *** homeless has joined #openstack-infra | 13:45 | |
| *** mbacchi has joined #openstack-infra | 13:46 | |
| fungi | looks surprisingly like when nodepool was hung yesterday talking to the tripleo hp region | 13:47 |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/requirements: Add pyscss and django_pyscss for Horizon https://review.openstack.org/94376 | 13:50 |
| anteaya | hmmmm | 13:51 |
| fungi | though all sockets from nodepool.o.o to region-b.geo-1.identity.hpcloudsvc.com are in a CLOSE_WAIT state, which is somewhat curious | 13:52 |
| fungi | as opposed to region-a.geo-1.identity.hpcloudsvc.com which is in ESTABLISHED | 13:53 |
| fungi | i'll see what happens if i clean up the CLOSE_WAIT sockets | 13:53 |
| SergeyLukjanov | fungi, I've assigned https://bugs.launchpad.net/openstack-ci/+bug/1322154 to myself presuming that I will have access to it soon :) | 13:55 |
| uvirtbot | Launchpad bug 1322154 in openstack-ci "Initial group member for cloudkitty-core" [Low,Confirmed] | 13:55 |
| *** pcrews__ has joined #openstack-infra | 13:55 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/requirements: Add xstatic and xstatic-jquery for Horizon https://review.openstack.org/94337 | 13:56 |
| *** zz_gondoi is now known as gondoi | 13:57 | |
| *** prad has joined #openstack-infra | 13:58 | |
| *** otherwiseguy has joined #openstack-infra | 13:58 | |
| *** blamar has quit IRC | 13:58 | |
| *** gokrokve has joined #openstack-infra | 14:01 | |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: Use pbr, pep8 checks & fixes, add "doc" tox target https://review.openstack.org/90455 | 14:01 |
| *** sandywalsh_ has quit IRC | 14:02 | |
| *** e0ne_ has joined #openstack-infra | 14:08 | |
| *** e0ne__ has joined #openstack-infra | 14:08 | |
| *** e0ne has quit IRC | 14:09 | |
| *** e0ne_ has quit IRC | 14:13 | |
| *** crc32 has joined #openstack-infra | 14:13 | |
| openstackgerrit | ayoung proposed a change to openstack/requirements: request-kerberos https://review.openstack.org/84740 | 14:14 |
| *** Ryan_Lane has joined #openstack-infra | 14:16 | |
| *** otherwiseguy has quit IRC | 14:17 | |
| *** gondoi is now known as zz_gondoi | 14:20 | |
| *** msabramo has quit IRC | 14:21 | |
| *** timrc-afk is now known as timrc | 14:22 | |
| *** afazekas_mtg has quit IRC | 14:24 | |
| *** lcheng_ has joined #openstack-infra | 14:24 | |
| *** david-lyle has joined #openstack-infra | 14:26 | |
| *** matrohon_ has joined #openstack-infra | 14:26 | |
| *** yfried__ has quit IRC | 14:27 | |
| *** afazekas has joined #openstack-infra | 14:29 | |
| *** mayu_ has quit IRC | 14:29 | |
| fungi | killing the old sockets did nothing actually. i wonder whether the fact that we had 400+ servers in a delete state in that provider but a max servers of 200 configured was causing nodepool not to bother reestablishing a connection to the provider because it wasn't going to try to create any new nodes? i'm manually nodepool-deleting the stale nodes in that provider now to see if that wakes it back up | 14:33 |
| fungi | once the total falls below 200 | 14:33 |
| *** zz_gondoi is now known as gondoi | 14:35 | |
| jeblair | fungi: are you sure the connection was dead? | 14:37 |
| jeblair | fungi: oh, yes you are | 14:37 |
| fungi | it was in a close_wait state | 14:37 |
| fungi | and packet captures showed no communication | 14:37 |
| fungi | and rechecking it a few times it was still the same sockets (same source ports on the nodepool side) so it's not like they were cycling | 14:38 |
| fungi | jeblair: there's also a thread dump in the debug log from before i started mass deleting | 14:39 |
| jeblair | fungi: i just did another and it's sitting at return self._sslobj.read(len) | 14:39 |
| *** wenlock has joined #openstack-infra | 14:40 | |
| jeblair | though i think it's running tasks... | 14:40 |
| fungi | what's odd is that, unlike yesterday's situation with tripleo, the api connection was not in an established state | 14:40 |
| jeblair | fungi: so these are http requests which mean they don't necessarily have to stay open.... | 14:41 |
| fungi | good point | 14:41 |
| sdague | anteaya: thanks | 14:41 |
| jeblair | it looks like region-b is running a 'GetImageTask' every second or so | 14:42 |
| jeblair | but nothing else | 14:42 |
| *** doug-fish has joined #openstack-infra | 14:42 | |
| *** _nadya_ has joined #openstack-infra | 14:43 | |
| fungi | the couple of delete tasks i sampled were all sitting waiting on listServers() but digging deeper i see there were probably more waiting in in submitTask(DeleteServerTask()) | 14:45 |
| *** msabramo has joined #openstack-infra | 14:45 | |
| *** yamahata has joined #openstack-infra | 14:46 | |
| jeblair | well, it seems to be creating servers now | 14:46 |
| fungi | around the time the delete node count fell below the max-servers limit for that provider | 14:47 |
| fungi | which makes sense | 14:48 |
| jeblair | how many delete threads were there? | 14:48 |
| jeblair | oh you know what... | 14:48 |
| jeblair | when the config changes, nodepool creates a new provider object | 14:48 |
| jeblair | so the deletes from before the revert revert revert were talking to that provider, but actions after the change should be talking to a new one | 14:49 |
| fungi | i count 1262 occurrences of 'Thread: NodeDeleter for ' in /var/log/nodepool/debug.log | 14:50 |
| jeblair | but the thread dumps only show one provider manager each time... | 14:51 |
| *** james_li has joined #openstack-infra | 14:51 | |
| jeblair | fungi: so i wonder if there's a bug here where the delete threads ended up using the old provider manager after it stopped | 14:51 |
| fungi | sounds like a plausible explanation at any rate\ | 14:52 |
| fungi | oh, my deleter thread count is a bit high because we both triggered stack dumps | 14:53 |
| jeblair | so 1/2 that sounds about right | 14:53 |
| jpich | Is anyone else seeing issues with paste.openstack.org? Someone sent me http://paste.openstack.org/show/81157/ but I seem to get Internal Server Error for every individual paste | 14:53 |
| *** crc32 has quit IRC | 14:54 | |
| fungi | jpich: i think something happened to the tracked cookies when it got moved into a trove database. try deleting all your browser cookies for that site | 14:54 |
| anteaya | sdague: does that help at all? | 14:54 |
| sdague | anteaya: in the general case it would, in my specific case, not really. | 14:54 |
| anteaya | :( | 14:55 |
| jpich | fungi: Did the trick. Thank you! | 14:55 |
| sdague | because I need '-' to be in the string | 14:55 |
| *** otherwiseguy has joined #openstack-infra | 14:55 | |
| anteaya | ah yeah, new gerrit doesn't interpret - as - | 14:55 |
| anteaya | even in quotes | 14:55 |
| anteaya | not sure how to escape it | 14:55 |
| anteaya | it splits on - | 14:55 |
| sdague | yep | 14:56 |
| sdague | such it is | 14:56 |
| jeblair | anteaya sdague: what's the context of your conversation? | 14:56 |
| anteaya | sdague is trying to feed a string to message: in gerrit searches | 14:56 |
| fungi | i believe gerrit tries to do web-style searches and expects "-" to indicate negation (or maybe just ignores it completely) | 14:56 |
| anteaya | gerrit message: doesn't like strings anymore | 14:56 |
| sdague | searching for the Depends-On: string in commit message in zuul was where I tripped on it | 14:56 |
| anteaya | only tokens+with+pluses | 14:56 |
| jeblair | oh | 14:56 |
| *** thedodd has joined #openstack-infra | 14:56 | |
| sdague | but I have a reasonable solution, which is searching for the idempotent id instead | 14:57 |
| jeblair | sdague: i think there's a 'header' search? | 14:57 |
| anteaya | and hyphens aren't hyphens anymore | 14:57 |
| sdague | and filtering | 14:57 |
| anteaya | they are whitespace | 14:57 |
| sdague | jeblair: message: searches the right field | 14:57 |
| sdague | just the tokenizing meant I need to do it in 2 steps | 14:57 |
| jeblair | hrm. at least i thought there was | 14:58 |
| *** otherwiseguy has quit IRC | 14:58 | |
| *** terryw has joined #openstack-infra | 14:58 | |
| *** dkranz has quit IRC | 14:58 | |
| jeblair | fungi: yeah, so once a delete thread starts, it keeps using the same provider object, even if that provider is stopped halfway through the deletion | 14:59 |
| jeblair | fungi: so we have 600 threads waiting on tasks that will never be run | 14:59 |
| *** andreykurilin_ has quit IRC | 14:59 | |
| fungi | that explains why they were all in delete state since a few hours prior to the config change | 15:00 |
| fungi | or between the config change and a few hours before anyway\ | 15:00 |
| anteaya | sdague: can you search for Depends+On ? | 15:01 |
| jeblair | fungi: yeah, and since the instances were all in error and hpcloud was deferring deletes, the delete threads could have run for hours | 15:01 |
| *** lcheng_ has quit IRC | 15:01 | |
| sdague | anteaya: doesn't look like | 15:01 |
| anteaya | :( | 15:01 |
| sdague | it's ok though, I have a solution there | 15:01 |
| anteaya | kk | 15:02 |
| *** zz_julien has left #openstack-infra | 15:06 | |
| *** igor_ has quit IRC | 15:07 | |
| *** moted has joined #openstack-infra | 15:07 | |
| *** zns has joined #openstack-infra | 15:07 | |
| *** igor has joined #openstack-infra | 15:07 | |
| *** mrodden has joined #openstack-infra | 15:09 | |
| *** dripton_ is now known as dripton | 15:10 | |
| *** igor has quit IRC | 15:12 | |
| *** mrodden has quit IRC | 15:12 | |
| *** mestery has quit IRC | 15:14 | |
| sdague | did anyone have a chance to read my -infra email about project local dashboards that could be approved by project cores? | 15:16 |
| sdague | mostly if we think we could build such a workflow | 15:16 |
| *** nati_ueno has joined #openstack-infra | 15:16 | |
| *** UtahDave has joined #openstack-infra | 15:16 | |
| *** ihrachyshka has quit IRC | 15:17 | |
| jeblair | sdague: yep, i read it and didn't immediately see a way to do that, so i thought i'd give it a bit to see if anything popped into my head. it hasn't :( | 15:18 |
| jeblair | sdague: i kind of think the best way to solve that is with better metadata in gerrit; if there were a priority field, or arbitrary tags, we could build searches on that | 15:19 |
| jeblair | sdague: of course reviewday basically exists because gerrit doesn't have that | 15:20 |
| jeblair | sdague: i wonder if you could use reviewday for that? | 15:20 |
| sdague | I was wondering if we could make like a .dashboards directory in the project, then trigger manage-projects on changes to that in a post job | 15:20 |
| sdague | jeblair: the problem with just pointing people at review day, is it doesn't update it's lists in realtime, and it drops you out of the rest of the gerrit interface | 15:21 |
| jeblair | sdague: i'm not sure i love the .dashboards idea -- just doesn't seem like the sort of thing that should be in a project (it's so temporal) | 15:23 |
| jeblair | sdague: oh | 15:23 |
| jeblair | sdague: what about using topics? | 15:23 |
| jeblair | sdague: since they are easy to change now, what if you used them to flag for priority review? | 15:24 |
| jeblair | (you can even change them through the api i believe) | 15:24 |
| sdague | so this was sort of carrot model on getting good topic use | 15:24 |
| sdague | because if we said, this week "blueprint X" is most important to review, we can set the topic there | 15:25 |
| sdague | sorry, we could set which topics we were using the in dashboard to pop things up to the top | 15:25 |
| *** nati_ueno has quit IRC | 15:25 | |
| jeblair | i see | 15:25 |
| *** asselin2 has joined #openstack-infra | 15:26 | |
| jeblair | sdague: so how about instead of setting that as a dashboard, you just email the link to the reviewers? | 15:26 |
| *** asselin2 has quit IRC | 15:26 | |
| sdague | jeblair: because my feeling is that's going to get lost, vs. a stable url | 15:26 |
| *** packet has joined #openstack-infra | 15:27 | |
| sdague | we had general policy decision that we'd try all looking at the same review view during the cycle | 15:27 |
| *** markmcclain has quit IRC | 15:27 | |
| *** zns has quit IRC | 15:27 | |
| sdague | I guess the other question is if we could change the permissions on refs/meta/dashboards for a project to let the cores post to it directly | 15:27 |
| sdague | because I see this as being in the tempest repo | 15:28 |
| sdague | not All-Projects | 15:28 |
| *** igor_ has joined #openstack-infra | 15:28 | |
| *** zns has joined #openstack-infra | 15:28 | |
| *** jooools has quit IRC | 15:29 | |
| *** tteggel has quit IRC | 15:30 | |
| sdague | mriedem: https://review.openstack.org/#/c/92369/ - thoughts on revisiting your -1, it's a blocker on trusty | 15:30 |
| sdague | because trusty ships a six new enough for us not to upgrade it, which explodes | 15:30 |
| *** MarkAtwood has quit IRC | 15:31 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Log stable/icehouse for sahara and remove obsolete https://review.openstack.org/94909 | 15:31 |
| *** chuck__ has quit IRC | 15:32 | |
| *** igor_ has quit IRC | 15:32 | |
| mriedem | sdague: looking.... | 15:32 |
| *** tteggel has joined #openstack-infra | 15:33 | |
| sdague | trusty packaged 1.5.2 fwiw | 15:33 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Refactor delete_jobs https://review.openstack.org/88646 | 15:33 |
| mriedem | sdague: we ship 1.5.2 and haven't seen that issue | 15:33 |
| sdague | I just exploded in grenade run | 15:34 |
| sdague | on cinder db sync | 15:34 |
| msabramo | zaro: Here ya go - https://review.openstack.org/88646 | 15:34 |
| sdague | I've force installed over it because I needed to get to the next real issue here | 15:35 |
| *** ominakov has quit IRC | 15:35 | |
| mriedem | sdague: just removed my score since idk | 15:37 |
| mriedem | and don't have time to know | 15:37 |
| sdague | ok, works for me | 15:37 |
| mriedem | thanks for at least doubling back :) | 15:37 |
| *** blamar has joined #openstack-infra | 15:38 | |
| sdague | yep, no prob | 15:38 |
| *** igor_ has joined #openstack-infra | 15:38 | |
| sdague | ok, I'm going to +A then, as it has 2 +2s | 15:39 |
| *** msabramo has quit IRC | 15:39 | |
| *** igor__ has joined #openstack-infra | 15:40 | |
| *** flaper87 is now known as flaper87|afk | 15:41 | |
| *** duran has joined #openstack-infra | 15:42 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Rename climate to blazar https://review.openstack.org/94912 | 15:43 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Remove obsolete #openstack-climate channel https://review.openstack.org/94913 | 15:43 |
| *** igor_ has quit IRC | 15:43 | |
| SergeyLukjanov | jeblair, do we already have changes for specs renaming? (I can do them if not) | 15:43 |
| *** igor__ has quit IRC | 15:44 | |
| jeblair | SergeyLukjanov: no, that would be great (i don't think we have changes for any of the renames) | 15:45 |
| SergeyLukjanov | jeblair, okay, I'll prepare CRs for renaming | 15:46 |
| *** dstanek is now known as dstanek_zzz | 15:47 | |
| *** yfried__ has joined #openstack-infra | 15:49 | |
| *** msabramo has joined #openstack-infra | 15:49 | |
| *** reed has joined #openstack-infra | 15:50 | |
| ttx | hrm. my "git-review -d 94330" call fails with ssh: Could not resolve hostname : Name or service not known | 15:52 |
| ttx | it's arguably the first time I use git-review -d with newgerrit though, so maybe I overlooked something | 15:53 |
| jeblair | ttx: has 'get review -s' been run? | 15:53 |
| jeblair | git review even | 15:53 |
| fungi | ttx: what does git review --version report? | 15:53 |
| ttx | it has been run a long time ago | 15:53 |
| ttx | git-review version 1.23 | 15:53 |
| *** mrodden has joined #openstack-infra | 15:54 | |
| pleia2 | good morning | 15:54 |
| fungi | ttx: and what url do you have for the gerrit remotes reported by git remote -v ? | 15:54 |
| ttx | The following command failed with exit code 255 | 15:54 |
| ttx | "ssh -x gerrit query --format=JSON --current-patch-set change:94330" | 15:54 |
| ttx | that space between -x and gerrit is a bit... concerning | 15:54 |
| jeblair | fungi: (unrelated: 93161 is ready for your attention) | 15:54 |
| ttx | gerritssh://ttx@review.openstack.org:29418/openstack-infra/storyboard-webclient.git (fetch) | 15:54 |
| ttx | gerritssh://ttx@review.openstack.org:29418/openstack-infra/storyboard-webclient.git (push) | 15:54 |
| fungi | ttx: and are you sure 94330 is a change number for the project corresponding to the git repository you're in? | 15:55 |
| * fungi checks | 15:55 | |
| ttx | https://review.openstack.org/#/c/94330 is a openstack-infra/storyboard-webclient change for sure | 15:55 |
| fungi | yeah, looks right | 15:55 |
| *** mestery has joined #openstack-infra | 15:55 | |
| *** mestery has quit IRC | 15:56 | |
| jeblair | 2014-05-22 08:55:57.481275 Running: ssh -xp29418 corvus@review.openstack.org gerrit query --format=JSON --current-patch-set change:94330 | 15:56 |
| jeblair | is what i get for that | 15:56 |
| *** mestery has joined #openstack-infra | 15:56 | |
| jeblair | ttx: run "git review -v -d 94330" and paste the output | 15:56 |
| *** hdd_ has quit IRC | 15:56 | |
| *** ihrachyshka has joined #openstack-infra | 15:56 | |
| ttx | jeblair: http://paste.ubuntu.com/7501943/ | 15:57 |
| *** doude has quit IRC | 15:57 | |
| ttx | oh. oh. | 15:58 |
| fungi | yeah, for some reason it's leaving the review.openstack.org hostname and port number out of the ssh call | 15:58 |
| *** jpeeler has quit IRC | 15:58 | |
| ttx | hah! got it | 15:58 |
| sdague | ttx bad .gitreview ? | 15:58 |
| ttx | no no. | 15:58 |
| fungi | "Found origin Push URL:" is probably the cause | 15:58 |
| ttx | "LANG=C git-review -d 94330" works. | 15:58 |
| fungi | huh | 15:58 |
| ttx | damn Ubuntu 14.04 has localized git output | 15:59 |
| sdague | hah | 15:59 |
| *** lcheng_ has joined #openstack-infra | 15:59 | |
| ttx | here is what git remote show gives: | 15:59 |
| ttx | http://paste.ubuntu.com/7501955/ | 15:59 |
| fungi | ttx: please test master of git-review if you have a moment and see if it's fixed. there have been several localization-related fixups merged recently. we're probably overdue for a release | 15:59 |
| ttx | "URL de rapatriement" :) | 15:59 |
| sdague | yeh, that's the problem with shell as a stable interface... all those people that configure their computers to not be english ;) | 16:00 |
| fungi | bwahaha | 16:00 |
| ttx | i suspect you grep on an english string | 16:00 |
| *** bknudson has quit IRC | 16:00 | |
| ttx | so yeah, it's the first time I run git review -d on my upgraded Ubuntu french setup | 16:01 |
| fungi | ttx: right, in master i believe we're setting the locale to c anywhere shell commands are being called | 16:01 |
| *** hashar has quit IRC | 16:01 | |
| msabramo | zaro: Python 3 support for python-jenkins is just a click away :-) https://review.openstack.org/90114 | 16:01 |
| fungi | ttx: so feedback on the upcoming release from someone working in a non-en environment would be very helpful | 16:02 |
| *** jpeeler has joined #openstack-infra | 16:02 | |
| *** jpeeler has quit IRC | 16:02 | |
| *** jpeeler has joined #openstack-infra | 16:02 | |
| ttx | fungi: what's the most convenient way of testing without overwriting my pip install ? clone git-review repo and python setup.py develop ? | 16:02 |
| *** ihrachyshka has quit IRC | 16:03 | |
| *** ihrachyshka has joined #openstack-infra | 16:03 | |
| *** _nadya_ has quit IRC | 16:03 | |
| fungi | ttx: i create a virtualenv, pip install . the checkout into it, and have a symlink from ~/bin/git-review into that virtualenv | 16:03 |
| fungi | about the only thing you miss out on that way is testing whether --help is working | 16:04 |
| fungi | since it relies on manpages being installed... though i could probably link ~/man/git-review.1 to the correct file in the venv too | 16:04 |
| fungi | assuming my manpath is configured accordingly | 16:05 |
| *** bknudson has joined #openstack-infra | 16:05 | |
| *** rockyg has joined #openstack-infra | 16:05 | |
| * ttx tries | 16:06 | |
| *** marcoemorais has joined #openstack-infra | 16:06 | |
| *** virmitio has joined #openstack-infra | 16:07 | |
| openstackgerrit | A change was merged to openstack-infra/config: Initial commit to add Aviator to Stackforge https://review.openstack.org/93663 | 16:08 |
| ttx | fungi: yes, master works | 16:08 |
| *** hogepodge has joined #openstack-infra | 16:08 | |
| *** rcarrillocruz has quit IRC | 16:08 | |
| fungi | ttx: great. all the more impetus for me to finish slogging through the outstanding fixes in review and stabilize for a new release | 16:08 |
| *** yfried__ has quit IRC | 16:09 | |
| *** gokrokve has quit IRC | 16:10 | |
| *** jgallard has quit IRC | 16:10 | |
| fungi | derekh_: if you're around, i've got a couple of undeletable instances in tripleo-test-cloud (e.g. "r1")... c553c607-f16f-4804-9828-678e45539686 and 499f8bb2-a1b8-4bb9-988a-4832f4802e74 | 16:11 |
| openstackgerrit | A change was merged to openstack-infra/config: Allow the neutron functional job to gate merges https://review.openstack.org/93161 | 16:11 |
| openstackgerrit | A change was merged to openstack-infra/git-review: Use gerrit 2.8.5 instead of gerrit 2.6.1 in tests https://review.openstack.org/93951 | 16:12 |
| *** gokrokve has joined #openstack-infra | 16:12 | |
| *** matrohon_ has quit IRC | 16:13 | |
| *** amcrn has joined #openstack-infra | 16:13 | |
| *** nati_ueno has joined #openstack-infra | 16:14 | |
| *** rcarrillocruz has joined #openstack-infra | 16:14 | |
| *** zns has quit IRC | 16:14 | |
| fungi | derekh_: also, it looks like we're still requesting 16384m ram when building an instance there, not 8192m... can you confirm we're actually building with the new flavor you created? | 16:15 |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Small refactor for preferences https://review.openstack.org/94332 | 16:15 |
| *** dstanek_zzz is now known as dstanek | 16:16 | |
| *** Hal_ has quit IRC | 16:16 | |
| *** atiwari has joined #openstack-infra | 16:16 | |
| *** esker has joined #openstack-infra | 16:16 | |
| *** gyee has joined #openstack-infra | 16:17 | |
| *** mwagner_lap has joined #openstack-infra | 16:20 | |
| clarkb | morning | 16:20 |
| *** zns has joined #openstack-infra | 16:20 | |
| clarkb | fungi: there is a change up to fix ttx's git review problem | 16:20 |
| *** zns has quit IRC | 16:20 | |
| *** rdopieralski has quit IRC | 16:21 | |
| clarkb | it sets LANG=C during git commands | 16:21 |
| openstackgerrit | A change was merged to openstack-infra/release-tools: Workaround LP timeouts in consolidate_page.py https://review.openstack.org/94852 | 16:21 |
| anteaya | morning clarkb | 16:21 |
| *** zns has joined #openstack-infra | 16:21 | |
| *** afazekas has quit IRC | 16:22 | |
| *** gokrokve has quit IRC | 16:24 | |
| *** yfried__ has joined #openstack-infra | 16:26 | |
| *** mestery has quit IRC | 16:27 | |
| clarkb | I have realized that I approved a bunch of specs repos and they need group membership to be done /me does that | 16:29 |
| *** mestery has joined #openstack-infra | 16:30 | |
| *** arnaud has joined #openstack-infra | 16:30 | |
| clarkb | hrm groups don't exist maybe project creation is really broken? | 16:30 |
| clarkb | no nevermind identity-specs exists | 16:30 |
| *** Hal_ has joined #openstack-infra | 16:31 | |
| *** alugovoi has joined #openstack-infra | 16:31 | |
| clarkb | looks like all are good except baremetal specs | 16:32 |
| *** wenlock has quit IRC | 16:33 | |
| clarkb | that explains it, there was no provided ACL file for baremetal-specs | 16:33 |
| *** wenlock has joined #openstack-infra | 16:33 | |
| clarkb | devananda: ^ | 16:33 |
| devananda | ahh | 16:34 |
| devananda | clarkb: whatcha need me to do? | 16:34 |
| clarkb | devananda: decide who should have +2 on baremetal-specs | 16:34 |
| clarkb | I can write a change to make it so | 16:34 |
| clarkb | some projects are using their core team, other projects are using a dedicated group | 16:34 |
| *** Hal_ has quit IRC | 16:36 | |
| clarkb | pbelanyi: jpich: yes for linting you shouldn't depend on devstack, thats a bit overkill. I would do something similar to what storyboard does instead | 16:36 |
| clarkb | devananda: and you can always change it later if needed | 16:36 |
| devananda | devananda: great - start with me, and I'll add folks later today | 16:37 |
| clarkb | I need to AFK at some point this morning to go shopping for new glasses btw | 16:37 |
| clarkb | devananda: ok so you want a dedicated group then? | 16:37 |
| clarkb | and not ironic-core for example? | 16:38 |
| *** praneshp has joined #openstack-infra | 16:38 | |
| devananda | yea. i beleive that's what i specified in the project creation | 16:38 |
| *** zns has quit IRC | 16:38 | |
| clarkb | devananda: you didn't specify anything | 16:38 |
| clarkb | which is why it has no ACLs | 16:38 |
| devananda | oh | 16:38 |
| clarkb | oh wait now I see | 16:39 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Added priorities to tasks https://review.openstack.org/91675 | 16:39 |
| clarkb | you made a file but the implicit lookup broke because ironic-specs != baremetal-specs | 16:39 |
| clarkb | I can fix that | 16:39 |
| *** jergerber has joined #openstack-infra | 16:39 | |
| openstackgerrit | Boris Pavlovic proposed a change to openstack-infra/config: Add missing export -f post_test_hook in rally-cli https://review.openstack.org/94601 | 16:39 |
| devananda | oh! right. i did a rename, but seems i missed that | 16:39 |
| *** blamar has quit IRC | 16:40 | |
| *** igor__ has joined #openstack-infra | 16:40 | |
| *** crc32 has joined #openstack-infra | 16:41 | |
| derekh_ | fungi: sorry am on a call but took a quick look, on R1 h1.large is still being used on R2 your using the new flavour for all but one instance | 16:41 |
| derekh_ | fungi: I'll reset those two instances now | 16:41 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Rename ironic-specs to baremetal-specs to fix ACLs https://review.openstack.org/94930 | 16:41 |
| clarkb | devananda: jeblair fungi ^ that should do it | 16:41 |
| *** duran has quit IRC | 16:42 | |
| derekh_ | fungi: done | 16:43 |
| *** arnaud has quit IRC | 16:43 | |
| fungi | derekh_: i wonder if our flavor filter is causing it to still select that one | 16:44 |
| *** arnaud has joined #openstack-infra | 16:44 | |
| *** zns has joined #openstack-infra | 16:44 | |
| fungi | i'll poke around in the account real quick | 16:44 |
| clarkb | fungi: nodepool bug? | 16:44 |
| fungi | clarkb: could just be a flavor name issue | 16:44 |
| clarkb | I know nodepool does processing of the flavor list by creating an ordered list of flavors | 16:44 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: Updated documentation to include instructions for prod-proxy mode. https://review.openstack.org/94279 | 16:45 |
| *** mestery has quit IRC | 16:45 | |
| *** e0ne has joined #openstack-infra | 16:45 | |
| *** igor__ has quit IRC | 16:45 | |
| *** lcheng_ has quit IRC | 16:46 | |
| *** esker has quit IRC | 16:46 | |
| *** esker has joined #openstack-infra | 16:47 | |
| fungi | so currently in that tenant we have a new h1.ci flavor with 8192 memory and an old h1.large flavor with 16384 memory | 16:48 |
| openstackgerrit | A change was merged to openstack-infra/storyboard-webclient: Some project ui/ux updates https://review.openstack.org/91196 | 16:48 |
| *** e0ne__ has quit IRC | 16:48 | |
| fungi | nodepool.yaml is looking for min-ram:8192 and name-filter:h1 which should theoretically cause it to select the h1.ci flavor | 16:49 |
| *** zns has quit IRC | 16:50 | |
| fungi | but the subsequent image update there seems to have continued to use the h1.large flavor instead | 16:50 |
| *** shardy is now known as shardy_afk | 16:50 | |
| *** jpich has quit IRC | 16:51 | |
| *** esker has quit IRC | 16:51 | |
| *** markmcclain has joined #openstack-infra | 16:52 | |
| clarkb | fungi: reading sb looks like potential provider bug in nodepool when reloading configs. Was that the same problem with tripleo cloud? | 16:52 |
| fungi | perhaps changes to min-ram don't cause the provider object to get recreated, and the previous choice of flavor stays cached as a result? | 16:52 |
| clarkb | er hp tripleo cloud | 16:52 |
| *** dstanek is now known as dstanek_zzz | 16:53 | |
| fungi | clarkb: symptoms seem entirely different, but could be related somehow i guess | 16:53 |
| clarkb | fungi: I am missing what the symptoms were for tripleo | 16:53 |
| clarkb | I got the connection state was different was that it? | 16:53 |
| fungi | clarkb: depends on which symptoms you're talking about | 16:53 |
| *** harlowja_away is now known as harlowja_ | 16:54 | |
| *** markmcclain1 has joined #openstack-infra | 16:54 | |
| clarkb | fungi: well in particular lots of nodes that nodepool isn't deleting | 16:54 |
| *** markmcclain1 has quit IRC | 16:54 | |
| fungi | clarkb: the symptoms we were looking into yesterday or what i'm looking at now? | 16:54 |
| fungi | ahh, that | 16:54 |
| *** markmcclain1 has joined #openstack-infra | 16:54 | |
| clarkb | flavors are maintained by ProviderManager and we create a new one according to jeblair in sb when the config reloads so I don't think we should be caching the flavor | 16:55 |
| fungi | clarkb: yesterday it seemed that there was a hung established connection to the provider endpoint and that it was stuck indefinitely waiting on the results of a nova list query | 16:55 |
| clarkb | ah ok so it was that thing again | 16:55 |
| fungi | yeah, for tripleo yesterday, causing the established connection to terminate woke things back up | 16:56 |
| clarkb | so that would be different | 16:56 |
| *** sarob has joined #openstack-infra | 16:56 | |
| fungi | today's symptom with hpcloud-region-b was nodepool not bothering to do anything in that provider because it already had too many nodes there, but all the delete threads were for a no longer valid provider object since the reconfiguration | 16:57 |
| openstackgerrit | A change was merged to openstack-infra/config: Rename ironic-specs to baremetal-specs to fix ACLs https://review.openstack.org/94930 | 16:57 |
| *** markmcclain has quit IRC | 16:57 | |
| clarkb | ya and that same no longer valid provider object should get us a new flavor list in the new provider | 16:57 |
| fungi | once those nodes were manually deleted down below the max-servers threshold, activity in that provider resumed | 16:57 |
| *** sarob has quit IRC | 16:57 | |
| anteaya | fungi: I'm looking for the patch that deals with the script to generate the electorate list | 16:57 |
| clarkb | fungi: using nodepool delete? | 16:57 |
| clarkb | or more manual than that? | 16:58 |
| *** sarob has joined #openstack-infra | 16:58 | |
| fungi | clarkb: yeah, nodepool delete --now | 16:58 |
| anteaya | fungi: I've looked in your open and your merged for April and March, I'm coming up empty | 16:58 |
| fungi | anteaya: what patch? | 16:59 |
| anteaya | the patch for generating teh electoral list | 16:59 |
| *** nati_ueno has quit IRC | 16:59 | |
| anteaya | we had it specify the gerrit preferred email | 16:59 |
| anteaya | which required a patch | 16:59 |
| fungi | oh, that might have gotten auto-abandoned... looking | 17:00 |
| *** dstanek_zzz is now known as dstanek | 17:00 | |
| *** sarob_ has joined #openstack-infra | 17:00 | |
| anteaya | fungi: oh okay | 17:01 |
| fungi | anteaya: https://review.openstack.org/85385 | 17:01 |
| *** cdub has joined #openstack-infra | 17:01 | |
| *** nati_ueno has joined #openstack-infra | 17:01 | |
| fungi | added back to my to-do list... thanks | 17:01 |
| anteaya | ta | 17:01 |
| anteaya | np | 17:01 |
| anteaya | putting together a bit of a report | 17:02 |
| clarkb | fungi: ooohhhhhh wait we only create a new ProviderManager when we change the config on our side, but the change happened on the rh side instead? | 17:02 |
| anteaya | making the change to gerrit preferred cut waaaaaay down on missing ballot follow up emails | 17:02 |
| clarkb | nodepool would not update its cache in that case | 17:02 |
| *** sarob has quit IRC | 17:02 | |
| *** sarob_ has quit IRC | 17:03 | |
| SpamapS | good morning folks | 17:03 |
| zaro | morning SpamapS | 17:03 |
| *** sarob has joined #openstack-infra | 17:03 | |
| clarkb | fungi: also reconfigureManagers doesn't appear to consider min-ram as a thing to reconfigure on | 17:04 |
| *** nati_ueno has quit IRC | 17:04 | |
| fungi | clarkb: no, so the new flavor was created, then the config change to lower the min-ram was merged, then i deleted the old images and nodepool built new ones there | 17:04 |
| fungi | clarkb: but yes, the latter was what i was suspecting | 17:05 |
| *** nati_ueno has joined #openstack-infra | 17:05 | |
| anteaya | morning SpamapS | 17:05 |
| fungi | which is why i theorized yesterday that lowering min-ram on that provider might need a complete nodepool restart before creating new images | 17:05 |
| *** w_ has joined #openstack-infra | 17:05 | |
| clarkb | right I think the first thing is only a problem if we update the config first, but with the reconfigureManagers not looking at image config changing its still a noop | 17:06 |
| *** nati_ueno has quit IRC | 17:06 | |
| *** nati_ueno has joined #openstack-infra | 17:06 | |
| *** derekh_ is now known as derekh_afk | 17:06 | |
| *** dangers_` is now known as dangers | 17:07 | |
| *** eglynn has quit IRC | 17:07 | |
| *** jlibosva has quit IRC | 17:08 | |
| *** olaph has quit IRC | 17:08 | |
| SpamapS | clarkb: can you give me a summary of where we're at, and let me know of anything I can try to address on our cloud? | 17:10 |
| *** jp_at_hp has quit IRC | 17:10 | |
| clarkb | SpamapS: I think fungi said hp tripleo cloud is happy again | 17:11 |
| *** jcoufal has quit IRC | 17:11 | |
| clarkb | and rh cloud is still using the wrong flavor because nodepool bug | 17:11 |
| fungi | clarkb: no, not for the "new" problem | 17:11 |
| clarkb | oh what is the new problem? | 17:11 |
| clarkb | the flavor? | 17:11 |
| fungi | happy from yesterday's deletion problem | 17:11 |
| fungi | not happy about the flavor change | 17:12 |
| SpamapS | Ok. I've started rolling out trusty compute nodes.. | 17:12 |
| fungi | broken from the flavor change (waaay over ram quota) | 17:12 |
| *** afazekas has joined #openstack-infra | 17:12 | |
| SpamapS | and I wanted to see some success from that before I make the change global. | 17:12 |
| *** bhuvan has joined #openstack-infra | 17:12 | |
| clarkb | ok, so that will require a nodepool restart to fix. Should we try writing a quick nodepool change to fix the bug so that in the future we don't need to do a restart? | 17:12 |
| clarkb | I am relatively familiar with the code looking at it right now if we want to do that | 17:12 |
| jeblair | clarkb: which bug? | 17:13 |
| fungi | clarkb: i believe so. if you can take a stab at it, i'm happy to review. much appreciated | 17:13 |
| fungi | jeblair: nodepool not reconfiguring the provider on flavor/image changes (min-ram in particular in this case) | 17:13 |
| jeblair | ack. sounds good. | 17:13 |
| clarkb | fungi: ok will do | 17:14 |
| *** esker has joined #openstack-infra | 17:14 | |
| fungi | that is to say, lowering min-ram on the tripleo labels did not have the desired effect of causing the subsequent image update to select the lower-memory flavor | 17:14 |
| *** ihrachyshka has quit IRC | 17:14 | |
| fungi | because nodepool doesn't currently see it as a reason to create a new provider object | 17:15 |
| clarkb | ya, I suppose the other workaround is to change a provider value not in images | 17:15 |
| fungi | right, we can (and have) done that in the past to get around restarting in situations like this | 17:16 |
| fungi | just hand-edit the config for something non-impacting which will cause it to create a new provider object, then change it back again (or let puppet change it back for you) | 17:16 |
| fungi | which i can go ahead and do now, to speed this along | 17:17 |
| *** Longgeek has quit IRC | 17:17 | |
| *** BadCub01_ has joined #openstack-infra | 17:18 | |
| *** e0ne has quit IRC | 17:18 | |
| *** BadCub01_ has left #openstack-infra | 17:19 | |
| *** esker has quit IRC | 17:20 | |
| fungi | okay, provider config tweaked for their two regions (temporarily bumped the max-servers count in them by 1) and old nodepool images there deleted | 17:20 |
| *** gokrokve has joined #openstack-infra | 17:21 | |
| *** duran has joined #openstack-infra | 17:22 | |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Correct update-image to image-update in samples https://review.openstack.org/85436 | 17:23 |
| *** BadCub01_ has joined #openstack-infra | 17:24 | |
| *** blamar has joined #openstack-infra | 17:24 | |
| *** yfried__ has quit IRC | 17:24 | |
| *** yamahata has quit IRC | 17:24 | |
| *** dims has quit IRC | 17:24 | |
| *** yamahata has joined #openstack-infra | 17:25 | |
| *** _nadya_ has joined #openstack-infra | 17:26 | |
| *** msabramo1 has joined #openstack-infra | 17:27 | |
| *** alugovoi has quit IRC | 17:28 | |
| *** ihrachyshka has joined #openstack-infra | 17:28 | |
| *** crc32 has quit IRC | 17:29 | |
| *** msabramo has quit IRC | 17:30 | |
| *** rcarrillocruz has left #openstack-infra | 17:30 | |
| *** nati_ueno has quit IRC | 17:30 | |
| fungi | SpamapS: derekh_afk: can one of you check whether tripleo-precise-140 | 17:30 |
| *** rcarrillocruz has joined #openstack-infra | 17:30 | |
| SpamapS | fungi: 140 ? | 17:30 |
| *** nati_ueno has joined #openstack-infra | 17:30 | |
| fungi | er, whether tripleo-precise-1400779768.template.openstack.org in the hp tripleo region is using the new h1.ci flavor? | 17:30 |
| fungi | just want to make sure jiggering it in configuration caused it to pick up the right flavor now | 17:31 |
| SpamapS | fungi: I don't see an instance with that name. | 17:31 |
| *** chuckC has quit IRC | 17:31 | |
| SpamapS | oops | 17:31 |
| SpamapS | ok now that I copied/pasted .. still don't see it :-P | 17:31 |
| clarkb | running tox on my change now | 17:32 |
| clarkb | its a bit ugly... | 17:32 |
| SpamapS | fungi: yeah, no instance by that name | 17:32 |
| fungi | SpamapS: looks like the instance id should be 936f03c3-cae7-4f10-8406-125e00ea5498 | 17:33 |
| fungi | ahh, the old one was already deleted due to an unrelated failure, which is why you didn't find it | 17:33 |
| SpamapS | | flavor | h1.ci (9d00764c-26a5-4878-9073-eeea0fbe6759) | | 17:34 |
| fungi | replaced by tripleo-precise-1400779810.template.openstack.org | 17:34 |
| fungi | SpamapS: excellent | 17:34 |
| fungi | SpamapS: as soon as these images finish building, memory usage in the tenant should fall dramatically | 17:34 |
| SpamapS | looks like it may have gotten eth0/eth1 in the wrong order tho | 17:34 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/nodepool: Create new provider managers on image data changes https://review.openstack.org/94946 | 17:35 |
| clarkb | how does that look? | 17:35 |
| fungi | SpamapS: yeah, about half of them do, then get killed and replaced until it gets a working one it can reach | 17:35 |
| fungi | SpamapS: i believe tripleo has an outstanding neutron bug filed on that | 17:35 |
| *** signed8bit has quit IRC | 17:36 | |
| *** signed8bit has joined #openstack-infra | 17:36 | |
| *** lcheng_ has joined #openstack-infra | 17:38 | |
| fungi | clarkb: i suppose you could have inverted the != to == and or's to and's and False to True in those conditionals, but you're at least preserving the previous phrasing i guess | 17:38 |
| clarkb | ya I wanted to make it look like the old code | 17:38 |
| fungi | fair enough (even though a bunch of or'ed negations is crazy-brain-twisty) | 17:39 |
| *** eglynn has joined #openstack-infra | 17:39 | |
| clarkb | I think there is a larger refactor that we can do to make the config classes inherit from dict or similar and make the named properties a thing then just do if config == config | 17:39 |
| *** crc32 has joined #openstack-infra | 17:39 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Move openstack-qa repo to attic https://review.openstack.org/94949 | 17:39 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Move melange to attic https://review.openstack.org/94950 | 17:39 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Move python-melangeclient to attic https://review.openstack.org/94951 | 17:39 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Move openstack-chef to attic https://review.openstack.org/94952 | 17:39 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Move database-api to attic https://review.openstack.org/94953 | 17:39 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Rename oslo.test to oslotest https://review.openstack.org/94954 | 17:39 |
| *** SumitNaiksatam has quit IRC | 17:39 | |
| * anteaya dusts the cobwebs off SergeyLukjanov's shoulder | 17:40 | |
| anteaya | dusty in that attic | 17:40 |
| *** bookwar has joined #openstack-infra | 17:40 | |
| SergeyLukjanov | anteaya, yup_ | 17:40 |
| *** ihrachyshka has quit IRC | 17:41 | |
| *** igor_ has joined #openstack-infra | 17:42 | |
| *** saper_ is now known as saper | 17:42 | |
| * SergeyLukjanov will add CRs for barbican rename and specs after the sahara meeting | 17:42 | |
| SergeyLukjanov | jeblair, ^^ | 17:42 |
| *** saper has quit IRC | 17:43 | |
| *** saper has joined #openstack-infra | 17:43 | |
| jeblair | SergeyLukjanov: thanks! | 17:43 |
| *** saper has quit IRC | 17:44 | |
| jeblair | clarkb, fungi: wait -- we don't currently stop the old provider manager? | 17:45 |
| jeblair | nm i see it | 17:45 |
| *** changbl has joined #openstack-infra | 17:46 | |
| fungi | for oldmanager in stop_managers: oldmanager.stop() | 17:46 |
| fungi | yeah | 17:46 |
| *** igor_ has quit IRC | 17:46 | |
| devananda | clarkb: so that patch landed, but I think I still need to be added to the baremetal-specs group | 17:46 |
| fungi | clarkb: lgtm | 17:46 |
| SpamapS | fungi: Right we do have a neutron bug on that. I'm still a bit confused why we need two networks.. but w'ever. :-P | 17:46 |
| fungi | devananda: i can do it | 17:46 |
| devananda | fungi: thanks | 17:46 |
| *** yfried__ has joined #openstack-infra | 17:47 | |
| fungi | devananda: done | 17:47 |
| *** chuckC has joined #openstack-infra | 17:48 | |
| SpamapS | fungi: bfb8c742-445d-46ac-a191-12b6197cb2a4 booted o-k | 17:49 |
| clarkb | fungi: woot. I have to run off to spend too much money on glasses now | 17:49 |
| zaro | jeblair: do you know how long it takes to run neutron coverage job? i've started one and i think it's been more than 20 mins. can't tell if it's hanging or it's just taking a real long time. | 17:49 |
| clarkb | fungi: feel free to update my change as necessary | 17:49 |
| fungi | clarkb: thanks again | 17:49 |
| clarkb | zaro: normal run is like 50 minutes | 17:50 |
| zaro | wow! | 17:50 |
| clarkb | add coverage and its probably more than an hour | 17:50 |
| jeblair | zaro: you could go find one that has already run in jenkins | 17:50 |
| *** liam__ has joined #openstack-infra | 17:50 | |
| zaro | probably not a good idea to move to check pipeline then? | 17:50 |
| clarkb | no they shouold fix their tests first | 17:50 |
| clarkb | marun: has done good work on that aiui | 17:51 |
| *** rockyg has left #openstack-infra | 17:51 | |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Create new provider managers on image data changes https://review.openstack.org/94946 | 17:51 |
| *** SumitNaiksatam has joined #openstack-infra | 17:51 | |
| *** rcarrill` has joined #openstack-infra | 17:52 | |
| marun | clarkb: if only it were easy :/ | 17:52 |
| *** _nadya_ has quit IRC | 17:52 | |
| marun | clarkb: i never run coverage tbh. does it run in parallel? | 17:52 |
| clarkb | it does | 17:53 |
| clarkb | and combines the results at the end | 17:53 |
| *** rcarrillocruz has quit IRC | 17:53 | |
| *** gyee has quit IRC | 17:53 | |
| marun | clarkb: ah, ok. good to know. | 17:54 |
| anteaya | SergeyLukjanov: I -1'd 4 patches in that series, they are missing acl file renames | 17:56 |
| anteaya | I did +1 two of them though | 17:56 |
| anteaya | clarkb: happy spending | 17:57 |
| SergeyLukjanov | anteaya, thanks for review | 17:57 |
| SergeyLukjanov | anteaya, I'll fix it | 17:57 |
| *** lcheng_ has quit IRC | 17:57 | |
| anteaya | SergeyLukjanov: sure, I added the links in my comments, and dhellmann added teh link for oslotest | 17:57 |
| *** alugovoi has joined #openstack-infra | 17:58 | |
| zaro | marun, clarkb : last neutron coverage run was 27mins. | 17:58 |
| SpamapS | fungi: is it possible to observe the template building process from anywhere? | 17:58 |
| SergeyLukjanov | I've not commited acl files somehow :( | 17:58 |
| anteaya | huh | 17:58 |
| jeblair | zaro: how long was the regular unit test run for the same change? | 17:58 |
| anteaya | two patches did need acl file changes | 17:58 |
| anteaya | SergeyLukjanov: melange-client and openstack-dev/qa.o.o didn't, the rest do | 17:59 |
| fungi | SpamapS: http://nodepool.openstack.org/ | 17:59 |
| SergeyLukjanov | anteaya, oh, sahara meeting time, I'll complete CRs after it | 17:59 |
| fungi | SpamapS: the image.log is interleaved output from all the image builds though, so you end up having to filter for the image/provider names you're curious about | 18:00 |
| *** KurtMartin has joined #openstack-infra | 18:00 | |
| SpamapS | fungi: cool ty | 18:01 |
| *** giulivo has quit IRC | 18:01 | |
| *** dstanek is now known as dstanek_zzz | 18:01 | |
| *** pelix has quit IRC | 18:02 | |
| zaro | jeblair: unit tests are not run with coverage because coverage runs in post. i can only find time on most recent test run. | 18:02 |
| *** morganfainberg_Z is now known as morganfainberg | 18:02 | |
| *** Ryan_Lane1 has joined #openstack-infra | 18:02 | |
| *** Ryan_Lane has quit IRC | 18:02 | |
| *** dstanek_zzz is now known as dstanek | 18:03 | |
| *** MarkAtwood has joined #openstack-infra | 18:03 | |
| zaro | jeblair: this one is pretty recent 30 mins: https://jenkins07.openstack.org/job/gate-neutron-python27/1308/ | 18:03 |
| jeblair | zaro: i meant how long is the regular unit test run without coverage. i would like to compare them because i am surprised that coverage takes 25 minutes and the normal run is said to take 50 minutes. | 18:03 |
| jeblair | ok | 18:03 |
| jeblair | it seems to have gotten a lot faster :) | 18:03 |
| *** _nadya_ has joined #openstack-infra | 18:04 | |
| *** sarob has quit IRC | 18:04 | |
| *** esker has joined #openstack-infra | 18:04 | |
| anteaya | SergeyLukjanov: /me nods | 18:06 |
| anteaya | happy meeting | 18:06 |
| SergeyLukjanov | anteaya, thx :) | 18:06 |
| *** doude has joined #openstack-infra | 18:07 | |
| *** nati_ueno has quit IRC | 18:10 | |
| *** arnaud has quit IRC | 18:13 | |
| *** giulivo has joined #openstack-infra | 18:13 | |
| *** andreaf has quit IRC | 18:13 | |
| *** lcostantino has quit IRC | 18:13 | |
| *** thomasbiege has joined #openstack-infra | 18:13 | |
| *** esker has quit IRC | 18:14 | |
| *** e0ne has joined #openstack-infra | 18:15 | |
| openstackgerrit | A change was merged to openstack-infra/zuul: Move status stylesheets into css file https://review.openstack.org/84921 | 18:16 |
| *** zns has joined #openstack-infra | 18:16 | |
| *** _nadya_ has quit IRC | 18:17 | |
| *** terryw has quit IRC | 18:17 | |
| *** gilliard has quit IRC | 18:17 | |
| *** liam__ has quit IRC | 18:18 | |
| *** jcoufal has joined #openstack-infra | 18:19 | |
| *** svarnau has joined #openstack-infra | 18:20 | |
| *** sarob has joined #openstack-infra | 18:20 | |
| *** lcostantino has joined #openstack-infra | 18:23 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/jenkins-job-builder: update URL references to project. https://review.openstack.org/94430 | 18:23 |
| *** dizquierdo has quit IRC | 18:23 | |
| jeblair | clarkb, fungi, sdague: check this out: https://review.openstack.org/#/c/91310/3/etc/status/public_html/zuul.app.js | 18:24 |
| *** dims has joined #openstack-infra | 18:24 | |
| jeblair | clarkb, fungi, sdague: there is a file rename with changes, and gerrit displays that as a deleted file and added file. since they are different, you can't actually see the diff | 18:24 |
| *** doude has quit IRC | 18:25 | |
| jeblair | clarkb, fungi, sdague: gertty notes the rename and diffs the old/new file so the diff is actually reviewable! | 18:25 |
| fungi | ha! | 18:26 |
| fungi | jgit fails once again | 18:26 |
| *** doude has joined #openstack-infra | 18:27 | |
| jeblair | mordred, mordred_: ^ (also, why are there two of you?) | 18:27 |
| *** zns has quit IRC | 18:27 | |
| *** zns has joined #openstack-infra | 18:28 | |
| *** talluri has joined #openstack-infra | 18:28 | |
| *** sarob has quit IRC | 18:28 | |
| *** jcoufal has quit IRC | 18:29 | |
| *** zns has quit IRC | 18:29 | |
| *** jcoufal has joined #openstack-infra | 18:30 | |
| james_li | anyone knows if gate supports docker? | 18:31 |
| *** thomasbiege has quit IRC | 18:31 | |
| *** doude has quit IRC | 18:31 | |
| anteaya | james_li: what do you mean by supports? | 18:31 |
| anteaya | we don't use docker in the gate if that is the question | 18:31 |
| james_li | anteaya: I mean docker-enabled kernel | 18:32 |
| jeblair | and the comments even showed up in the right place | 18:32 |
| *** lcheng_ has joined #openstack-infra | 18:32 | |
| *** sarob has joined #openstack-infra | 18:32 | |
| jeblair | james_li: the latest OSes we have are f20 and precise; we're working on trusty | 18:33 |
| *** e0ne has quit IRC | 18:33 | |
| *** dstanek is now known as dstanek_zzz | 18:33 | |
| james_li | jeblair: is the kernel ver of precise is 3.8? | 18:34 |
| *** amcrn_ has joined #openstack-infra | 18:34 | |
| *** zns has joined #openstack-infra | 18:34 | |
| *** praneshp_ has joined #openstack-infra | 18:35 | |
| *** amcrn has quit IRC | 18:35 | |
| *** gilliard has joined #openstack-infra | 18:36 | |
| jeblair | 3.2.0-61-generic | 18:36 |
| *** praneshp has quit IRC | 18:37 | |
| *** praneshp_ is now known as praneshp | 18:37 | |
| *** doug-fish1 has joined #openstack-infra | 18:37 | |
| *** doug-fish has quit IRC | 18:39 | |
| *** Igrsrolqak has joined #openstack-infra | 18:39 | |
| *** cdub has quit IRC | 18:39 | |
| SergeyLukjanov | jeblair, folks, do we like to keep ACL when moving to the attic? | 18:40 |
| SpamapS | are there any plans for adding trusty testing? | 18:40 |
| *** chuckC has quit IRC | 18:40 | |
| jeblair | SpamapS: mordred is on it | 18:40 |
| *** eglynn has quit IRC | 18:40 | |
| *** chuckC has joined #openstack-infra | 18:40 | |
| jeblair | SergeyLukjanov: no, actually we'll want to completely disable those projects | 18:41 |
| SpamapS | jeblair: sweet | 18:41 |
| SergeyLukjanov | jeblair, so, to remove ACLs for them? | 18:41 |
| SergeyLukjanov | jeblair, ACL files I mean | 18:41 |
| *** lcheng_ has quit IRC | 18:42 | |
| *** igor_ has joined #openstack-infra | 18:42 | |
| *** alexpilotti has joined #openstack-infra | 18:42 | |
| jeblair | SergeyLukjanov: i think we'll want an acl file that's mostly empty -- no permissions and sets the project state to 'read only' instead of 'active'. | 18:43 |
| SergeyLukjanov | oh, sounds like we should just keep it empty | 18:43 |
| *** james_li_ has joined #openstack-infra | 18:43 | |
| *** james_li has quit IRC | 18:43 | |
| *** james_li_ is now known as james_li | 18:43 | |
| *** arnaud___ has joined #openstack-infra | 18:43 | |
| *** arnaud___ has quit IRC | 18:44 | |
| *** Igrsrolqak has left #openstack-infra | 18:44 | |
| *** chuckC has quit IRC | 18:45 | |
| james_li | jeblair: thanks. how do I know my gate is precise or f20? | 18:45 |
| jeblair | james_li: it's whatever the jobs are configured to run on | 18:46 |
| james_li | got it | 18:46 |
| jeblair | james_li: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/ | 18:46 |
| jeblair | james_li: there's the configuration | 18:46 |
| alexpilotti | hi guys | 18:46 |
| james_li | jeblair: thx! | 18:46 |
| alexpilotti | I’m doing a session at a conference about the Hyper-V CI | 18:46 |
| *** igor_ has quit IRC | 18:47 | |
| alexpilotti | I’d like to send in a patch to gerrit demoing the full gerrit / zuul “roundtrip” | 18:47 |
| alexpilotti | is there an established way to send in test jobs? | 18:47 |
| alexpilotti | as in “test do not merge” ion teh subject, etc | 18:48 |
| jeblair | alexpilotti: you can use the openstack-dev/sandbox repo | 18:48 |
| alexpilotti | jeblair: nice tx! | 18:48 |
| jeblair | alexpilotti: we actually ask new ci systems to start with that now | 18:48 |
| jeblair | so that they can exercise without leaving possibly incorrect comments on real projects | 18:48 |
| *** vtapia has quit IRC | 18:55 | |
| *** jcoufal has quit IRC | 18:55 | |
| *** rcarrillocruz has joined #openstack-infra | 18:56 | |
| SergeyLukjanov | jeblair, could you please confirm acl for attic project? https://etherpad.openstack.org/p/attic-project-acl | 18:57 |
| *** rcarrill` has quit IRC | 18:59 | |
| *** _nadya_ has joined #openstack-infra | 19:00 | |
| jeblair | SergeyLukjanov: looks good. i just manually changed melange to be certain: https://review.openstack.org/gitweb?p=openstack/melange.git;a=blob;f=project.config;hb=54f07d307de3cfef43b7cf5d5d11499bf9919332 | 19:01 |
| *** mestery has joined #openstack-infra | 19:01 | |
| SergeyLukjanov | jeblair, yup, it looks good | 19:02 |
| SergeyLukjanov | https://review.openstack.org/#/admin/projects/openstack/melange | 19:02 |
| SergeyLukjanov | jeblair, thank you | 19:02 |
| *** dangers is now known as dangers_away | 19:02 | |
| *** dangers_away has quit IRC | 19:02 | |
| *** che-arne has quit IRC | 19:03 | |
| openstackgerrit | A change was merged to openstack-infra/zuul: Factor out common code between cli utilities https://review.openstack.org/85565 | 19:04 |
| *** unicell has quit IRC | 19:05 | |
| *** dangers_away has joined #openstack-infra | 19:06 | |
| *** dangers_away is now known as dangers | 19:06 | |
| *** sarob has quit IRC | 19:08 | |
| *** sarob has joined #openstack-infra | 19:09 | |
| clarkb | ok back ish | 19:09 |
| clarkb | lunch may happen | 19:09 |
| pleia2 | oh hey, lunch | 19:10 |
| *** _nadya_ has quit IRC | 19:10 | |
| *** sarob has quit IRC | 19:13 | |
| anteaya | clarkb: do you like your glasses? | 19:15 |
| clarkb | anteaya: well I have to wait 7-10 business days now | 19:15 |
| anteaya | ah | 19:15 |
| anteaya | you had the fun of paying lots of money for them today | 19:15 |
| clarkb | yes | 19:15 |
| anteaya | wooo | 19:16 |
| anteaya | hope that was lots of fun | 19:16 |
| clarkb | the staring at a wall of choices and agonizing over which one is best | 19:16 |
| msabramo1 | zaro: https://review.openstack.org/90455 | 19:16 |
| anteaya | yeehaw | 19:16 |
| *** melwitt1 has joined #openstack-infra | 19:17 | |
| anteaya | as long as you look good in a hat, should you choose to wear one | 19:17 |
| *** melwitt has quit IRC | 19:18 | |
| *** eglynn has joined #openstack-infra | 19:20 | |
| *** GheRivero has quit IRC | 19:23 | |
| *** dstanek_zzz is now known as dstanek | 19:24 | |
| *** rlandy has quit IRC | 19:26 | |
| *** alexpilotti has quit IRC | 19:27 | |
| *** zns has quit IRC | 19:29 | |
| *** GheRivero has joined #openstack-infra | 19:30 | |
| *** lakshmiS has joined #openstack-infra | 19:31 | |
| *** KurtMartin has quit IRC | 19:31 | |
| *** eglynn has quit IRC | 19:32 | |
| *** sarob has joined #openstack-infra | 19:33 | |
| *** serverascode has quit IRC | 19:35 | |
| *** serverascode has joined #openstack-infra | 19:37 | |
| *** dprince has quit IRC | 19:40 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: The process for making a Third-Party CI voting needs docs https://review.openstack.org/93562 | 19:41 |
| anteaya | reed you around? | 19:42 |
| *** flaper87|afk is now known as flaper87 | 19:42 | |
| reed | anteaya, o/ | 19:42 |
| anteaya | hi | 19:42 |
| *** eglynn has joined #openstack-infra | 19:43 | |
| anteaya | so where are we on having a new etherpad to talk about the physical meeting? | 19:43 |
| anteaya | can we do that? | 19:43 |
| *** igor_ has joined #openstack-infra | 19:43 | |
| *** andreaf has joined #openstack-infra | 19:44 | |
| reed | anteaya, I'm still trying to make sure that I have budget for it and that the main recipient of the class is interested in it | 19:45 |
| anteaya | fair enough | 19:45 |
| anteaya | any objection to me creating an new etherpad? | 19:45 |
| anteaya | that costs nothin | 19:45 |
| anteaya | g | 19:45 |
| reed | anteaya, this week is nasty and next week is super busy too (most of the Foundation folks are going to be flying to Europe) | 19:45 |
| reed | anteaya, :) go ahead | 19:45 |
| anteaya | fair enough | 19:46 |
| anteaya | great thanks | 19:46 |
| reed | thank you for moving this forward | 19:46 |
| anteaya | np | 19:46 |
| anteaya | you need to feel supported | 19:46 |
| anteaya | and I want you to know I support you | 19:46 |
| anteaya | I'm not alone in this, I'm just the most vocal | 19:47 |
| james_li | how to effectively set DEVSTACK_GATE_VIRT_DRIVER? I export it in my pre_test_hook, but still see VIRT_DRIVER=libvirt in localrc | 19:47 |
| *** eglynn has quit IRC | 19:47 | |
| lifeless | sdague: derekh_afk: infra folk can requeue, and do so regularly for gate; but it can be done on any queue and has been done on tripleo-check before precisely for this reason to avoid waste | 19:47 |
| *** igor_ has quit IRC | 19:47 | |
| fungi | james_li: you'll need to set it in the job configuration | 19:47 |
| *** hashar has joined #openstack-infra | 19:47 | |
| james_li | what I did is : export DEVSTACK_GATE_VIRT_DRIVER=docker | 19:47 |
| anteaya | okay etherpad cpu usage has jumped significantly: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=12&page=2 | 19:48 |
| fungi | james_li: it has to be exported in the builder definition before the wrap script is invoked | 19:48 |
| *** zns has joined #openstack-infra | 19:48 | |
| *** zns has quit IRC | 19:49 | |
| anteaya | and so has the load average | 19:49 |
| james_li | fungi: thanks. ok so in modules/openstack_project/files/jenkins_job_builder/config/solum.yaml ? | 19:49 |
| anteaya | are we getting hit from somewhere on etherpad? | 19:49 |
| fungi | james_li: yes | 19:49 |
| james_li | fungi: thanks! | 19:49 |
| *** zns has joined #openstack-infra | 19:49 | |
| anteaya | fungi: can you take a look at what is happening on the etherpad server? | 19:50 |
| anteaya | something is going on | 19:50 |
| fungi | anteaya: looks like someone tried to import a word document (best guess), and there's a stuck/runaway abiword plugin process running under the eplite account because of it | 19:52 |
| mordred | fungi: we should disable that | 19:52 |
| anteaya | nice | 19:53 |
| fungi | clarkb: any hints on how to do that? | 19:53 |
| openstackgerrit | james-li-3 proposed a change to openstack-infra/config: set DEVSTACK_GATE_VIRT_DRIVER for solum https://review.openstack.org/94994 | 19:53 |
| openstackgerrit | A change was merged to stackforge/python-jenkins: Make it work on Python 3 https://review.openstack.org/90114 | 19:54 |
| SpamapS | fungi: ok I see h1.ci slaves on the hp cloud | 19:54 |
| fungi | also, killing the abiword process was like a central american military coup taking out a leader. another just cropped up to take its place | 19:54 |
| clarkb | mordred: that was one of the requirements of the move to etherpad lite | 19:54 |
| anteaya | fungi: ha ha ha | 19:54 |
| mordred | clarkb: oh. well, piddle | 19:54 |
| james_li | fungi: https://review.openstack.org/#/c/94994/ please | 19:54 |
| clarkb | mordred: I am all for disabling it but the import export stuff was something I was told to make work | 19:54 |
| clarkb | fungi: you can probably restart the server itself if it is really cranky | 19:54 |
| anteaya | I'm for restarting | 19:54 |
| hashar | jeblair: thank you for the recent Zuul approvals :-) | 19:55 |
| *** e0ne has joined #openstack-infra | 19:55 | |
| *** gyee has joined #openstack-infra | 19:55 | |
| fungi | yeah, the new abiword process is right back to eating 100% of a cpu again | 19:55 |
| clarkb | and you can disable it in the config.js | 19:55 |
| fungi | so i suspect etherpad is "smart" and restarts it if it dies | 19:55 |
| clarkb | iirc you have to give it a path to abiword otherwise its disabled | 19:55 |
| mordred | timrc: hey! I hear you're going to help with the dib-nodepool work | 19:55 |
| SpamapS | | ef6c69fe-1385-4104-ba46-c69436b70924 | tripleo-precise-1400780542.template.openstack.org | qcow2 | bare | | queued | | 19:56 |
| SpamapS | hm | 19:56 |
| SpamapS | does that mean something is stalled while trying to make a snapshot? | 19:56 |
| fungi | SpamapS: possibly. i noticed the precise image builds were taking waaay too long (the f20 builds finished more than an hour ago) but switched gears to etherpad troubleshooting now | 19:57 |
| *** esker has joined #openstack-infra | 19:58 | |
| SpamapS | the compute node it is running on is having fits and starts .. the precursor of total mellanox fail | 19:58 |
| SpamapS | I think I've found my next candidate for trusty upgrade | 19:58 |
| clarkb | SpamapS: does trusty fix mellanox or just differently break it? | 19:59 |
| mordred | timrc: SpamapS and jeblair will both be happy about progress there - and you for sure want to chat with jeblair about how to get glanceclient stuff into nodepool | 19:59 |
| fungi | clarkb: restarting etherpad and killing the abiword plugin process caused it to respawn, but it's no longer eating an entire processor | 19:59 |
| clarkb | fungi: caused abiword to respawn? | 19:59 |
| fungi | clarkb: yeah, i think maybe it always runs | 19:59 |
| clarkb | I wouldn't expect etherpad-lite to maintain that state across reboots | 19:59 |
| clarkb | oh maybe | 19:59 |
| SpamapS | clarkb: fixes it | 19:59 |
| SpamapS | clarkb: the mellanox fail in trusty is that infiniband actually works now, which confuses trusty and our special automation stuff. | 20:00 |
| clarkb | so differently broken :) | 20:00 |
| SpamapS | clarkb: But that only causes issues on first boot | 20:00 |
| SpamapS | clarkb: no I think it is exposing some assumptions in our dhcp-all-interfaces element | 20:01 |
| *** rcarrill` has joined #openstack-infra | 20:01 | |
| jeblair | yeah, it's a constant surprise when openstack works ;) | 20:01 |
| mordred | jeblair: openstack works? | 20:01 |
| SpamapS | jinx | 20:01 |
| fungi | clarkb: yeah, behavior was that before restarting etherpad it kept respawning an abiword plugin which was maxing a cpu. after the etherpad restart it was still respawning the abiword plugin if i killed it, but no longer actually consuming substantial processor time | 20:01 |
| clarkb | I need to check on our quota bump | 20:01 |
| clarkb | then we can see if we can make openstack work | 20:01 |
| *** MarkAtwood has quit IRC | 20:02 | |
| *** gondoi is now known as zz_gondoi | 20:02 | |
| *** rcarrillocruz has quit IRC | 20:02 | |
| *** talluri has quit IRC | 20:03 | |
| *** homeless has quit IRC | 20:04 | |
| fungi | SpamapS: so worth noting, the precise image builds in both tripleo regions seem to be in the same boat. 2014-05-22 reached the sync+sleep at the end of the prep scripts a couple hours ago but never transitioned from building to ready | 20:04 |
| *** homeless has joined #openstack-infra | 20:04 | |
| fungi | er, ignore that stray date i apparently included for fun | 20:04 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/gerritlib: Doc generation support https://review.openstack.org/94861 | 20:04 |
| SpamapS | :) | 20:05 |
| openstackgerrit | A change was merged to openstack-dev/cookiecutter: Fix documentation template issues https://review.openstack.org/94492 | 20:05 |
| hashar | dims: you are just too fast :) | 20:06 |
| *** SumitNaiksatam has quit IRC | 20:06 | |
| fungi | SpamapS: any guesses as to why this would be affecting precise image builds in both regions and not f20? | 20:07 |
| SergeyLukjanov | sorry folks, review bomb | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Move database-api to attic https://review.openstack.org/94953 | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Move openstack-chef to attic https://review.openstack.org/94952 | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Rename oslo.test to oslotest https://review.openstack.org/94954 | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Move openstack-qa repo to attic https://review.openstack.org/94949 | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Move python-melangeclient to attic https://review.openstack.org/94951 | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Move melange to attic https://review.openstack.org/94950 | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Remove obsolete #openstack-climate channel https://review.openstack.org/94913 | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Rename climate to blazar https://review.openstack.org/94912 | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Move barbican from stackforge to barbican https://review.openstack.org/94997 | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Rename glance-specs to image-specs https://review.openstack.org/94998 | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Rename neutron-specs to networking specs https://review.openstack.org/94999 | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Rename oslo-specs to common-libraries-specs https://review.openstack.org/95000 | 20:07 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Restrict releasing of blazar to blazar-ptl group https://review.openstack.org/95001 | 20:07 |
| mordred | wow. SergeyLukjanov is a machine | 20:08 |
| *** SumitNaiksatam has joined #openstack-infra | 20:08 | |
| *** ravips has joined #openstack-infra | 20:08 | |
| dims | :) | 20:08 |
| mihgen | :O | 20:09 |
| *** andreaf has quit IRC | 20:11 | |
| SergeyLukjanov | etherpad for tomorrows renaming - https://etherpad.openstack.org/p/repo-renaming-2014-05-23 | 20:11 |
| SergeyLukjanov | mordred, hehe, vending machine :) | 20:11 |
| ravips | I mistyped username in gerrit and I'm unable to rename it, can someone reset my login in gerrit? | 20:12 |
| *** esker has quit IRC | 20:13 | |
| *** hogepodge has quit IRC | 20:13 | |
| *** esker has joined #openstack-infra | 20:13 | |
| anteaya | reed when you are around: https://etherpad.openstack.org/p/juno-openid-training-session | 20:14 |
| anteaya | let's play here for a bit | 20:14 |
| anteaya | ravips: what is the account number for your account? | 20:14 |
| anteaya | sign in, go to settings, it is the account id number | 20:15 |
| ravips | anteaya: 11324 | 20:15 |
| anteaya | fungi: ^^ | 20:15 |
| anteaya | when fungi gets a chance he will find your account in the db | 20:15 |
| anteaya | what username do you want it set to? | 20:15 |
| ravips | anteaya: fungi: ravips instead of ravp | 20:16 |
| *** dims has quit IRC | 20:16 | |
| * anteaya nods | 20:16 | |
| openstackgerrit | A change was merged to openstack-infra/git-review: Topic: do not use '(detached' when detached https://review.openstack.org/90466 | 20:17 |
| sdague | jeblair: I feel like gerrit used to sometimes handle the rename case | 20:17 |
| anteaya | ravips: when fungi gets a moment he will help you out, but it may take some time depending on what he is working on | 20:17 |
| ravips | anteaya: ok, not a problem..thanks for helping me out | 20:18 |
| *** hogepodge has joined #openstack-infra | 20:18 | |
| *** esker has quit IRC | 20:18 | |
| anteaya | ravips: np, I don't have access to the db, but we have been through this before and I have a pretty good idea what data fungi needs to make the change | 20:19 |
| anteaya | he will ping you if he needs more, so stick around | 20:19 |
| *** esker has joined #openstack-infra | 20:20 | |
| *** mihgen has left #openstack-infra | 20:20 | |
| ravips | anteaya: ok, sure ping me for any more details | 20:20 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/config: Uncomment gate-python-jenkins-python33 https://review.openstack.org/95004 | 20:21 |
| anteaya | kk | 20:21 |
| fungi | ravips: anteaya: resetting ssh account names is not a gerrit feature, and involves directly editing tables in the database so we try not to do it unless absolutely necessary. 'git config --global gitreview.username ravp' should make your system log into review.openstack.org as ravp from now on | 20:21 |
| *** MarkAtwood has joined #openstack-infra | 20:21 | |
| jeblair | sdague: i know it detects straight renames, but i don't know at what threshold it considers it an add/delete. | 20:21 |
| SpamapS | fungi: precise is about 50MB bigger than fedora ... maybe the template image is over some sort of boundary causing slowness or full-something | 20:22 |
| fungi | ravips: anteaya: i can make a note to clear the username from your account instead, but am in the middle of other things right now, so no guarantees on when i'll get to it | 20:22 |
| ravips | fungi: ok, that will be helpful..thx | 20:23 |
| sdague | jeblair: yeh, when I did the governance repo reformat, some were detected, others were not | 20:23 |
| sdague | it was odd | 20:23 |
| SpamapS | 2014-05-22 13:57:47.018 26826 ERROR nova.compute.manager [req-f8474238-f3a2-4618-b5b3-879dc1d4ce2e d5af62d2183d431796d74c5bb119ec9f e01e473a9250498883955b80966a1e58] [instance: bfb8c742-445d-46ac-a191-12b6197cb2a4] Error while trying to clean up image ef6c69fe-1385-4104-ba46-c69436b70924 | 20:24 |
| SpamapS | 2014-05-22 13:57:47.018 26826 TRACE nova.compute.manager [instance: bfb8c742-445d-46ac-a191-12b6197cb2a4] This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. | 20:24 |
| openstackgerrit | Antoine Musso proposed a change to openstack-infra/config: Bring docs jobs for openstack-infra/gerritlib https://review.openstack.org/95006 | 20:24 |
| fungi | SpamapS: that looks decidedly broken | 20:25 |
| SpamapS | fungi: I wonder if it took longer and something (token?) timed out | 20:25 |
| fungi | SpamapS: want me to have it retry? | 20:25 |
| SpamapS | fungi: if that is easy, yes | 20:25 |
| *** mjseger has quit IRC | 20:25 | |
| SpamapS | fungi: I'll keep an eye on the compute node that it lands on | 20:25 |
| *** therve has quit IRC | 20:25 | |
| anteaya | fungi: *nod, thanks | 20:25 |
| *** igor_ has joined #openstack-infra | 20:25 | |
| fungi | SpamapS: deleted, and new images are building now | 20:26 |
| *** miqui has quit IRC | 20:26 | |
| fungi | SpamapS: 750f0b6e-5f05-4a9c-8f9e-58dd79a1a6fa in r1 and 84db4f02-278d-4f47-8d66-5a2e9eda3007 in r2 | 20:26 |
| msabramo1 | zaro: Let's turn on py33 gate jobs for python-jenkins? https://review.openstack.org/95004 | 20:27 |
| *** jooools has joined #openstack-infra | 20:27 | |
| *** igor__ has joined #openstack-infra | 20:27 | |
| clarkb | so I have an email saying our quota was bumped for openstackjenkins2 but asking neutron quota-show shows the old numebrs | 20:28 |
| clarkb | I am going to try creatin ga new router and see what happens | 20:28 |
| SpamapS | fungi: heh, a290fd8f-7c26-4c51-81cc-522187f0f977 now in r1 | 20:29 |
| SpamapS | silly neutron bugs | 20:29 |
| fungi | SpamapS: that tends to get caught and retried fairly quickly at least | 20:29 |
| SpamapS | fungi: true | 20:29 |
| *** igor_ has quit IRC | 20:30 | |
| clarkb | nope cannot create a router | 20:30 |
| fungi | the "i've fallen and i can't get up" error it hit earlier seems to need manual intervention on the other hand | 20:30 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Rename neutron-specs to networking specs https://review.openstack.org/94999 | 20:31 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Restrict releasing of blazar to blazar-ptl group https://review.openstack.org/95001 | 20:31 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Rename oslo-specs to common-libraries-specs https://review.openstack.org/95000 | 20:31 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Remove obsolete #openstack-climate channel https://review.openstack.org/94913 | 20:31 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Rename climate to blazar https://review.openstack.org/94912 | 20:31 |
| *** zz_gondoi is now known as gondoi | 20:32 | |
| *** esker has quit IRC | 20:32 | |
| *** igor__ has quit IRC | 20:32 | |
| *** dangers is now known as dangers_away | 20:32 | |
| *** amcrn_ has quit IRC | 20:32 | |
| *** gondoi is now known as zz_gondoi | 20:32 | |
| *** esker has joined #openstack-infra | 20:32 | |
| * anteaya heads out for a walk | 20:32 | |
| *** asselin has quit IRC | 20:32 | |
| *** sarob has quit IRC | 20:32 | |
| SpamapS | fungi: aaaand one more time | 20:33 |
| *** mwagner_lap has quit IRC | 20:33 | |
| fungi | SpamapS: eh? | 20:33 |
| SpamapS | fungi: it happened again | 20:34 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/config: Uncomment gate-python-jenkins-python33 https://review.openstack.org/95004 | 20:34 |
| SpamapS | Waiting up to 60 more seconds for network configuration... | 20:34 |
| SpamapS | Booting system without full network configuration... | 20:34 |
| SlickNik | clarkb: Rebased https://review.openstack.org/#/c/92755/, and also fixed the issue you hinted at earlier. Thanks! | 20:34 |
| *** lcheng_ has joined #openstack-infra | 20:34 | |
| fungi | SpamapS: eww | 20:34 |
| SpamapS | fungi: takes about 5 min I think to detect that one. | 20:34 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Textareas now autoresize their height. https://review.openstack.org/92939 | 20:35 |
| *** therve has joined #openstack-infra | 20:35 | |
| SpamapS | fungi: thats the swapped ethernets (the other one is a straight up neutron "but I am le-tired" problem that results in ERROR. | 20:35 |
| fungi | SpamapS: both regions or just one of them? | 20:35 |
| SpamapS | fungi: just r1.. I suppose I should check r2 as well | 20:35 |
| SpamapS | fungi: we're in a weird place right now, because the most functional compute node for this would be novacompute5 .. but it is full of old slaves doing nothing. :-P | 20:35 |
| timrc | mordred, Hi. Yep. Spoke with Cody a bit earlier about it. | 20:36 |
| fungi | okay, deleted again from r1 so maybe it'll get lucky this time around | 20:36 |
| *** esker has quit IRC | 20:37 | |
| fungi | SpamapS: ab7f6e30-83c3-47da-b926-c3c6b31f700f is the new best hope | 20:37 |
| SpamapS | fungi: sadly, I don't even know exactly how to login to r2 .. it has.. you know.. been working | 20:37 |
| fungi | heh | 20:37 |
| SpamapS | fungi: ab7f6e30-83c3-47da-b926-c3c6b31f700f got network.. yaaaay | 20:37 |
| SpamapS | fungi: Ok, its running, on the same compute node that failed last time.. | 20:38 |
| SpamapS | fungi: so help me understand the tubes a bit more.. right now, nodepool wants new templates.. but why isn't zuul/jenkins running jobs on the existing nodes? | 20:39 |
| SpamapS | once we want a new template.. all old nodes are discarded? | 20:39 |
| fungi | SpamapS: i believe those are all f20 nodes and it doesn't need any more of them | 20:40 |
| SpamapS | oh now that is pretty interesting isn't it? like maybe there's something specifically breaking precise? | 20:40 |
| stevebaker | sdague: hey | 20:40 |
| SpamapS | PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND | 20:40 |
| SpamapS | 5094 libvirt- 20 0 13.1g 1.1g 9.9m S 30 1.2 1:07.02 qemu-system-x86 | 20:41 |
| SpamapS | Alright, its doing "stuff" | 20:41 |
| stevebaker | is pypi.openstack.org used by devstack gating currently? I saw a comment about pypi CDN | 20:41 |
| fungi | SpamapS: yeah, 7 tripleo-f20 building, 3 delete | 20:41 |
| clarkb | stevebaker: it is | 20:41 |
| *** sarob has joined #openstack-infra | 20:41 | |
| SpamapS | fungi: and that jives with what I see in the zuul queue, which is f20's getting done, and no precises getting done | 20:41 |
| clarkb | stevebaker: but not for all projects. things that live outside of openstack/ use the mirror and pypi proper | 20:41 |
| fungi | SpamapS: yeah, because all the old remaining precise nodes got used up from the previous image which used the wrong flavor | 20:42 |
| stevebaker | clarkb: so I need to get diskimage-builder to use that, plus a local pip mirror which contains packages built from git for things being gated on | 20:42 |
| *** che-arne has joined #openstack-infra | 20:42 | |
| *** esker has joined #openstack-infra | 20:43 | |
| clarkb | stevebaker: why do you need packages from git for things being gated on? can we not just do source installs? all of the existing tests do source installs | 20:44 |
| fungi | stevebaker: clarkb: it's worth noting that jobs running on openstack/requirements changes also hit pypi.python.org instead of pypi.openstack.org (since they may be trying to add new packages/versions we don't mirror) | 20:45 |
| stevebaker | clarkb: feedback from lifeless, he would prefer these projects get into the image via pip rather than git source install | 20:45 |
| *** W00die is now known as w00dy | 20:45 | |
| *** w00dy is now known as W00die | 20:45 | |
| clarkb | lifeless: can we not special case this and be consistent? | 20:45 |
| SpamapS | Man, I kind of wish the image building process would log to /dev/console so I didn't have to re-download image.log again to see whats going on. :-P | 20:45 |
| *** W00die is now known as w00dy | 20:45 | |
| *** w00dy is now known as Guest33461 | 20:46 | |
| *** Guest33461 is now known as W00die | 20:46 | |
| clarkb | stevebaker: if you do need the local mirror you should look at the pbr integration test as it builds one in order to do some specific functional testing | 20:47 |
| clarkb | stevebaker: I will find a link for you | 20:47 |
| *** eglynn has joined #openstack-infra | 20:47 | |
| clarkb | stevebaker: https://git.openstack.org/cgit/openstack-dev/pbr/tree/tools/integration.sh#n54 starting from about that line | 20:48 |
| *** derekh_afk has quit IRC | 20:48 | |
| *** rdopieralski has joined #openstack-infra | 20:48 | |
| *** andreaf has joined #openstack-infra | 20:49 | |
| *** derekh_ has joined #openstack-infra | 20:49 | |
| jeblair | stevebaker: they can still be installed via pip, just "pip install ." from inside the source dir | 20:49 |
| stevebaker | I think lifeless is recovering from summit flu | 20:49 |
| *** ramashri has joined #openstack-infra | 20:50 | |
| *** e0ne has quit IRC | 20:50 | |
| stevebaker | clarkb: thanks, I'm hoping building a full local mirror won't be necessary, but if it is then some of that should be done in devstack-gate | 20:51 |
| stevebaker | jeblair: yes, but in this case the source dir is outside the (diskimage-builder) image being build | 20:52 |
| SpamapS | fungi: I see sync / sleep 5 for the rh1 precise .. | 20:52 |
| SpamapS | fungi: is that what is done before snapshotting? | 20:52 |
| clarkb | jeblair: fungi mordred the quota bump was made against the wrong account (my personal one apparently) so I am now waiting for that to get sorted | 20:52 |
| jeblair | stevebaker: how would the local mirror be made available? | 20:52 |
| clarkb | when it does get sorted I will create 4 new routers and 4 new networks and we can try region b again | 20:52 |
| jeblair | clarkb: ++ | 20:53 |
| mordred_ | clarkb: haha | 20:53 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Drop formatStatusHTML() methods https://review.openstack.org/92585 | 20:53 |
| fungi | stevebaker: the challenge there is that mirror-building is time-consuming, and we can't predict in advance which versions of git repositories which were involved in the change being tested (taking other changes ahead of in in a dependent pipeline into account) should be turned into packages and added to the local mirror | 20:53 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Fixed several typos in the codebase https://review.openstack.org/94056 | 20:53 |
| jeblair | clarkb: as long as they bill you personally. :) | 20:53 |
| mordred_ | jeblair: I have two ids because I'm experimenting with phone clients | 20:53 |
| stevebaker | jeblair: the pypi element makes the local mirror ~/.cache/image-create/pypi available inside the image | 20:53 |
| clarkb | jeblair: I'm ok with that if mordred_ approves the expense report :) | 20:53 |
| mordred_ | but not ready to, you know, change anything | 20:54 |
| clarkb | mordred_: weechat an be a proxy for you and there is an android weechat protocol speaking client | 20:54 |
| fungi | SpamapS: yes, i believe that's the end of the template setup, and then the template instance is stopped and a snapshot taken | 20:54 |
| mordred_ | clarkb: I have never gotten that to work. so far, this is GREAT on the phone side | 20:54 |
| *** mordred_ is now known as mordred_phone | 20:55 | |
| stevebaker | mordred_: bip proxy works nice for me | 20:55 |
| SpamapS | ooo | 20:56 |
| *** rdopieralski has quit IRC | 20:56 | |
| SpamapS | curl --verbose -C - -o image.log http://nodepool.openstack.org/image.log | 20:56 |
| SpamapS | that works | 20:56 |
| SpamapS | only dl's the new bits | 20:56 |
| stevebaker | fungi: I'll be trying to avoid building a full mirror unless that happens during node building | 20:56 |
| fungi | stevebaker: mainly just pointing out one of the main reasons we install from source--because we're testing the source and need to make decisions once the job is running about what source we install | 20:57 |
| fungi | SpamapS: great tip! | 20:57 |
| mtreinish | fungi: quick question is there a guide for pushing tags to a repo. I need to tag the start of using branchless tempest | 20:57 |
| mtreinish | also any suggestions on a naming convention | 20:57 |
| fungi | mtreinish: https://wiki.openstack.org/wiki/GerritJenkinsGit#Tagging_a_Release | 20:57 |
| openstackgerrit | A change was merged to openstack-infra/git-review: Allow to specify default scheme in .gitreview file https://review.openstack.org/93552 | 20:58 |
| mtreinish | I was just going todo 2014.1 and so on but sdague thinks that will be confusing | 20:58 |
| mtreinish | because it doesn't match up with the other project releases | 20:58 |
| mtreinish | fungi: cool tanks | 20:58 |
| fungi | mtreinish: 1.0? | 20:58 |
| mtreinish | *thanks | 20:58 |
| *** lttrl has joined #openstack-infra | 20:58 | |
| Shrews | fungi: clarkb: Are there any infra issues that would cause this? http://logs.openstack.org/36/94436/1/check/check-grenade-dsvm-neutron-icehouse/0e737b3/console.html#_2014-05-22_15_07_36_937 | 20:58 |
| fungi | mtreinish: (it's a nice, round number) | 20:58 |
| stevebaker | fungi: agreed, I'll be running setup.py sdist from source, then pushing that into the image via pip | 20:58 |
| mtreinish | fungi: yeah I guess that works, but what is the major version the year? or every new tag? | 20:58 |
| *** zns has quit IRC | 20:59 | |
| sdague | stevebaker: I think we should start with the install straight from git path, and not build intermediate packages. That's been our model so far. | 20:59 |
| *** alexpilotti has joined #openstack-infra | 20:59 | |
| sdague | and it has worked very well | 20:59 |
| clarkb | mtreinish: fungi I think I would make them line up with the releases | 20:59 |
| sdague | clarkb: the issue is, they won't | 20:59 |
| fungi | mtreinish: i vote good old-fashioned human decision-making, maybe based on semver if you want something process-oriented to help you make the decision | 20:59 |
| clarkb | sdague: why not? | 20:59 |
| mtreinish | clarkb: we're pushing a tag each release and eol | 20:59 |
| sdague | because we're going to have 4 releases a year | 20:59 |
| clarkb | sdague: I don't think that changes anything | 21:00 |
| clarkb | 2014.1 then 15 months later 2014.1.eol | 21:00 |
| sdague | clarkb: so what would you call the july release? | 21:00 |
| *** homeless_ has joined #openstack-infra | 21:00 | |
| sdague | clarkb: we don't have branches | 21:00 |
| clarkb | sdague: doesn't matter | 21:00 |
| sdague | it's not 2014.1.eol | 21:00 |
| clarkb | tags are arbitrary | 21:00 |
| sdague | right, sure | 21:00 |
| sdague | but if we call this one 2014.1 | 21:01 |
| fungi | i think you should avoid painting yourselves into a corner of devising a non-extensible versioning scheme on the assumption that you'll only ever release quarterly | 21:01 |
| sdague | then the july one is 2014.2 | 21:01 |
| sdague | then the oct one is 2014.3 | 21:01 |
| stevebaker | sdague, clarkb: hmm, in that case we need to have this conversation when lifeless is sentient | 21:01 |
| clarkb | sdague: why? | 21:01 |
| *** aconrad has joined #openstack-infra | 21:01 | |
| *** sarob has quit IRC | 21:01 | |
| mtreinish | clarkb: it's eol of havana | 21:01 |
| mtreinish | not anything in tempest | 21:01 |
| clarkb | mtreinish: right so why would you make it 2014.3 | 21:01 |
| sdague | no, 2014.3 would be the 3rd release this year | 21:02 |
| sdague | which would be coincident with the 2014.2 os release | 21:02 |
| clarkb | right and I am saying don't do that | 21:02 |
| *** yamahata has quit IRC | 21:02 | |
| clarkb | make the tags mean someting against the releases they test(ed) | 21:02 |
| mtreinish | clarkb: oh I see what your saying. We just tag havana eol as something more descriptive | 21:02 |
| fungi | Shrews: that failure doesn't look related to the system on which the job ran... looks more like the neutron install in devstack wasn't configured for a large enough address pool (or didn't return some it should have, or what have you) | 21:03 |
| clarkb | yes havana was 2013.2 right? so 2013.2.eol | 21:03 |
| sdague | so we release 2014.1 ? | 21:03 |
| sdague | then release 2013.2.eol | 21:03 |
| sdague | ? | 21:03 |
| clarkb | yse | 21:03 |
| sdague | which is actually newer than 2014.1 | 21:03 |
| *** homeless has quit IRC | 21:03 | |
| clarkb | think of it less as a release and more a marker in time | 21:03 |
| sdague | which then confuses people because they think 2014.1 is ++ 2013.2.eol | 21:04 |
| fungi | sdague: i believe 2013.2.eol will end up being interpreted as a prerelease | 21:04 |
| mtreinish | sdague: we could do 2014.1.2013.2.eol :) | 21:04 |
| clarkb | at this point in time you could test foo, and at this other point in time you no longer test bar | 21:04 |
| mordred_phone | stevebaker: I'd like to be part of it too | 21:04 |
| stevebaker | mordred_phone: ok | 21:04 |
| sdague | I think people are going to read way more into these versions | 21:04 |
| *** jooools has quit IRC | 21:04 | |
| fungi | sdague: mtreinish: clarkb: i think if the discussion of how to embed meaning into your version numbers is this complex, you should just go with incrementing versions and not try to have them mean anything special beyond what normal semver would imply | 21:05 |
| mordred_phone | sdague: yes they will | 21:05 |
| stevebaker | mordred_phone, clarkb, sdague, sounds like this needs to be a mailing list thread | 21:05 |
| mordred_phone | stevebaker: about pypi mirrors? | 21:05 |
| sdague | stevebaker: which complicated discussion? dib or tempest names? | 21:05 |
| fungi | sdague: mtreinish: clarkb: including not using year-based version numbers, because people will think they mean something special too | 21:06 |
| stevebaker | dib | 21:06 |
| sdague | stevebaker: sure | 21:06 |
| mtreinish | fungi: yeah that's a good point | 21:06 |
| mtreinish | 1.0 it is | 21:06 |
| *** boris-42 has quit IRC | 21:06 | |
| * mordred_phone actually thinks its an irc convo | 21:06 | |
| mordred_phone | it's gonna get complex quickly | 21:07 |
| clarkb | fungi: the problem with your scheme is now you need a lookup table | 21:07 |
| mordred_phone | I'd dive in now, but ... phone | 21:07 |
| clarkb | and no one will understand that | 21:07 |
| mtreinish | clarkb: is that a problem though | 21:07 |
| clarkb | mtreinish: yes, its already a problem for the clients | 21:07 |
| fungi | clarkb: i think it's going to need a lookup table regardless, because the list of openstack features and releases against which a given version of tempest gets tested and confirmed to work will change over time and depend on tempest support and stable branch maintenance policy decisions | 21:07 |
| clarkb | fungi: my scheme is te lookup table though | 21:08 |
| *** boris-42 has joined #openstack-infra | 21:08 | |
| sdague | stevebaker: so I'll give you my openning argument here. dib & friends have largely lived in an alternate reality which was gateless. As we want to use them in existing gate jobs we should really bring them into a gateful jobs. As such, we should really as much as possible use the model that we've already proven here, at least as a starting point. | 21:09 |
| fungi | clarkb: you're suggesting to tag with a range of openstack releases (ignoring stable point releases) i guess? | 21:09 |
| sdague | but I'm happy to have the conversation in either irc or email | 21:10 |
| *** lcheng_ has quit IRC | 21:10 | |
| clarkb | fungi: I am suggesting that we have pairs of tags. The first denotes start of support and the second denotes removal of support | 21:10 |
| *** lcheng_ has joined #openstack-infra | 21:10 | |
| sdague | clarkb: well the problem is that's not the only change between those 2 tags | 21:10 |
| clarkb | fungi: so that via git tag | grep you can know very easily if a given release is supported | 21:10 |
| sdague | we also had 500 new tests | 21:10 |
| clarkb | if I git clone tempest and want to test my havana cloud I do git tag | grep 2013.2.eol && git checkout 2013.2.eol or checkout master | 21:11 |
| sdague | clarkb: I'm not sure that 2014.2 would tell you it could test 2014.1 | 21:11 |
| clarkb | and I am good to go | 21:11 |
| clarkb | sdague: it doesn't | 21:11 |
| stevebaker | sdague: yes, that is what I had assumed, and my first attempt implemented just that but the approach was -2ed https://review.openstack.org/#/c/92035/ (even though most tripleo elements install openstack components from git :/ ) | 21:12 |
| clarkb | 2014.2 tells you support for koala started | 21:12 |
| clarkb | or whatever reelase we are on. I have them mixed up. is 2014.2 juno? | 21:12 |
| *** MarkAtwood has quit IRC | 21:12 | |
| fungi | juno | 21:13 |
| *** aysyd has quit IRC | 21:13 | |
| SpamapS | fungi: hm, so same place now.. I see the image snapshot as "queued" | 21:13 |
| sdague | stevebaker: right, so those reviews seem completely about the tripleo side, not using these tools outside of tripleo. | 21:14 |
| fungi | clarkb: kauai will be 2015.1 | 21:14 |
| sdague | which is the alternate universe problem | 21:14 |
| SpamapS | kauai France is so lovely. | 21:14 |
| fungi | isn't it though? | 21:14 |
| clarkb | do we have a list of k names yet? | 21:15 |
| SpamapS | hah, france is like HK with I | 21:15 |
| fungi | SpamapS: so, the snapshot call getting queued on the nova side sounds like cloud brokenness to me | 21:15 |
| stevebaker | sdague: the DIB_REPO* in https://review.openstack.org/#/c/92260/1/lib/heat is the other half, of that universe | 21:15 |
| clarkb | ttx should get on that since france is his country | 21:15 |
| SpamapS | there are no cities in france that start with K | 21:15 |
| SpamapS | http://www.french-at-a-touch.com/Index_Pages/index_alphabetical_to_french_towns.htm#K | 21:15 |
| clarkb | clearly we need to start picking venues with a consideration for release naming | 21:15 |
| *** dstanek is now known as dstanek_zzz | 21:16 | |
| stevebaker | SpamapS: choose from Belgium, just to piss them off http://users.skynet.be/pjansse1/genealogy/beplaatsen.html#K | 21:16 |
| *** hashar has quit IRC | 21:16 | |
| clarkb | Koblenz | 21:16 |
| sdague | stevebaker: right, and I'm holding the -2 on that part of the tree until we decide these tools exist independently and can be used in the gate outside of tripleo. | 21:16 |
| clarkb | it is near to france | 21:16 |
| SpamapS | stevebaker: they're all french right? ;) | 21:17 |
| SpamapS | http://www.francethisway.com/indexes/indexofplaces-k.php | 21:17 |
| SpamapS | thats it | 21:17 |
| fungi | clarkb: yes, i guess lānaʻi will need to be 2015.2 | 21:17 |
| SpamapS | Kintzheim | 21:17 |
| *** lcheng_ has quit IRC | 21:17 | |
| SpamapS | Alsace ... of course | 21:17 |
| SpamapS | Kaysersberg | 21:17 |
| stevebaker | Koolskamp | 21:17 |
| SpamapS | say that 10 times fast | 21:17 |
| sdague | there is a keppler street in paris, I think that's what ttx thought would win the day | 21:18 |
| mtreinish | sdague: shouldn't win be in quotes... | 21:18 |
| *** andreykurilin_ has joined #openstack-infra | 21:18 | |
| sdague | sure? | 21:18 |
| * fungi is a fan of johannes kepler and would totally keep mistyping that release name | 21:19 | |
| sdague | fungi: or the right spelling | 21:19 |
| stevebaker | Its Rue Kepler | 21:19 |
| sdague | never trust my spelling | 21:19 |
| fungi | oh, in that case i'm all in | 21:19 |
| mtreinish | sdague: I guess air quotes, just hinting that he'd rig it | 21:20 |
| clarkb | we have too many changes | 21:20 |
| clarkb | I keep running into changes that don't just conflict at a file level btu conflict on a fundamental level | 21:20 |
| reed | Kaysersberg | 21:21 |
| mtreinish | fungi: it's ok, I think Ice House street was 2 words, not 1 like the release is | 21:22 |
| reed | a typical French name | 21:22 |
| mtreinish | so we can tweak it a little | 21:22 |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/devstack-gate: Modify horizon log copy for Fedora https://review.openstack.org/93251 | 21:23 |
| *** melwitt1 has quit IRC | 21:23 | |
| *** pdmars has quit IRC | 21:23 | |
| *** melwitt has joined #openstack-infra | 21:23 | |
| *** mbacchi has quit IRC | 21:23 | |
| ianw | sdague: thanks! https://review.openstack.org/#/c/93251/ is the last one, just a rebase because other stuff got merged in front of it | 21:25 |
| *** dstanek_zzz is now known as dstanek | 21:25 | |
| ianw | clarkb: re https://review.openstack.org/#/c/93862/, adding txt before wouldn't be possible at least with the webui? we could make it match "workflow" if you want | 21:26 |
| *** igor_ has joined #openstack-infra | 21:28 | |
| clarkb | ianw: I think workflow is too restrictive but if I wrote a comment that was foo\n\nrecheck no bug and didn't leave a workflow comment that would match right? | 21:28 |
| clarkb | ianw: in any case I don't think it matters in reality | 21:28 |
| *** sarob has joined #openstack-infra | 21:29 | |
| *** esker has quit IRC | 21:30 | |
| *** esker has joined #openstack-infra | 21:31 | |
| SpamapS | fungi: agreed it looks like brokenness.. but I'm having a little trouble figuring out how to debug said breakgae | 21:31 |
| SpamapS | breakage | 21:31 |
| *** Sukhdev has joined #openstack-infra | 21:32 | |
| fungi | SpamapS: trying to make a snapshot of a vm is generally working there i guess? (odd that it worked for the f20 image) | 21:32 |
| fungi | ravips: i've cleared your gerrit ssh username. you should be able to try setting it again now | 21:32 |
| openstackgerrit | A change was merged to openstack-infra/config: django_openstack_auth: make the Python 3.3 gate voting https://review.openstack.org/94442 | 21:32 |
| SpamapS | fungi: I'm testing that now | 21:32 |
| *** igor_ has quit IRC | 21:32 | |
| mtreinish | clarkb, fungi, sdague: I think I'm just going to going go with 1.0. I think the other way would be a bit more confusing. | 21:32 |
| fungi | SpamapS: not over quota on snapshot space/count or out of disk on the backend or anything? | 21:33 |
| ravips | fungi: awesome, it worked...thanks | 21:34 |
| *** gokrokve has quit IRC | 21:34 | |
| *** sarob has quit IRC | 21:34 | |
| SpamapS | fungi: I don't see quotas for glance | 21:35 |
| SpamapS | fungi: actually I was just able to create a snap of a vm I booted | 21:36 |
| *** esker has quit IRC | 21:36 | |
| SpamapS | oh! | 21:36 |
| SpamapS | fungi: the template worked | 21:36 |
| SpamapS | snap finished | 21:36 |
| clarkb | msabramo1: I just reviewed your change for making jenkins-job-builder-coverage. let me know if you have questions about my suggestions. Happy to help sort that out | 21:36 |
| SpamapS | | 02edc86e-fbef-4811-a177-7d648545e15f | tripleo-precise-1400790950.template.openstack.org | qcow2 | bare | 5130092544 | active | | 21:36 |
| *** duran has quit IRC | 21:38 | |
| SpamapS | fungi: ok, so should I expect to see nodepool clear out all the old precise nodes and create new nodes? | 21:38 |
| openstackgerrit | A change was merged to openstack-infra/reviewday: Add Marconi and its client to ReviewDay https://review.openstack.org/93901 | 21:39 |
| *** signed8bit has quit IRC | 21:39 | |
| *** nati_ueno has joined #openstack-infra | 21:40 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Gate on Postrgres in trove https://review.openstack.org/94307 | 21:42 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Add tempest-dsvm-postgres-full into the integrated gate https://review.openstack.org/94311 | 21:42 |
| SlickNik | SergeyLukjanov: Thanks for fixing the trove-postgres rebase! | 21:42 |
| SlickNik | SergeyLukjanov: Was about to look into it, and there you were. :) | 21:42 |
| clarkb | fungi: 94601 is tested? | 21:43 |
| clarkb | fungi: I will approve that nowish if yo uare happy with it | 21:43 |
| SergeyLukjanov | SlickNik, np :) | 21:43 |
| *** mfer has quit IRC | 21:44 | |
| msabramo1 | clarkb: Thanks! | 21:44 |
| openstackgerrit | A change was merged to openstack-infra/config: Add missing export -f post_test_hook in rally-cli https://review.openstack.org/94601 | 21:44 |
| *** signed8bit has joined #openstack-infra | 21:45 | |
| mtreinish | fungi, clarkb: one last question before I push the tag. If I push it and we want to change the tag to use clarkb's scheme instead is it possible to drop the tag once it's out in the wild? | 21:46 |
| *** malini is now known as malini_afk | 21:46 | |
| mtreinish | I'm debating whether I should ask on the ML for opinions first | 21:46 |
| clarkb | ok I have quota, creating networks and routers now | 21:46 |
| mtreinish | instead of having people complain to me after I push something that is confusing down the road | 21:46 |
| clarkb | let me know if yo usee anything funny with region b nodes but this should be a noop for them | 21:46 |
| *** lcostantino has quit IRC | 21:47 | |
| clarkb | mtreinish: no you shouldn't drop tags | 21:47 |
| clarkb | tags should be treated as permanent | 21:47 |
| mtreinish | yeah that's what I figured | 21:47 |
| ianw | clarkb: i don't think so http://paste.openstack.org/show/81188/? | 21:47 |
| clarkb | because there is no way to make a downstream repo pick up the tag | 21:47 |
| clarkb | er pick up the deletion of a tag | 21:47 |
| clarkb | ianw: if you remove the Workflow-1 from the comment | 21:48 |
| clarkb | ianw: which would happe nif you don't vote | 21:48 |
| jerryz | clarkb: is there a way to skip specific test job for a branch in zuul? not just non-voting but skip it | 21:49 |
| clarkb | jerryz: yes see the branch: listings under jobs in the layout.yaml | 21:49 |
| clarkb | lololol | 21:49 |
| clarkb | so you want to know one reason for things possibly failing in region b? | 21:49 |
| clarkb | the existing network is only a /24 | 21:49 |
| jeblair | ha! | 21:50 |
| jeblair | what did i say yesterday? "oh about 50 hosts more than what we're doing now which is 200" :) | 21:50 |
| ianw | clarkb: ah, yeah. so i did that global match in case something else got inserted in there it wouldn't need to change again | 21:50 |
| ianw | clarkb: (?i)^(Patch Set [0-9]+:)?( Workflow[\w\\-]*)?(\n\n)?\s*check experimental\s*$ could work too? | 21:51 |
| ianw | but be a bit less flexible | 21:51 |
| clarkb | jeblair: I am going to create new networks as /24 as well and do 10.0.[01234].0/24 | 21:51 |
| clarkb | ianw: ya Ithink your global match is better | 21:51 |
| clarkb | we are just slightly changing what is acceptable | 21:51 |
| SpamapS | \o/ looks like tripleo precise jobs are flowing again | 21:52 |
| SpamapS | 99 tripleo patches on the wall.. 99 tripleo patches... | 21:52 |
| jerryz | clarkb: "voting: false" is turning off the vote or disable the job? | 21:53 |
| clarkb | jerryz: that turns off the voting but the job still runs | 21:53 |
| jerryz | clarkb: that is what i was concerned about, because i don't want to waste my test resource on a patch definately to fail | 21:54 |
| clarkb | jerryz: right so you can use branch: regex to select which branch to run again | 21:55 |
| openstackgerrit | A change was merged to openstack-infra/config: Gate on Postrgres in trove https://review.openstack.org/94307 | 21:56 |
| sdague | mtreinish: well the advantage of going with 1.0 | 21:56 |
| sdague | is that 2014.1 is definitely a bigger number than it | 21:56 |
| *** rfolco has quit IRC | 21:57 | |
| sdague | so if you want to move to that kind of release versioning later, it doesn't break people that compare versions | 21:57 |
| sdague | and installer the bigger one | 21:57 |
| mtreinish | sdague: yeah that's a good point. | 21:58 |
| jerryz | clarkb: i don't get it. i mean is there an option like: branch: regex running: false | 21:58 |
| mtreinish | but I think I'll still send it out to the ML for opinions first. | 21:58 |
| mtreinish | It doesn't hurt to wait a couple more days | 21:58 |
| mtreinish | because I'd rather not see things change once we decide on a scheme | 21:59 |
| jerryz | clarkb: sorry, i didn't look closely. now i get it | 22:00 |
| *** chuckC has joined #openstack-infra | 22:00 | |
| SpamapS | Hm..I still have 30 h1.large precise nodes on tripleo cloud r1 | 22:00 |
| openstackgerrit | A change was merged to openstack-infra/config: Add gnocchi project on StackForge https://review.openstack.org/93465 | 22:00 |
| SpamapS | oh, but I have one h1.ci | 22:01 |
| *** reed has quit IRC | 22:01 | |
| clarkb | ok I cheated and used horizon to create the networks and thigns mostly because remembering all the stuff that is needed vs what isn't needed is hard and horizon draws a nice topology thing | 22:01 |
| *** ramashri has quit IRC | 22:01 | |
| clarkb | but we now have five /24 networks with external connectivity via Ext-Net | 22:01 |
| clarkb | going to propose a change to split up the network usage in nodepool now | 22:02 |
| openstackgerrit | A change was merged to openstack-infra/config: Remove job compute-api-v2.0-wadl https://review.openstack.org/94104 | 22:02 |
| *** sarob has joined #openstack-infra | 22:02 | |
| *** mkoderer has quit IRC | 22:02 | |
| *** esker has joined #openstack-infra | 22:02 | |
| fungi | SpamapS: yes, seems all filled of workingishness now | 22:03 |
| *** sarob has quit IRC | 22:03 | |
| * fungi catches up on scrollback | 22:03 | |
| *** sarob has joined #openstack-infra | 22:03 | |
| openstackgerrit | A change was merged to openstack-infra/config: Handle Workflow in comment matching https://review.openstack.org/93862 | 22:04 |
| *** sarob_ has joined #openstack-infra | 22:04 | |
| *** mburned is now known as mburned_out | 22:04 | |
| *** sarob_ has quit IRC | 22:04 | |
| fungi | clarkb: no idea about 94601 (were you maybe asking about a different change than you pasted?) | 22:05 |
| clarkb | fungi: the find on static.o.o one | 22:05 |
| clarkb | 94610 | 22:06 |
| clarkb | sorry typing is hard | 22:06 |
| clarkb | SpamapS: why do we give your clouds two network ids? | 22:06 |
| *** marcoemorais has quit IRC | 22:06 | |
| clarkb | I am tryin gto understand how we can use this with hpcloud and I am beginning to wonder if this will work the way I expect | 22:06 |
| *** marcoemorais has joined #openstack-infra | 22:07 | |
| clarkb | fungi: ^ do you know? | 22:07 |
| *** marcoemorais has quit IRC | 22:07 | |
| *** marcoemorais has joined #openstack-infra | 22:08 | |
| *** sarob has quit IRC | 22:08 | |
| *** sarob_ has joined #openstack-infra | 22:08 | |
| SpamapS | clarkb: because our baremetal nodes are on a special provider network | 22:08 |
| derekh_ | clarkb: its because you are creating instances with 2 nics , one on the default network and the other on the test network (where it talks to the other VM's in the ci job) | 22:08 |
| fungi | clarkb: tested as closely as i can without actually modifying the cron job directly, yeah | 22:08 |
| clarkb | ah ok so we are fine as we just have teh default | 22:08 |
| clarkb | fungi: great I will finish reviewing and approve once I finish this nodepool config update | 22:09 |
| mattoliverau | morning | 22:09 |
| SpamapS | clarkb: the testenv boxes are the ones running the heavy lifting / pxe boot vms .. and they're not on the 10.x network, they're on a 192.x flat network | 22:09 |
| fungi | clarkb: i did at least check to see what would get matched so we know what else is going to be deleted when it runs this weekend | 22:09 |
| *** marcoemorais has quit IRC | 22:09 | |
| *** marcoemorais has joined #openstack-infra | 22:09 | |
| anteaya | morning mattoliverau | 22:09 |
| *** jamielennox|away is now known as jamielennox | 22:11 | |
| *** eharney has quit IRC | 22:11 | |
| *** andreaf has quit IRC | 22:12 | |
| *** esker has quit IRC | 22:15 | |
| SpamapS | derekh_: I thought the "test network" was required because the testenvs are baremetal, not vms, and that is a flat network. No? | 22:15 |
| *** boris-42 has quit IRC | 22:17 | |
| SpamapS | Ok so now that we have CI flowing again, we also found that most of our F20 jobs were failing .. we've merged a fix, but we have to recheck basically everything.. | 22:18 |
| *** smarcet has quit IRC | 22:18 | |
| SpamapS | so, can has a queue clear for check-tripleo ? | 22:18 |
| *** david-lyle has quit IRC | 22:18 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Use network specific hpcloud region-b providers https://review.openstack.org/95027 | 22:18 |
| *** david-lyle has joined #openstack-infra | 22:18 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: The process for making a Third-Party CI voting needs docs https://review.openstack.org/93562 | 22:18 |
| *** david-lyle has quit IRC | 22:19 | |
| clarkb | mordred_phone: jeblair fungi SergeyLukjanov et al ^ | 22:19 |
| clarkb | note I left the old region-b provider in place as I think that will avoid the bug we ran into yesterday | 22:19 |
| clarkb | I just remove it from providing node flavors | 22:19 |
| clarkb | this change will also need to build new images so there will be a longer lag period | 22:20 |
| jeblair | clarkb: can we give them shorter names? :) | 22:20 |
| clarkb | jeblair: sure, what do you want to call them? | 22:20 |
| clarkb | b1, b2, etc? | 22:20 |
| jeblair | is 'region-b' still the right thing to call that? i can never remember. | 22:20 |
| clarkb | yes they are all in region-b | 22:20 |
| jeblair | yeah, so i think "hpcloud-b1" gets all the interesting bits in there... | 22:21 |
| derekh_ | SpamapS: maybe , I'm not sure it was required although iirc keeping both net's separate was preferred | 22:21 |
| clarkb | jeblair: ya, I can rename them to that | 22:21 |
| *** boris-42 has joined #openstack-infra | 22:21 | |
| jeblair | clarkb: (i'm mostly thinking about what the full hostname will look like :) | 22:22 |
| *** BadCub01_ has quit IRC | 22:24 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Use network specific hpcloud region-b providers https://review.openstack.org/95027 | 22:24 |
| clarkb | jeblair: ^ renamed | 22:24 |
| *** flaper87 is now known as flaper87|afk | 22:24 | |
| *** esker has joined #openstack-infra | 22:25 | |
| *** nati_ueno has quit IRC | 22:25 | |
| *** nati_ueno has joined #openstack-infra | 22:25 | |
| *** lakshmiS has quit IRC | 22:26 | |
| SpamapS | clarkb: who or how do I ask for a clear of the check-tripleo queue? | 22:27 |
| clarkb | SpamapS: uh | 22:27 |
| clarkb | you want everything to be evicted? | 22:27 |
| SpamapS | well if you have a way of saying "all of the patches that failed f20 checks already" | 22:27 |
| clarkb | or do you just need a few tests at the front? | 22:27 |
| *** chuckC has quit IRC | 22:27 | |
| jeblair | SpamapS: would you mind writing the "zuul dequeue" command? :) | 22:27 |
| SpamapS | clarkb: that is 60 out of the 97 | 22:28 |
| SpamapS | so a full flush is fine. | 22:28 |
| clarkb | SpamapS: I ask because we can't do that without restarting zuul | 22:28 |
| clarkb | and so am trying to udnerstand what you really need | 22:28 |
| SpamapS | jeblair: I don't mind putting some time in to make this easier if you have a spec in mind. :) | 22:28 |
| *** igor_ has joined #openstack-infra | 22:28 | |
| SpamapS | clarkb: so we had a bad patch making the f20 test fail.. | 22:29 |
| SpamapS | clarkb: meanwhile all the precise nodes were dead because FAIL .. | 22:29 |
| clarkb | SpamapS: sure, so do we simply need to get the fix for f20 merged? | 22:29 |
| SpamapS | clarkb: now we have precise nodes catching up, but they have about 60 already failed f20's to get through, and those have to be rechecked again. | 22:29 |
| SpamapS | clarkb: fix is already merged | 22:29 |
| *** esker has quit IRC | 22:30 | |
| SpamapS | actually I stand corrected, it is not merged yet.. https://review.openstack.org/#/c/94800/ is the fix | 22:30 |
| SpamapS | clarkb: so the fix will go in soon enough | 22:31 |
| *** giulivo has quit IRC | 22:31 | |
| jeblair | SpamapS: you can push new patchsets for all the changes | 22:31 |
| *** jergerber has quit IRC | 22:32 | |
| fungi | SpamapS: or, openstack-infra/zuul needs a patch adding the inverse of zuul/cmd/client.py:Client.enqueue() added | 22:32 |
| SpamapS | jeblair: will that cancel in-progress CI? | 22:32 |
| jeblair | SpamapS: (and the dequeue command should mirror the existing enqueue command) | 22:32 |
| jeblair | SpamapS: yes | 22:32 |
| SpamapS | Or we could just let the queue flush out | 22:33 |
| *** changbl has quit IRC | 22:33 | |
| fungi | SpamapS: go take a nap and it will fix itself ;) | 22:33 |
| *** igor_ has quit IRC | 22:33 | |
| *** esker has joined #openstack-infra | 22:33 | |
| SpamapS | yeah thats what I'm tempted to let happen | 22:33 |
| SpamapS | actually | 22:33 |
| SpamapS | I will go to a Billy Joel concert and it will fix itself. :) | 22:33 |
| fungi | though a zuul dequeue command *would* be really useful, ftr | 22:33 |
| jeblair | fungi: ++ | 22:33 |
| * SpamapS clones zuul and takes a look | 22:34 | |
| jeblair | SpamapS: i think you should write the dequeue command while at a billy joel concert. | 22:35 |
| SpamapS | jeblair: I'd rather laugh with the sinners than dequeue patches with the saints. The sinners are much more fun. | 22:35 |
| SpamapS | only the good patches die young! | 22:35 |
| clarkb | mordred_phone: you don't happen to be around do you? | 22:36 |
| clarkb | I think it would be great to test this network stuff | 22:36 |
| clarkb | jeblair: fungi: thoughts on that? probably best to do it when we are not renaming projects | 22:36 |
| *** thedodd has quit IRC | 22:36 | |
| jeblair | clarkb: no time like the present | 22:37 |
| *** marcoemorais has quit IRC | 22:37 | |
| clarkb | ++ | 22:37 |
| *** arborism has joined #openstack-infra | 22:37 | |
| *** marcoemorais has joined #openstack-infra | 22:37 | |
| jeblair | oh i suppose i should formalize that with a +2 | 22:38 |
| clarkb | :) | 22:38 |
| clarkb | fungi: you too ^ | 22:38 |
| jesusaurus | clarkb: jeblair: do we still want to try to merge that refactor today? | 22:38 |
| clarkb | fungi: I am approving your find changes now | 22:38 |
| clarkb | jesusaurus: it has my +2 | 22:38 |
| clarkb | if someone else reviews it I am happy to | 22:38 |
| jesusaurus | it currently has 2 +2's | 22:39 |
| clarkb | oh | 22:39 |
| clarkb | 897XX? | 22:39 |
| jeblair | i'm around for about 1 more hour so i can't commit to being around to hit the revert button for that anymore | 22:39 |
| fungi | clarkb: just finished reviewing and lgtm, but i didn't actually use neutronclient to verify the net-ids | 22:40 |
| jeblair | clarkb: so... one more thing -- that's going to kick off 25 simultaneous image builds | 22:40 |
| clarkb | fungi: I copy pasta'd them and have a high degree of confidence they are correct | 22:40 |
| clarkb | jeblair: oh right, so maybe we hold off on jesusaurus' thing otherwise we may break all those uilds | 22:40 |
| jeblair | clarkb: we may want to reduce our overall max to 570 | 22:41 |
| *** signed8bit has quit IRC | 22:41 | |
| jeblair | clarkb: well, we could use all those builds to test jesusaurus's thing :) | 22:41 |
| clarkb | jeblair: 114 max servers for eac? | 22:41 |
| jeblair | clarkb: yeah | 22:41 |
| clarkb | I can make that change | 22:41 |
| jeblair | clarkb: cool. and hopefully after the image refactor we'll be back up to 600 | 22:41 |
| *** mwagner_lap has joined #openstack-infra | 22:42 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Use network specific hpcloud region-b providers https://review.openstack.org/95027 | 22:42 |
| fungi | wfm | 22:42 |
| clarkb | there it is | 22:42 |
| *** dstanek is now known as dstanek_zzz | 22:42 | |
| jeblair | aprvd | 22:43 |
| clarkb | jeblair: if this hpcloud thing goes well I am happy to be around to approve your change | 22:43 |
| clarkb | er jesusaurus ^ | 22:43 |
| clarkb | I just don't want to have too many pots on the stove | 22:43 |
| jesusaurus | okay, cool | 22:44 |
| fungi | oh, heh, double-approved ;) | 22:44 |
| *** bknudson has quit IRC | 22:44 | |
| jeblair | jesusaurus: i think we're still working out the logistics of this. fwiw, i think it's totally cool if you show up and say "merge my change now!". i think when we leave a comment in gerrit saying "ping us when you're ready" we're more or less ready to merge it whenever... | 22:45 |
| anteaya | so how many nodes to we have now? | 22:45 |
| clarkb | anteaya: ~370 more | 22:45 |
| anteaya | the graph has space for 1500 | 22:45 |
| anteaya | yay for more | 22:46 |
| clarkb | but if this works we will be killing off hpcloud 1.0 | 22:46 |
| clarkb | and losing about that many | 22:46 |
| anteaya | what was it before the 370 more? | 22:46 |
| jeblair | jesusaurus: basically trying to put you in the drivers seat as much as possible | 22:46 |
| clarkb | anteaya: I don't know the hard numbers | 22:46 |
| anteaya | hpcloud 1.0 is az-1, -2, and -3 yes? | 22:46 |
| clarkb | anteaya: yes | 22:46 |
| anteaya | k, I don't feel so bad for not knowing myself then | 22:46 |
| anteaya | cool | 22:46 |
| *** salv-orlando has quit IRC | 22:48 | |
| anteaya | pleia2: the docs look better now: https://review.openstack.org/#/c/93562/ | 22:48 |
| anteaya | I needed to add an empty line before the bullet points began | 22:49 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added search icon to typeahead fields https://review.openstack.org/94273 | 22:50 |
| jesusaurus | jeblair: ah, okay. i was taking it more as a "we dont have time for this right now" because i don't think im in much of a position to debug nodepool if changing single_use_slave has an unforseen issue | 22:52 |
| *** andreykurilin_ has quit IRC | 22:52 | |
| *** mrodden has quit IRC | 22:52 | |
| pleia2 | anteaya: cool, thanks | 22:53 |
| *** wenlock_ has joined #openstack-infra | 22:55 | |
| jesusaurus | jeblair: im pretty much ready whenever to watch zuul and puppetboard like a hawk | 22:55 |
| clarkb | jesusaurus: in particular the nodepool change relies on working jenkins puppet | 22:55 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Don't accept tasks for stopped managers https://review.openstack.org/95034 | 22:56 |
| clarkb | to build new slave images | 22:56 |
| jesusaurus | clarkb: yeah, i was dealing with similar issues a few days ago with hp's downstream nodepool + zuul deployment | 22:58 |
| jesusaurus | but im still learning how all this works together | 22:58 |
| jeblair | jesusaurus: yeah, i think you may be being too nice and waiting for a nice quiet time for us to do it :). it's probably better earlier in the day as it usually takes several hours for problems to manifest. so the takeaway is: yell at us early and often. :) | 22:58 |
| *** blamar has quit IRC | 22:58 | |
| jesusaurus | ha, okay, i'll try to be louder earlier | 22:59 |
| *** esker has quit IRC | 22:59 | |
| anteaya | pleia2: thanks for spotting it | 22:59 |
| jeblair | clarkb, fungi: https://review.openstack.org/95034 is a quick and dirty "fix" for the problems from this morning. | 22:59 |
| *** esker has joined #openstack-infra | 22:59 | |
| jhesketh | Morning | 23:00 |
| clarkb | hrm a config pep8 is stuck in the gate so the nodepool config change isnt going through | 23:01 |
| fungi | jeblair: clarkb: if we can get that merged too, i think we probably have reasons for nodepool and zuul restarts coincident with the gerrit restart for tomorrow's project renames | 23:01 |
| * SpamapS is about to turn into a pumpkin | 23:02 | |
| jeblair | fungi: ++ | 23:02 |
| clarkb | ++ I will review now | 23:02 |
| *** wenlock_ has quit IRC | 23:03 | |
| clarkb | I am going to manually kill that pep8 job then reapprove that change | 23:04 |
| *** SumitNaiksatam has quit IRC | 23:04 | |
| *** dstanek_zzz is now known as dstanek | 23:04 | |
| fungi | jeblair: i take it ProviderManager.submitTask() was just unused cruft? | 23:04 |
| jeblair | fungi: afaict | 23:04 |
| fungi | k | 23:04 |
| jeblair | i think the tests cover that enough to tell us | 23:04 |
| *** esker has quit IRC | 23:04 | |
| fungi | agreed | 23:05 |
| anteaya | morning jhesketh | 23:06 |
| *** chuckC has joined #openstack-infra | 23:06 | |
| *** MarkAtwood has joined #openstack-infra | 23:06 | |
| * anteaya hands SpamapS a glass shoe, just in case | 23:06 | |
| *** lttrl has quit IRC | 23:07 | |
| jhesketh | hello anteaya | 23:07 |
| jhesketh | did I miss anything fun? | 23:07 |
| openstackgerrit | A change was merged to openstack-infra/config: Update puppet-openstack gerritbot configuration https://review.openstack.org/94748 | 23:07 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack-infra/zuul: Add a dequeue command https://review.openstack.org/95035 | 23:07 |
| SpamapS | bibbety bobbety dequeue | 23:07 |
| SpamapS | I have _NO_ idea if that will work | 23:07 |
| SpamapS | but it looks good to my layman eyes | 23:08 |
| clarkb | jeblair: fungi I approved te nodepool change | 23:08 |
| lifeless | clarkb: this is consistent | 23:08 |
| openstackgerrit | A change was merged to openstack-infra/config: Use network specific hpcloud region-b providers https://review.openstack.org/95027 | 23:09 |
| fungi | SpamapS: yay! | 23:09 |
| lifeless | clarkb: its not consistent with *devstack*, but this isn't devstack. | 23:09 |
| clarkb | here we go, should update in ~5 minutes | 23:09 |
| clarkb | lifeless: its not consistent with any of the testing we do | 23:10 |
| clarkb | lifeless: devstack or not its all from source | 23:10 |
| lifeless | so warning - I have some nasty lurgy and my brain is basically out of commission | 23:10 |
| openstackgerrit | A change was merged to openstack-infra/config: Adding openstack-sdk-dotnet, openstack-cli-powershell, openstackclient, and python-openstacksdk to gerritbot config. https://review.openstack.org/94683 | 23:11 |
| lifeless | clarkb: install from . and install from published package test slightly different things | 23:11 |
| davidlenwell | lifeless: nasty lurgy? | 23:11 |
| anteaya | jhesketh: well nodepool was having an interesting time with hpcloud region-b today | 23:11 |
| anteaya | you were there for the start of our awareness around that | 23:11 |
| lifeless | clarkb: we strive for production equivalence, and the only things we install straight from ./git are those things that don't do releases. | 23:11 |
| lifeless | davidlenwell: diarrhoea, nausea, fever, painful stomach bloat | 23:11 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Don't accept tasks for stopped managers https://review.openstack.org/95034 | 23:12 |
| davidlenwell | lifeless: conference plauge | 23:12 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack-infra/zuul: Add a dequeue command https://review.openstack.org/95035 | 23:12 |
| lifeless | davidlenwell: other folk have reported the same symptoms? | 23:12 |
| davidlenwell | lifeless: I had them | 23:12 |
| *** derekh_ has quit IRC | 23:12 | |
| *** blamar has joined #openstack-infra | 23:12 | |
| lifeless | davidlenwell: ugh; hope you're better now, cause its a doozy | 23:12 |
| anteaya | lifeless: this is the first conf in a long time I haven't been sick | 23:12 |
| anteaya | lifeless: sorry to hear you are | 23:12 |
| anteaya | :( | 23:13 |
| davidlenwell | lifeless: I'm at the tail end .. only a flemmy cough remains | 23:13 |
| davidlenwell | I drank like 5 gallons of water in two days and slept a lot | 23:13 |
| lifeless | when I drink my stomach growls for like 10 minutes, its most unpleasant | 23:13 |
| lifeless | i'm just managing to force gastrolyte down | 23:13 |
| openstackgerrit | A change was merged to openstack-infra/config: Remove old non-gz files in log archive maintenance https://review.openstack.org/94610 | 23:14 |
| clarkb | lifeless: I'm not qute sure I grok what you intend to do with say nova | 23:14 |
| clarkb | since it does releases but not pacakges | 23:14 |
| *** UtahDave has quit IRC | 23:15 | |
| lifeless | clarkb: it doesn't do releases often enough to be useful, so we install that (and other things in its category directly) | 23:16 |
| lifeless | clarkb: but everything else we take published releases | 23:16 |
| *** dstanek is now known as dstanek_zzz | 23:16 | |
| ArxCruz | guys, I know you're from infra, but probably someone know about scheduler | 23:17 |
| ArxCruz | I have 4 nodes, one with twice the memory of the others | 23:17 |
| anteaya | hey ArxCruz we missed you at summit | 23:17 |
| ArxCruz | and all vm's are being instantiated in this node, untill there's less ram available than the others | 23:17 |
| anteaya | Paris dude, Paris | 23:17 |
| ArxCruz | anteaya: I will be in Paris for sure :) | 23:18 |
| *** pcm__ has quit IRC | 23:18 | |
| ArxCruz | then we can drink a beer \o/ | 23:18 |
| *** james_li has quit IRC | 23:18 | |
| ArxCruz | need to follow sdague in the beer list | 23:18 |
| ArxCruz | anteaya: you know about scheduler? I need help, my vm's are being spawned only in the node with twice ram | 23:19 |
| clarkb | ArxCruz: try nova? | 23:19 |
| ArxCruz | clarkb: nova-scheduler you mean? | 23:19 |
| *** openstackgerrit has quit IRC | 23:19 | |
| anteaya | no, I don't know anything about scheduler, sorry | 23:20 |
| *** esker has joined #openstack-infra | 23:20 | |
| ArxCruz | :( | 23:20 |
| anteaya | hey openstackgerrit, come back | 23:20 |
| clarkb | ArxCruz: I mean ask nova | 23:20 |
| anteaya | jhesketh: these are kind of important: https://review.openstack.org/#/q/status:open+project:openstack-infra/config+branch:master+topic:renaming-may-23,n,z | 23:20 |
| *** SumitNaiksatam has joined #openstack-infra | 23:20 | |
| ArxCruz | clarkb: okay, thanks | 23:20 |
| anteaya | jhesketh: since we will be renaming tomorrow/today depending on your pov | 23:20 |
| jhesketh | thanks anteaya, I'll take a look | 23:20 |
| anteaya | thanks | 23:20 |
| anteaya | so openstackgerrit left | 23:21 |
| clarkb | fungi: is puppet disabled on nodepool.o.o? | 23:21 |
| clarkb | how does one check? | 23:21 |
| *** openstackgerrit has joined #openstack-infra | 23:21 | |
| anteaya | oh, there he is | 23:21 |
| clarkb | I really don't care for puppet agent --enable|disable | 23:21 |
| clarkb | I wish there was a more explicit way of denoting that | 23:21 |
| fungi | clarkb: hrm... looking | 23:22 |
| *** nati_ueno has quit IRC | 23:23 | |
| fungi | clarkb: does not seem to be disabled because the minor tweaks i made to the max-servers for tripleo regions (to get nodepoold to create new provider objects for them) have been magically reverted as i hoped | 23:24 |
| *** virmitio has quit IRC | 23:24 | |
| clarkb | k | 23:24 |
| clarkb | maybe puppet doesn't run as quickly as I expect | 23:24 |
| fungi | so probably just need to wait | 23:24 |
| clarkb | the new region b providers are there now | 23:24 |
| clarkb | images are building | 23:24 |
| clarkb | and nodes are attached to my new networks | 23:25 |
| *** atiwari has quit IRC | 23:26 | |
| *** markmcclain1 has quit IRC | 23:29 | |
| *** markmcclain has joined #openstack-infra | 23:29 | |
| *** igor_ has joined #openstack-infra | 23:29 | |
| *** sarob_ has quit IRC | 23:29 | |
| anteaya | SergeyLukjanov: this needs another patchset: https://review.openstack.org/#/c/94912/ | 23:31 |
| anteaya | you had barbican in your fingers in the blazar patchset | 23:31 |
| *** Sukhdev has quit IRC | 23:32 | |
| *** sarob has joined #openstack-infra | 23:32 | |
| *** lcheng_ has joined #openstack-infra | 23:33 | |
| *** igor_ has quit IRC | 23:34 | |
| clarkb | still building images | 23:34 |
| anteaya | so we have russellb's scripts: http://git.openstack.org/cgit/openstack-infra/reviewstats/tree/ | 23:39 |
| anteaya | do we run and publish them anywhere? | 23:39 |
| anteaya | russellb publishes them on his server, but they haven't been updated in a while and I don't want to bug him, he is a bit busy | 23:40 |
| *** mburned_out is now known as mburned | 23:40 | |
| *** fanhe has joined #openstack-infra | 23:41 | |
| clarkb | we dont | 23:42 |
| *** praneshp has quit IRC | 23:43 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack-infra/zuul: Add a dequeue command https://review.openstack.org/95035 | 23:43 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Add oslo.db to the devstack gate https://review.openstack.org/94173 | 23:43 |
| *** praneshp has joined #openstack-infra | 23:44 | |
| SpamapS | Ok.. so.. can we talk about check-tripleo and precise nodes? | 23:44 |
| SpamapS | I only see 5 running slaves | 23:44 |
| SpamapS | But I see 30 ACTIVE nodes on r1 | 23:45 |
| anteaya | clarkb: k | 23:46 |
| *** esker has quit IRC | 23:46 | |
| *** lcheng_ has quit IRC | 23:46 | |
| *** esker has joined #openstack-infra | 23:46 | |
| anteaya | I'm going to pack it in for the night | 23:49 |
| anteaya | g'night see y'all tomorrow | 23:49 |
| pleia2 | night anteaya | 23:49 |
| clarkb | SpamapS: r1 is hp? | 23:49 |
| *** marcoemorais has quit IRC | 23:49 | |
| *** marcoemorais has joined #openstack-infra | 23:50 | |
| clarkb | or is that rh1? | 23:50 |
| *** marcoemorais has quit IRC | 23:50 | |
| *** r-daneel has quit IRC | 23:50 | |
| *** marcoemorais has joined #openstack-infra | 23:50 | |
| clarkb | SpamapS: most of the nodes I see are in a building state | 23:51 |
| clarkb | which would explain why they are not being used for tests | 23:51 |
| *** esker has quit IRC | 23:51 | |
| SpamapS | r1 == hp yes | 23:53 |
| SpamapS | clarkb: Ok, how long do we time out from building? | 23:53 |
| clarkb | SpamapS: I am not sure, and it does vary | 23:54 |
| SpamapS | clarkb: I have nodes that have been ACTIVE for 24+ hours | 23:54 |
| clarkb | oh I don't think the ones I have are that old | 23:54 |
| SpamapS | a1d793fd-dd96-4d1b-b453-a911f73872cd | 23:54 |
| SpamapS | | updated | 2014-05-21T16:08:26Z | | 23:54 |
| SpamapS | never got a floating ip | 23:54 |
| clarkb | the oldest nodes I see are 6 hours old | 23:54 |
| SpamapS | clarkb: so that UUID, you don't have it listed? | 23:55 |
| SpamapS | tripleo-precise-tripleo-test-cloud-4121749.slave.openstack.org | 23:55 |
| clarkb | nope | 23:56 |
| clarkb | let me serach by name | 23:56 |
| SpamapS | weird | 23:56 |
| *** jgrimm has quit IRC | 23:56 | |
| jeblair | nova has this habit of returning errors and then creating instances anyway | 23:56 |
| clarkb | nothing by name either | 23:56 |
| SpamapS | clarkb: so maybe I just need to nova delete that one? | 23:56 |
| jeblair | it's pretty much impossible to keep proper accounting in that kind of situation | 23:56 |
| SpamapS | I have 28 more tripleo-precise-tripleo-test nodes that have no floating ip | 23:57 |
| SpamapS | I suspect those are all things that nodepool thought were failed | 23:57 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Removed tabs, changed to four spaces. https://review.openstack.org/94280 | 23:58 |
| *** sarob has quit IRC | 23:59 | |
| SpamapS | the fun part is that is eating away at nodepool's quota | 23:59 |
| clarkb | jeblair: (stats_counts.nodepool.launch.provider.hpcloud-b*.ready is that not a valid glob in graphite? I a mgetting no data | 23:59 |
| SpamapS | but nodepool doesn't know it exists | 23:59 |
| ianw | does anyone know if there's an existing bug about tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_compute_with_volumes failures ... not seeing one (http://paste.openstack.org/show/81191/ is the log output) | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!