| *** sabeen2 has quit IRC | 00:01 | |
| fungi | madorn: the upstream gerrit issue is tracked as https://code.google.com/p/gerrit/issues/detail?id=3831 | 00:03 |
|---|---|---|
| fungi | took me a few minutes to hunt that down | 00:03 |
| madorn | fungi: great, yeah i was lookin too, thanks | 00:03 |
| *** davideagnello has quit IRC | 00:04 | |
| openstackgerrit | Clint Adams proposed openstack-infra/project-config: Add periodic job to autogenerate devstack plugins list https://review.openstack.org/269921 | 00:05 |
| *** krotscheck1 has joined #openstack-infra | 00:06 | |
| *** salv-orl_ has quit IRC | 00:08 | |
| *** tiswanso has joined #openstack-infra | 00:08 | |
| *** krotscheck1 has quit IRC | 00:08 | |
| *** dims has quit IRC | 00:10 | |
| *** Swami has joined #openstack-infra | 00:12 | |
| *** annegentle has quit IRC | 00:12 | |
| *** tiswanso has quit IRC | 00:13 | |
| clarkb | https://jenkins01.openstack.org/job/gate-tempest-dsvm-multinode-full/2526/console is running on bluebox | 00:14 |
| *** vmbrasseur_ has joined #openstack-infra | 00:14 | |
| *** notnownikki has quit IRC | 00:20 | |
| *** annegentle has joined #openstack-infra | 00:21 | |
| *** jpr has joined #openstack-infra | 00:21 | |
| *** mtanino_ has joined #openstack-infra | 00:22 | |
| *** mtanino has quit IRC | 00:24 | |
| *** annegentle has quit IRC | 00:25 | |
| *** vmbrasseur has quit IRC | 00:29 | |
| *** vmbrasseur_ is now known as vmbrasseur | 00:29 | |
| *** MarkAtwood has quit IRC | 00:32 | |
| *** ZZelle_ has quit IRC | 00:32 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Remove hpcloud from clouds.yaml listings https://review.openstack.org/274950 | 00:36 |
| fungi | clarkb: ^ any idea if there was a pending change for that already? | 00:36 |
| fungi | or i guess i could check gerrit's conflicting changes list | 00:36 |
| fungi | anyway, that's a new development which has caused ansible to cease puppeting us entirely, from what i can tell in the logs | 00:36 |
| clarkb | fungi: no I don't recall a pending change for that already | 00:36 |
| *** gildub has quit IRC | 00:37 | |
| clarkb | we can cleanup all our secret data files too | 00:37 |
| fungi | less urgent, but sure | 00:37 |
| *** eil397 has quit IRC | 00:41 | |
| clarkb | fungi: I guess that means they truly shut the cloud down | 00:45 |
| clarkb | one thing I have noticed about neutron and our overlay networks is they circumvent some things | 00:45 |
| clarkb | which is :| | 00:45 |
| fungi | rolled down the security door and turned off the fluorescents | 00:46 |
| ianw | fungi: when you have time, re centos dib images, maybe you could take a look at https://review.openstack.org/#/c/272857/ (fix for yum.conf) | 00:46 |
| clarkb | I set IPs in the first /24 of 172.24.4.0/23 as our hypervisor IPs and looks like neutron goes through and sets them out of the second /24 because thta is where we floating ip | 00:46 |
| clarkb | so everything still works, its just not what I wanted | 00:47 |
| clarkb | oh that is because its not dvr so the second hypervisor doesn't have a thing | 00:47 |
| *** devananda has quit IRC | 00:47 | |
| clarkb | ok maybe I care less then | 00:47 |
| *** stevebaker has quit IRC | 00:52 | |
| *** stevebaker has joined #openstack-infra | 00:52 | |
| *** unicell has joined #openstack-infra | 00:52 | |
| clarkb | the bluebox with vxlan multinode test failed but ran to completion | 00:55 |
| *** jamielennox|away is now known as jamielennox | 00:55 | |
| clarkb | does not appear to have failed on networking but rather timeouts | 00:55 |
| openstackgerrit | Merged openstack-infra/system-config: Don't exclude python-setuptools https://review.openstack.org/272857 | 00:55 |
| *** annegentle has joined #openstack-infra | 00:56 | |
| openstackgerrit | Merged openstack/diskimage-builder: Skip centos functional testing https://review.openstack.org/274937 | 00:56 |
| *** SumitNaiksatam has quit IRC | 01:00 | |
| *** MarkAtwood has joined #openstack-infra | 01:00 | |
| *** ashleighfarnham has quit IRC | 01:01 | |
| *** unicell1 has joined #openstack-infra | 01:02 | |
| openstackgerrit | Merged openstack-infra/system-config: Remove hpcloud from clouds.yaml listings https://review.openstack.org/274950 | 01:04 |
| *** unicell has quit IRC | 01:04 | |
| *** [1]Thelo has joined #openstack-infra | 01:04 | |
| *** thorst has quit IRC | 01:05 | |
| *** krotscheck1 has joined #openstack-infra | 01:05 | |
| *** thorst has joined #openstack-infra | 01:06 | |
| openstackgerrit | Merged openstack-infra/system-config: Add $YUM to pickup dnf on Fedora>=22 https://review.openstack.org/252221 | 01:06 |
| *** Thelo has quit IRC | 01:06 | |
| *** [1]Thelo is now known as Thelo | 01:06 | |
| *** shashank_hegde has quit IRC | 01:07 | |
| *** doug-fish has quit IRC | 01:10 | |
| *** bhunter71 has quit IRC | 01:10 | |
| *** doug-fish has joined #openstack-infra | 01:10 | |
| *** tiswanso has joined #openstack-infra | 01:10 | |
| *** EricGonczer_ has quit IRC | 01:12 | |
| *** doug-fis_ has joined #openstack-infra | 01:12 | |
| *** piet has quit IRC | 01:12 | |
| *** krotscheck1 has quit IRC | 01:13 | |
| *** thorst has quit IRC | 01:14 | |
| *** doug-fi__ has joined #openstack-infra | 01:14 | |
| *** doug-fish has quit IRC | 01:14 | |
| nibalizer | fungi: I haven't been reading all the scrollback | 01:15 |
| nibalizer | did we resolve the issue with ansible | 01:15 |
| nibalizer | I get errors when running ansible all --list-hosts | 01:15 |
| *** markvoelker has joined #openstack-infra | 01:15 | |
| *** doug-fi__ has quit IRC | 01:15 | |
| clarkb | nibalizer: https://review.openstack.org/274950 should resolve it | 01:16 |
| *** piet has joined #openstack-infra | 01:16 | |
| fungi | except if we're maybe in a catch-22 | 01:16 |
| *** doug-fish has joined #openstack-infra | 01:16 | |
| *** MarkAtwood has quit IRC | 01:16 | |
| clarkb | since puppet won't apply that change | 01:16 |
| fungi | will that get applied to the puppetmaster if the run_all script is breaking? | 01:16 |
| *** doug-fish has quit IRC | 01:16 | |
| *** doug-fis_ has quit IRC | 01:16 | |
| clarkb | but if we get it merged we can do it by hand | 01:16 |
| clarkb | fungi: it won't | 01:16 |
| nibalizer | fungi: we can puppet apply by hand | 01:16 |
| fungi | well, it merged 10 minutes ago | 01:16 |
| nibalizer | also... | 01:16 |
| clarkb | fungi: we could fix that by changing run_all to localhost and not ansible | 01:16 |
| clarkb | fungi: maybe do that first | 01:16 |
| nibalizer | oh is that what we broke | 01:17 |
| nibalizer | nice | 01:17 |
| clarkb | er still ansible but just use local connection without inventory | 01:17 |
| fungi | well, it's the second thing which broke | 01:17 |
| fungi | first we had 2+ hour timeouts trying to reach the pypi mirror in hpcloud to puppet it, then we started breaking on being able to authenticate to hpcloud | 01:17 |
| nibalizer | we could run puppet apply at the top of run_all | 01:17 |
| fungi | nibalizer: since you're on there running stuff already, wanna give it a go? | 01:18 |
| fungi | but yeah, any way to future-proof the existing process would also be great | 01:18 |
| *** krotscheck1 has joined #openstack-infra | 01:19 | |
| *** markvoelker has quit IRC | 01:20 | |
| clarkb | so I think vxlan does work around the bluebox problems | 01:20 |
| clarkb | we can switch to vxlan and/or work with bluebox to make gre work more reliably | 01:20 |
| clarkb | however I think there may be performance issues on bluebox running those tests as they don't seem to be very reliable | 01:21 |
| nibalizer | sure | 01:21 |
| clarkb | bluebox has a much higher fail rate in general | 01:21 |
| *** tiswanso has quit IRC | 01:22 | |
| clarkb | but ost of that appears related to vxlan/gre trouble | 01:22 |
| *** jamielennox is now known as jamielennox|away | 01:23 | |
| *** mtanino_ has quit IRC | 01:27 | |
| clarkb | all that to say I think we should consider the switch at least temporarily while we work through the rest | 01:28 |
| clarkb | https://review.openstack.org/#/c/274821/1 | 01:29 |
| nibalizer | clouds.yaml updated | 01:33 |
| fungi | thanks, nibalizer! | 01:34 |
| nibalizer | note that you have to git pull manually to update the system-config checkout | 01:34 |
| nibalizer | and if the problem was in a module, you'd have to run install_modules.sh | 01:34 |
| clarkb | nibalizer: be careful that you don't implicit merge | 01:34 |
| *** Daisy has joined #openstack-infra | 01:35 | |
| nibalizer | it look like I didn't | 01:35 |
| nibalizer | in the future I should run git fetch, get checkout master or so | 01:35 |
| clarkb | I don't think I ever use git pull | 01:35 |
| fungi | i tend to: git remote update ; git checkout master ; git reset --hard origin/master | 01:38 |
| fungi | just to be clear that i'm on an actual commit from the origin's copy of that branch and not some local hork-up | 01:38 |
| *** angdraug has quit IRC | 01:39 | |
| *** marekd has quit IRC | 01:40 | |
| *** marekd has joined #openstack-infra | 01:40 | |
| *** krotscheck1 has quit IRC | 01:40 | |
| *** Sukhdev has quit IRC | 01:41 | |
| *** annegentle has quit IRC | 01:46 | |
| *** sflanigan has quit IRC | 01:46 | |
| *** annegentle has joined #openstack-infra | 01:46 | |
| *** julim has joined #openstack-infra | 01:49 | |
| *** annegentle has quit IRC | 01:49 | |
| *** tiswanso has joined #openstack-infra | 01:49 | |
| *** tiswanso has quit IRC | 01:49 | |
| *** jgriffith is now known as jgriffith_away | 01:50 | |
| *** tiswanso has joined #openstack-infra | 01:50 | |
| *** Apoorva has quit IRC | 01:52 | |
| *** jgriffith_away is now known as jgriffith | 01:53 | |
| *** tphummel has quit IRC | 01:53 | |
| *** bpokorny has quit IRC | 01:55 | |
| *** Swami has quit IRC | 01:56 | |
| *** jpr has quit IRC | 01:59 | |
| *** krotscheck1 has joined #openstack-infra | 01:59 | |
| *** pvaneck has quit IRC | 01:59 | |
| *** krotscheck1 has quit IRC | 02:01 | |
| *** Jeffrey4l has joined #openstack-infra | 02:03 | |
| *** Sukhdev has joined #openstack-infra | 02:04 | |
| *** jgriffith is now known as jgriffith_away | 02:04 | |
| *** Sukhdev has quit IRC | 02:05 | |
| *** piet has quit IRC | 02:05 | |
| *** krotscheck1 has joined #openstack-infra | 02:07 | |
| *** thorst has joined #openstack-infra | 02:09 | |
| *** dims has joined #openstack-infra | 02:11 | |
| *** thorst has quit IRC | 02:12 | |
| *** thorst has joined #openstack-infra | 02:13 | |
| *** markvoelker has joined #openstack-infra | 02:16 | |
| *** markvoelker has quit IRC | 02:20 | |
| *** thorst has quit IRC | 02:21 | |
| *** EricGonczer_ has joined #openstack-infra | 02:24 | |
| *** krotscheck1 has quit IRC | 02:26 | |
| *** coolsvap|away is now known as coolsvap | 02:26 | |
| *** yamahata has quit IRC | 02:28 | |
| *** krotscheck1 has joined #openstack-infra | 02:28 | |
| *** krotscheck1 has quit IRC | 02:31 | |
| *** krotscheck1 has joined #openstack-infra | 02:33 | |
| prometheanfire | ianw: I think I know why that revert happened, it's in another branch that was 'fixed' | 02:38 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: add new cloud-init element https://review.openstack.org/273764 | 02:41 |
| *** jamielennox|away is now known as jamielennox | 02:44 | |
| openstackgerrit | Ben Swartzlander proposed openstack/requirements: Raise manilaclient upper to 1.6.1 https://review.openstack.org/274983 | 02:44 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: fix gentoo hardened support https://review.openstack.org/273795 | 02:44 |
| *** zz_dimtruck is now known as dimtruck | 02:44 | |
| *** EricGonczer_ has quit IRC | 02:47 | |
| *** EricGonczer_ has joined #openstack-infra | 02:48 | |
| *** annegentle has joined #openstack-infra | 02:48 | |
| *** gildub has joined #openstack-infra | 02:49 | |
| *** MarkAtwood has joined #openstack-infra | 02:49 | |
| ianw | prometheanfire: i don't want to send your reviews around again, but could you please fix up your subjects to have a captial letter, and reflow them (emacs ctl-q if you use that) so they're indented/wrapped properly | 02:49 |
| prometheanfire | ianw: do you mean the commit message? | 02:50 |
| ianw | yes | 02:50 |
| prometheanfire | ianw: any one specifically? | 02:50 |
| ianw | prometheanfire: the outstanding ones. basically, make them look like all the other commit messages | 02:51 |
| prometheanfire | that doesn't tell me what's wrong with them, also I use vim, so don't have that reflow | 02:51 |
| prometheanfire | ianw: any one without workflow? | 02:52 |
| ianw | https://review.openstack.org/#/c/273764/ | 02:52 |
| ianw | https://review.openstack.org/#/c/273795/ | 02:52 |
| *** EricGonczer_ has quit IRC | 02:53 | |
| prometheanfire | ok, what is wrong with them? max line length is 79 for https://review.openstack.org/#/c/273795/ | 02:54 |
| *** sbalukoff has quit IRC | 02:55 | |
| prometheanfire | the other one just needs more info? | 02:55 |
| ianw | prometheanfire: i don't know why but everything after the first line is indented | 02:56 |
| prometheanfire | I did that on purpose for readability :| | 02:56 |
| ianw | The subject should start with a captial letter if it's a regular word | 02:56 |
| prometheanfire | those two things it? | 02:57 |
| prometheanfire | or are there other minor nits? | 02:57 |
| *** annegentle has quit IRC | 02:58 | |
| prometheanfire | ok, commiting that one then | 02:59 |
| ianw | promethanfire: i think that's it | 02:59 |
| *** coolsvap is now known as coolsvap|away | 02:59 | |
| prometheanfire | and I'll caps the https://review.openstack.org/#/c/273764/ | 02:59 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Fix Gentoo hardened support https://review.openstack.org/273795 | 02:59 |
| ianw | i think the hardened one is ok, presumably the default won't switch to hardened? because it seems that where that check is would be too late if that were the case | 02:59 |
| prometheanfire | default isn't hardened | 03:00 |
| prometheanfire | though I kinda would like that | 03:00 |
| * prometheanfire is part of the gentoo hardened team | 03:00 | |
| clarkb | fungi: jeblair we only need room for the mirror host on the openstackci account right? | 03:01 |
| *** annegentle has joined #openstack-infra | 03:01 | |
| * bkero is curious what kind of stuff grsec is going to be grumpy about WRT Ironic and friends. | 03:02 | |
| clarkb | fungi: jeblair trying to make sure we split up our quota properly on osic and don't sacrifice too much of it to unnecessarily oversized mirrors | 03:02 |
| *** wei-wang has joined #openstack-infra | 03:02 | |
| clarkb | I will go ahead with that then we can rebalance later if we need more for mirrors | 03:03 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Add new cloud-init element https://review.openstack.org/273764 | 03:04 |
| *** Daisy_ has joined #openstack-infra | 03:05 | |
| *** krotscheck1 has quit IRC | 03:06 | |
| clarkb | nibalizer: did you make progress running tests on infra cloud? were we able to get the test run times down to something similar to the gate? | 03:06 |
| *** wei-wang has quit IRC | 03:06 | |
| nibalizer | best I got was 78 minutes | 03:07 |
| nibalizer | jeblair aksmed to poke around in the test directory to find specifics on timing | 03:07 |
| nibalizer | but I wasn't able to glean any information out of that (mostly because I don't really know what I am looking for) | 03:08 |
| *** piet has joined #openstack-infra | 03:09 | |
| *** yamamoto has joined #openstack-infra | 03:09 | |
| *** Daisy_ has quit IRC | 03:09 | |
| *** krotscheck1 has joined #openstack-infra | 03:10 | |
| *** amotoki has quit IRC | 03:10 | |
| *** sridhar_ram has quit IRC | 03:10 | |
| ianw | nibalizer: devstack at the end prints out some timing, could be quickly compared to see what's out-of-line with others | 03:11 |
| clarkb | nibalizer: I would look at the devstack log and the tempest log to see where the majority of time is spent | 03:11 |
| clarkb | ya then compare that to rax/ovh/internap/bluebox et al | 03:11 |
| nibalizer | http://paste.openstack.org/show/485690/ | 03:13 |
| openstackgerrit | Merged openstack/diskimage-builder: Add support for Gentoo to source-repositories https://review.openstack.org/273776 | 03:15 |
| openstackgerrit | Merged openstack/diskimage-builder: Add Gentoo support to growroot https://review.openstack.org/273769 | 03:15 |
| prometheanfire | neat | 03:15 |
| *** links has joined #openstack-infra | 03:15 | |
| openstackgerrit | Merged openstack/diskimage-builder: Remove outdated translation files https://review.openstack.org/274457 | 03:16 |
| openstackgerrit | Merged openstack/diskimage-builder: Fix package-installs for python3 https://review.openstack.org/273782 | 03:16 |
| *** krotscheck1 has quit IRC | 03:18 | |
| *** thorst has joined #openstack-infra | 03:19 | |
| prometheanfire | if someone else can review this that'd be nice as it's the thing the other reviews generally need (and one specifically) https://review.openstack.org/#/c/273795/ | 03:19 |
| *** yamamoto has quit IRC | 03:20 | |
| prometheanfire | ianw: you don't happen to be a core for glean do you? | 03:21 |
| *** krotscheck1 has joined #openstack-infra | 03:21 | |
| prometheanfire | are you, not do you | 03:21 |
| *** yuanying_ has joined #openstack-infra | 03:21 | |
| ianw | prometheanfire: i am not core for glean, yolanda / greghaynes / modred probably people to ping | 03:22 |
| prometheanfire | ya, need one more +2 there, but I'll bug them tomorrow | 03:22 |
| prometheanfire | ya, guess I need more other dib cores at this point | 03:24 |
| *** yuanying has quit IRC | 03:24 | |
| clarkb | nibalizer: and that was on an image built with cached stuff? 414 seconds for apt get seems slow | 03:26 |
| clarkb | but I would need to compare | 03:26 |
| *** thorst has quit IRC | 03:26 | |
| nibalizer | clarkb: http://paste.openstack.org/show/485691/ | 03:26 |
| nibalizer | I don't think the image had caching no | 03:26 |
| *** krotscheck1 has quit IRC | 03:26 | |
| *** jgriffith_away is now known as jgriffith | 03:26 | |
| nibalizer | just an ubuntu-minimal vm | 03:26 |
| ianw | greghaynes: dib released, right ... | 03:27 |
| openstackgerrit | Sachi King proposed openstack-dev/pbr: Don't attempt to test with 0.6c11 with Py3 https://review.openstack.org/273358 | 03:27 |
| greghaynes | ianw: nope :( will tomorrow am | 03:27 |
| greghaynes | I spaced on that | 03:28 |
| prometheanfire | greghaynes: ohai :P | 03:28 |
| ianw | greghaynes: ok ... yeah f23 needs it for growroot support so it stops running out of disk | 03:28 |
| greghaynes | prometheanfire: hey, I'm EOD so not likely to review anything until tomorrow | 03:29 |
| *** zhurong has joined #openstack-infra | 03:29 | |
| greghaynes | ianw: got it | 03:29 |
| *** krotscheck1 has joined #openstack-infra | 03:30 | |
| *** annegentle has quit IRC | 03:30 | |
| prometheanfire | greghaynes: np, what tz are you btw, don't want to bother when off | 03:31 |
| greghaynes | PST | 03:31 |
| *** unicell1 has quit IRC | 03:31 | |
| *** krotscheck1 has quit IRC | 03:32 | |
| prometheanfire | k, cst here | 03:32 |
| *** krotscheck1 has joined #openstack-infra | 03:33 | |
| clarkb | nibalizer: you should compare with one of our images (either buold it there or cp from nodepool.o.o) | 03:33 |
| *** Daisy has quit IRC | 03:34 | |
| clarkb | so ya thats ~5 minutes we can save right there | 03:34 |
| clarkb | andaybe more with git caches | 03:34 |
| *** sdake has joined #openstack-infra | 03:35 | |
| prometheanfire | will be nice when the osic stuff comes online, perf wise | 03:35 |
| *** amotoki has joined #openstack-infra | 03:35 | |
| *** annegentle has joined #openstack-infra | 03:36 | |
| *** tphummel has joined #openstack-infra | 03:36 | |
| *** annegentle has quit IRC | 03:41 | |
| *** yuanying has joined #openstack-infra | 03:41 | |
| *** krotscheck1 has quit IRC | 03:43 | |
| *** achanda has joined #openstack-infra | 03:43 | |
| *** yuanying_ has quit IRC | 03:44 | |
| *** SumitNaiksatam has joined #openstack-infra | 03:48 | |
| *** samyewell_ has joined #openstack-infra | 03:50 | |
| *** samyewell has quit IRC | 03:51 | |
| *** samyewell_ is now known as samyewell | 03:51 | |
| *** dims has quit IRC | 03:53 | |
| *** MarkAtwood has quit IRC | 03:58 | |
| *** amotoki has quit IRC | 04:01 | |
| *** armax has quit IRC | 04:02 | |
| *** flwang1 has quit IRC | 04:03 | |
| *** achanda has quit IRC | 04:04 | |
| *** armax has joined #openstack-infra | 04:05 | |
| *** yuanying has quit IRC | 04:05 | |
| *** yuanying has joined #openstack-infra | 04:06 | |
| *** coolsvap|away is now known as coolsvap | 04:07 | |
| *** yuanying_ has joined #openstack-infra | 04:07 | |
| *** yuanying has quit IRC | 04:07 | |
| *** yamamoto_ has joined #openstack-infra | 04:08 | |
| *** diana_clarke has quit IRC | 04:08 | |
| *** jamielennox is now known as jamielennox|away | 04:10 | |
| *** mrmartin has joined #openstack-infra | 04:11 | |
| *** achanda has joined #openstack-infra | 04:11 | |
| fungi | clarkb: yeah, just the mirror server for now (and actually very little for that? just enough to hold a warm afs cache probably...) | 04:14 |
| *** shashank_hegde has joined #openstack-infra | 04:14 | |
| *** mriedem has left #openstack-infra | 04:15 | |
| *** woodster_ has quit IRC | 04:16 | |
| *** markvoelker has joined #openstack-infra | 04:17 | |
| *** sbalukoff has joined #openstack-infra | 04:18 | |
| *** markvoelker has quit IRC | 04:21 | |
| *** isaacb has joined #openstack-infra | 04:23 | |
| *** thorst has joined #openstack-infra | 04:24 | |
| *** thorst has quit IRC | 04:31 | |
| *** tiswanso has quit IRC | 04:33 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/puppet-nodepool: Add option to install diskimage-builder from git https://review.openstack.org/275004 | 04:37 |
| *** rguillebert has quit IRC | 04:37 | |
| *** jamielennox|away is now known as jamielennox | 04:39 | |
| *** isaacb has quit IRC | 04:49 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config: Use diskimage-builder from git for nodepool https://review.openstack.org/275006 | 04:55 |
| *** aukhan has joined #openstack-infra | 04:55 | |
| aukhan | Hi Folks, could you kindly share how to install "Zuul" (i.e. without using puppet) ? | 04:57 |
| prometheanfire | figure out what puppet's doing, and do that :P | 04:58 |
| prometheanfire | https://github.com/openstack-infra/zuul | 04:59 |
| openstackgerrit | Jake Yip proposed openstack/os-testr: correct typo https://review.openstack.org/275008 | 04:59 |
| aukhan | Thanks @prometheanfire, I was looking for a quick way to install it :P | 04:59 |
| aukhan | I think i am just being too lazy | 04:59 |
| *** gongysh has joined #openstack-infra | 05:00 | |
| jroll | puppet is the easy way :P | 05:00 |
| aukhan | I thought there was a way to like do a "apt-get install zuul" and get done :P | 05:00 |
| aukhan | I agree with you both @jroll/@prometheanfire | 05:00 |
| jroll | so, pip install zuul should just work, but there's a handful of services it looks like | 05:01 |
| prometheanfire | ya, there's a setup.py | 05:01 |
| *** harlowja_at_home has joined #openstack-infra | 05:01 | |
| aukhan | I did try isntallin using the setup.py | 05:01 |
| aukhan | but had to resolve a lot of dependencies manually | 05:01 |
| nibalizer | aukhan: i think you'd do best to use this http://docs.openstack.org/infra/openstackci/ | 05:02 |
| prometheanfire | pip should do deps for you | 05:02 |
| aukhan | pip did not i think there was some issue with my pip version or something | 05:03 |
| *** fedexo has joined #openstack-infra | 05:03 | |
| aukhan | hello @nibalizer, thanks will check that out | 05:04 |
| aukhan | too | 05:04 |
| aukhan | thanks a ton guys for the quick response | 05:04 |
| *** aopincar has joined #openstack-infra | 05:04 | |
| *** aopincar has quit IRC | 05:04 | |
| *** amotoki has joined #openstack-infra | 05:05 | |
| *** sdake has quit IRC | 05:05 | |
| *** unicell has joined #openstack-infra | 05:07 | |
| nibalizer | aukhan: also assuming you work at hpe, asselin is a great resource to work with | 05:07 |
| *** jamielennox is now known as jamielennox|away | 05:13 | |
| *** shashank_hegde has quit IRC | 05:17 | |
| *** Sukhdev has joined #openstack-infra | 05:17 | |
| *** thorst has joined #openstack-infra | 05:30 | |
| jeblair | nibalizer: ah, yeah, if you're working without a cache on infra cloud, then you'll want to only directly compare the tempest runtimes as printed out e.g. here http://logs.openstack.org/12/272412/2/check/gate-tempest-dsvm-full/f20896d//console.html#_2016-02-02_04_25_41_450 | 05:31 |
| jeblair | nibalizer: the first number there -- 1144 in that example is the most important for comparison | 05:32 |
| *** gyee has quit IRC | 05:32 | |
| jeblair | clarkb: yeah, i've been setting aside 100-200G for afs caches but only configured it to use 50G so far (and it's using way less than that). | 05:35 |
| jeblair | clarkb: see http://docs-draft.openstack.org/73/273673/1/check/gate-infra-specs-docs/f9e8572//doc/build/html/specs/unified_mirrors.html#servers | 05:35 |
| jeblair | clarkb: and i've been doing 2G servers | 05:36 |
| *** thorst has quit IRC | 05:36 | |
| *** Daisy has joined #openstack-infra | 05:36 | |
| *** exploreshaifali has joined #openstack-infra | 05:39 | |
| *** AzherKhan has joined #openstack-infra | 05:44 | |
| *** piet has quit IRC | 05:44 | |
| *** aukhan has quit IRC | 05:44 | |
| *** sileht has quit IRC | 05:44 | |
| *** piet has joined #openstack-infra | 05:45 | |
| *** Swami has joined #openstack-infra | 05:49 | |
| *** bradm_ is now known as bradm | 05:54 | |
| *** changbl has quit IRC | 05:56 | |
| *** kushal has quit IRC | 05:57 | |
| *** vivekd has joined #openstack-infra | 05:57 | |
| *** dimtruck is now known as zz_dimtruck | 06:00 | |
| *** hdd has quit IRC | 06:02 | |
| *** gongysh has quit IRC | 06:04 | |
| *** oomichi has joined #openstack-infra | 06:04 | |
| *** abregman has joined #openstack-infra | 06:06 | |
| *** abregman has quit IRC | 06:06 | |
| *** vgridnev has joined #openstack-infra | 06:06 | |
| openstackgerrit | Shamail Tahir proposed openstack-infra/irc-meetings: Add new bi-weekly APAC meeting for user committee https://review.openstack.org/269427 | 06:07 |
| openstackgerrit | Shamail Tahir proposed openstack-infra/irc-meetings: Add new bi-weekly APAC meeting for user committee https://review.openstack.org/269427 | 06:09 |
| *** piet has quit IRC | 06:11 | |
| *** abregman has joined #openstack-infra | 06:11 | |
| *** harlowja_at_home has quit IRC | 06:11 | |
| *** kushal has joined #openstack-infra | 06:11 | |
| *** fawadkhaliq has joined #openstack-infra | 06:11 | |
| *** gildub has quit IRC | 06:11 | |
| *** fawadkhaliq has quit IRC | 06:11 | |
| openstackgerrit | Shamail Tahir proposed openstack-infra/irc-meetings: Add new bi-weekly APAC meeting for user committee https://review.openstack.org/269427 | 06:14 |
| *** jtomasek has quit IRC | 06:15 | |
| *** jyuso1 has joined #openstack-infra | 06:16 | |
| *** markvoelker has joined #openstack-infra | 06:18 | |
| *** diana_clarke has joined #openstack-infra | 06:18 | |
| *** fedexo has quit IRC | 06:18 | |
| *** aopincar has joined #openstack-infra | 06:19 | |
| *** rcernin has joined #openstack-infra | 06:22 | |
| *** markvoelker has quit IRC | 06:22 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/275027 | 06:22 |
| *** boris-42 has quit IRC | 06:23 | |
| openstackgerrit | daisy-ycguo proposed openstack-infra/puppet-translation_checksite: Puppet module to provide a translation checksite Approved specification: translation_check_site.rst https://review.openstack.org/274701 | 06:24 |
| *** shashank_hegde has joined #openstack-infra | 06:26 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/273907 | 06:26 |
| *** Swami has quit IRC | 06:28 | |
| openstackgerrit | Zhenyu Zheng proposed openstack-infra/project-config: Add py27 and docs job to terracotta. https://review.openstack.org/275031 | 06:35 |
| *** thorst has joined #openstack-infra | 06:35 | |
| openstackgerrit | Zhenyu Zheng proposed openstack-infra/project-config: Add py27 and docs job to terracotta. https://review.openstack.org/275031 | 06:39 |
| *** vgridnev has quit IRC | 06:40 | |
| *** vgridnev has joined #openstack-infra | 06:41 | |
| *** thorst has quit IRC | 06:41 | |
| *** tphummel has quit IRC | 06:45 | |
| *** rcernin has quit IRC | 06:46 | |
| *** shashank_hegde has quit IRC | 06:48 | |
| *** dalgaaf has joined #openstack-infra | 06:51 | |
| *** amotoki_ has joined #openstack-infra | 06:51 | |
| *** exploreshaifali has quit IRC | 07:00 | |
| *** followinfrared has joined #openstack-infra | 07:02 | |
| *** armax has quit IRC | 07:03 | |
| *** amotoki has quit IRC | 07:03 | |
| *** keedya has joined #openstack-infra | 07:04 | |
| AJaeger | Sam-I-Am: I can delete files on docs.openstack.org - as can annegentle. Send us an email... | 07:04 |
| *** eliqiao has quit IRC | 07:10 | |
| *** eliqiao has joined #openstack-infra | 07:12 | |
| openstackgerrit | daisy-ycguo proposed openstack-infra/puppet-translation_checksite: Puppet module to provide a translation checksite Approved specification: translation_check_site.rst https://review.openstack.org/274701 | 07:13 |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/275027 | 07:13 |
| *** _nadya_ has joined #openstack-infra | 07:16 | |
| *** coolsvap is now known as coolsvap|away | 07:16 | |
| *** shashank_hegde has joined #openstack-infra | 07:18 | |
| *** _nadya_ has quit IRC | 07:20 | |
| openstackgerrit | Akihiro Motoki proposed openstack-infra/irc-meetings: Drop I18N Translation test meeting https://review.openstack.org/275045 | 07:21 |
| *** zaro has joined #openstack-infra | 07:23 | |
| openstackgerrit | Zhenyu Zheng proposed openstack-infra/project-config: Add py27, docs and publishing to pypi jobs to terracotta. https://review.openstack.org/275031 | 07:24 |
| *** keedya has quit IRC | 07:26 | |
| AJaeger | amotoki_: do you agree to push https://review.openstack.org/#/c/274421/ forward? Today's imports look fine - https://review.openstack.org/#/q/status:open+topic:zanata/translations,n,z | 07:28 |
| * AJaeger just double checks jenkins log files... | 07:28 | |
| amotoki_ | AJaeger: looking | 07:29 |
| *** zul has joined #openstack-infra | 07:29 | |
| *** sileht_ has joined #openstack-infra | 07:29 | |
| *** yamahata has joined #openstack-infra | 07:29 | |
| *** ianw has quit IRC | 07:29 | |
| *** ianw has joined #openstack-infra | 07:30 | |
| zaro | ~ | 07:30 |
| nibalizer | jeblair: http://paste.openstack.org/show/485697/ | 07:31 |
| *** scheuran has joined #openstack-infra | 07:33 | |
| *** sileht_ is now known as sileht | 07:35 | |
| *** abregman has quit IRC | 07:35 | |
| *** Sukhdev has quit IRC | 07:38 | |
| *** esikachev has joined #openstack-infra | 07:39 | |
| *** thorst has joined #openstack-infra | 07:39 | |
| *** achanda has quit IRC | 07:40 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config: Use diskimage-builder from git for nodepool https://review.openstack.org/275006 | 07:41 |
| *** zul has quit IRC | 07:41 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use new translation setup for all python projects https://review.openstack.org/274421 | 07:41 |
| AJaeger | amotoki_: here's an update ^. Thanks for reviewing | 07:41 |
| AJaeger | nibalizer, ianw: Could you review 274421 as next step for translation setup, please? We're starting to clean up now ... | 07:42 |
| *** achanda has joined #openstack-infra | 07:43 | |
| nibalizer | sure | 07:43 |
| nibalizer | AJaeger: sorry but I don't really understand that well enough to review quickly | 07:45 |
| nibalizer | after my talk today I'll try to cirlce around | 07:45 |
| AJaeger | nibalizer: understood. What are you talking about? | 07:46 |
| *** thorst has quit IRC | 07:46 | |
| *** fhubik has joined #openstack-infra | 07:49 | |
| *** sileht has quit IRC | 07:50 | |
| *** sileht has joined #openstack-infra | 07:50 | |
| nibalizer | AJaeger: ansible+puppet | 07:51 |
| AJaeger | cool | 07:52 |
| scheuran | Good morning, is there a known problem with the gate-tempest-dsvm-layer4 job? My tempest patch always fails with: Details: No "_member_" role found. I don't think that this is related to my change... | 07:54 |
| *** flepied has quit IRC | 07:55 | |
| *** achanda has quit IRC | 07:55 | |
| AJaeger | scheuran: better ask on #openstack-qa - I'm not aware of anything | 08:00 |
| scheuran | AJaeger, ok, thanks! | 08:00 |
| *** diana_clarke has quit IRC | 08:02 | |
| *** amotoki_ is now known as amotoki | 08:03 | |
| *** fhubik has quit IRC | 08:05 | |
| *** sdake has joined #openstack-infra | 08:09 | |
| esikachev | hi all! i need requirements cores. take a look around:) | 08:10 |
| *** jistr has joined #openstack-infra | 08:11 | |
| *** oomichi has quit IRC | 08:12 | |
| *** ildikov has quit IRC | 08:13 | |
| *** jtomasek_ has joined #openstack-infra | 08:13 | |
| *** oomichi has joined #openstack-infra | 08:13 | |
| *** samuelBartel has joined #openstack-infra | 08:14 | |
| *** matrohon has joined #openstack-infra | 08:14 | |
| *** vivekd has quit IRC | 08:16 | |
| *** boris-42 has joined #openstack-infra | 08:17 | |
| *** markvoelker has joined #openstack-infra | 08:18 | |
| *** nikil89_ has joined #openstack-infra | 08:19 | |
| *** nikil89_ is now known as naggappan | 08:19 | |
| *** mrmartin has quit IRC | 08:19 | |
| *** shashank_hegde has quit IRC | 08:19 | |
| *** fhubik has joined #openstack-infra | 08:20 | |
| *** mikelk has joined #openstack-infra | 08:22 | |
| *** vivekd has joined #openstack-infra | 08:22 | |
| *** markvoelker has quit IRC | 08:22 | |
| *** zeih has joined #openstack-infra | 08:23 | |
| *** zeih has quit IRC | 08:23 | |
| *** tlbr_ has quit IRC | 08:23 | |
| *** zeih has joined #openstack-infra | 08:24 | |
| *** k4n0 has joined #openstack-infra | 08:26 | |
| *** mrmartin has joined #openstack-infra | 08:26 | |
| *** k4n0 has quit IRC | 08:26 | |
| *** cznewt has joined #openstack-infra | 08:27 | |
| *** dingyichen has quit IRC | 08:29 | |
| *** flepied has joined #openstack-infra | 08:32 | |
| *** mrmartin has quit IRC | 08:34 | |
| *** fhubik_real has joined #openstack-infra | 08:37 | |
| yolanda | morning | 08:37 |
| yolanda | AJaeger, things are a bit quieter today, i can start looking at pending things | 08:37 |
| *** hashar has joined #openstack-infra | 08:38 | |
| *** e0ne has joined #openstack-infra | 08:38 | |
| AJaeger | morning, yolanda. 274421 is my only one right now ;) Thanks! | 08:38 |
| *** mrmartin has joined #openstack-infra | 08:39 | |
| *** fhubik has quit IRC | 08:40 | |
| *** fhubik_real has quit IRC | 08:42 | |
| AJaeger | yolanda: There's the usual round of open reviews, nothing critical to point to... Thanks! | 08:42 |
| yolanda | ok going to do that round | 08:42 |
| *** tlbr has joined #openstack-infra | 08:43 | |
| *** vivekd has quit IRC | 08:43 | |
| yolanda | AJaeger about https://review.openstack.org/#/c/275031 ... this says it adds docs jobs, but i'm not seeing those on the change | 08:44 |
| *** thorst has joined #openstack-infra | 08:45 | |
| *** rubasov has joined #openstack-infra | 08:45 | |
| *** yaume has joined #openstack-infra | 08:47 | |
| AJaeger | they are part of the template added | 08:47 |
| *** kzaitsev_mb has joined #openstack-infra | 08:47 | |
| AJaeger | The python-jobs templates add docs jobs to check and gate | 08:48 |
| yolanda | ah ok | 08:48 |
| *** vivekd has joined #openstack-infra | 08:48 | |
| openstackgerrit | Merged openstack-infra/project-config: Add growroot element to Fedora 23 https://review.openstack.org/271907 | 08:49 |
| *** marekd has quit IRC | 08:50 | |
| *** abregman has joined #openstack-infra | 08:50 | |
| *** thorst has quit IRC | 08:51 | |
| *** marekd has joined #openstack-infra | 08:52 | |
| *** yamahata has quit IRC | 08:52 | |
| *** exploreshaifali has joined #openstack-infra | 08:53 | |
| *** abregman is now known as abregman|intervi | 08:54 | |
| openstackgerrit | Merged openstack-infra/project-config: Add py27, docs and publishing to pypi jobs to terracotta. https://review.openstack.org/275031 | 08:55 |
| *** zul has joined #openstack-infra | 08:56 | |
| ianw | AJaeger: "In addition, enable new setup for vmware-nsx and ironic-inspector" ... is that in there? | 08:56 |
| *** aopincar has quit IRC | 08:56 | |
| *** rossella_s has joined #openstack-infra | 08:56 | |
| openstackgerrit | Evgeny Antyshev proposed openstack-infra/ciwatch: Ability to select specific CI results https://review.openstack.org/274412 | 08:56 |
| *** jlanoux has joined #openstack-infra | 08:57 | |
| *** abregman has joined #openstack-infra | 08:58 | |
| *** ifarkas has joined #openstack-infra | 08:58 | |
| *** abregman|intervi has quit IRC | 08:58 | |
| *** marekd has quit IRC | 09:01 | |
| *** dprince has joined #openstack-infra | 09:03 | |
| openstackgerrit | Merged openstack-infra/project-config: zuul/layout: optimize puppet jobs https://review.openstack.org/274137 | 09:05 |
| *** Kevin_Zheng has joined #openstack-infra | 09:06 | |
| AJaeger | ianw: let me double check... | 09:06 |
| *** jyuso1 has quit IRC | 09:07 | |
| AJaeger | ianw: https://review.openstack.org/#/c/274421/3/jenkins/scripts/propose_translation_update.sh line 239 in old has: | 09:07 |
| AJaeger | ironic-inspector|networking-*|neutron-*|vmware-nsx) | 09:08 |
| AJaeger | but new line 188 only | 09:08 |
| openstackgerrit | Merged openstack-infra/glean: enable glean in gentoo's init on install https://review.openstack.org/274505 | 09:08 |
| AJaeger | networking-*|neutron-*) | 09:08 |
| ianw | AJaeger: ok, cool | 09:09 |
| AJaeger | thanks, ianw | 09:10 |
| *** vgridnev has joined #openstack-infra | 09:10 | |
| ianw | AJaeger: lgtm, but i didn't +w as i'm not super confident in the area, feel free to self approve | 09:10 |
| AJaeger | ianw: will do - and monitor as usual ;) | 09:11 |
| *** abregman is now known as abregman|intervi | 09:11 | |
| *** jordanP has joined #openstack-infra | 09:15 | |
| openstackgerrit | Merged openstack-infra/project-config: Added three new repos for fuel-plugins https://review.openstack.org/269007 | 09:16 |
| *** mrmartin has quit IRC | 09:18 | |
| *** dizquierdo has joined #openstack-infra | 09:19 | |
| *** mrmartin has joined #openstack-infra | 09:20 | |
| *** ildikov has joined #openstack-infra | 09:20 | |
| openstackgerrit | Merged openstack-infra/project-config: Use new translation setup for all python projects https://review.openstack.org/274421 | 09:20 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Re-Enable django_openstack_auth/designate-dashboard translations https://review.openstack.org/274832 | 09:25 |
| *** HeOS_ has quit IRC | 09:25 | |
| *** achanda has joined #openstack-infra | 09:25 | |
| *** jcoufal has joined #openstack-infra | 09:28 | |
| *** dalgaaf has quit IRC | 09:28 | |
| *** zhurong has quit IRC | 09:30 | |
| *** achanda has quit IRC | 09:30 | |
| prometheanfire | yolanda: thanks for the review :D | 09:32 |
| yolanda | anytime | 09:32 |
| *** vivekd has quit IRC | 09:33 | |
| *** vivekd_ has joined #openstack-infra | 09:33 | |
| *** vivekd_ is now known as vivekd | 09:33 | |
| *** andymaier has joined #openstack-infra | 09:34 | |
| *** mrmartin has quit IRC | 09:35 | |
| *** wei-wang has joined #openstack-infra | 09:36 | |
| *** krotscheck1 has joined #openstack-infra | 09:37 | |
| *** zeih has quit IRC | 09:37 | |
| *** wei-wang has quit IRC | 09:37 | |
| *** zeih has joined #openstack-infra | 09:38 | |
| *** _nadya_ has joined #openstack-infra | 09:39 | |
| nibalizer | ss | 09:39 |
| *** followin_ has joined #openstack-infra | 09:39 | |
| *** ihrachys has joined #openstack-infra | 09:40 | |
| *** amotoki has quit IRC | 09:40 | |
| nibalizer | jeblair: so the difference is 1883.0000 - 1144 = 739 | 09:41 |
| nibalizer | which is about 12 minutes | 09:41 |
| *** notnownikki has joined #openstack-infra | 09:41 | |
| *** followinfrared has quit IRC | 09:42 | |
| *** zeih has quit IRC | 09:43 | |
| yolanda | hi nibalizer, AJaeger, have you ever used curl with gerrit REST API? | 09:43 |
| *** _nadya_ has quit IRC | 09:43 | |
| yolanda | i'm trying to fix the documentation of third party, but i tested authenticating with --digest and fails for me | 09:43 |
| *** kzaitsev_mb has quit IRC | 09:43 | |
| yolanda | i always did it just being authenticated on a browser, so i may be doing something wrong | 09:43 |
| nibalizer | did you have /a/ in the url? | 09:44 |
| yolanda | yes | 09:44 |
| *** hashar_ has joined #openstack-infra | 09:44 | |
| nibalizer | :shrug: | 09:44 |
| nibalizer | yolanda: i used python with https with gerrit | 09:45 |
| yolanda | curl --digest --user yolanda.robla:<<pass>> -X GET https://review.openstack.org/a/accounts/self | 09:45 |
| yolanda | i get 401 | 09:45 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Ignore empty django files https://review.openstack.org/275090 | 09:45 |
| nibalizer | yolanda: https://review.openstack.org/#/c/216877/1/jeepyb/cmd/branch_mgmt.py is the script Iwas using | 09:46 |
| *** hashar has quit IRC | 09:47 | |
| yolanda | nibalizer, i use digest as well, and i send my gerrit username | 09:49 |
| *** ihrachys has quit IRC | 09:49 | |
| yolanda | going to try with projects endpoint instead of accounts... | 09:49 |
| *** thorst has joined #openstack-infra | 09:49 | |
| yolanda | same thing | 09:50 |
| yolanda | and i have to write instructions for curl | 09:50 |
| nibalizer | i wonder if that used to work for 2.8 and something changed in 2.11? | 09:50 |
| yolanda | then we have another problem apart from writing the documentation :( | 09:51 |
| *** zeih has joined #openstack-infra | 09:51 | |
| yolanda | i cannot see anything related on gerrit logs | 09:51 |
| yolanda | mm, maybe pass needs to be hashed | 09:52 |
| *** rhallisey has joined #openstack-infra | 09:53 | |
| *** mrmartin has joined #openstack-infra | 09:54 | |
| *** amotoki has joined #openstack-infra | 09:55 | |
| *** gongysh has joined #openstack-infra | 09:56 | |
| *** thorst has quit IRC | 09:57 | |
| yolanda | no luck :( | 09:57 |
| *** abregman|intervi is now known as abregman | 09:58 | |
| *** mrmartin has quit IRC | 09:58 | |
| *** vivekd has quit IRC | 10:03 | |
| *** abregman has quit IRC | 10:03 | |
| *** alivigni has joined #openstack-infra | 10:04 | |
| *** rguillebert has joined #openstack-infra | 10:08 | |
| *** amotoki has quit IRC | 10:10 | |
| *** dprince has quit IRC | 10:12 | |
| *** mkwiek has joined #openstack-infra | 10:13 | |
| mkwiek | hi guys! is there anything else I need to do to get this patch merged to master in fuel-web https://review.openstack.org/#/c/274034/ ? It was in merge-check for several minutes, but now I can't see it anywhere on zuul, nor I can see it merged to master. | 10:14 |
| *** abregman has joined #openstack-infra | 10:15 | |
| *** openstackgerrit has quit IRC | 10:17 | |
| *** openstackgerrit has joined #openstack-infra | 10:17 | |
| openstackgerrit | Moshe Levi proposed openstack/diskimage-builder: Adding InfiniBand Support https://review.openstack.org/274781 | 10:19 |
| *** markvoelker has joined #openstack-infra | 10:19 | |
| *** esikachev has quit IRC | 10:21 | |
| *** esikachev has joined #openstack-infra | 10:22 | |
| *** markvoelker has quit IRC | 10:23 | |
| AJaeger | yolanda: no, never | 10:23 |
| *** amotoki has joined #openstack-infra | 10:24 | |
| yolanda | let's see if any other infra-root knows | 10:24 |
| AJaeger | amotoki: translation import works fine for python projects. Just noticed one issue on django: https://review.openstack.org/275090 | 10:25 |
| *** gongysh has quit IRC | 10:25 | |
| AJaeger | mkwiek: merge-check has nothing to do with merge - changes are in to check that they can merge, this is triggered by another change merging. | 10:28 |
| AJaeger | mkwiek: you need to check gate queue | 10:28 |
| AJaeger | mkwiek: your change has a depends-on on another change - and your change was approved before the other merged. | 10:29 |
| mkwiek | oh, so if +1 workflow is removed and added again it should merge? | 10:29 |
| AJaeger | mkwiek: since the jobs do not share a queue, there'S no notification of this and so somebody else needs to approve your change again. | 10:29 |
| mkwiek | cool, thank you AJaeger | 10:29 |
| AJaeger | mkwiek: Exactly, another +W or toggling the existing +W will start it | 10:29 |
| *** amotoki_ has joined #openstack-infra | 10:30 | |
| *** _nadya_ has joined #openstack-infra | 10:31 | |
| *** rossella_s has quit IRC | 10:31 | |
| *** amotoki has quit IRC | 10:31 | |
| *** rossella_s has joined #openstack-infra | 10:32 | |
| *** vivekd has joined #openstack-infra | 10:33 | |
| *** yamamoto_ has quit IRC | 10:37 | |
| openstackgerrit | Vincent Françoise proposed openstack-infra/project-config: Enable multinode testing in Watcher https://review.openstack.org/274068 | 10:38 |
| *** jcoufal has quit IRC | 10:39 | |
| *** AzherKhan has quit IRC | 10:42 | |
| *** amotoki_ has quit IRC | 10:42 | |
| *** followinfrared has joined #openstack-infra | 10:45 | |
| *** followinfrared has quit IRC | 10:45 | |
| *** amotoki has joined #openstack-infra | 10:45 | |
| *** followinfrared has joined #openstack-infra | 10:45 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:47 | |
| *** rfolco has joined #openstack-infra | 10:47 | |
| *** followin_ has quit IRC | 10:48 | |
| *** followinfrared has quit IRC | 10:50 | |
| *** ildikov has quit IRC | 10:51 | |
| *** electrofelix has joined #openstack-infra | 10:53 | |
| *** dtantsur|afk is now known as dtantsur | 10:54 | |
| *** followinfrared has joined #openstack-infra | 10:54 | |
| *** thorst has joined #openstack-infra | 10:56 | |
| *** ociuhandu has quit IRC | 10:58 | |
| *** thorst has quit IRC | 11:01 | |
| *** dims_ has joined #openstack-infra | 11:02 | |
| *** _nadya_ has quit IRC | 11:03 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Provide fallback mirrors for centos pin https://review.openstack.org/274287 | 11:03 |
| *** aysyd has joined #openstack-infra | 11:03 | |
| *** [1]Thelo has joined #openstack-infra | 11:04 | |
| *** _amrith_ is now known as amrith | 11:05 | |
| *** Thelo has quit IRC | 11:07 | |
| *** [1]Thelo is now known as Thelo | 11:07 | |
| *** exploreshaifali has quit IRC | 11:09 | |
| *** samuelBartel has quit IRC | 11:15 | |
| openstackgerrit | Merged openstack-infra/release-tools: don't require launchpad milestone for stable branch https://review.openstack.org/273133 | 11:17 |
| *** ildikov has joined #openstack-infra | 11:19 | |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Use 'Tasks' Icon for Tasks https://review.openstack.org/275130 | 11:21 |
| *** jistr has quit IRC | 11:22 | |
| *** jlanoux_ has joined #openstack-infra | 11:27 | |
| *** jlanoux has quit IRC | 11:28 | |
| *** achanda has joined #openstack-infra | 11:29 | |
| *** amotoki has quit IRC | 11:30 | |
| *** ociuhandu has joined #openstack-infra | 11:34 | |
| *** achanda has quit IRC | 11:34 | |
| *** MarkAtwood has joined #openstack-infra | 11:38 | |
| *** zeih has quit IRC | 11:39 | |
| *** esp has joined #openstack-infra | 11:46 | |
| *** tcammann has joined #openstack-infra | 11:46 | |
| *** dizquierdo is now known as dizquierdo_afk | 11:47 | |
| *** gus_ is now known as gus | 11:47 | |
| *** NobodyCa1 has joined #openstack-infra | 11:48 | |
| *** tcammann has quit IRC | 11:51 | |
| *** tzn has joined #openstack-infra | 11:52 | |
| *** NobodyCa1 has quit IRC | 11:53 | |
| *** esp has quit IRC | 11:53 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Re-enable python and networking repos for translation https://review.openstack.org/275140 | 11:54 |
| *** jlanoux_ has quit IRC | 11:55 | |
| *** exploreshaifali has joined #openstack-infra | 11:56 | |
| *** jlanoux has joined #openstack-infra | 11:58 | |
| *** kzaitsev_mb has quit IRC | 11:58 | |
| AJaeger | yolanda: The change above and https://review.openstack.org/275090 are ready for review, please. These are again translation related. | 11:59 |
| *** thorst has joined #openstack-infra | 12:00 | |
| *** thorst has quit IRC | 12:07 | |
| *** dalgaaf has joined #openstack-infra | 12:08 | |
| *** maishsk has joined #openstack-infra | 12:10 | |
| *** vikram has joined #openstack-infra | 12:10 | |
| *** Daisy has quit IRC | 12:10 | |
| *** rhallisey has quit IRC | 12:13 | |
| *** kzaitsev_mb has joined #openstack-infra | 12:14 | |
| *** andymaier has quit IRC | 12:15 | |
| *** yamamoto has joined #openstack-infra | 12:15 | |
| *** _nadya_ has joined #openstack-infra | 12:16 | |
| *** vikram has quit IRC | 12:17 | |
| amaretskiy | AJaeger: hi | 12:18 |
| AJaeger | hi amaretskiy | 12:18 |
| *** zeih has joined #openstack-infra | 12:18 | |
| amaretskiy | AJaeger: have you looked at http://amaretskiy.droppages.com/lost-seconds-while-setting-up-dsvm-workspace.html ? | 12:18 |
| AJaeger | yes, seen it - and the comments by fungi and jeblair yesterday. Thanks. | 12:18 |
| amaretskiy | AJaeger: I prepared some stat regarding the problem with stuck requests | 12:18 |
| amaretskiy | AJaeger: i missed comments (being offline) :( | 12:19 |
| AJaeger | amaretskiy: use eavesdrop, let me give you a link... | 12:19 |
| *** vikram_ has joined #openstack-infra | 12:19 | |
| AJaeger | amaretskiy: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-02-01.log.html#t2016-02-01T17:12:55 | 12:19 |
| *** markvoelker has joined #openstack-infra | 12:20 | |
| AJaeger | Question is how to move forward with this and figure out the root cause. fungi and jeblair gave some suggestions, perhaps you want to followup with them? I won't have time to do this myself. | 12:20 |
| *** amotoki has joined #openstack-infra | 12:20 | |
| amaretskiy | I found comment sin history, great | 12:20 |
| amaretskiy | AJaeger: I could dig this deeper but maybe a bit later, too busy now with the current task | 12:22 |
| *** yamamoto has quit IRC | 12:22 | |
| amaretskiy | AJaeger: I keep this in mind, maybe will be back to this soon | 12:23 |
| *** yamamoto has joined #openstack-infra | 12:23 | |
| *** weshay_xchat has joined #openstack-infra | 12:24 | |
| *** chaitu has joined #openstack-infra | 12:24 | |
| *** markvoelker has quit IRC | 12:24 | |
| *** thiagop_ has joined #openstack-infra | 12:25 | |
| chaitu | I am trying to setup fresh ci deployment when i start nodepool service, I dont see any image bulding in nodepool | 12:25 |
| openstackgerrit | Timur Sufiev proposed openstack/requirements: Exclude xvfbwrapper 0.2.8 from global requirements https://review.openstack.org/275155 | 12:26 |
| chaitu | My nodepool image.log is also empty | 12:26 |
| *** bnemec has joined #openstack-infra | 12:26 | |
| *** thiagop has quit IRC | 12:26 | |
| chaitu | can someone please tell me how could i make sure that my image is building | 12:27 |
| *** yamamoto has quit IRC | 12:27 | |
| *** thiagop_ is now known as thiagop | 12:30 | |
| *** vikram_ has quit IRC | 12:31 | |
| *** dizquierdo_afk is now known as dizquierdo | 12:33 | |
| *** julim_ has joined #openstack-infra | 12:34 | |
| *** j_king_ has joined #openstack-infra | 12:35 | |
| *** vikram_ has joined #openstack-infra | 12:35 | |
| *** gordc has joined #openstack-infra | 12:35 | |
| *** jlanoux_ has joined #openstack-infra | 12:36 | |
| *** j_king has quit IRC | 12:36 | |
| *** julim has quit IRC | 12:36 | |
| *** DuncanT has quit IRC | 12:36 | |
| *** links has quit IRC | 12:36 | |
| *** anthonyper has quit IRC | 12:36 | |
| *** ptoohill has quit IRC | 12:36 | |
| *** AJaeger has quit IRC | 12:36 | |
| *** AJaeger has joined #openstack-infra | 12:36 | |
| *** jlanoux has quit IRC | 12:36 | |
| *** bswartz has quit IRC | 12:36 | |
| *** DuncanT has joined #openstack-infra | 12:37 | |
| *** ptoohill has joined #openstack-infra | 12:38 | |
| *** anthonyper has joined #openstack-infra | 12:38 | |
| *** links has joined #openstack-infra | 12:39 | |
| *** lucasagomes is now known as lucas-hungry | 12:39 | |
| openstackgerrit | Vincent Françoise proposed openstack-infra/project-config: Watcher Dashboard project creation https://review.openstack.org/275167 | 12:41 |
| *** thorst has joined #openstack-infra | 12:42 | |
| *** boris-42 has quit IRC | 12:43 | |
| *** yamamoto has joined #openstack-infra | 12:46 | |
| openstackgerrit | Vincent Françoise proposed openstack-infra/project-config: Watcher Dashboard project creation https://review.openstack.org/275167 | 12:47 |
| openstackgerrit | Mateusz Matuszkowiak proposed openstack-infra/project-config: Added three new repos for fuel-plugins https://review.openstack.org/269567 | 12:49 |
| *** deva_ has joined #openstack-infra | 12:49 | |
| *** tcammann has joined #openstack-infra | 12:49 | |
| *** esp has joined #openstack-infra | 12:50 | |
| *** baoli has joined #openstack-infra | 12:53 | |
| *** andymaier has joined #openstack-infra | 12:53 | |
| *** yamamoto has quit IRC | 12:56 | |
| *** tcammann has quit IRC | 12:56 | |
| *** erlon has joined #openstack-infra | 12:57 | |
| *** esp has quit IRC | 12:57 | |
| *** trown|outttypeww is now known as trown | 12:57 | |
| *** dtardivel has joined #openstack-infra | 12:57 | |
| *** deva_ has quit IRC | 12:58 | |
| *** NobodyCa1 has joined #openstack-infra | 12:59 | |
| *** anteaya has joined #openstack-infra | 12:59 | |
| *** lxsli has joined #openstack-infra | 12:59 | |
| *** amotoki has quit IRC | 13:00 | |
| *** tcammann_ has joined #openstack-infra | 13:01 | |
| openstackgerrit | Ondřej Nový proposed openstack/requirements: Bump PyECLib version to 1.0.8 https://review.openstack.org/275181 | 13:02 |
| AJaeger | fungi, anteaya: Did you ever have see a followup from angdraug to the fuel plugin discussion from 2015-12-29? I commented now on 269567 as well... | 13:02 |
| *** flepied has quit IRC | 13:05 | |
| *** oomichi has quit IRC | 13:05 | |
| *** ayoung has quit IRC | 13:05 | |
| *** tcammann_ has quit IRC | 13:05 | |
| *** anteaya has quit IRC | 13:06 | |
| *** exploreshaifali has quit IRC | 13:06 | |
| *** lxsli has quit IRC | 13:06 | |
| *** NobodyCa1 has quit IRC | 13:06 | |
| *** doug-fish has joined #openstack-infra | 13:07 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul: Add poll.poll() timeout https://review.openstack.org/274445 | 13:08 |
| *** rlandy has joined #openstack-infra | 13:09 | |
| *** dizquierdo has quit IRC | 13:09 | |
| openstackgerrit | Victor Ryzhenkin proposed openstack-infra/project-config: Make murano tempest job voring and add it to gate pipeline https://review.openstack.org/275191 | 13:11 |
| *** maishsk_ has joined #openstack-infra | 13:12 | |
| *** maishsk has quit IRC | 13:12 | |
| *** maishsk_ is now known as maishsk | 13:12 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: WIP/PoC: Live-updating kanban boards https://review.openstack.org/275193 | 13:13 |
| *** hashar_ is now known as hashar | 13:13 | |
| *** amotoki has joined #openstack-infra | 13:16 | |
| *** ihrachys has joined #openstack-infra | 13:16 | |
| *** MarkAtwood has quit IRC | 13:17 | |
| *** thiagop has quit IRC | 13:18 | |
| *** samuelBartel has joined #openstack-infra | 13:20 | |
| clarkb | yolanda make sure that you arent mixing up accounts and that the password from the web ui belongs to that user | 13:20 |
| *** markvoelker has joined #openstack-infra | 13:21 | |
| *** davidlenwell has quit IRC | 13:24 | |
| *** thiagop has joined #openstack-infra | 13:24 | |
| *** xyang1 has joined #openstack-infra | 13:25 | |
| *** gongysh has joined #openstack-infra | 13:25 | |
| *** markvoelker has quit IRC | 13:26 | |
| *** markvoelker_ has joined #openstack-infra | 13:26 | |
| *** yamamoto has joined #openstack-infra | 13:27 | |
| openstackgerrit | fumihiko kakuma proposed openstack-infra/project-config: Check branch-override in ofagent job-template https://review.openstack.org/274930 | 13:27 |
| *** yamamoto has quit IRC | 13:27 | |
| *** yamamoto has joined #openstack-infra | 13:28 | |
| *** davidlenwell has joined #openstack-infra | 13:29 | |
| *** FallenPegasus has joined #openstack-infra | 13:30 | |
| *** vivekd has quit IRC | 13:31 | |
| *** mikelk has quit IRC | 13:31 | |
| *** achanda has joined #openstack-infra | 13:33 | |
| *** zeih has quit IRC | 13:33 | |
| *** vivekd has joined #openstack-infra | 13:33 | |
| krotscheck1 | clarkb: One of the things I've been asked to do for wheel mirrors is to generate a short lsb_release slug that's reasonably unique. I've got the code, but given that it's going to be used in both builders and in configure_mirror.sh, I'm wondering if there's a place where I can just have that slug written into the env. So I can reuse it elsewhere. Any suggestions? | 13:34 |
| krotscheck1 | clarkb: For reference: http://paste.openstack.org/show/485734/ | 13:36 |
| *** pradk has joined #openstack-infra | 13:36 | |
| *** pradk has quit IRC | 13:36 | |
| krotscheck1 | clarkb: Sorry, quick syntax revision : http://paste.openstack.org/show/485735/ | 13:37 |
| AJaeger | krotscheck1: can't we use /etc/os-release nowadays? Aren't all our supported distros new enough to support that instead? | 13:37 |
| *** edmondsw has joined #openstack-infra | 13:37 | |
| *** tlbr has quit IRC | 13:37 | |
| amotoki | AJaeger: neutron-fwaas translation patch just has been put into the gate | 13:37 |
| AJaeger | amotoki: yippie! Great! | 13:37 |
| *** achanda has quit IRC | 13:38 | |
| AJaeger | krotscheck1: os-release has variables that are ease to parse... | 13:38 |
| *** thiagop has quit IRC | 13:39 | |
| *** keedya has joined #openstack-infra | 13:40 | |
| jesusaur | clarkb: when you're around can you take a look at https://review.openstack.org/254992 ? | 13:40 |
| krotscheck1 | AJaeger: Lemme check. | 13:40 |
| *** annegentle has joined #openstack-infra | 13:40 | |
| krotscheck1 | AJaeger: fedora23 has it. CHecking centos and trusty | 13:41 |
| *** thiagop has joined #openstack-infra | 13:41 | |
| openstackgerrit | Flavio Percoco proposed openstack-infra/irc-meetings: Remove glance-drivers meeting https://review.openstack.org/275209 | 13:41 |
| openstackgerrit | Flavio Percoco proposed openstack-infra/irc-meetings: Change chair for Glance's meeting https://review.openstack.org/275210 | 13:41 |
| *** andymaier has quit IRC | 13:42 | |
| *** burgerk has joined #openstack-infra | 13:43 | |
| *** pradk has joined #openstack-infra | 13:43 | |
| *** pradk has quit IRC | 13:43 | |
| *** tzn has quit IRC | 13:44 | |
| AJaeger | krotscheck1: not sure whether there's a best practice for usage in OpenStack infra. | 13:44 |
| AJaeger | krotscheck1: centos7 should have it. | 13:45 |
| *** regXboi has joined #openstack-infra | 13:45 | |
| *** zul has quit IRC | 13:45 | |
| krotscheck1 | AJaeger: They all do. The only catch is that ubuntu doesn't include 'trusty' in any of the vars, so I have to get that using lsb_release | 13:46 |
| *** zeih has joined #openstack-infra | 13:46 | |
| krotscheck1 | AJaeger: I have to generate a slug that's less than 13 characters long. | 13:47 |
| *** hichihara has joined #openstack-infra | 13:47 | |
| *** tzn has joined #openstack-infra | 13:47 | |
| krotscheck1 | AJaeger: Oh, doh. I need to include the architecture as well. | 13:48 |
| krotscheck1 | So, trusty64, centos7-x64 | 13:48 |
| clarkb | krotscheck1: you can put it in project-cpnfig/jenkins/scripts then any script can source it | 13:48 |
| *** zeih has quit IRC | 13:48 | |
| krotscheck1 | clarkb: Gotcha | 13:49 |
| clarkb | also we have been using a - between distro and release elsewhere and you need to encode the cpu arch too iirc | 13:49 |
| AJaeger | clarkb: good idea, write a simple script and place it there... | 13:49 |
| Sam-I-Am | mornings | 13:49 |
| *** zeih has joined #openstack-infra | 13:49 | |
| clarkb | there is possibility we may have arm at some point | 13:49 |
| AJaeger | morning, Sam-I-Am | 13:49 |
| *** BobBall_ has joined #openstack-infra | 13:50 | |
| *** BobBall_ is now known as Guest98155 | 13:50 | |
| *** lucas-hungry is now known as lucasagomes | 13:50 | |
| *** Guest98155 is now known as Bobba | 13:51 | |
| AJaeger | project-config cores, translation unification is working fine, we're switching everything over to new setup and cleanup. Could I get reviews for https://review.openstack.org/#/c/274832 , please? | 13:51 |
| *** Bobba is now known as BobBall1 | 13:51 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: Fix on username generation https://review.openstack.org/275215 | 13:51 |
| *** krtaylor has quit IRC | 13:51 | |
| *** amrith is now known as _amrith_ | 13:51 | |
| AJaeger | 275090 and 274359 are also related but have less priority | 13:51 |
| *** keedya has quit IRC | 13:51 | |
| *** amitgandhinz has joined #openstack-infra | 13:52 | |
| esikachev | ttx: hi! | 13:52 |
| *** BobBall has quit IRC | 13:53 | |
| esikachev | ttx: patch from your comment updated https://review.openstack.org/#/c/270676/ | 13:53 |
| *** anteaya has joined #openstack-infra | 13:53 | |
| *** jtomasek_ has quit IRC | 13:55 | |
| pabelanger | morning | 13:56 |
| anteaya | morning pabelanger | 13:56 |
| openstackgerrit | Alexey Shtokolov proposed openstack-infra/project-config: Add fuel-plugin-etckeeper repository for Fuel https://review.openstack.org/260208 | 13:57 |
| *** jgriffith is now known as jgriffith_away | 13:58 | |
| AJaeger | hi anteaya, feeling well again? | 13:58 |
| *** zul has joined #openstack-infra | 13:59 | |
| anteaya | AJaeger: not a 100% but better than yesterday, thanks for asking | 13:59 |
| anteaya | my weechat server went away :( | 13:59 |
| *** BobBall1 is now known as BobBall | 13:59 | |
| *** jcoufal has joined #openstack-infra | 14:00 | |
| anteaya | I'm trying to acertain whether there is or is not a neutron meeting today | 14:00 |
| anteaya | or if it was yesterday and I missed it | 14:00 |
| *** baoli has quit IRC | 14:00 | |
| AJaeger | seems there is - was just mentioned on #openstack-neutron | 14:00 |
| *** yaume_ has joined #openstack-infra | 14:00 | |
| anteaya | woooo | 14:00 |
| *** dkranz has joined #openstack-infra | 14:00 | |
| anteaya | thanks | 14:00 |
| *** woodster_ has joined #openstack-infra | 14:01 | |
| *** baoli has joined #openstack-infra | 14:01 | |
| *** salv-orlando has joined #openstack-infra | 14:01 | |
| *** Guest37814 has quit IRC | 14:01 | |
| *** yaume has quit IRC | 14:02 | |
| *** BobBall has quit IRC | 14:04 | |
| *** krtaylor has joined #openstack-infra | 14:05 | |
| *** msuriar has left #openstack-infra | 14:06 | |
| *** tzn has quit IRC | 14:07 | |
| *** changbl has joined #openstack-infra | 14:07 | |
| *** tzn has joined #openstack-infra | 14:07 | |
| *** FallenPegasus has quit IRC | 14:08 | |
| *** jtomasek__ has joined #openstack-infra | 14:09 | |
| krotscheck1 | Can anyone here help me assemble a list of possible output for uname -m? I've got 'x86_64', I need output from a 32bit system, as well as an ARM processor. | 14:10 |
| * krotscheck1 can't believe he just asked that. | 14:10 | |
| *** tlbr has joined #openstack-infra | 14:11 | |
| *** sdake has quit IRC | 14:11 | |
| *** thomasem-away is now known as thomasem | 14:13 | |
| *** rhallisey has joined #openstack-infra | 14:13 | |
| *** dmsimard has quit IRC | 14:15 | |
| *** piet has joined #openstack-infra | 14:16 | |
| *** Hal has joined #openstack-infra | 14:16 | |
| *** Hal is now known as Guest12806 | 14:16 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Re-enable repos for translation https://review.openstack.org/275140 | 14:17 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Cleanup translation scripts https://review.openstack.org/275225 | 14:17 |
| AJaeger | krotscheck1: for a 32-bit x86 system, just run "linux32 uname -a" ;) | 14:18 |
| AJaeger | "Linux byrd 4.4.0-6.g9b438ee-default #1 SMP PREEMPT Mon Jan 25 17:01:54 UTC 2016 (9b438ee) i686 i686 i386 GNU/Linux" | 14:18 |
| *** achanda has joined #openstack-infra | 14:18 | |
| *** zz_dimtruck is now known as dimtruck | 14:18 | |
| openstackgerrit | Victor Ryzhenkin proposed openstack-infra/project-config: Make murano tempest job voting and add it to gate pipeline https://review.openstack.org/275191 | 14:18 |
| krotscheck1 | AJaeger: i386, thanks | 14:19 |
| *** annegentle has quit IRC | 14:20 | |
| *** sgordon` is now known as sgordon | 14:21 | |
| pabelanger | yolanda: do you mind helping approve 274900? Adds functional tests into the gate | 14:22 |
| *** tiswanso has joined #openstack-infra | 14:22 | |
| *** _skolekonov is now known as skolekonov | 14:22 | |
| *** jsavak has joined #openstack-infra | 14:23 | |
| *** Guest12806 has quit IRC | 14:25 | |
| openstackgerrit | Merged openstack-infra/openstackid: Fix on username generation https://review.openstack.org/275215 | 14:25 |
| *** peristeri has joined #openstack-infra | 14:26 | |
| *** mikelk has joined #openstack-infra | 14:26 | |
| SotK | I'm looking into what is causing all the rogue storyboard-worker-daemon processes on storyboard.openstack.org, and I was wondering if anyone can perhaps please explain how `puppet apply` gets invoked to update storyboard? | 14:27 |
| *** achanda has quit IRC | 14:27 | |
| * SotK finds http://docs.openstack.org/infra/system-config/puppet.html immediately after asking that | 14:28 | |
| *** piet has quit IRC | 14:29 | |
| Zara | :) | 14:29 |
| *** dizquierdo has joined #openstack-infra | 14:30 | |
| *** dizquierdo_ has joined #openstack-infra | 14:30 | |
| *** dizquierdo_ has quit IRC | 14:30 | |
| *** dizquierdo has quit IRC | 14:30 | |
| *** jistr has joined #openstack-infra | 14:31 | |
| *** rossella_s has quit IRC | 14:31 | |
| *** dizquierdo has joined #openstack-infra | 14:31 | |
| *** rossella_s has joined #openstack-infra | 14:32 | |
| *** zul has quit IRC | 14:33 | |
| *** jsavak has quit IRC | 14:33 | |
| *** tiswanso has quit IRC | 14:33 | |
| *** tiswanso has joined #openstack-infra | 14:34 | |
| *** piet has joined #openstack-infra | 14:35 | |
| *** gnuoy_ has joined #openstack-infra | 14:36 | |
| *** gnuoy_ has quit IRC | 14:36 | |
| yolanda | pabelanger, done | 14:41 |
| *** dimtruck is now known as zz_dimtruck | 14:41 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for manila / cinder no valid host bug 1540947 https://review.openstack.org/275236 | 14:42 |
| openstack | bug 1540947 in Manila "gate-manila-tempest-dsvm-neutron fails randomly due to no valid host from cinder (insufficient free space)" [Undecided,Confirmed] https://launchpad.net/bugs/1540947 | 14:42 |
| *** zz_dimtruck is now known as dimtruck | 14:42 | |
| *** mriedem has joined #openstack-infra | 14:43 | |
| *** pradk has joined #openstack-infra | 14:43 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for magnum API test timeout https://review.openstack.org/274744 | 14:45 |
| *** keedya has joined #openstack-infra | 14:47 | |
| *** Daisy has joined #openstack-infra | 14:48 | |
| openstackgerrit | Merged openstack-infra/project-config: Add functional tests for windmill https://review.openstack.org/274900 | 14:48 |
| pabelanger | yolanda: danke | 14:48 |
| *** sabeen1 has joined #openstack-infra | 14:50 | |
| *** annegentle has joined #openstack-infra | 14:50 | |
| *** jsavak has joined #openstack-infra | 14:51 | |
| *** _amrith_ is now known as amrith | 14:54 | |
| *** sabeen1 has quit IRC | 14:54 | |
| *** rbrndt has joined #openstack-infra | 14:55 | |
| *** dimtruck is now known as zz_dimtruck | 14:55 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-lodgeit: fix puppet-lodgeit to support latest mysql https://review.openstack.org/275244 | 14:56 |
| *** zul has joined #openstack-infra | 14:57 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-etherpad_lite: fix module to work with latest mysql version https://review.openstack.org/275247 | 14:57 |
| *** followinfrared is now known as infrared | 14:58 | |
| pabelanger | yolanda: could I also trouble you for 274694 too | 14:58 |
| *** infrared is now known as infra-red | 14:58 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for manila / cinder no valid host bug 1540947 https://review.openstack.org/275236 | 14:59 |
| openstack | bug 1540947 in Manila "gate-manila-tempest-dsvm-neutron fails randomly due to no valid host from cinder (insufficient free space)" [Undecided,Confirmed] https://launchpad.net/bugs/1540947 | 14:59 |
| yolanda | ah, for accessbot we need to be sure that there are no meetings going on in a wide period of time | 14:59 |
| *** angdraug has joined #openstack-infra | 14:59 | |
| yolanda | there are things going on now | 14:59 |
| *** naggappan has quit IRC | 15:00 | |
| *** esker has joined #openstack-infra | 15:00 | |
| pabelanger | yolanda: ah, good to know. Didn't know that. But make sense | 15:01 |
| *** annegentle has quit IRC | 15:02 | |
| *** dmsimard has joined #openstack-infra | 15:02 | |
| *** bryan_att has joined #openstack-infra | 15:03 | |
| *** krtaylor has quit IRC | 15:03 | |
| *** vikram_ has quit IRC | 15:03 | |
| *** annegentle has joined #openstack-infra | 15:04 | |
| yolanda | i gave a +2 without approval | 15:05 |
| AJaeger | yolanda: you can do project-config changes AFAIK for accessbot - the problem is the system-config update for meetbot to enable eavesdropping logging support | 15:05 |
| AJaeger | yolanda: so 274694 is fine to merge | 15:06 |
| yolanda | ah i mixed concepts there | 15:06 |
| AJaeger | ;) | 15:06 |
| yolanda | ok approved then | 15:07 |
| *** acruz has quit IRC | 15:08 | |
| *** piet has quit IRC | 15:08 | |
| *** Hal has joined #openstack-infra | 15:08 | |
| fungi | yolanda: meetbot, not accessbot | 15:08 |
| fungi | oh, AJaeger just said that (nevermind!) | 15:08 |
| *** Hal is now known as Guest67386 | 15:08 | |
| * fungi goes back to getting things done | 15:08 | |
| eantyshev | clarkb, yolanda, anteaya: Hello! Can I get some attention to my fix for periodic pipelines in Zuul: https://review.openstack.org/#/c/272123/ | 15:09 |
| *** zul has quit IRC | 15:10 | |
| yolanda | fungi, need your knowledge on using curl with gerrit rest api. I am writing the doc for third party, and jeblair suggested to add full curl commands there. But i tried that, using auth digest, and i always get a 401 | 15:10 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 15:10 | |
| yolanda | have you ever used that? all times that i used api , i have done it from browser, with some plugins to send requests | 15:10 |
| fungi | yolanda: using the "a/" prefix on your queries? | 15:10 |
| fungi | those are necessary for authenticated api calls | 15:11 |
| yolanda | fungi i tried that | 15:11 |
| *** rajinir has joined #openstack-infra | 15:11 | |
| *** links has quit IRC | 15:11 | |
| *** zul has joined #openstack-infra | 15:11 | |
| yolanda | curl -v --digest --user --user yolanda.robla:<<my_pass>> https://review.openstack.org/a/projects for example | 15:12 |
| yolanda | and i get 401 | 15:12 |
| yolanda | i double checked pass and username, and is correct | 15:12 |
| yolanda | one --user flag, not two... | 15:12 |
| fungi | curl --digest --user yolanda:somepassword | 15:13 |
| fungi | i think that's what i've done in the past | 15:13 |
| yolanda | fungi, no , doesn't work | 15:13 |
| *** hichihara has quit IRC | 15:13 | |
| *** rbrndt has quit IRC | 15:14 | |
| yolanda | pass should be sso pass right? | 15:14 |
| *** ihrachys has quit IRC | 15:14 | |
| *** jed56 has joined #openstack-infra | 15:14 | |
| fungi | no, http api password | 15:14 |
| yolanda | ah , so i shall configure it? | 15:14 |
| yolanda | ok, i see now | 15:14 |
| fungi | generate one at https://review.openstack.org/#/settings/http-password if you don't have one already | 15:14 |
| yolanda | ah, i didn't know about htat | 15:15 |
| *** krtaylor has joined #openstack-infra | 15:15 | |
| *** eharney has joined #openstack-infra | 15:15 | |
| yolanda | fungi, works now! | 15:15 |
| fungi | great! | 15:15 |
| yolanda | first time i used that http pass... | 15:15 |
| fungi | also be aware that gerrit prefixes the json with ~4 bytes of garbage you'll need to strip | 15:16 |
| fungi | if you don't trim it off, you won't be able to parse the response as valid json | 15:16 |
| yolanda | it's just for writing documentation about how to disable third party ci accounts, and didn't want to write something without ensuring it will work | 15:16 |
| fungi | oh, yep! good call | 15:16 |
| fungi | though you can also disable them through the ssh api | 15:16 |
| fungi | (gerrit set-account --inactive xxxxx) | 15:17 |
| yolanda | well yes, but the section i'm writing talks about both | 15:17 |
| fungi | very cool! | 15:17 |
| yolanda | fungi, can you get account active details by ssh? haven't seen the command for it | 15:17 |
| *** baoli has quit IRC | 15:17 | |
| *** ajmiller has joined #openstack-infra | 15:18 | |
| *** zz_dimtruck is now known as dimtruck | 15:18 | |
| *** zul has quit IRC | 15:18 | |
| *** ayoung has joined #openstack-infra | 15:20 | |
| *** dmsimard has quit IRC | 15:21 | |
| fungi | yolanda: not that i'm aware, no. at least i don't see it listed at https://review.openstack.org/Documentation/cmd-index.html#_server so the rest api does provide a useful mechanism to validate your work | 15:22 |
| yolanda | fungi yes, i was looking at the ssh equivalent for rest api command and i was surprised to don't find it. I thought both apis should be pretty equivalent | 15:23 |
| *** jaosorior has joined #openstack-infra | 15:23 | |
| fungi | the ssh api is missing at least a few informational methods | 15:24 |
| *** amotoki has quit IRC | 15:24 | |
| fungi | but i agree that's the one which surprises me the most too | 15:24 |
| *** esikachev has quit IRC | 15:25 | |
| *** mtanino has joined #openstack-infra | 15:27 | |
| *** placko_ has joined #openstack-infra | 15:27 | |
| *** zul has joined #openstack-infra | 15:30 | |
| *** jpr has joined #openstack-infra | 15:36 | |
| *** zul has quit IRC | 15:36 | |
| krotscheck1 | fungi: Can I have an underscore in a node name? | 15:36 |
| AJaeger | yolanda: great! | 15:37 |
| *** baoli has joined #openstack-infra | 15:38 | |
| *** jpr has quit IRC | 15:39 | |
| *** tiswanso has quit IRC | 15:39 | |
| *** jpr has joined #openstack-infra | 15:39 | |
| yolanda | fungi, also in the article jeblair points that i should show some command to retrieve the account numeric id, given the email or username. I haven't found any way in REST API to get that (i can only think about mysql) . Do you know if that's possible? | 15:39 |
| *** piet has joined #openstack-infra | 15:39 | |
| anteaya | krotscheck1: looks like hyphens or periods are preferred in node names: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=181 | 15:39 |
| AJaeger | fungi, anteaya: Did you ever have see a followup from angdraug to the fuel plugin discussion from 2015-12-29? I commented on 269567 as well and wonder how to continue with that one... | 15:40 |
| anteaya | yolanda: yes: curl -X GET 'https://review.openstack.org/accounts/pci-ci@intel.com' | 15:40 |
| AJaeger | fungi, anteaya : as well with 260208 | 15:40 |
| fungi | krotscheck1: depends on what you mean by "node name" but if it's going into dns then no, you really shouldn't (it violates rfc 952/1123, and is clarified in 1912 particularly) | 15:40 |
| krotscheck1 | fungi: Thanks | 15:41 |
| anteaya | AJaeger: I have not seen a followup, mind you I have been away for a week at nova mid-cycle and have lost my weechat server, so the possibility exists | 15:41 |
| fungi | krotscheck1: i'm guessing you mean "hostname" | 15:41 |
| *** infra-red has quit IRC | 15:41 | |
| AJaeger | anteaya: I haven't seen anything either, just wanted to double check | 15:41 |
| *** tiswanso has joined #openstack-infra | 15:41 | |
| anteaya | AJaeger: yup, good idea | 15:41 |
| krotscheck1 | fungi: I'm trying to figure out if it's possible to have the hostname contain the same string output from uname -m, | 15:41 |
| krotscheck1 | It's not critical. | 15:42 |
| AJaeger | anteaya: regarding voting jobs only in the check queue: Something to discuss in the team meeting today? Here's an example: https://review.openstack.org/#/c/274182/ | 15:42 |
| fungi | yolanda: you can query accounts by e-mail address, username, full name, et cetera and one of the details returned is the account id | 15:42 |
| fungi | (in the rest api that is) | 15:42 |
| angdraug | AJaeger: still working on that followup | 15:42 |
| yolanda | fungi, anteaya, thx | 15:42 |
| AJaeger | angdraug: then I'll wait with further reviews... | 15:43 |
| anteaya | AJaeger: that only leaves one neutron gate job | 15:43 |
| fungi | krotscheck1: probably needs to be some transformation of that (replace any characters not in the hostname regex with a hyphen?) | 15:43 |
| * yolanda was trying incorrectly, with /a/accounts/email/detail | 15:43 | |
| anteaya | AJaeger: I would like to hear what tempest reviewers have to say on that patch | 15:44 |
| AJaeger | anteaya: there're several templates with gate jobs... | 15:44 |
| krotscheck1 | fungi: Gimme 20 minutes, I'll update my patches for you to doublecheck. | 15:44 |
| anteaya | AJaeger: I already voiced my opinion about communication drift in neutron in today's meeting so I'm not popular with neutron at the moment | 15:44 |
| AJaeger | anteaya: do you want to add them? | 15:44 |
| *** tcammann_ has joined #openstack-infra | 15:44 | |
| *** pradk_ has joined #openstack-infra | 15:44 | |
| *** pradk has quit IRC | 15:44 | |
| *** pradk_ is now known as pradk | 15:44 | |
| anteaya | AJaeger: I can comment on the patch that I would like to see tempest reviewers opinion on the patch | 15:45 |
| anteaya | but I'm not feeling obliged to track them down | 15:45 |
| AJaeger | ;) | 15:45 |
| fungi | krotscheck1: _technically_ the domain part can contain other characters, they're really only forbidden as part of the last (hostname) component as far as i know, but convention is to avoid them throughout the entire fqdn for records which reference hosts by name (there are other record types for which underscores are okay) | 15:45 |
| *** sfinucan has joined #openstack-infra | 15:45 | |
| *** NobodyCa1 has joined #openstack-infra | 15:45 | |
| anteaya | AJaeger: thanks for bringing that patch to my attention | 15:45 |
| *** esp has joined #openstack-infra | 15:46 | |
| anteaya | I now have to do battle with networks to see if I can get a server up, wish me luck | 15:46 |
| AJaeger | anteaya: as discussed last week ;) | 15:46 |
| anteaya | AJaeger: yup, thank you :) | 15:46 |
| * AJaeger wishes anteaya luck to master the network daemons ;) | 15:46 | |
| anteaya | I don't know about master, if I can get one server up I'll be happy | 15:47 |
| *** baoli has quit IRC | 15:47 | |
| AJaeger | anteaya: One step at a time to mastery ;) | 15:47 |
| *** baoli has joined #openstack-infra | 15:48 | |
| anteaya | ha ha ha | 15:48 |
| tristanC | yolanda: thank you for the zuul poll timeout review! | 15:49 |
| *** bhunter71 has joined #openstack-infra | 15:49 | |
| *** tcammann_ has quit IRC | 15:50 | |
| *** esp has quit IRC | 15:50 | |
| *** ilyashakhat has joined #openstack-infra | 15:50 | |
| *** NobodyCa1 has quit IRC | 15:51 | |
| krotscheck1 | When configure_mirror.sh is executed (nodepool), does it have access to jenkins/slave_scripts? | 15:51 |
| * krotscheck1 is guessing not | 15:51 | |
| *** ihrachys has joined #openstack-infra | 15:51 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/system-config: Add instructions on how to disable a third party ci account https://review.openstack.org/265359 | 15:53 |
| *** placko_ has quit IRC | 15:53 | |
| *** zeih has quit IRC | 15:53 | |
| *** _nadya_ has quit IRC | 15:54 | |
| *** dims has joined #openstack-infra | 15:54 | |
| clarkb | krotscheck1: it will beacuse we install all of the slave_scripts on all of our images | 15:54 |
| *** dims_ has quit IRC | 15:54 | |
| clarkb | krotscheck1: however the path will be the installed path and not in the local dir | 15:55 |
| *** jlanoux_ has quit IRC | 15:55 | |
| krotscheck1 | clarkb: Gotcha | 15:55 |
| krotscheck1 | clarkb: Thanks | 15:55 |
| clarkb | /usr/local/jenkins/scripts or something, can double check with puppet to find the rooted path | 15:56 |
| *** ihrachys has quit IRC | 15:58 | |
| *** maishsk has quit IRC | 15:58 | |
| *** devananda has joined #openstack-infra | 16:00 | |
| openstackgerrit | Al Miller proposed openstack/requirements: Add neutron-lbaas-dashboard to projects.txt https://review.openstack.org/275291 | 16:01 |
| *** david-lyle has joined #openstack-infra | 16:02 | |
| *** tphummel has joined #openstack-infra | 16:02 | |
| *** samuelBartel has quit IRC | 16:02 | |
| *** jsavak has quit IRC | 16:02 | |
| *** jsavak has joined #openstack-infra | 16:03 | |
| *** yamamoto has quit IRC | 16:05 | |
| AJaeger | anteaya, yolanda, clarkb: Could either of you review the stack starting with https://review.openstack.org/#/c/274832 on translation changes, please? | 16:06 |
| *** rbrndt has joined #openstack-infra | 16:07 | |
| AJaeger | whenever it's convenient for you - anteaya, get your network server up first ;) | 16:07 |
| *** yamamoto has joined #openstack-infra | 16:07 | |
| *** kzaitsev_mb has quit IRC | 16:08 | |
| *** yamamoto has quit IRC | 16:08 | |
| yolanda | AJaeger i cannot now but i'll take a look as soon as i can | 16:09 |
| anteaya | AJaeger: thanks | 16:09 |
| anteaya | btw Sam-I-Am is magic | 16:09 |
| anteaya | just pure magic | 16:09 |
| *** sfinucan has quit IRC | 16:09 | |
| AJaeger | Sam-I-Am: what did he do? Yeah, he is... | 16:10 |
| *** sfinucan has joined #openstack-infra | 16:10 | |
| *** pigmej has joined #openstack-infra | 16:10 | |
| pigmej | yolanda: hey, could you please look at https://review.openstack.org/#/c/267453/ ? It just needs one more + | 16:11 |
| Sam-I-Am | hey! | 16:11 |
| AJaeger | yolanda: no worries, thanks | 16:11 |
| jeblair | yolanda: just GET /a/accounts/emc.isilon.ci@emc.com | 16:12 |
| jeblair | yolanda: returns {"_account_id":15942,"name":"EMC Isilon CI","email":"emc.isilon.ci@emc.com","username":"e"} | 16:12 |
| *** esikachev has joined #openstack-infra | 16:12 | |
| * AJaeger will be back later... | 16:12 | |
| yolanda | jeblair yes, i know now... i was misunderstandng the documentation and was trying with /a/accounts/email/detail | 16:13 |
| yolanda | and was failing | 16:13 |
| *** gokrokve has joined #openstack-infra | 16:13 | |
| *** annegentle has quit IRC | 16:13 | |
| *** vgridnev has quit IRC | 16:14 | |
| jeblair | ok | 16:15 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Add wheel mirror to configure_mirror.sh https://review.openstack.org/267117 | 16:15 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Added wheel-release job https://review.openstack.org/273549 | 16:15 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Create jobs for a wheel mirror https://review.openstack.org/164927 | 16:15 |
| *** mattt has left #openstack-infra | 16:17 | |
| *** Daisy has quit IRC | 16:17 | |
| *** gongysh has quit IRC | 16:18 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Temporarily stop releasing after bandersnatch https://review.openstack.org/275299 | 16:18 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Revert "Temporarily stop releasing after bandersnatch" https://review.openstack.org/275300 | 16:18 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Revert "Temporarily switch to AFS RW mirrors" https://review.openstack.org/275301 | 16:18 |
| jeblair | fungi: i'd like to go aheand and land https://review.openstack.org/275299 for safety | 16:19 |
| *** Daisy has joined #openstack-infra | 16:19 | |
| fungi | jeblair: lgtm | 16:19 |
| fungi | i agree that's a good idea for now | 16:19 |
| *** samuelBartel has joined #openstack-infra | 16:20 | |
| fungi | also didn't get far on the mirror reviews last night but trying to squeeze some time in between my current meeting and the team meeting to get through them | 16:20 |
| jeblair | last night i cleaned up the mess i made yesterday and started a new vos release in k5start+screen. i'm expecting that to finish wed night | 16:20 |
| krotscheck1 | Ok, I _think_ that last project_config patch chain will work as advertised. | 16:20 |
| krotscheck1 | Also, added bonus, generic afs-release builder. | 16:20 |
| *** sdake has joined #openstack-infra | 16:21 | |
| *** esikachev has quit IRC | 16:22 | |
| krotscheck1 | jeblair, fungi: Should I WIP the wheel mirror patches until AFS things settle down? | 16:22 |
| jeblair | krotscheck1: nope | 16:22 |
| krotscheck1 | Alrightey | 16:22 |
| jeblair | krotscheck1: none of the volume replication stuff we're working through on the pypi mirrors should affect the wheel mirror work | 16:22 |
| *** abregman has quit IRC | 16:23 | |
| krotscheck1 | jeblair: Different volume? | 16:23 |
| jeblair | yep | 16:23 |
| openstackgerrit | ice4o@hotmail.com proposed openstack-infra/jenkins-job-builder: Add HockeyApp Plugin support. https://review.openstack.org/275304 | 16:24 |
| *** bhunter71 has quit IRC | 16:24 | |
| jeblair | i don't understand the difference between the "packages" and "user_packages" variables in modules/openstack_project/manifests/users_install.pp | 16:24 |
| krotscheck1 | Alright then, I'll pester y'all when all the builds pass. | 16:24 |
| *** ilyashakhat has quit IRC | 16:24 | |
| jeblair | oh... | 16:25 |
| jeblair | we only install user_packages if we install users | 16:25 |
| jeblair | this is strange, but oh well. | 16:25 |
| *** ilyashakhat has joined #openstack-infra | 16:26 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add iftop to all Ubuntu servers https://review.openstack.org/275308 | 16:26 |
| *** infra-red has joined #openstack-infra | 16:26 | |
| *** woodster_ has quit IRC | 16:26 | |
| jeblair | how have we lived without that ^ ? | 16:26 |
| *** ilyashakhat has quit IRC | 16:27 | |
| jeblair | iftop -f "not port domain" -P | 16:27 |
| jeblair | is my preference | 16:27 |
| *** jlanoux has joined #openstack-infra | 16:29 | |
| *** fesp has joined #openstack-infra | 16:29 | |
| *** mdenny has quit IRC | 16:29 | |
| *** ddecapit has joined #openstack-infra | 16:30 | |
| *** ddecapit is now known as DuaneDeC7 | 16:30 | |
| *** jpr has quit IRC | 16:31 | |
| *** jpr has joined #openstack-infra | 16:31 | |
| fungi | woah! | 16:31 |
| fungi | it's like openbsd's pftop | 16:32 |
| *** amaretskiy has quit IRC | 16:32 | |
| fungi | which i'd regularly found myself desiring an equivalent of for linux | 16:32 |
| clarkb | jeblair: greghaynes we need a release of gear for that double disconnect thing right? | 16:33 |
| fungi | though i guess it's somewhere between pftop and ntop feature wise | 16:33 |
| jeblair | i just ran it on the afs fileservers, and noticed that one of them the bar graph goes to 954Mb, but the other only goes to 95.4Mb. i wonder how it decides how long it should be... | 16:33 |
| * clarkb is trying to make a list of nodepool imge builder related items that should be done in the near future | 16:33 | |
| jeblair | clarkb: yeah, but that's just a test code cleanup basically | 16:34 |
| EmilienM | why http://git.openstack.org/cgit/openstack-infra/ansible-puppet/ does not use http://docs.ansible.com/ansible/puppet_module.html ? Is there any plan to migrate to the ansible module? | 16:34 |
| jeblair | clarkb: i'll go ahead and release it | 16:34 |
| jeblair | EmilienM: they're the same code | 16:34 |
| clarkb | EmilienM: I think because ansible-puppet is the source for that puppet module and we just haven't switched over yet | 16:35 |
| clarkb | jeblair: thank you | 16:35 |
| jeblair | EmilienM: we'll probably move to it eventually, but we still keep our own copy so it's easy for us to CD it without CDing ansible | 16:35 |
| EmilienM | ok no problem, I was just asking :-) | 16:35 |
| jeblair | np, good question | 16:35 |
| EmilienM | I was actually for that kind of tool and I found it :) | 16:36 |
| *** ilyashakhat has joined #openstack-infra | 16:36 | |
| *** DuaneDeC7 has quit IRC | 16:37 | |
| *** ilyashakhat has quit IRC | 16:38 | |
| *** ilyashakhat has joined #openstack-infra | 16:38 | |
| clarkb | our last image builds were from saturday, I think we must still be having errors which cause the nodepoold process to think builds are in progress when the builder has nothing queued | 16:38 |
| *** infra-red has quit IRC | 16:38 | |
| clarkb | time for me to go review the changes that should address disconnects | 16:38 |
| *** infra-red has joined #openstack-infra | 16:39 | |
| clarkb | infra-root https://review.openstack.org/#/c/274821/1 should hopefully make multinode more reliable for us | 16:39 |
| *** gabor_antal_ has joined #openstack-infra | 16:39 | |
| jeblair | clarkb: yes, the disconnect stuff is still a problem | 16:39 |
| freerunner | Hi jeblair fungi clarkb AJaeger and project-config folks! Could you please review it https://review.openstack.org/#/c/275191/2 ? | 16:40 |
| fungi | clarkb: last pass on 274821 failed the multinode-full job? | 16:41 |
| *** mgkwill_training is now known as mgkwill | 16:41 | |
| jeblair | freerunner: bare review requests like that aren't helpful | 16:41 |
| clarkb | jeblair: it is worth noting that on internap we now have a decent number of multinode passes. I think updating the security groups there may have been the ticket | 16:41 |
| *** infra-re_ has joined #openstack-infra | 16:41 | |
| clarkb | fungi: it did, but not due to networking from what I can tell. Just general performance issues | 16:41 |
| fungi | ahh | 16:42 |
| clarkb | fungi: it ran on bluebox too, I think the broken networking may be masking another issue there | 16:42 |
| *** esker has quit IRC | 16:42 | |
| jeblair | clarkb: woot! mgagne ^ | 16:42 |
| *** fesp has quit IRC | 16:42 | |
| mgagne | clarkb :D | 16:42 |
| gabor_antal_ | Hello there, I'd like to commit into gerrit. Can someone help me with some docs, or anything like that? | 16:43 |
| jeblair | clarkb: should we ask bluebox folks why our gre packets might be getting dropped? maybe they can check on whether conntrack_gre is present or needed? | 16:43 |
| mgagne | jeblair you can get a smiley face too :D | 16:43 |
| freerunner | jeblair Oh. So, this patch will replace our (murano) not-tempest gate job with a tempest one. | 16:43 |
| *** bpokorny has joined #openstack-infra | 16:43 | |
| clarkb | jeblair: that would be great | 16:43 |
| *** ianw has quit IRC | 16:43 | |
| clarkb | jeblair: but I think vxlan is a viable workaround if we want to punt | 16:44 |
| mgagne | jeblair any instance UUID I can check? | 16:44 |
| fungi | gabor_antal_: you mean specifically for the openstack community's gerrit code review instance? or something else? | 16:44 |
| mgagne | jeblair want to check host for conntrack helper presence of not | 16:44 |
| mgagne | or not* | 16:44 |
| *** infra-red has quit IRC | 16:44 | |
| clarkb | mgagne: I think internap may be working now. I will keep an eye on it though | 16:44 |
| clarkb | mgagne: just bluebox seems to have trouble now | 16:45 |
| mgagne | any instance you know of I can check? | 16:45 |
| clarkb | ya let me dig up the uuid for this one I have a fail log on | 16:45 |
| mgagne | clarkb we haven't officially loaded the gre conntrack helper on our hosts | 16:45 |
| *** bpokorny has quit IRC | 16:45 | |
| fungi | gabor_antal_: presumably you're looking for http://docs.openstack.org/infra/manual/developers.html | 16:46 |
| mgagne | fail at internap or bluebox? I don't control bluebox host =) | 16:46 |
| *** sc68cal has joined #openstack-infra | 16:46 | |
| clarkb | mgagne: internap | 16:46 |
| *** bpokorny has joined #openstack-infra | 16:46 | |
| mgagne | ok, some are failing and others not? | 16:46 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Deal with a blank ZUUL_CHANGES later https://review.openstack.org/271559 | 16:46 |
| *** gibi_ is now known as gibi | 16:47 | |
| clarkb | mgagne: 614a9590-9efe-44de-b398-a99bf6d6d4f6 and 2a631ff9-8e50-41d4-9059-fa598f3cef55 were the internap pair. Yes we now have a 50% pass rate over the last 10 days but those two ran a job recently and it failed to ssh over the overlay network | 16:47 |
| *** dims_ has joined #openstack-infra | 16:47 | |
| mgagne | I would like to know a pair that worked too to compare | 16:47 |
| clarkb | mgagne: the fail rate was higher before I opened up the security group with unspecified protocol types | 16:47 |
| clarkb | mgagne: I can dig that up too if I have more time | 16:48 |
| gabor_antal_ | fungi: I also got that documents, any of the infra module's are the same as other OpenStack modules? | 16:48 |
| mgagne | could fail be unrelated to provider? like race or whatever? trying to see if I need to dig more in hosts or not. | 16:48 |
| jeblair | greghaynes, clarkb: gear 0.5.9 pushed | 16:48 |
| *** jaosorior has quit IRC | 16:49 | |
| *** Swami has joined #openstack-infra | 16:49 | |
| *** dims has quit IRC | 16:49 | |
| fungi | gabor_antal_: i don't understand the context of your question. i thought you were asking if someone could help you find documentation on submitting changes to our code review system. can you be more specific about what you're looking for help with? | 16:49 |
| *** DuaneDeC7 has joined #openstack-infra | 16:50 | |
| clarkb | hrm not so easy to find the stuff in the logs that the health dashboard claims succeeded | 16:51 |
| mgagne | :-/ | 16:51 |
| clarkb | I wonder if the health dashboard is buggy, I see that the same job is recordred twice, once as success and once as failure | 16:51 |
| clarkb | whcih would explain the roughtly 1:1 ratio | 16:51 |
| clarkb | mgagne: looking at http://status.openstack.org/openstack-health/#/g/node_provider/internap-nyj01?groupKey=node_provider&resolutionKey=hour fwiw | 16:51 |
| mgagne | I had difficulty finding actual jobs associated to a provider | 16:51 |
| clarkb | so maybe I need to do more digging there | 16:51 |
| mgagne | ok, what I know for sure is that gre conntrack helper isn't loaded on the hosts the jobs ran | 16:52 |
| *** armax has joined #openstack-infra | 16:52 | |
| *** infra-re_ has quit IRC | 16:52 | |
| clarkb | mgagne: ok, from your perspective would it be better if we used vxlan? | 16:52 |
| clarkb | because that is the other alternative I have been working | 16:53 |
| mgagne | clarkb does vxlan require a conntrack helper? | 16:53 |
| openstackgerrit | Vincent Françoise proposed openstack-infra/project-config: Watcher Dashboard project creation https://review.openstack.org/275167 | 16:53 |
| *** bhunter71 has joined #openstack-infra | 16:53 | |
| *** bnemec has quit IRC | 16:53 | |
| mgagne | clarkb just trying to see if gre/vxlan requires collaboration from providers or not | 16:53 |
| *** jistr has quit IRC | 16:53 | |
| *** rhallisey has quit IRC | 16:53 | |
| mgagne | haven't tried any of them, well, tried vxlan on grizzly install 1-2 years ago | 16:53 |
| clarkb | mgagne: I don't think so, it is udp and the endpoints should manage their own state | 16:54 |
| mgagne | our netadmin was flabbergasted that I didn't need his help to setup | 16:54 |
| *** reed_ has joined #openstack-infra | 16:55 | |
| clarkb | mgagne: we aren't using neutron fwiw. Its just an ovs bridge connected to another ovs brdige via gre/vxlan | 16:55 |
| mgagne | I don't have a strong opinion. whatever works without involving providers is fine tbh | 16:55 |
| *** e0ne has quit IRC | 16:55 | |
| clarkb | the big downside to vxlan is that it will use multicast for broadcast among members but almost nothing supports multicast | 16:55 |
| clarkb | which makes arp less fun aiui but it should mostly work in these small setups | 16:56 |
| *** annegentle has joined #openstack-infra | 16:56 | |
| *** achanda has joined #openstack-infra | 16:57 | |
| *** matrohon has quit IRC | 16:57 | |
| fungi | yeah, i'm honestly surprised that multicast groups are working where something as simple as a less common ip protocol is being filtered | 16:57 |
| mgagne | I have yet to fully understand the underlying details =) | 16:57 |
| *** scheuran has quit IRC | 16:57 | |
| clarkb | fungi: well mutlicast doesn't work, it fails back to unicast iirc | 16:57 |
| fungi | clarkb: or are you saying that multicast doesn't come into play because we're using it for a point-to-point tunnel anyway? | 16:57 |
| fungi | that, got it | 16:57 |
| gabor_antal_ | fungi: I'd like to ask that is there any special things with the infra modules? Or is it the same as anywhere else | 16:58 |
| sc68cal | clarkb: it can use multicast for broadcast, otherwise it'll just flood all the nodes | 16:58 |
| sc68cal | sadly I don't know how well it is tested, that implies hardware that can do IGMP | 16:58 |
| fungi | gabor_antal_: code contribution process for all official openstack projects is consistent, including the infra projects | 16:58 |
| *** infra-red has joined #openstack-infra | 16:58 | |
| jeblair | krotscheck1: would you mind trying to avoid rebasing so frequently? | 16:58 |
| *** mikelk has quit IRC | 16:58 | |
| jeblair | krotscheck1: it's basically impossible for me to tell whether anything was changed between PS 26 and PS 27 of https://review.openstack.org/164927 | 16:59 |
| gabor_antal_ | fungi: okay, thanks | 16:59 |
| fungi | gabor_antal_: also you had mentioned in a private irc message you were wanting to assist with code quality in openstack. there is actually a quality assurance team, if you haven't talked to them already. they tend to hang out in the #openstack-qa channel | 16:59 |
| *** baoli has quit IRC | 16:59 | |
| *** Guest67386 has quit IRC | 16:59 | |
| openstackgerrit | Igor Belikov proposed openstack-infra/system-config: Get debian kernel headers based on architecture https://review.openstack.org/275323 | 16:59 |
| gabor_antal_ | fungi: thanks, we've already talked with them | 17:00 |
| *** kzaitsev_mb has joined #openstack-infra | 17:00 | |
| *** vivekd has quit IRC | 17:00 | |
| fungi | very cool. welcome to the community! | 17:00 |
| jeblair | krotscheck1: basically it's most helpful to only rebase if you have to because there is a conflict. | 17:00 |
| *** baoli has joined #openstack-infra | 17:00 | |
| *** kzaitsev_mb has quit IRC | 17:01 | |
| *** kzaitsev_mb has joined #openstack-infra | 17:01 | |
| *** kzaitsev_mb has quit IRC | 17:01 | |
| *** kzaitsev_mb has joined #openstack-infra | 17:01 | |
| *** samuelBartel has quit IRC | 17:01 | |
| *** ashleighfarnham has joined #openstack-infra | 17:01 | |
| *** kzaitsev_mb has quit IRC | 17:02 | |
| *** kzaitsev_mb has joined #openstack-infra | 17:02 | |
| *** vivekd has joined #openstack-infra | 17:02 | |
| krotscheck1 | jeblair: I'll try, but I may be too ocd to feel comfortable with that. | 17:02 |
| fungi | krotscheck1: if it helps, you can be *even more( ocd about trying to rebase onto conflicting changes only | 17:03 |
| clarkb | jeblair: greghaynes https://review.openstack.org/#/c/272384/ depends on the new gear release. Is that a strict dependency and we need to restart the geard before updating nodepool? | 17:03 |
| fungi | i get pretty ocd about introducing the smallest divergence possible in updating my changes | 17:03 |
| *** jsavak has quit IRC | 17:04 | |
| *** jsavak has joined #openstack-infra | 17:04 | |
| jeblair | krotscheck1: what fungi said -- and seriously -- as a reviewer, when i work through changes it's important to see how they evolve over time. when there is a rebase + change in a single update, the entire flow is lost and there is no narrative connection with the previous patchsets | 17:04 |
| openstackgerrit | Merged openstack-infra/git-restack: Initial commit of git-restack https://review.openstack.org/259647 | 17:04 |
| *** ilyashakhat has quit IRC | 17:04 | |
| jeblair | krotscheck1: so the most ideal thing is updates only without rebasing; then if something has a conflict, a rebase only (ideally with a comment saying so), and then if an update is needed, a separate patchset to do that | 17:05 |
| jeblair | krotscheck1: i'm working on making the git-restack tool to make that a bit easier for long-running patch series | 17:05 |
| fungi | when the rebase is ugly enough, i've even gone so far as to do back-to-back patchsets to first perform the rebase and then make the new modifications i needed | 17:05 |
| zaro | morning | 17:05 |
| fungi | (review comments pointing out the rebase patchset are also very helpful for that) | 17:05 |
| *** achanda has quit IRC | 17:06 | |
| *** ashleighfarnham has quit IRC | 17:06 | |
| greghaynes | clarkb: it depends on the client only | 17:07 |
| fungi | zaro: not sure if you saw my discussion with madorn yesterday, but curious if you had recommendations for how best to clean up the faux "registered users" groups file entries in our gerrit once manage-projects is no longer creating them | 17:07 |
| greghaynes | clarkb: that test fail | 17:07 |
| greghaynes | clarkb: is wierd | 17:07 |
| clarkb | greghaynes: cool so that will get updated when we update nodepool itself | 17:07 |
| jeblair | krotscheck1: PS27 in 164927 is in fact a rebase + a significant change | 17:07 |
| jeblair | krotscheck1: so there's no good way for me to review that | 17:07 |
| jeblair | krotscheck1: would you like to restack those without the rebases, or shall i? | 17:07 |
| fungi | zaro: especially with regards to whether there is a way to do it live without taking gerrit offline | 17:07 |
| clarkb | jeblair: git review has a thing that works when there aren't conflicts | 17:07 |
| *** ashleighfarnham has joined #openstack-infra | 17:08 | |
| zaro | fungi: just got myself back in channel today after the HP cloud going offline | 17:08 |
| clarkb | -m ? | 17:08 |
| jeblair | clarkb: how do i leave comments with git review? | 17:08 |
| fungi | zaro: or if it's going to be easier to do that cleanup in a maintenance window | 17:08 |
| openstackgerrit | Vincent Françoise proposed openstack-infra/project-config: Watcher Dashboard project creation https://review.openstack.org/275167 | 17:08 |
| * zaro looks at yesterday's backscroll | 17:08 | |
| clarkb | jeblair: you don't, but it gives you the ability to review the delta in an easier to understand diff | 17:08 |
| anteaya | zaro: Sam-I-Am just helped me get a server up, I'm working on instructions to share | 17:09 |
| clarkb | and the line numbers should be static for the "new" side | 17:09 |
| fungi | zaro: discussion started around 23:47 utc | 17:09 |
| krotscheck1 | jeblair: restack on... what, the previous commit? | 17:09 |
| dhellmann | fungi : do you think you'll have time this week to look at setting up releases.openstack.org? the patch series is https://review.openstack.org/#/q/topic:releases-openstack-org but I'm sure there are some steps to handle outside of that (DNS?) | 17:09 |
| clarkb | krotscheck1: my trick is to git rebase -i HEAD~X | 17:09 |
| jeblair | krotscheck1: yeah, PS26 was based on 8ef6e559e802f9e8969be9f3d7d4ff5261b2bcb6; so we need to cherry-pick cd5fe79762a7251420a33e64b1aecf3c37ff6ec9 (PS27) on top of that | 17:10 |
| fungi | dhellmann: i think adding a couple cname entries in dns is all we need if it's being hosted from static.openstack.org like i remember | 17:10 |
| fungi | dhellmann: if so, i can add those now--there's no need to wait and it only takes a moment | 17:10 |
| dhellmann | fungi : is that something I can do with a patch, or is that a manual step? | 17:11 |
| dhellmann | fungi : ah, cool | 17:11 |
| krotscheck1 | Doesn't git review do rewinds anyway? | 17:11 |
| jeblair | krotscheck1: nope | 17:11 |
| * krotscheck1 can try | 17:11 | |
| fungi | dhellmann: manual process since rackspace doesn't use openstack to host dns | 17:11 |
| *** fesp has joined #openstack-infra | 17:11 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Add wheel mirror to configure_mirror.sh https://review.openstack.org/267117 | 17:11 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Added wheel-release job https://review.openstack.org/273549 | 17:11 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Create jobs for a wheel mirror https://review.openstack.org/164927 | 17:11 |
| dhellmann | fungi : ack | 17:11 |
| *** fesp has quit IRC | 17:12 | |
| openstackgerrit | Igor Belikov proposed openstack-infra/puppet-jenkins: Use 'ruby' instead of 'ruby1.9.1' https://review.openstack.org/275329 | 17:12 |
| fungi | dhellmann: though if we also need https then i'll have to get a cert issued, which means i may need to step up priority for looking into switching the cert authority i've been using | 17:12 |
| *** sridhar_ram has joined #openstack-infra | 17:12 | |
| jeblair | krotscheck1: perfect, thanks! | 17:12 |
| dhellmann | fungi : I hadn't thought about https explicitly, but it makes sense. ttx, thoughts? | 17:12 |
| fungi | dhellmann: we can always add it a little later if it wasn't an up-front requirement, but i think it should be added soonish | 17:13 |
| fungi | i don't want to hold up the deployment for that though | 17:13 |
| dhellmann | fungi : I agree we should do it, and I'm OK with not blocking on it if adding it later isn't a big deal | 17:13 |
| zaro | anteaya: ugh! that hp cloud down was a big headache for me | 17:13 |
| *** diana_clarke has joined #openstack-infra | 17:13 | |
| *** dtantsur is now known as dtantsur|afk | 17:13 | |
| anteaya | zaro: yup | 17:14 |
| fungi | dhellmann: yeah, i already set up https for other important sources of information (governance, tarballs, security) and i think releases falls into a similar category | 17:14 |
| jeblair | git-restack 1.0.0 pushed | 17:14 |
| dhellmann | fungi : ++ | 17:14 |
| *** salv-orlando has quit IRC | 17:14 | |
| *** ifarkas has quit IRC | 17:14 | |
| fungi | dhellmann: so anyway, if the change series is close to being landable i can add whatever else we need to my pile for this week | 17:15 |
| *** sdake has quit IRC | 17:15 | |
| *** tphummel has quit IRC | 17:16 | |
| *** baoli has quit IRC | 17:16 | |
| dhellmann | fungi : the system-config and project-config patches have one +2 each; the doc change to add the redirect is blocked waiting for those to land (makes sense); the work in the releases repo to reorganize the pages is done | 17:16 |
| *** Daisy has quit IRC | 17:16 | |
| dhellmann | fungi : so I think we're ready to go when you have time to do it | 17:16 |
| clarkb | greghaynes: looking at that failure it looks like dib was just slow | 17:16 |
| greghaynes | clarkb: yep, I rechecked and left a comment | 17:17 |
| pabelanger | yolanda: fungi: I incorrectly rechecked 274694, it had a +1 workflow on it, but I think my recheck stopped the merge. Do either of you mind checking it out? | 17:17 |
| fungi | dhellmann: awesome! it may be in your best interests to prod me about it again tomorrow since today is meetingsmeetingsmeetings | 17:17 |
| greghaynes | clarkb: those gearman logs in console.log are odd | 17:17 |
| greghaynes | clarkb: but I think it is just a red herring | 17:17 |
| dhellmann | fungi : I'll schedule a reminder ;-) | 17:17 |
| fungi | pabelanger: recheck shouldn't stop anything already being tested | 17:17 |
| clarkb | though the image log cuts off before the job finishes | 17:18 |
| pabelanger | clarkb: greghaynes: looking forward to afs mirrors for yum and apt. That should help speed up somethings | 17:18 |
| clarkb | but it was going slowly before that so could be IO issues | 17:18 |
| *** dalgaaf has quit IRC | 17:18 | |
| greghaynes | clarkb: Yep, I see it died while dib was downloading packages | 17:18 |
| pabelanger | fungi: Ya, I thought that too, but seems stuck on +1 workflow | 17:18 |
| *** zul has joined #openstack-infra | 17:18 | |
| fungi | pabelanger: zuul never left a jenkins comment about enqueuing it | 17:18 |
| fungi | pabelanger: we may have something else going on. i'll check the zuul logs | 17:19 |
| pabelanger | fungi: That is right, forgot about that | 17:19 |
| chaitu | Iam trying setup fresh ci setup i could not shh to the instance created in openstack env and here is mynodepool log file http://paste.openstack.org/show/485758/ | 17:19 |
| chaitu | can some one help me out with it | 17:19 |
| *** gabor_antal_ has quit IRC | 17:20 | |
| pabelanger | chaitu: are you using the right ssh public / private keys? | 17:20 |
| zaro | fungi, madorn : i don't think removing a group can be done live. it will require downtime because the only way to remove groups is to remove it directly from db. there's no gerrit feature to do that. here the script to remove groups: https://review.openstack.org/#/c/270537/ | 17:20 |
| pabelanger | fungi: just did now | 17:20 |
| pabelanger | fungi: zuul left the message | 17:21 |
| pabelanger | Oh | 17:21 |
| fungi | zaro: oh, it's also in the db tables? not just an entry in the groups files in the meta/config ref of each repo? | 17:21 |
| pabelanger | yolanda: redid her workflow | 17:21 |
| yolanda | pabelanger, yes, i redid before fungi told that | 17:21 |
| fungi | zaro: i mean, i know in the normal case there is a database entry for created groups, just wondering about in this special case of it having the same name as a system group, whether the database entries actually got created | 17:21 |
| jeblair | krotscheck1: i think https://review.openstack.org/164927 looks good but i left comments on two things where i think the documentation is wrong -- my guess is they are harmless and left over from a previous iteration where you were generating the short name automatically rather than us generating it manually by convention. | 17:22 |
| ttx | dhellmann: https is better, but we should not block on it, since we don't have it currently | 17:22 |
| krotscheck1 | jeblair: Cool, time for a rebase! ;) | 17:22 |
| *** baoli has joined #openstack-infra | 17:22 | |
| jeblair | krotscheck1: let me know if that's correct. i think, on balance, it's probably worth one more ps to correct them. | 17:22 |
| dhellmann | ttx: that's what fungi and I agreed, so we'll add that in the near future | 17:22 |
| jeblair | krotscheck1: :| | 17:22 |
| chaitu | pabelanger: yes iam using the right keys | 17:23 |
| jeblair | git-restack is on pypi | 17:23 |
| *** reed_ has quit IRC | 17:24 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Add wheel mirror to configure_mirror.sh https://review.openstack.org/267117 | 17:24 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Added wheel-release job https://review.openstack.org/273549 | 17:24 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Create jobs for a wheel mirror https://review.openstack.org/164927 | 17:24 |
| yolanda | pabelanger, i can see gate jobs starting now | 17:25 |
| zaro | fungi: it's actually created as a normal group. | 17:25 |
| *** reed_ has joined #openstack-infra | 17:25 | |
| pabelanger | yolanda: fungi: odd how zuul never started before | 17:25 |
| *** yamahata has joined #openstack-infra | 17:25 | |
| zaro | fungi: they call it "internal group" | 17:25 |
| pabelanger | chaitu: not sure, nodepool cannot connent. So either your keys are wrong, username is wrong, or user does not exist | 17:25 |
| fungi | pabelanger: yolanda: i don't see any mention of a comment-added event for 274694,2 at 15:07 utc when the original workflow +1 was recorded | 17:25 |
| pabelanger | fungi: yolanda: network blip with gerrit? | 17:26 |
| *** salv-orlando has joined #openstack-infra | 17:26 | |
| fungi | trying to figure that out now | 17:26 |
| *** annegentle has quit IRC | 17:26 | |
| yolanda | looks as it didn't pick the event | 17:26 |
| *** gabor_antal has joined #openstack-infra | 17:26 | |
| *** mdenny has joined #openstack-infra | 17:26 | |
| *** Daisy has joined #openstack-infra | 17:26 | |
| *** maishsk has joined #openstack-infra | 17:27 | |
| openstackgerrit | Merged openstack-infra/project-config: Add IRC notification for windmill https://review.openstack.org/274694 | 17:28 |
| fungi | there is a traceback at 15:07:12 about a dependency loop, but it's between 202459 and 202440 | 17:28 |
| fungi | so unless as a side effect it dropped the rest of its buffer or something, i don't think that was related | 17:28 |
| openstackgerrit | Igor Belikov proposed openstack-infra/project-config: Add debian-jessie image to nodepool https://review.openstack.org/264726 | 17:28 |
| *** alivigni has quit IRC | 17:28 | |
| *** infra-re_ has joined #openstack-infra | 17:29 | |
| *** infra-r__ has joined #openstack-infra | 17:29 | |
| fungi | yeah, i'm not finding anything to explain the missing event | 17:30 |
| fungi | i guess it's possible gerrit never emitted one--i'll see if there's an obvious needle in its log haystack | 17:31 |
| *** infra-red has quit IRC | 17:31 | |
| *** Daisy has quit IRC | 17:31 | |
| *** reed_ has quit IRC | 17:31 | |
| *** openstackgerrit has quit IRC | 17:32 | |
| *** openstackgerrit has joined #openstack-infra | 17:32 | |
| *** infra-re_ has quit IRC | 17:33 | |
| *** Jeffrey4l has quit IRC | 17:33 | |
| jeblair | krotscheck1: so i should create mirror.wheel.trustyx64 and mount it at /afs/openstack.org/mirror/wheel/ubuntu-14.04-x86_64 correct? | 17:33 |
| *** maishsk has quit IRC | 17:34 | |
| *** dalgaaf has joined #openstack-infra | 17:34 | |
| *** Apoorva has joined #openstack-infra | 17:35 | |
| *** infra-r__ has quit IRC | 17:37 | |
| clarkb | if you scroll through a diff in new gerrit ui it expands the context automagically which is :( | 17:37 |
| *** Daisy has joined #openstack-infra | 17:37 | |
| clarkb | * scroll through with keyboard | 17:37 |
| fungi | pabelanger: yolanda: nothing useful around that time in gerrit's error log either | 17:38 |
| *** tphummel has joined #openstack-infra | 17:38 | |
| pabelanger | fungi: yolanda: I wonder of the +2 approve followed up with the +1 workflow on the same user might have something to do with it. | 17:39 |
| krotscheck1 | jeblair: Yes | 17:39 |
| pabelanger | either way, I cannot think of a reason for the event to be missing | 17:39 |
| pabelanger | never seen that before | 17:39 |
| igorbelikov | pabelanger: hi, regarding https://review.openstack.org/#/c/264726/ - debian-minimal is still busted in latest diskimage-builder release, so Is it ok to use non-minimal image for now? My patch to fix it was merged to master | 17:40 |
| *** annegentle has joined #openstack-infra | 17:40 | |
| fungi | pabelanger: that pattern has worked fine in the past | 17:41 |
| *** Daisy has quit IRC | 17:41 | |
| *** unicell has quit IRC | 17:41 | |
| openstackgerrit | Igor Belikov proposed openstack-infra/project-config: Add debian-jessie image to nodepool https://review.openstack.org/264726 | 17:42 |
| clarkb | greghaynes: commented on https://review.openstack.org/#/c/272384/7 but I think it can be addressed in afollow up | 17:42 |
| clarkb | greghaynes: and the existing child fails pep8 and tests | 17:42 |
| greghaynes | clarkb: Yep, I have a tox going right now... | 17:43 |
| jeblair | clarkb, fungi: can you take a sec to look at this release announcement for git-restack? https://etherpad.openstack.org/p/iriJVixL9z | 17:43 |
| pabelanger | igorbelikov: Ya, I think we want -minimal for all images, mordred will know more. Keep in mind, it took fedora 23 a few months to land because of a series of issue around -minimal. So, you'll likely need to get that working correctly before jessie can land. We already do ubuntu-minimal, so it shouldn't be to much work for debian-minimal | 17:45 |
| *** SumitNaiksatam has quit IRC | 17:46 | |
| pabelanger | igorbelikov: if you fixed it in diskimage-builder, you can add a Depend-on header to the commit. We run from git I believe with diskimage-builder | 17:46 |
| clarkb | jeblair: seems fine | 17:47 |
| clarkb | explains what it is used for and how i tworks | 17:47 |
| igorbelikov | pabelanger: ok, I'll update it with minimal in a couple of minutes, thanks | 17:47 |
| *** esikachev has joined #openstack-infra | 17:47 | |
| *** sdake has joined #openstack-infra | 17:48 | |
| pabelanger | igorbelikov: nice. Like I said, do you mind posting the output online some place. Won't mind looking it over see what is different between fedora and debian. | 17:48 |
| zaro | fungi, madorn : also the upstream gerrit fix has not been merged yet, i just asked DavidP to take a another look. https://gerrit-review.googlesource.com/#/c/74312/ | 17:48 |
| igorbelikov | pabelanger: yeah, will do. I'll start building minimal one now, will update request with link to the logs | 17:49 |
| fungi | jeblair: lgtm | 17:49 |
| *** smarcet has joined #openstack-infra | 17:49 | |
| pabelanger | igorbelikov: thanks | 17:50 |
| smarcet | fungi: afternoon | 17:50 |
| fungi | jeblair: a great explanation of why people should want to use it | 17:50 |
| smarcet | fungi: there is any way to fix an already commited tag message on infra github ? | 17:50 |
| fungi | smarcet: hey there. jimmy mentioned earlier that you may need help with something time-sensitive today | 17:50 |
| *** sdake has quit IRC | 17:50 | |
| *** salv-orl_ has joined #openstack-infra | 17:50 | |
| smarcet | fungi: i tried but it reject it | 17:50 |
| fungi | smarcet: the message is part of the tag. for a new message you need a new tag | 17:50 |
| smarcet | oh ok , well not big deal | 17:51 |
| smarcet | yes we have a new version of openstackid | 17:51 |
| *** krotscheck1 has quit IRC | 17:51 | |
| smarcet | ready to go to production (Openid connect) | 17:51 |
| *** hdd has joined #openstack-infra | 17:51 | |
| clarkb | mordred: /etc/openstack/clouds.yaml is 0600 root, what do you think about opening that up to a common infra-root group? | 17:52 |
| fungi | basically git combines the tag name, tag message and the object reference being tagged, then creates a hash of that data and the result is what you cryptographically sign | 17:52 |
| clarkb | mgagne: sorry that is on puppetmaster | 17:52 |
| smarcet | fungi: ok got it, thx for info :) | 17:52 |
| clarkb | gah mordred ^ mgagne I apologize and you may ignore | 17:52 |
| smarcet | fungi: basically to deploy openstackid to production, from what i understand, i need to update tag vesion here modules/openstack_project/manifests/openstackid_prod.pp | 17:53 |
| smarcet | that would be enough to fire the deployment process | 17:53 |
| *** rhallisey has joined #openstack-infra | 17:53 | |
| *** infra-red has joined #openstack-infra | 17:53 | |
| *** salv-orlando has quit IRC | 17:54 | |
| fungi | smarcet: yes, i believe you change it to $release = '1.0.7', | 17:54 |
| fungi | er | 17:54 |
| fungi | stray newline there | 17:54 |
| *** sfinucan has quit IRC | 17:54 | |
| smarcet | fungi : cool | 17:54 |
| fungi | change it to $release = 'your_new_tag', | 17:54 |
| jeblair | 11 new messages arrived on openstack-dev in the time it took me to write that one | 17:55 |
| clarkb | mordred: also `venv/bin/openstack --help` fails if it can't read the clouds.yaml | 17:55 |
| clarkb | I will file a bug for that | 17:55 |
| smarcet | fungi: roger that for info :) | 17:55 |
| *** zul has quit IRC | 17:56 | |
| *** infra-red has quit IRC | 17:58 | |
| clarkb | https://bugs.launchpad.net/python-openstackclient/+bug/1541047 has been filed | 17:58 |
| openstack | Launchpad bug 1541047 in python-openstackclient "openstack --help fails if clouds.yaml cannot be read" [Undecided,New] | 17:58 |
| clarkb | greghaynes: you making a dib release today? | 18:00 |
| greghaynes | clarkb: yes! ty for reminding me | 18:00 |
| greghaynes | clarkb: after this phone call... | 18:00 |
| *** yamahata has quit IRC | 18:00 | |
| jeblair | krotscheck1's irc bouncer situtation is not optimal | 18:02 |
| *** rossella_s has quit IRC | 18:02 | |
| jeblair | fungi, greghaynes: any idea how much space a single-arch wheel mirror built from our requirements ought to be? | 18:03 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 18:03 | |
| *** sridhar_ram1 has joined #openstack-infra | 18:03 | |
| *** sridhar_ram has quit IRC | 18:03 | |
| *** simondodsley has joined #openstack-infra | 18:03 | |
| *** rossella_s has joined #openstack-infra | 18:04 | |
| *** diana_clarke has quit IRC | 18:04 | |
| *** annegentle has quit IRC | 18:05 | |
| *** rhallisey has quit IRC | 18:05 | |
| fungi | i really don't recall what it used to look like size wise. i wonder if we can recheck a requirements change, hold the node and then look at the resulting wheel cache | 18:06 |
| *** tzn has quit IRC | 18:06 | |
| jeblair | meh, i'll just set it to 20G for now and we can watch the wheel build job | 18:06 |
| clarkb | fungi: that would be very close yes (remember that we leave old wheels in place so it will grow over time) | 18:06 |
| clarkb | 20GB should be plenty to start | 18:06 |
| fungi | good point. we definitely need room to grow | 18:06 |
| *** rfolco has quit IRC | 18:07 | |
| *** yaume_ has quit IRC | 18:07 | |
| *** davideagnello has joined #openstack-infra | 18:09 | |
| *** jsavak has quit IRC | 18:09 | |
| *** yamahata has joined #openstack-infra | 18:09 | |
| *** jtomasek__ has quit IRC | 18:10 | |
| *** jsavak has joined #openstack-infra | 18:10 | |
| *** rbrndt has quit IRC | 18:11 | |
| *** anteaya has quit IRC | 18:11 | |
| *** ybathia has joined #openstack-infra | 18:11 | |
| *** anteaya has joined #openstack-infra | 18:12 | |
| *** jordanP has quit IRC | 18:12 | |
| jeblair | /afs/openstack.org/mirror/wheel/ubuntu-14.04-x86_64 exists | 18:13 |
| jeblair | and is replicated on both fileservers :) | 18:13 |
| *** Swami has quit IRC | 18:14 | |
| * fungi cheers | 18:14 | |
| *** annegentle has joined #openstack-infra | 18:14 | |
| *** gibi has quit IRC | 18:14 | |
| *** kzaitsev_mb has quit IRC | 18:15 | |
| *** annegentle has quit IRC | 18:15 | |
| *** rfolco has joined #openstack-infra | 18:15 | |
| *** ianw has joined #openstack-infra | 18:15 | |
| *** gibi has joined #openstack-infra | 18:15 | |
| *** pvaneck has joined #openstack-infra | 18:16 | |
| *** jcoufal has quit IRC | 18:16 | |
| *** unicell has joined #openstack-infra | 18:16 | |
| *** SumitNaiksatam has joined #openstack-infra | 18:17 | |
| openstackgerrit | Shamail Tahir proposed openstack-infra/system-config: Adding meetbot and statusbot support for openstack-product. https://review.openstack.org/275364 | 18:18 |
| openstackgerrit | Merged openstack-infra/system-config: Temporarily stop releasing after bandersnatch https://review.openstack.org/275299 | 18:18 |
| openstackgerrit | Shamail Tahir proposed openstack-infra/project-config: Adding gerritbot and accessbot support for openstack-product. https://review.openstack.org/275365 | 18:19 |
| *** achanda has joined #openstack-infra | 18:19 | |
| anteaya | the meeting topic in -alt needs a refresh if anyone has a moment | 18:19 |
| *** alivigni has joined #openstack-infra | 18:20 | |
| *** annegentle has joined #openstack-infra | 18:20 | |
| anteaya | now that I have the ability to work again, I have some backscroll to read | 18:20 |
| *** dimtruck is now known as zz_dimtruck | 18:21 | |
| *** yamahata has quit IRC | 18:22 | |
| fungi | anteaya: thanks--done! | 18:22 |
| anteaya | thank you | 18:22 |
| zaro | anteaya: trying to setup now | 18:24 |
| *** jsavak has quit IRC | 18:25 | |
| prometheanfire | greghaynes: hi :P | 18:25 |
| *** baoli has quit IRC | 18:25 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/git-restack: Remove argparse from requirements https://review.openstack.org/270411 | 18:26 |
| pabelanger | jeblair: fungi: is there any reason not to move pep8 jobs from bare-trusty to ubuntu-trusty images? | 18:26 |
| pabelanger | or are we waiting until bare-precise is removed | 18:27 |
| fungi | pabelanger: mostly waiting until dib uploads to rackspace are stable | 18:27 |
| anteaya | zaro: thanks, share your feedback in a pm if you would | 18:27 |
| anteaya | as this is internal | 18:27 |
| *** salv-orl_ has quit IRC | 18:28 | |
| *** salv-orlando has joined #openstack-infra | 18:28 | |
| fungi | pabelanger: but also, the jobs need to use a different job template to call the install-distro-packages builder first, which also implies reordering the other builders since we need to checkout the change, then install system packages, then revoke sudo, then invoke tox | 18:29 |
| *** vivekd has quit IRC | 18:29 | |
| clarkb | fungi: we can in theory do that before the switch | 18:29 |
| *** kzaitsev_mb has joined #openstack-infra | 18:29 | |
| fungi | yep | 18:29 |
| clarkb | not sure if the extra effort is worth it | 18:29 |
| fungi | i have an example in the experimental-jobs.yaml of doing that for nova unit tests | 18:29 |
| anteaya | clarkb: as you your comment from last night, yes they did shut down the cloud, as you surmised | 18:29 |
| *** gokrokve has quit IRC | 18:29 | |
| *** annegentle has quit IRC | 18:30 | |
| fungi | er, experimental-workers.yaml | 18:31 |
| *** angdraug has quit IRC | 18:31 | |
| *** salv-orlando has quit IRC | 18:31 | |
| *** esikachev has quit IRC | 18:31 | |
| kzaitsev_mb | folks, who should I ask to remove stable/juno from murano repositories? =) | 18:32 |
| *** salv-orlando has joined #openstack-infra | 18:32 | |
| pabelanger | fungi: okay, let me take a look into that | 18:32 |
| pabelanger | fungi: curious what is all needed to do the switch | 18:32 |
| *** ilyashakhat has joined #openstack-infra | 18:32 | |
| kzaitsev_mb | I wrote a letter to openstack-infra last week, but got now response =) | 18:32 |
| anteaya | kzaitsev_mb: is is timing | 18:32 |
| *** Swami has joined #openstack-infra | 18:33 | |
| anteaya | kzaitsev_mb: it needs to be an infra-root and last week there weren't many around | 18:33 |
| fungi | pabelanger: also we should switch the bindep-fallback-ubuntu-trusty job to voting and add similar jobs for any other worker types where we may want to use the same job templates | 18:33 |
| fungi | pabelanger: that will make sure we don't break the world by merging a broken fallback package list | 18:34 |
| anteaya | kzaitsev_mb: this week there are more but they are catching up from what they missed last week, thanks for your patience | 18:34 |
| *** DuaneDeC7 has quit IRC | 18:34 | |
| *** amitgandhinz has quit IRC | 18:34 | |
| pabelanger | fungi: Ya, I need to see what that job does. Haven't been following bindep much | 18:34 |
| kzaitsev_mb | anteaya: thanks for update =) there's obviously no rush from our side of things =) | 18:34 |
| *** ilyashakhat has quit IRC | 18:34 | |
| fungi | kzaitsev_mb: i saw your e-mail and have put it into my tentative to-do pile, in case nobody else gets to it before i am freed up | 18:34 |
| kzaitsev_mb | fungi: thanks = ) | 18:35 |
| *** esikachev has joined #openstack-infra | 18:35 | |
| *** ilyashakhat has joined #openstack-infra | 18:35 | |
| fungi | pabelanger: it basically exercises the same script which jobs would use to invoke bindep, but overrides it to test proposed changes to our fallback list or proposed changes to bindep accordingly | 18:35 |
| *** amitgandhinz has joined #openstack-infra | 18:35 | |
| fungi | pabelanger: that way we get it verified with the tool which would consume it | 18:36 |
| kzaitsev_mb | just wanted to be sure, that I, say wrote to the correct ML and didn't mix things up ) | 18:36 |
| *** annegentle has joined #openstack-infra | 18:36 | |
| kzaitsev_mb | thanks again for info | 18:36 |
| *** gyee has joined #openstack-infra | 18:37 | |
| *** DuaneDeC7 has joined #openstack-infra | 18:37 | |
| fungi | pabelanger: and covers corner cases such that we can't merge a fallback list which the installed bindep on our workers is unable to use, nor change bindep such that it can't work with the current state of the fallback list on our workers | 18:37 |
| fungi | belt meet suspenders | 18:37 |
| timrc | zaro, Khai long time not speak. Are you around for a quick chat about jjb? | 18:37 |
| *** exploreshaifali has joined #openstack-infra | 18:38 | |
| *** trown is now known as trown|lunch | 18:38 | |
| *** zz_dimtruck is now known as dimtruck | 18:38 | |
| fungi | pabelanger: no doubt there are other corner cases we'll discover as we start relying on this more and more, but i at least tried to plan for the obvious ones | 18:38 |
| *** shashank_hegde has joined #openstack-infra | 18:38 | |
| fungi | pabelanger: i've really not had the available time i've wanted to spend on it, so welcome any assistance you end up being able to render | 18:40 |
| prometheanfire | think I scared him off | 18:40 |
| fungi | seems from his earlier messages that he's on a call | 18:40 |
| *** infra-red has joined #openstack-infra | 18:40 | |
| *** ilyashakhat has quit IRC | 18:41 | |
| *** jsavak has joined #openstack-infra | 18:41 | |
| *** ilyashakhat has joined #openstack-infra | 18:42 | |
| pabelanger | fungi: Sure, don't mind diving into it. Let me setup some time to look into it and see what is needed to move it forward | 18:42 |
| *** salv-orlando has quit IRC | 18:43 | |
| fungi | pabelanger: my hero! | 18:44 |
| *** esikachev has quit IRC | 18:44 | |
| *** kzaitsev_mb has quit IRC | 18:44 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Switch ansible-role-diskimage-builder jobs voting https://review.openstack.org/275373 | 18:45 |
| *** zul has joined #openstack-infra | 18:45 | |
| *** salv-orlando has joined #openstack-infra | 18:45 | |
| *** jlanoux has quit IRC | 18:46 | |
| *** tzn has joined #openstack-infra | 18:47 | |
| AJaeger | dhellmann: could you review https://review.openstack.org/#/c/243295/, please? that's the solum releasenotes addition... | 18:47 |
| dhellmann | AJaeger : looking | 18:48 |
| *** tzn has quit IRC | 18:48 | |
| *** annegentle has quit IRC | 18:49 | |
| *** angdraug has joined #openstack-infra | 18:50 | |
| *** annegentle has joined #openstack-infra | 18:53 | |
| pabelanger | fungi: Ya, so I'd be curios which job would be a good place to gate on bindep. We have an experimental job on project-config, maybe start by moving that to non-voting | 18:53 |
| pabelanger | fungi: I should also see what is needed to run this on centos / fedora | 18:54 |
| *** changbl has quit IRC | 18:54 | |
| AJaeger | johnthetubaguy, sdague : could you review https://review.openstack.org/#/c/273326 , please? This adds a voting ceph gate job for nova. | 18:55 |
| fungi | pabelanger: plan was to adjust that bindep-fallback project-template so that it was running in check and gate instead of experimental. i've already tested it on multiple changes for both project-config and bindep so it should be stable | 18:55 |
| *** Sukhdev has joined #openstack-infra | 18:55 | |
| pabelanger | fungi: okay, I can start on that | 18:55 |
| *** changbl has joined #openstack-infra | 18:55 | |
| fungi | pabelanger: so i guess move the ubuntu-trusty version of the job to check and gate, add centos-7 and fedora-23 (or whatever the dib node labels are for those) to experimental | 18:55 |
| pabelanger | fungi: right | 18:56 |
| anteaya | AJaeger: did you want to address the comment on https://review.openstack.org/#/c/274359 | 18:56 |
| fungi | pabelanger: and we can switch those to check and gate too once they're proven on a few changes in experimental | 18:56 |
| AJaeger | anteaya: I'll do some more cleanups on that file soon and can do it at that time... | 18:57 |
| AJaeger | anteaya: thanks for asking | 18:57 |
| fungi | pabelanger: though maybe only if we're actually consistently building working images for those labels--don't want to block development on project-config and bindep if we have trouble satisfying demand for check and gate jobs with those platforms | 18:57 |
| zaro | timrc: i can try to answer your question. but tbh, i've been out of touch lately. | 18:57 |
| AJaeger | anteaya: if that's fine with you | 18:57 |
| fungi | pabelanger: since that can leave us in a catch-22 where we need to intervene manually | 18:57 |
| zaro | anteaya: i sent you an email with feeback. | 18:57 |
| anteaya | AJaeger: that's fine with me | 18:58 |
| anteaya | zaro: thank you | 18:58 |
| pabelanger | fungi: okay | 18:58 |
| SpamapS | jeblair: ^5 for restack! | 18:58 |
| timrc | zaro: Who's in touch these days? I'm interested in the future of jjb as openstack moves away. Has anyone expressed interested in maintaining it? | 18:58 |
| *** kdas_ has joined #openstack-infra | 18:59 | |
| jeblair | SpamapS: v5 down low! | 18:59 |
| timrc | zaro: It seems quite a few shops now use it. Blue Box being one fo them. We're not going to move away from Jenkins at the same time as Openstack, so I'd like to know that patches are still going to get reviewed / merged | 18:59 |
| jeblair | did that work? i've never tried that on irc | 18:59 |
| *** tiswanso has quit IRC | 18:59 | |
| openstackgerrit | Merged openstack-infra/project-config: Update App Catalog dead link checker https://review.openstack.org/269935 | 18:59 |
| prometheanfire | SpamapS: hi :P | 18:59 |
| fungi | meeting time! | 19:00 |
| fungi | -> #openstack-meeting | 19:00 |
| *** electrofelix has quit IRC | 19:00 | |
| *** tiswanso has joined #openstack-infra | 19:01 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add centos7 and fedora-23 to bindep-fallback https://review.openstack.org/275376 | 19:01 |
| *** DuaneDeC7 has quit IRC | 19:01 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add wheel mirror to configure_mirror.sh https://review.openstack.org/267117 | 19:02 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Added wheel-release job https://review.openstack.org/273549 | 19:02 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Create jobs for a wheel mirror https://review.openstack.org/164927 | 19:02 |
| greghaynes | clarkb: ianw dib released | 19:02 |
| *** jsavak has quit IRC | 19:02 | |
| openstackgerrit | Merged openstack-infra/project-config: Ignore empty django files https://review.openstack.org/275090 | 19:02 |
| *** jsavak has joined #openstack-infra | 19:03 | |
| *** rbrndt has joined #openstack-infra | 19:03 | |
| openstackgerrit | Shamail Tahir proposed openstack-infra/system-config: Adding meetbot and statusbot support for openstack-product. The openstack-product IRC channel will be used by the OpenStack Product Working Group for additional conversations when the meetings take longer than our allocated time on openstack-meeting-alt. https://review.openstack.org/275364 | 19:03 |
| *** weshay_xchat is now known as weshay | 19:03 | |
| zaro | timrc: get in touch with waynr, zxiiro and electrofelix. | 19:03 |
| *** DuaneDeC7 has joined #openstack-infra | 19:04 | |
| *** matrohon has joined #openstack-infra | 19:04 | |
| clarkb | timrc: zaro I don't think we had any intention of killing the project. It has users that maintain it and so on | 19:04 |
| zaro | timrc: i know waynr is doing a lot of work with new features for jjb. | 19:04 |
| *** sridhar_ram1 has quit IRC | 19:04 | |
| *** Swami_ has joined #openstack-infra | 19:04 | |
| zxiiro | timrc: My organization will depend on JJB for a long time and we hope to keep it maintained even if OpenStack stops using it. | 19:05 |
| *** esker has joined #openstack-infra | 19:05 | |
| *** crinkle_ has joined #openstack-infra | 19:05 | |
| *** asselin_ has joined #openstack-infra | 19:05 | |
| *** xwizard__ has joined #openstack-infra | 19:06 | |
| *** crinkle has quit IRC | 19:06 | |
| *** crinkle_ is now known as crinkle | 19:07 | |
| *** YorikSar_ has joined #openstack-infra | 19:07 | |
| *** jamespd_ has joined #openstack-infra | 19:07 | |
| *** mrmartin has joined #openstack-infra | 19:07 | |
| *** sridhar_ram has joined #openstack-infra | 19:07 | |
| timrc | So let's just take https://review.openstack.org/#/c/266970/ for an example... | 19:07 |
| timrc | Where are the cores at? How do we get their attention? | 19:07 |
| mgagne | when your coworker compares cloud-init to django... | 19:08 |
| openstackgerrit | Shamail Tahir proposed openstack-infra/system-config: Adding meetbot and statusbot support for openstack-product https://review.openstack.org/275364 | 19:09 |
| zaro | timrc: other projects have consumed my attention these days :( | 19:10 |
| *** flepied has joined #openstack-infra | 19:10 | |
| zxiiro | timrc: I feel you, folks are busy and reviews aren't quick. It's something I've complained on several occasions myself. Now that I'm a core I try to keep up with reviews but the day job unfortunately takes priority so I can only review a few patches each week. | 19:10 |
| timrc | So... I'll put something on next Tuesday's meeting. | 19:10 |
| *** luv_ has joined #openstack-infra | 19:10 | |
| *** jpr has quit IRC | 19:10 | |
| timrc | We need to find people that can actively review and merge stuff to jjb. | 19:10 |
| *** sbadia_ has joined #openstack-infra | 19:10 | |
| *** f1ller- has joined #openstack-infra | 19:10 | |
| *** jpr has joined #openstack-infra | 19:10 | |
| *** itsuugo_ has joined #openstack-infra | 19:10 | |
| *** lbragstad_ has joined #openstack-infra | 19:11 | |
| timrc | Now, I must BBQ in... Seattle | 19:11 |
| *** bknudson_ has joined #openstack-infra | 19:11 | |
| *** lucasagomes_ has joined #openstack-infra | 19:11 | |
| *** mari0jv has joined #openstack-infra | 19:11 | |
| openstackgerrit | Shamail Tahir proposed openstack-infra/project-config: Adding gerritbot and accessbot support for openstack-product https://review.openstack.org/275365 | 19:11 |
| *** htruta` has joined #openstack-infra | 19:11 | |
| *** pabelanger_ has joined #openstack-infra | 19:11 | |
| *** coreycb` has joined #openstack-infra | 19:11 | |
| zxiiro | timrc: for what it's worth I'm mentoring a student this upcoming summer to work on JJB. Hoping he can help with some of the reviews. | 19:11 |
| jeblair | timrc: thank you for volunteering | 19:11 |
| *** charz has joined #openstack-infra | 19:11 | |
| *** mrodden_ has joined #openstack-infra | 19:11 | |
| *** Swami has quit IRC | 19:12 | |
| *** kushal has quit IRC | 19:12 | |
| *** Kennan has quit IRC | 19:12 | |
| *** luv has quit IRC | 19:12 | |
| *** hogepodge has quit IRC | 19:12 | |
| *** lucasagomes has quit IRC | 19:12 | |
| *** pcrews has quit IRC | 19:12 | |
| *** waynr has quit IRC | 19:12 | |
| *** xwizard_ has quit IRC | 19:12 | |
| *** pabelanger has quit IRC | 19:12 | |
| *** asselin has quit IRC | 19:12 | |
| *** akuznetsova has quit IRC | 19:12 | |
| *** katyafervent_awa has quit IRC | 19:12 | |
| *** mordred has quit IRC | 19:12 | |
| *** bknudson has quit IRC | 19:12 | |
| *** mrunge has quit IRC | 19:12 | |
| *** sbadia has quit IRC | 19:12 | |
| *** lindsayk- has quit IRC | 19:12 | |
| *** pavel_bondar has quit IRC | 19:12 | |
| *** htruta has quit IRC | 19:12 | |
| *** itsuugo has quit IRC | 19:12 | |
| *** coreycb has quit IRC | 19:12 | |
| *** f1ller has quit IRC | 19:12 | |
| *** jamespd has quit IRC | 19:12 | |
| *** YorikSar has quit IRC | 19:12 | |
| *** pkarikh has quit IRC | 19:12 | |
| *** mariojv has quit IRC | 19:12 | |
| *** charz_ has quit IRC | 19:12 | |
| *** michaelxin has quit IRC | 19:12 | |
| *** thingee has quit IRC | 19:12 | |
| *** lbragstad has quit IRC | 19:12 | |
| *** mrodden has quit IRC | 19:12 | |
| *** krotscheck1 has joined #openstack-infra | 19:12 | |
| *** lbragstad_ is now known as lbragstad | 19:12 | |
| AJaeger | it's 502 gerrit time for me ;( | 19:12 |
| *** sbadia_ is now known as sbadia | 19:12 | |
| *** mrodden_ is now known as mrodden | 19:12 | |
| *** luv_ is now known as luv | 19:12 | |
| *** pabelanger_ is now known as pabelanger | 19:12 | |
| *** itsuugo_ is now known as itsuugo | 19:12 | |
| *** maishsk has joined #openstack-infra | 19:12 | |
| *** matrohon has quit IRC | 19:13 | |
| *** mrunge has joined #openstack-infra | 19:13 | |
| openstackgerrit | Shamail Tahir proposed openstack-infra/system-config: Adding meetbot and statusbot support for openstack-product https://review.openstack.org/275364 | 19:13 |
| *** pkarikh has joined #openstack-infra | 19:13 | |
| *** katyafervent_awa has joined #openstack-infra | 19:14 | |
| *** htruta` is now known as htruta | 19:14 | |
| *** akuznetsova has joined #openstack-infra | 19:14 | |
| *** mordred has joined #openstack-infra | 19:15 | |
| *** pavel_bondar has joined #openstack-infra | 19:15 | |
| *** waynr has joined #openstack-infra | 19:16 | |
| *** pcrews has joined #openstack-infra | 19:16 | |
| *** Kennan has joined #openstack-infra | 19:16 | |
| *** boris-42 has joined #openstack-infra | 19:19 | |
| *** esikachev has joined #openstack-infra | 19:19 | |
| anteaya | AJaeger: :( time for a break? | 19:19 |
| AJaeger | anteaya: just not multitasking ;) | 19:20 |
| AJaeger | reloads work... | 19:20 |
| anteaya | ah | 19:20 |
| AJaeger | jeblair, krotscheck1 : I have a question on https://review.openstack.org/#/c/164927 - uname -m gives i686 for me | 19:21 |
| *** kdas_ has quit IRC | 19:21 | |
| krotscheck1 | AJaeger: Ubuntu? | 19:21 |
| *** dizquierdo has quit IRC | 19:21 | |
| AJaeger | krotscheck1: openSUSE | 19:22 |
| AJaeger | uname -i gives i386 | 19:22 |
| AJaeger | krotscheck1: What OS are you on? | 19:22 |
| krotscheck1 | AJaeger: Hrm. I'm on an ubuntu trusty cloud node, and I have x86_64 | 19:22 |
| AJaeger | krotscheck1: try "linux32 uname -i" | 19:22 |
| clarkb | AJaeger: krotscheck1 its going to be install dependent | 19:23 |
| clarkb | if you installed i686 you will get that | 19:23 |
| AJaeger | linux32 gives you a 32-bit environment... | 19:23 |
| krotscheck1 | AJaeger: i686 | 19:23 |
| AJaeger | for -i? Oh... | 19:23 |
| *** hogepodge has joined #openstack-infra | 19:24 | |
| ianw | greghaynes: cool , was it just then? assuming latest builds missed it | 19:24 |
| *** exploreshaifali has quit IRC | 19:24 | |
| *** kushal has joined #openstack-infra | 19:24 | |
| AJaeger | krotscheck1: if we always run it on same environment it won't matter ;) But code would be more robust if we check for i?86 and replace with whatever you want;) | 19:25 |
| krotscheck1 | AJaeger: I'm not too worried about it in that particular case. The wheel slave will build wheels for system/version/architecture, and a slave will only resolve from its appropriate system/version/architecture. | 19:25 |
| *** kushal has quit IRC | 19:25 | |
| clarkb | ianw: I can trigger fedora 23 after meeting | 19:25 |
| *** salv-orlando has quit IRC | 19:25 | |
| *** thingee has joined #openstack-infra | 19:26 | |
| krotscheck1 | AJaeger: If the slave doesn't match exactly, it won't get a mirror. | 19:26 |
| AJaeger | krotscheck1: ah, ok. Then let's try it... | 19:26 |
| * AJaeger will +2 but let jeblair hit the +A | 19:26 | |
| ianw | clarkb: that would be good, thanks. i have nothing else outstanding -- the latest dib should have growroot support for f23 so we should have enough disk-space to run a devstack+tempest | 19:26 |
| krotscheck1 | jeblair: Well, let's let the jobs pass first ;) | 19:27 |
| clarkb | ianw: reviews on https://review.openstack.org/#/c/272384/7 would also help reliability of image builds/uploads | 19:27 |
| *** esp_ has joined #openstack-infra | 19:27 | |
| mrmartin | hi | 19:28 |
| *** jesusaurus has joined #openstack-infra | 19:28 | |
| *** infra-red has quit IRC | 19:29 | |
| *** NobodyCa1 has joined #openstack-infra | 19:30 | |
| mrmartin | someone could help me today to make a backup form ask.o.o's database because I want to fix a bug on ask-staging.o.o and I need to make sure that the bug was caused by an upstream repo branch issue, so it is not an app error. | 19:30 |
| *** deva_ has joined #openstack-infra | 19:30 | |
| krotscheck1 | I meant AJaeger ^^ | 19:31 |
| *** dmsimard has joined #openstack-infra | 19:32 | |
| * dansmith throws something rotten at gerrit | 19:32 | |
| AJaeger | Ah, all dansmith's fault now ;) | 19:33 |
| timrc | zxiiro: Hey good to know you're a core. That's excellent | 19:33 |
| dansmith | AJaeger: I'm throwing in response to the issues, not in order to cause them :P | 19:33 |
| *** trown|lunch is now known as trown | 19:33 | |
| *** esp_ has quit IRC | 19:34 | |
| zxiiro | timrc: yep. I've been a bit slower on reviews lately because my day job is in the middle of a big release cycle in 3 weeks so all my focus unfortunately's been with that. I'll be back to reviewing as much as I can once we're done with that though. | 19:35 |
| *** NobodyCa1 has quit IRC | 19:35 | |
| mrmartin | AJaeger: are you a core? | 19:35 |
| *** jesusaurus has quit IRC | 19:35 | |
| *** deva_ has quit IRC | 19:36 | |
| *** vgridnev has joined #openstack-infra | 19:36 | |
| AJaeger | mrmartin: which repo? You know that you can find the cores yourself? | 19:36 |
| clarkb | ianw: looks like paths may have changed sufficiently to make the per image log files not work properly. Liekyl something I missed in splitting the logging configs | 19:36 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config: Use diskimage-builder from git for nodepool https://review.openstack.org/275006 | 19:36 |
| mrmartin | AJaeger, I just like to get a backup from ask.o.o's db and move a copy to ask-staging.o.o | 19:37 |
| *** dmsimard has quit IRC | 19:37 | |
| AJaeger | mrmartin: that needs infra-root AFAIK - and I'm not one | 19:37 |
| ianw | pabelanger: ^ so another question might be, why did that *not* fail all the other apply tests... | 19:37 |
| mrmartin | oh ok. | 19:37 |
| *** dtardivel has quit IRC | 19:38 | |
| ianw | clarkb: ok, i haven't checked today. i can take a look in a bit | 19:38 |
| AJaeger | mrmartin: clarkb, fungi, jeblair are around and can help. | 19:38 |
| pabelanger | ianw: good question, I'd have to look | 19:38 |
| AJaeger | mrmartin: but right now is infra meeting... | 19:38 |
| mrmartin | I know. | 19:38 |
| timrc | zxiiro: Hey I probably came off as a bit rude, so my apologies there. I completely understand trying to balance responsibilities. | 19:38 |
| zxiiro | timrc: not at all. I didn't take any offense whatsoever :) | 19:39 |
| zxiiro | timrc: I understand where your coming from, I wish JJB reviews would be quicker myself too. | 19:40 |
| SpamapS | Shrews: hey, can you check out https://review.openstack.org/274360 and verify for me that we are, in fact, going to remove munch some day? | 19:40 |
| *** dmsimard has joined #openstack-infra | 19:41 | |
| *** dmsimard has quit IRC | 19:41 | |
| pabelanger | ianw: so, puppet-apply did catch the problem it seems however jenkins_dev is only run on trusty. I think you want to add the change for 275006 some other place | 19:42 |
| *** sridhar_ram1 has joined #openstack-infra | 19:43 | |
| *** maishsk has quit IRC | 19:43 | |
| krotscheck1 | jeblair: Shifting into daycare mode. Having some trouble with the project-config builds, something's not syntaxing right. | 19:43 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add centos7 and fedora-23 to bindep-fallback https://review.openstack.org/275376 | 19:43 |
| krotscheck1 | KeyError: 'name' | 19:43 |
| krotscheck1 | I'll look at it after 3ish if possible. | 19:43 |
| *** krotscheck1 has quit IRC | 19:44 | |
| *** _nadya_ has joined #openstack-infra | 19:44 | |
| *** sridhar_ram has quit IRC | 19:44 | |
| *** maishsk has joined #openstack-infra | 19:44 | |
| pabelanger | ianw: I think you need to update puppet-openstackci | 19:44 |
| timrc | zxiiro: I'll start reviewing more in this area if it helps any. Might be a good way of priotizing what to look at for +2 if there are multiple +1's. | 19:45 |
| *** dmsimard has joined #openstack-infra | 19:46 | |
| AJaeger | krotscheck1, jeblair: I'm looking at the wheel syntax now.. | 19:47 |
| openstackgerrit | Igor Belikov proposed openstack-infra/puppet-jenkins: Use 'ruby' instead of 'ruby1.9.1' https://review.openstack.org/275329 | 19:48 |
| timrc | clarkb: Also, I went with the "crowd" to Pike's BBQ... really, really bad in terms of brisket. I had to know though. | 19:48 |
| *** zxiiro_ has joined #openstack-infra | 19:49 | |
| clarkb | timrc: I have never been | 19:49 |
| clarkb | but will likely make my own this weekend for sportsball | 19:49 |
| timrc | clarkb: You are not missing anything unless you like disappointment. | 19:49 |
| Clint | and who doesn't like disappointment | 19:49 |
| timrc | I dunno.. alderwood. I feel sorry for people here that can't get post oak. | 19:49 |
| clarkb | I have done hole in the wall and their brisket sandwich is decent | 19:49 |
| clarkb | timrc: apple | 19:49 |
| *** simondodsley_ has joined #openstack-infra | 19:50 | |
| timrc | This brisket sandwhich tasted like sloppy joes. It was... disconcerting. | 19:50 |
| bkero | Can we just get some better bbq in Austin? | 19:50 |
| *** dkranz has quit IRC | 19:50 | |
| prometheanfire | bkero: come down to san antonio if you want real bbq | 19:51 |
| *** flepied has quit IRC | 19:51 | |
| *** zxiiro has quit IRC | 19:51 | |
| *** simondodsley has quit IRC | 19:51 | |
| clarkb | eh I just make it myself because then I don't have to go to texas :) | 19:51 |
| timrc | bkero: If you expanded into Central Texas you have many more options :) If you want very good bbq run by !hipster Cooper's in Llano. Worth the drive. | 19:51 |
| *** simondodsley_ is now known as simondodsley | 19:51 | |
| bkero | Whatever, that's a short drive | 19:51 |
| bkero | (San Antonio) | 19:51 |
| *** esker has quit IRC | 19:51 | |
| *** zxiiro_ is now known as zxiiro | 19:51 | |
| prometheanfire | yep | 19:51 |
| bkero | timrc: Sadly I think afterwards I might be headed easterly instead. | 19:51 |
| prometheanfire | still don't understand why we are having a summit in austin | 19:51 |
| prometheanfire | it's not gonna be a fun time | 19:51 |
| bkero | That's where it was the first time? | 19:52 |
| SpamapS | prometheanfire: oh hi, you said hi. How's the gentoo stuff going? | 19:52 |
| *** michaelxin has joined #openstack-infra | 19:52 | |
| prometheanfire | SpamapS: need a couple of +2/+w | 19:52 |
| prometheanfire | since gate is passing again | 19:52 |
| prometheanfire | or was 12h ago | 19:52 |
| timrc | prometheanfire: It's going to be a grand time for me. And a 20 minute drive from my house. | 19:52 |
| *** maishsk has quit IRC | 19:52 | |
| *** flepied has joined #openstack-infra | 19:53 | |
| *** lindsayk has joined #openstack-infra | 19:53 | |
| prometheanfire | timrc: so that's 2 blocks? | 19:53 |
| prometheanfire | I'm going to book my hotel this weekend most likely | 19:53 |
| *** amrith is now known as _amrith_ | 19:53 | |
| prometheanfire | going for 9 on the map | 19:53 |
| *** esikachev has quit IRC | 19:54 | |
| *** _amrith_ is now known as amrith | 19:54 | |
| prometheanfire | SpamapS: want links? | 19:54 |
| *** Sukhdev has quit IRC | 19:54 | |
| *** resker has joined #openstack-infra | 19:54 | |
| prometheanfire | https://review.openstack.org/#/c/273795/ is the important one | 19:54 |
| *** esikachev has joined #openstack-infra | 19:55 | |
| SpamapS | prometheanfire: I'll run the whole queue soon, I think I see some :) | 19:55 |
| prometheanfire | yep | 19:56 |
| prometheanfire | I -1 one waiting on the lined review | 19:56 |
| *** maishsk has joined #openstack-infra | 19:56 | |
| *** gyee has quit IRC | 19:57 | |
| *** ashleigh_ has joined #openstack-infra | 19:59 | |
| *** ashleighfarnham has quit IRC | 19:59 | |
| *** ashleigh_ has quit IRC | 20:00 | |
| pabelanger | fungi: AJaeger: Indeed, 2 talks submitted! So, once voting starts, I'll be sure to email out the talks for people to vote on :) | 20:01 |
| *** jaypipes has joined #openstack-infra | 20:01 | |
| greghaynes | ianw: yep, the latest builds likely did miss it | 20:02 |
| *** Sukhdev has joined #openstack-infra | 20:02 | |
| AJaeger | pabelanger: you submitted an old version of my abstract, I updated - and I capitalized Infrastructure in the title this morning. Could you double check, please? | 20:02 |
| openstackgerrit | Igor Belikov proposed openstack-infra/puppet-jenkins: Use 'ruby' instead of 'ruby1.9.1' for Debian https://review.openstack.org/275329 | 20:02 |
| *** Sukhdev has quit IRC | 20:03 | |
| *** aysyd has quit IRC | 20:04 | |
| *** Sukhdev has joined #openstack-infra | 20:04 | |
| *** reed_ has joined #openstack-infra | 20:04 | |
| SpamapS | fungi, clarkb, jeblair: regarding swift being unsuitable for docs hosting.. this seems like a huge failure of swift. I wonder if we should take a small detour and make sure these things are recorded as bugs. | 20:04 |
| *** rfolco has quit IRC | 20:04 | |
| AJaeger | jeblair: found the bug in the wheel change, currently testing locally... | 20:05 |
| clarkb | SpamapS: I have spent the last couple years trying to explain our use case to swift :) | 20:05 |
| fungi | SpamapS: i don't know that it's entirely a failing of swift | 20:05 |
| clarkb | but as fungi says its not necessarily swifts fault, we just expected something different of it | 20:05 |
| *** ilyashakhat has quit IRC | 20:05 | |
| fungi | more like our organically-grown documentation management and arrangement does not easily map to what swift provides | 20:06 |
| SpamapS | I'd expect that it would have a way to take a local directory with index.html's pre-created, and host that. | 20:06 |
| clarkb | SpamapS: it does, but then for every container you do that with you hvae to update dns | 20:06 |
| fungi | SpamapS: sure, it does. we could build the entire site and then upload it to swift, no problem | 20:06 |
| clarkb | (granted that particular thing is an implementation detail) | 20:06 |
| *** tiswanso has quit IRC | 20:07 | |
| fungi | the problem domain here is around building and updating, not around how to serve http | 20:07 |
| prometheanfire | SpamapS: about removing the interface files, really I could just not do anything there as they don't exist by default, but if another element ever adds them they will be missed | 20:07 |
| *** tiswanso has joined #openstack-infra | 20:07 | |
| fungi | once we're already stuck maintaining a copy of all the content, it's easier to just apt-get install apache2 | 20:07 |
| SpamapS | so it sounds like it's as much the CDN failing as swift itself. | 20:07 |
| clarkb | SpamapS: its a "service" failing imo | 20:08 |
| AJaeger | anteaya: you asked me earlier about https://review.openstack.org/#/c/274359 - did you on purpose not vote or hit a 503? ;) | 20:08 |
| *** jsavak has quit IRC | 20:08 | |
| prometheanfire | gerrit just got really slow | 20:08 |
| SpamapS | No I'd want to build a dir, upload it to a container, and then tell the frontend "that container is now /docs/deployment-guide/" | 20:08 |
| *** adrienverge has joined #openstack-infra | 20:08 | |
| clarkb | we hvae many services but the overlap between what tehy do and what users want is often not very large | 20:08 |
| prometheanfire | and better now | 20:09 |
| *** jsavak has joined #openstack-infra | 20:09 | |
| Shrews | SpamapS: will look later tonight | 20:09 |
| anteaya | AJaeger: I just wanted to know if you wanted to comment, which you did, so I will review, thank you | 20:09 |
| clarkb | but that is based on my biased opinion formed with one sample | 20:10 |
| *** ociuhandu has quit IRC | 20:10 | |
| fungi | SpamapS: from a docs publication standpoint, a lot of the challenges are based around the fact that there is no clear separation between what build processes own which files | 20:11 |
| openstackgerrit | Mark Vanderwiel proposed openstack-infra/project-config: Adding integration test support for lbaas v2 dashboard https://review.openstack.org/275394 | 20:11 |
| *** achanda has quit IRC | 20:11 | |
| SpamapS | clarkb: there's a whole excited dev community over on AWS who host everything in their UI on s3+cloudfront.. so I'm kind of lamenting that swift doesn't provide that. | 20:11 |
| openstackgerrit | Igor Belikov proposed openstack-infra/project-config: Add debian-jessie image to nodepool https://review.openstack.org/264726 | 20:12 |
| *** mrmartin has quit IRC | 20:12 | |
| SpamapS | fungi: wait, they're not segmented into dirs? | 20:12 |
| fungi | it's gotten a lot cleaner over the course of the past year or two, but i gather there's still muddiness over multiple jobs updating overlapping/interleaved bits of documents | 20:12 |
| SpamapS | ohh | 20:12 |
| SpamapS | well damn | 20:12 |
| clarkb | SpamapS: there is no such thing as dirs with swift | 20:12 |
| AJaeger | jeblair: I commented on https://review.openstack.org/#/c/273549 on the two problems I found but now I have another one that confuses me ;( So, giving up and let you fix it if you want - or krotscheck1 later | 20:12 |
| clarkb | SpamapS: just key value pairs essentially | 20:12 |
| *** HeOS has joined #openstack-infra | 20:12 | |
| SpamapS | clarkb: no, there are containers though, and I'm kind of hoping containers can be mapped to paths by a CDN. | 20:12 |
| SpamapS | I am realizing now | 20:13 |
| * prometheanfire goes to do a rain dance waiting for patches to merge | 20:13 | |
| SpamapS | the problem is the frontend | 20:13 |
| SpamapS | not swift per se | 20:13 |
| *** ashleighfarnham has joined #openstack-infra | 20:13 | |
| fungi | right, swift does a great job of storing blobs of data and letting you retrieve them again on demand | 20:13 |
| SpamapS | well and grouping them actually | 20:13 |
| SpamapS | just, one level group | 20:13 |
| fungi | but then again, so does a filesystem | 20:13 |
| SpamapS | eh.. a filesystem does a much much much better job. But .. it's a static HTML site. You shouldn't need much of that. | 20:14 |
| fungi | and providing access controls around writing to them and so on | 20:14 |
| SpamapS | But yeah, I am 100% seeing where if you have to actually build up the whole thing.. you may as well just let the FS that you just built do it. | 20:14 |
| *** moravec has quit IRC | 20:16 | |
| anteaya | fungi clarkb bluebox is in fairly heavy error state | 20:16 |
| clarkb | SpamapS: as the user you don't know what the shared secret between swift and cdn is so you cna't just map the two | 20:16 |
| clarkb | SpamapS: there is some other non openstack api thing you can hit for that iirc | 20:16 |
| *** moravec has joined #openstack-infra | 20:16 | |
| clarkb | ianw: I have image uploads for fedora 23 queued to the rax regions. I notice that we do not hvae fedora 23 in the other clouds, want to add it to them? | 20:17 |
| ianw | clarkb: yep, on my todo list ... just trying to get one working first :) | 20:18 |
| clarkb | ianw: thats fair | 20:18 |
| clarkb | ianw: wasn't sure if that was a known todo or not | 20:18 |
| clarkb | and rax uploads are broken :/ | 20:19 |
| fungi | also in some situations we're going to need a real webserver to do things that simply aren't going to be supported by the cdn. actually having apache there gives us greater control for things like proxying to other systems, et cetera | 20:19 |
| clarkb | which as an aside go into swift <_< | 20:19 |
| ianw | greghaynes / clarkb : dropped a query on the disconnect change ... i'm no gear client expert | 20:19 |
| greghaynes | ianw: hrm? | 20:20 |
| clarkb | OpenStackCloudException: Failed at action (create_container) is the error | 20:20 |
| greghaynes | :( | 20:21 |
| SpamapS | clarkb: well for that I'd probably just have the job that built the site signal a worker somewhere "hey container X is the new /nova" right? But meh.. non openstack API makes me sad too ;) | 20:21 |
| greghaynes | seems like the api said "no" | 20:21 |
| clarkb | greghaynes: yes I think thats basically it | 20:21 |
| clarkb | unauthorized though so maybe its something with oscc | 20:21 |
| fungi | did we exceed a quota there possibly? | 20:21 |
| *** jgriffith_away is now known as jgriffith | 20:21 | |
| greghaynes | timeout? | 20:21 |
| pabelanger | AJaeger: I sure, I had to do some trimming because of the char limit | 20:21 |
| clarkb | its 401 unauthorized | 20:22 |
| clarkb | so I don't think it was timeout or quota | 20:22 |
| pabelanger | AJaeger: you should be able to make edits on the talk directly | 20:22 |
| AJaeger | pabelanger: I did... | 20:22 |
| clarkb | could be our old friend expiring tokens is back | 20:22 |
| clarkb | or I fixed it in one place but needed two | 20:22 |
| greghaynes | yea, sorry, thats what I mean | 20:22 |
| AJaeger | pabelanger: that's why I asked for double checking | 20:22 |
| *** HeOS has quit IRC | 20:22 | |
| greghaynes | token timeout ;) | 20:22 |
| greghaynes | We have way too many types of timeout | 20:23 |
| fungi | oh, right. session management for long glance uploads is still meh | 20:23 |
| greghaynes | fungi: Yep, and swift too | 20:23 |
| fungi | i meant long uploads into swift for glance to use | 20:23 |
| pabelanger | AJaeger: Ah, okay. Looking now | 20:23 |
| *** kzaitsev_mb has joined #openstack-infra | 20:24 | |
| fungi | glance assigns the location as part of the task, right? | 20:24 |
| notmorgan | clarkb: http://logs.openstack.org/27/275327/1/check/gate-tempest-dsvm-full/3c4f1df/logs/apache/keystone.txt.gz that look better from a timestamp perspective? | 20:24 |
| pabelanger | AJaeger: Ya, we are -252 over the char limit now | 20:24 |
| greghaynes | fungi: you just upload it to swift first and then tell glance where it is | 20:24 |
| notmorgan | clarkb: that is removing microseconds | 20:24 |
| fungi | i'm still slightly fuzzy on the signalling between glance and swift for task-based uploads | 20:24 |
| pabelanger | AJaeger: not sure what will happen | 20:24 |
| AJaeger | pabelanger: oops, let me check... | 20:24 |
| clarkb | SpamapS: so this would be another issue with it that we have found more recently. The client tooling is really hard to use | 20:24 |
| fungi | oh, so it's a normal swift upload, nothing special, glance has no idea you're doing anything until you tell it afterward? | 20:24 |
| ianw | greghayens: 272384 | 20:24 |
| clarkb | fungi: yes | 20:24 |
| clarkb | fungi: so problem here is swiftclient doesn't use sessions | 20:25 |
| greghaynes | fungi: Yep. The only specialish aspect is that our files are large enough we have to break them up | 20:25 |
| clarkb | fungi: so in a daemon that runs for days your token expires and it stops working | 20:25 |
| SpamapS | ugggghhhh | 20:25 |
| *** krotscheck1 has joined #openstack-infra | 20:25 | |
| fungi | got it. so the expectation is that the consuming system should pick idle times to reauthorize and refresh its token? | 20:25 |
| notmorgan | clarkb: sorry, it's on my long list of things to fix... | 20:25 |
| clarkb | notmorgan: that'll work, though milliseconds are nice if you have them but not a deal breaker | 20:25 |
| * SpamapS twitches when considering swift image upload in RAX | 20:25 | |
| greghaynes | fungi: I think so | 20:25 |
| AJaeger | jeblair, krotscheck1 I fixed the wheel jobs! | 20:25 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Add wheel mirror to configure_mirror.sh https://review.openstack.org/267117 | 20:25 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Added wheel-release job https://review.openstack.org/273549 | 20:25 |
| notmorgan | clarkb: can't do milliseconds, doesn't have the flexibility afaict | 20:26 |
| notmorgan | it's microsecond or no subsecond | 20:26 |
| *** bexelbie_ has joined #openstack-infra | 20:26 | |
| dmsimard | Is there somewhere I could download the centos 7 image that runs in the gate ? I am unable to reproduce an issue that happens in a job there on a local centos image. | 20:26 |
| krotscheck1 | AJaeger: Neat, thanks! | 20:26 |
| clarkb | notmorgan: kk | 20:26 |
| notmorgan | clarkb: i'll revisit the swift thing soon. | 20:26 |
| notmorgan | SpamapS: ^ | 20:26 |
| notmorgan | it really isn't far down my list | 20:26 |
| pabelanger | fungi: is -infra revisiting the use case of swift? I think that was mentioned last meeting? | 20:26 |
| notmorgan | just... not at the top atm with some higher prio cleanups | 20:26 |
| *** bexelbie is now known as Guest49277 | 20:26 | |
| *** bexelbie_ is now known as bexelbie | 20:26 | |
| *** ybathia has quit IRC | 20:26 | |
| *** bexelbie- has joined #openstack-infra | 20:27 | |
| notmorgan | getting swift on session is a must. | 20:27 |
| pabelanger | dmsimard: you could, but you wouldn't be able to log into it | 20:27 |
| notmorgan | swiftclient* | 20:27 |
| pabelanger | dmsimard: you should be able to build one | 20:27 |
| SpamapS | also remember there are really two swiftclients in swiftclient | 20:27 |
| dmsimard | pabelanger: why not ? no cloudinit ? | 20:27 |
| SpamapS | there's the easy one, and the functional one | 20:27 |
| pabelanger | dmsimard: well, we use SSH keys | 20:27 |
| dmsimard | pabelanger: that's fine, I can boot it in a local openstack environment with my key set up | 20:28 |
| *** salv-orlando has joined #openstack-infra | 20:28 | |
| *** piet has quit IRC | 20:28 | |
| fungi | pabelanger: i think that was some of the feedback on jhesketh's latest plans was that we may be expending unnecessary effort to map posix filesystem semantics onto swift containers, and that we should revisit better-suited alternatives to make sure the complexity is worthwhile | 20:28 |
| dmsimard | I can try and build it if it's not rocket science, I just thought downloading it would've been simpler :) | 20:28 |
| AJaeger | pabelanger: I'm on it... | 20:29 |
| timrc | prometheanfire: I actually live in Lakeway. I just drive wrecklessly. | 20:30 |
| pabelanger | dmsimard: look at tools/build-image.sh to setup it up locally. Easier then having infra expose the images to download | 20:30 |
| pabelanger | dmsimard: under project-config | 20:30 |
| fungi | dmsimard: the dib elements and envvars used to build our devstack-centos7 image are listed at http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml#n491 | 20:30 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Create jobs for a wheel mirror https://review.openstack.org/164927 | 20:31 |
| krotscheck1 | AJaeger: I found a bug in the first patch in that chain. | 20:31 |
| dmsimard | Ok, I'll try that. Thanks. | 20:31 |
| fungi | dmsimard: as you can see, it's based on dib's centos7 element primarily | 20:31 |
| krotscheck1 | Oh, bloody.... | 20:31 |
| krotscheck1 | silly rebase | 20:31 |
| pabelanger | dmsimard: DISTRO=devstack-centos7 ./tools/build-image.sh should do the trick | 20:32 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Create jobs for a wheel mirror https://review.openstack.org/164927 | 20:32 |
| *** piet has joined #openstack-infra | 20:32 | |
| *** tiswanso has quit IRC | 20:32 | |
| krotscheck1 | There. | 20:32 |
| clarkb | ok I can't start tugging on this swift upload thread | 20:32 |
| clarkb | I won't get anything else done this week if I do | 20:32 |
| clarkb | ianw: the lazy in my would suggest adding that image to ovh and internap and bluebox | 20:33 |
| clarkb | ianw: and worry about rax when we can figure out uploads to them | 20:33 |
| *** infra-red has joined #openstack-infra | 20:34 | |
| AJaeger | krotscheck1: krotscheck1 you need to rebase the whole series... | 20:34 |
| krotscheck1 | AJaeger: I know, working on it. | 20:34 |
| krotscheck1 | Gerrit s not cooperating | 20:34 |
| *** tiswanso has joined #openstack-infra | 20:35 | |
| AJaeger | pabelanger: ok to remove the companies from our names? Those are mentioned in the speaker bio anyhow... | 20:35 |
| * krotscheck1 peers at that patch chain. | 20:35 | |
| * krotscheck1 has no idea how that happened. | 20:35 | |
| AJaeger | krotscheck1: what happened? | 20:36 |
| krotscheck1 | AJaeger: It thinks I removed the wheel builder from the macros file. | 20:36 |
| krotscheck1 | It was never there to begin with. | 20:36 |
| pabelanger | AJaeger: sure | 20:37 |
| krotscheck1 | Oh | 20:37 |
| krotscheck1 | THat's a change jeblair made | 20:37 |
| * krotscheck1 sighs | 20:37 | |
| * krotscheck1 reverts all the things. | 20:37 | |
| ianw | dmsimard: do you have a rax account ... i could probably build one and share the image with you | 20:38 |
| *** reed_ has quit IRC | 20:38 | |
| dmsimard | ianw: don't think I do, it's alright, I'm working on building it :) | 20:38 |
| *** infra-re_ has joined #openstack-infra | 20:38 | |
| * mgagne opens one and get hit with 50$ infra monthly fee | 20:38 | |
| *** simondodsley has quit IRC | 20:38 | |
| ianw | dmsimard: yeah, you probably want to add the devuser element, and set something like DIB_DEV_USER_PASSWORD="..." DIB_DEV_USER_PWDLESS_SUDO=1 | 20:39 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Create jobs for a wheel mirror https://review.openstack.org/164927 | 20:39 |
| AJaeger | pabelanger: updated, please double check | 20:40 |
| clarkb | greghaynes: I think ianw's comment on https://review.openstack.org/#/c/272384/7 may be on to something. We indisciminately remove the db record even though the job may have failed | 20:40 |
| clarkb | greghaynes: should we set it up to onComplete remove the record instead? | 20:40 |
| krotscheck1 | AJaeger: I've reverted ^^ back to jeblair's patch (PS30). | 20:40 |
| *** infra-red has quit IRC | 20:41 | |
| krotscheck1 | AJaeger: All patches in the chain seem to have fixed themselves. | 20:41 |
| HenryG | gerrit responsiveness is pants | 20:42 |
| *** krotscheck1 has quit IRC | 20:42 | |
| *** zeih has joined #openstack-infra | 20:42 | |
| AJaeger | krotscheck1: no, they have not. If you go up, it's broken. Let me do it... | 20:42 |
| pabelanger | AJaeger: +1 | 20:42 |
| *** reed_ has joined #openstack-infra | 20:42 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Added wheel-release job https://review.openstack.org/273549 | 20:42 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Add wheel mirror to configure_mirror.sh https://review.openstack.org/267117 | 20:43 |
| *** eil397 has joined #openstack-infra | 20:43 | |
| *** bexelbie has quit IRC | 20:44 | |
| *** bexelbie- is now known as bexelbie | 20:44 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Add wheel mirror to configure_mirror.sh https://review.openstack.org/267117 | 20:47 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Added wheel-release job https://review.openstack.org/273549 | 20:47 |
| *** sdake has joined #openstack-infra | 20:47 | |
| clarkb | the GC time on gerrit has jumped in the last day | 20:47 |
| *** resker has quit IRC | 20:48 | |
| *** reed_ has quit IRC | 20:48 | |
| jeblair | clarkb: were we going to increase the ram or something? | 20:48 |
| openstackgerrit | Merged openstack/diskimage-builder: Add new cloud-init element https://review.openstack.org/273764 | 20:49 |
| clarkb | its something I have mentoned .I think we have to do it to keep up with gerrits ram needs | 20:49 |
| *** _nadya_ has quit IRC | 20:49 | |
| clarkb | but it is tricky because of the don't change IP or all corps break problem | 20:49 |
| openstackgerrit | Merged openstack/diskimage-builder: Fix Gentoo hardened support https://review.openstack.org/273795 | 20:49 |
| AJaeger | pabelanger: great - time to call it done (from my side ;) | 20:49 |
| *** diana_clarke has joined #openstack-infra | 20:49 | |
| pabelanger | AJaeger: now we wait | 20:50 |
| jeblair | clarkb: can we grow this instance? | 20:50 |
| clarkb | jeblair: we can, I am always wary of that because it doesn't allow us to set downtime expectations ahead of time | 20:50 |
| jeblair | yep | 20:50 |
| clarkb | but that is another options | 20:50 |
| AJaeger | pabelanger: yes ;) | 20:50 |
| fungi | we can grow it? | 20:51 |
| clarkb | fungi: ya we should be able to resize it up a flavor or three | 20:51 |
| fungi | double-checking. thought they removed that when they transitioned off their legacy environment | 20:51 |
| clarkb | nova supports resize | 20:51 |
| clarkb | but maybe rax doesn't anymore | 20:52 |
| fungi | i know _nova_ does | 20:52 |
| *** Sukhdev has quit IRC | 20:52 | |
| fungi | resize is greyed out in the actions manage drop-down section for review.o.o anyway | 20:52 |
| *** achanda has joined #openstack-infra | 20:52 | |
| greghaynes | clarkb: ianw re: deleting images change - there was much discussion in IRC which I failed at transcribing in that patch (so I should do that). tl;dr is there are several bad options for how to delete an image - we could try and estimate if there is a corresponding delete job we can call but that is likely to be incorrect and err on the side of keeping images, or we can err on the side of always | 20:52 |
| greghaynes | calling delete which will result in some hanging jobs | 20:52 |
| timrc | zaro: Can I bribe you with brisket at the Summit to get https://review.openstack.org/#/c/271543/3 merged so https://review.openstack.org/#/c/252094/ merges? | 20:52 |
| fungi | clarkb: the only instances i'm aware you can still resize in rax are trove instances | 20:53 |
| clarkb | fungi: gotcha | 20:53 |
| clarkb | well then, our option is to breaks HPs firewall rules :) | 20:53 |
| greghaynes | clarkb: ianw we figured it is easier to make deleteImage always call delete, and then in a later patch we can have that inspect a cache of what jobs are listed on the server to know wheter or not we can call the gearman delete | 20:53 |
| *** mriedem is now known as mriedem_machell | 20:53 | |
| clarkb | greghaynes: so this is a best effort as far as deleting off of disk? | 20:54 |
| greghaynes | clarkb: Yep | 20:54 |
| clarkb | ok | 20:54 |
| fungi | so i've been assuming we're stuck with an ip address change if we need more ram. which is not terrible, but for the sake of courtesy we should provide some (a few weeks?) notice | 20:54 |
| clarkb | my +2 is still applied | 20:54 |
| jeblair | fungi: we can also have the conversation about whether we need to worry so much now that https submission is a thing | 20:55 |
| jeblair | 3rd party ci is still a concern of course | 20:55 |
| fungi | yes, great point | 20:55 |
| anteaya | +1 bigger instance | 20:55 |
| zaro | timrc: mmm, brisket bribery. i'm a fan! | 20:56 |
| anteaya | shall I work on an email? | 20:56 |
| fungi | clearly anyone who needs to ask their corporate firewall fascists to open ssh to our server should just give in and use https if possible | 20:56 |
| *** jgriffith is now known as jgriffith_away | 20:56 | |
| *** dkranz has joined #openstack-infra | 20:56 | |
| fungi | which for everything but stream-events is fine | 20:56 |
| * anteaya will just copy paste the last email and let fungi insert new ip addresses | 20:56 | |
| zaro | timrc: have you verified that it works? | 20:56 |
| clarkb | fungi: don't forget https://github.com/pypa/pip/pull/3435 | 20:58 |
| fungi | clarkb: yep, i went so far as to run the unit tests locally, now just need to figure out what amount of tweaking will accommodate null versions | 20:58 |
| timrc | zaro: Yeah it's been awhile though. | 20:59 |
| timrc | zaro: If that's the criteria to getting this merged, I'll happily re-setup the experiment to verify moving to the latest stable version of gearman-plugin does not break current version of Jankins. | 20:59 |
| fungi | clarkb: which is the easy part. the hard part is deciding whether i push an incremental update based on that commit or a new replacement for that commit | 20:59 |
| clarkb | fungi: I think it depends on the project | 21:00 |
| clarkb | fungi: some wants amends/squahse others want the dirty history | 21:00 |
| timrc | zaro: I really want to see these zuul metrics come to life though :) | 21:00 |
| fungi | clarkb: and discard the old pull request and generate another pull request, or figure out if you can update an existing pull request | 21:00 |
| clarkb | fungi: you use the same pull request it automagically happens if you use th esame branch | 21:00 |
| *** j_king_ is now known as j_king | 21:01 | |
| *** DuaneDeC7 has quit IRC | 21:01 | |
| fungi | though to use the same branch i need to push --force or delete and rebranch | 21:01 |
| fungi | i guess that's expected in githubland? | 21:01 |
| clarkb | or push a fast forwardable commit | 21:01 |
| clarkb | but yes force or push fast forward are both normal | 21:01 |
| fungi | ffable implies not squashed | 21:01 |
| clarkb | yup | 21:02 |
| clarkb | so there are two trains of thought and it dpeends on the project you are PRing | 21:02 |
| fungi | i'll push --force and wait for screams | 21:02 |
| *** thiagop has quit IRC | 21:02 | |
| fungi | i have a feeling this project's school of thought for a one-line bug fix is "whatever gets the job done" | 21:02 |
| anteaya | I'm working here: https://etherpad.openstack.org/p/WciQEwLsue | 21:03 |
| clarkb | zaro: is there some rough maths we can do to come up with a new server size? | 21:03 |
| *** jsavak has quit IRC | 21:03 | |
| openstackgerrit | Adrien Vergé proposed openstack-infra/project-config: Lint YAML files https://review.openstack.org/271845 | 21:04 |
| openstackgerrit | Adrien Vergé proposed openstack-infra/project-config: Fix linting errors in YAML files https://review.openstack.org/271844 | 21:04 |
| *** vgridnev has quit IRC | 21:04 | |
| *** flepied1 has joined #openstack-infra | 21:04 | |
| *** jsavak has joined #openstack-infra | 21:04 | |
| *** salv-orl_ has joined #openstack-infra | 21:04 | |
| *** salv-orlando has quit IRC | 21:04 | |
| *** [1]Thelo has joined #openstack-infra | 21:04 | |
| clarkb | it is worth noting I don't think this is the only cause of the 500 errors | 21:05 |
| clarkb | but I do think it is the cause for the frequent errors as we GC more and pile up threads | 21:05 |
| anteaya | clarkb: can you add details to https://etherpad.openstack.org/p/WciQEwLsue | 21:05 |
| anteaya | clarkb: I'm mostly generalizing | 21:06 |
| *** flepied has quit IRC | 21:06 | |
| clarkb | anteaya: oh I think the message is fine I just wanted to point out that this likely won't fix it all | 21:06 |
| *** DuaneDeC7 has joined #openstack-infra | 21:06 | |
| anteaya | right, I just said we hope this will address the user experience | 21:06 |
| anteaya | not that we think this will fix the underlying cause | 21:06 |
| anteaya | and thanks for looking | 21:07 |
| *** Thelo has quit IRC | 21:07 | |
| *** [1]Thelo is now known as Thelo | 21:07 | |
| *** salv-orl_ has quit IRC | 21:08 | |
| *** adrienverge has quit IRC | 21:08 | |
| *** dalgaaf has quit IRC | 21:08 | |
| anteaya | added that as I realize my prior sentences could be interpreted either way | 21:08 |
| *** salv-orlando has joined #openstack-infra | 21:08 | |
| fungi | perhaps a qualifier like "most" would help | 21:08 |
| *** ihrachys has joined #openstack-infra | 21:09 | |
| pabelanger | fungi: here is another review that nibalizer +1 workflow but zuul never enqueued. 273678 | 21:09 |
| *** Apoorva has quit IRC | 21:09 | |
| pabelanger | fungi: that was 4 days ago | 21:09 |
| zaro | timrc: yeah, i think that's all it takes. | 21:09 |
| fungi | pabelanger: was it approved before its depends-on merged? | 21:09 |
| timrc | zaro: Okay cool. Will give me an entry point back into this magical world. | 21:09 |
| anteaya | fungi: I'll stick a most in there | 21:09 |
| pabelanger | fungi: I think it was | 21:10 |
| anteaya | added a most | 21:10 |
| fungi | pabelanger: known issue for projects who don't share a change queue for the gate pipeline. just needs workflow +1 (re)applied | 21:10 |
| pabelanger | fungi: Ah, okay. I didn't know that | 21:10 |
| pabelanger | fungi: do you mind toggling it? | 21:10 |
| zaro | clarkb: sorry, i totally forgot about that. been bogged down trying to fix upstream gerrit bugs. | 21:10 |
| zaro | and the hp cloud thingys | 21:11 |
| *** infra-re_ has quit IRC | 21:11 | |
| fungi | pabelanger: i've added a workflow +1 now | 21:11 |
| clarkb | zaro: I am just wondering if 2x is enough or if we should plan for more | 21:11 |
| pabelanger | fungi: danke | 21:12 |
| fungi | pabelanger: looks like it was approved a minute after its dependency, but 7 minutes before that dependency merged | 21:12 |
| zaro | clarkb: would be a good try. but i'm not sure that would be optimum atm. can maybe give you a better answer tomorrow | 21:13 |
| bknudson_ | gerrit seems a little slow (getting occasional proxy errors) | 21:13 |
| *** dkranz has quit IRC | 21:13 | |
| *** kgiusti has left #openstack-infra | 21:13 | |
| anteaya | bknudson_: we are discussing moving the server | 21:13 |
| bknudson_ | great, thanks | 21:14 |
| anteaya | bknudson_: we don't seem to be able to get it to run error free | 21:14 |
| anteaya | bknudson_: thanks | 21:14 |
| jeblair | clarkb: should we restart it? | 21:14 |
| *** annegentle has quit IRC | 21:14 | |
| *** yamahata has joined #openstack-infra | 21:15 | |
| *** reed_ has joined #openstack-infra | 21:15 | |
| *** dkranz has joined #openstack-infra | 21:16 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Roll out Fedora 23 to more providers https://review.openstack.org/275411 | 21:16 |
| *** tzn has joined #openstack-infra | 21:16 | |
| clarkb | jeblair: to address the current issues thats probably the best course of action | 21:16 |
| ianw | clarkb: ^ you know i don't think i've tried raw images | 21:16 |
| clarkb | zaro: if you can do that that would be great | 21:16 |
| ianw | i don't have accounts on any of these providers so debugging is harder | 21:17 |
| *** amrith is now known as _amrith_ | 21:18 | |
| AJaeger | ianw: your change conflicts with the stack at https://review.openstack.org/#/c/274162 - should we mrege that first? | 21:18 |
| *** diana_clarke has quit IRC | 21:18 | |
| clarkb | ianw: I am not too worried about the other providers, they mostly just work bceause dhcp | 21:18 |
| AJaeger | ian, please see https://review.openstack.org/#/c/274169/1 | 21:18 |
| *** diana_clarke has joined #openstack-infra | 21:18 | |
| ianw | clarkb: ha ha, famous last words :) | 21:18 |
| jeblair | fungi: topic:wheel_mirror is ready and passing, do you want to do a pass through and +A the system-config ones. once the mirrors exist we can +A the project-config ones. | 21:19 |
| *** smarcet has quit IRC | 21:19 | |
| *** tcammann_ has joined #openstack-infra | 21:19 | |
| AJaeger | ianw: isn't 274169 a duplicate? | 21:19 |
| clarkb | ianw: eh its amazing how much better dhcp is for getting reliable networking | 21:19 |
| fungi | jeblair: thanks for the heads up, switching back to that now | 21:19 |
| *** tcammann has joined #openstack-infra | 21:19 | |
| openstackgerrit | Merged openstack-infra/project-config: Gate on fedora-23 for openstack-infra/system-config https://review.openstack.org/273678 | 21:19 |
| *** NobodyCa1 has joined #openstack-infra | 21:20 | |
| *** esikachev has quit IRC | 21:20 | |
| *** bhunter71 has quit IRC | 21:20 | |
| clarkb | uh do we have working ^ machines for that | 21:20 |
| ianw | AJaeger: hmm, yeah, what's going on with that | 21:20 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Add support for OpenRC to dib-init-system https://review.openstack.org/273772 | 21:20 |
| openstackgerrit | Merged openstack-infra/project-config: Add lib-forward-testing for osprofiler https://review.openstack.org/274472 | 21:21 |
| AJaeger | ianw: 274162 is not approved yet... | 21:21 |
| ianw | ahhh, ok let me check out 274162 and get this all sorted out | 21:21 |
| AJaeger | jeblair: is the stack at 274162 ready? It has 2 +2s and we want to give you the honour to merge it... | 21:21 |
| *** gyee has joined #openstack-infra | 21:22 | |
| prometheanfire | SpamapS: ianw: greghaynes https://review.openstack.org/#/c/273772/ is the last one for now | 21:22 |
| *** esker has joined #openstack-infra | 21:22 | |
| *** gyee has quit IRC | 21:22 | |
| *** ubuntu has joined #openstack-infra | 21:23 | |
| *** ubuntu is now known as Guest40848 | 21:23 | |
| *** annegentle has joined #openstack-infra | 21:23 | |
| *** jsavak has quit IRC | 21:23 | |
| fungi | clarkb: er, yeah, i guess i should have looked more closely at that change before approving. granted it already had another workflow +1 and i didn't cr +2, but still my fault. happy to push up a revert and also do the manual steps to un-wedge us if necessary | 21:23 |
| jeblair | AJaeger: i think so, but ianw said something about 274169 at the end of that stack | 21:24 |
| AJaeger | jeblair, ianw: please figure it out together ;) | 21:24 |
| clarkb | fungi: if we don't need much disk then it may just work | 21:24 |
| *** tcammann_ has quit IRC | 21:24 | |
| fungi | i have a feeling the puppet apply jobs need at least some nontrivial diskspace to install all the modules | 21:24 |
| *** tcammann has quit IRC | 21:24 | |
| clarkb | fungi: however https://review.openstack.org/#/c/275411/1/nodepool/nodepool.yaml should get us fedora-23 machines with disk | 21:25 |
| jeblair | clarkb: any reason you +2d that instead of https://review.openstack.org/274169 ? | 21:25 |
| *** NobodyCa1 has quit IRC | 21:25 | |
| clarkb | jeblair: only because ianw pushed it just now and pointed it out | 21:26 |
| *** EricGonczer_ has joined #openstack-infra | 21:26 | |
| jeblair | ianw: any reason you pushed that rather than pointed clarkb at the older change you already +2d? | 21:26 |
| *** jsavak has joined #openstack-infra | 21:27 | |
| jeblair | i think it would be great if we could review changes rather than all of us just pushing the same change over and over again | 21:27 |
| clarkb | I agree, but we also sometimes do it and it isn't a huge deal (see hpcloud shutdown changes) | 21:28 |
| ianw | jeblair: yeah, i've just got confused as to what was committed, sorry, me cupla | 21:28 |
| *** esker has quit IRC | 21:28 | |
| anteaya | smcginnis: reading the novel, very digestable, well done | 21:29 |
| ianw | ok, i abandoned https://review.openstack.org/#/c/275411/ | 21:29 |
| *** sdake has quit IRC | 21:29 | |
| *** jsavak has quit IRC | 21:29 | |
| *** jsavak has joined #openstack-infra | 21:30 | |
| clarkb | ianw: looks like we are just waiting to switch devstack-centos7 over to dib in ovh, know of any reason to not go ahead and do that? | 21:31 |
| clarkb | oh you beat me to the +A | 21:31 |
| ianw | clarkb: i just approved that one through, looks fine. i just got a bit confused in the stacking of the change i think | 21:31 |
| jeblair | clarkb: that's true, though the older hp shutdown change was materially different than what we ended up merging due to the somewhat catastrophic situation we were in. | 21:31 |
| *** zaro has quit IRC | 21:32 | |
| anteaya | smcginnis: cheesecake ha ha ha | 21:32 |
| clarkb | jeblair: they were functionally identical and oculd have been merged together or in steps | 21:32 |
| clarkb | jeblair: but the end result was the same | 21:32 |
| *** Apoorva has joined #openstack-infra | 21:32 | |
| clarkb | (no more hpcloud in nodepool.yaml) | 21:32 |
| jeblair | clarkb: right, this is a series of things that i thought rather carefully about and put together last week and could have been reviewed and merged at any time by people interested in it, but it has languished | 21:32 |
| *** jpr has quit IRC | 21:33 | |
| *** jpr has joined #openstack-infra | 21:33 | |
| jeblair | at any rate, i'm seeing more instances of us not seeing the right patches | 21:34 |
| jeblair | i think that's true for all of us | 21:34 |
| AJaeger | yolanda and myself could use some help with reviews of project-config - we have 150+ open patches but many merge conflicts... | 21:35 |
| jeblair | AJaeger: merge conflicts are easy, right? ignore them :) | 21:35 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add windmill-jobs for ansible-role-zuul https://review.openstack.org/275419 | 21:35 |
| *** zul has quit IRC | 21:35 | |
| ianw | AJaeger: yeah, sorry, i've been slow there, and also slow on devstack since new year | 21:35 |
| clarkb | so I think there are a couple parts to the problem. The sheer volume involved is hard to deal with, but also everyone patches languish as a result | 21:35 |
| jeblair | AJaeger: and i'm going to help you by not reviewing those patches and writing zuulv3 instead so that there are far fewer of them to review in the future. | 21:35 |
| ianw | i feel like i might be getting to the end of fire-fighting images, however | 21:35 |
| clarkb | so you end up in a feedback loop of psuing the same changes | 21:36 |
| *** ybathia has joined #openstack-infra | 21:36 | |
| openstackgerrit | Anne Gentle proposed openstack-infra/project-config: Adds fairy-slipper build jobs for api-site repo https://review.openstack.org/269809 | 21:36 |
| *** gyee has joined #openstack-infra | 21:37 | |
| AJaeger | jeblair: sure, merges are ignored. | 21:37 |
| *** zaro has joined #openstack-infra | 21:37 | |
| *** zaro has quit IRC | 21:38 | |
| *** baoli has joined #openstack-infra | 21:38 | |
| greghaynes | There is also a strong negative feedback where the longer a patch sits the more effort it takes to merge (more likely to conflict, harder to fix bugs, more likely to get out of date). There is a really neat tool in russelb's reviewstats for measuring things related to that languish time... | 21:38 |
| greghaynes | I have used it in other projects with some success before | 21:38 |
| greghaynes | http://russellbryant.net/openstack-stats/infra-openreviews.html being the link | 21:39 |
| AJaeger | jeblair: looking forward to zuulv3... thanks for working on it. | 21:39 |
| *** zaro has joined #openstack-infra | 21:39 | |
| jeblair | however, i think right now we're talking about some patches related to an effort we're all working on | 21:39 |
| clarkb | greghaynes: looks like the pep8 fail for 273703 shouldn't affect the tests | 21:39 |
| jeblair | and we all ought to be aware of | 21:39 |
| greghaynes | clarkb: yea, for some reason the test is timing out | 21:40 |
| openstackgerrit | Merged openstack-infra/project-config: Remove devstack-centos7-dib in favor of devstack-centos7 https://review.openstack.org/274162 | 21:40 |
| jeblair | so maybe we can just try to keep this in mind that we have several people working on this and communicate a bit more | 21:40 |
| jeblair | maybe we should set topics better | 21:40 |
| *** zeih has quit IRC | 21:40 | |
| *** reed__ has joined #openstack-infra | 21:40 | |
| jeblair | on that topic | 21:41 |
| clarkb | greghaynes: I wonder if it isn't detecting the disconnect quickly enough because tcp? | 21:41 |
| jeblair | fungi: you may also want to review "topic:afs-mirror" | 21:41 |
| *** dkranz has quit IRC | 21:41 | |
| fungi | jeblair: thanks, adding | 21:42 |
| greghaynes | clarkb: I think I see the issue - the logic change to deleteImage breaks the test | 21:42 |
| * AJaeger waves good night... | 21:42 | |
| *** ihrachys has quit IRC | 21:42 | |
| greghaynes | clarkb: the disconnect happens, we try to submit the delete job to gearman but we are disconnected so that fails so the image sits forever | 21:42 |
| greghaynes | jeblair: ^ | 21:42 |
| ianw | jeblair: on the plus side, we got a bit tangled up in those image changes, but the tooling showed us the conflict, it was noticed quickly, we got on the same page and everything is moving ... in other places i've been, this would have probably required about 3 hours of meetings :) | 21:42 |
| openstackgerrit | Merged openstack-infra/project-config: Remove unused images https://review.openstack.org/274164 | 21:43 |
| *** reed__ has quit IRC | 21:43 | |
| clarkb | greghaynes: does it need to be a in try finally block then? | 21:43 |
| openstackgerrit | Merged openstack-infra/project-config: Add fedora-23 to bluebox, internap, and ovh https://review.openstack.org/274169 | 21:43 |
| greghaynes | http://logs.openstack.org/03/273703/5/check/gate-nodepool-python27/840734e/console.html.gz#_2016-01-29_18_48_17_345 | 21:43 |
| clarkb | greghaynes: so that db is always cleaned up but we log the exceptional state | 21:43 |
| greghaynes | clarkb: This is a bug in the algorithm we didnt consider | 21:43 |
| greghaynes | clarkb: so I am less thrilled with this solution | 21:43 |
| *** zeih has joined #openstack-infra | 21:44 | |
| *** reed_ has quit IRC | 21:44 | |
| greghaynes | is seems like in the case of a disconnect we will always leave the image hanging on the builder | 21:44 |
| greghaynes | I guess we can just say we will deal with that and periodically clean up? | 21:44 |
| clarkb | greghaynes: we do have the periodic cron that should resubmit the delete job if the image remains in the db | 21:45 |
| greghaynes | clarkb: oh. So put the delete in a finally | 21:45 |
| greghaynes | clarkb: yes | 21:45 |
| *** EricGonczer_ has quit IRC | 21:45 | |
| clarkb | well if you put it in a finally then it will get removed from the db and the next cron run won't try to delte it again | 21:45 |
| greghaynes | clarkb: er, in an else | 21:46 |
| clarkb | I think you can keep the code as is since it raises and doesn't remove the image | 21:46 |
| clarkb | it just never submits the job which breaks the test | 21:46 |
| sc68cal | rob: lol | 21:46 |
| sc68cal | whoops | 21:47 |
| greghaynes | clarkb: oh! the nodepool loop is not running | 21:48 |
| greghaynes | clarkb: that is the problem | 21:48 |
| *** sputnik13 has joined #openstack-infra | 21:48 | |
| *** achanda has quit IRC | 21:48 | |
| pabelanger | review.o.o issues? Getting random 503 errors now | 21:48 |
| *** EricGonczer_ has joined #openstack-infra | 21:48 | |
| anteaya | clarkb: I think you make a good point about volume | 21:49 |
| anteaya | pabelanger: been happening all day to folks | 21:50 |
| jeblair | ianw: true, i am fussing about the last 3% of efficiency :) | 21:50 |
| pabelanger | anteaya: Ah, must be luck then. Only started for me about 5 mins ago | 21:50 |
| jeblair | ianw, clarkb: so what is the deal with fedora23 -- do we expect it to work in the other providers? | 21:50 |
| *** ayoung has quit IRC | 21:51 | |
| jeblair | i thought the only issue was it requires dhcp, and those providers have it, so.. | 21:51 |
| clarkb | jeblair: yes, we cannot upload anything to rackspce right now but everyone else should be fine | 21:51 |
| anteaya | pabelanger: you win the lottery | 21:51 |
| jeblair | clarkb: yeah, and rax is the one without dhcp so it'd be broken anyway, right? | 21:51 |
| clarkb | jeblair: no, the glean stuff should work in rax now. Its purely a uploads to swift not working problem | 21:51 |
| clarkb | and the uploads to swift not working appears to still be the same token expiration issue | 21:52 |
| jeblair | oh ok | 21:52 |
| *** flepied1 has quit IRC | 21:52 | |
| jeblair | clarkb: is that a shade bug? | 21:52 |
| clarkb | jeblair: it is a swiftclient bug that shade has not worked around | 21:52 |
| jeblair | (i mean, even so, bug is arguable when we're talking about the proprietary rax upload protocol) | 21:52 |
| greghaynes | Yea, its kind of a bug in both :( | 21:53 |
| clarkb | it isn't failing on the proprietary bits, swiftclient as used by shade doesn't make new tokens when it needs to | 21:53 |
| *** esker has joined #openstack-infra | 21:53 | |
| greghaynes | if it is the same deal | 21:53 |
| clarkb | swiftclient supports a way to do that but shade wants to do it the standard way so doesn't do it he way swiftclient supports | 21:53 |
| jeblair | oh is this the thing notmorgan has on his plate? | 21:53 |
| notmorgan | jeblair: yeah | 21:54 |
| jeblair | cool, i feel like i'm caught up now, thanks :) | 21:54 |
| *** flepied has joined #openstack-infra | 21:54 | |
| clarkb | if shade were to hand off all of the credential info instead of handing over atoken this should just work | 21:54 |
| notmorgan | but that is generally bad(tm) - for the sake of swiftclient it isn't awful [since it's the "right" approach for now] | 21:55 |
| notmorgan | but simply making swift use session and be able to be constructed with session will solve it | 21:55 |
| clarkb | as an aside this is another reason why consistency between clouds is nice | 21:55 |
| clarkb | it means we don't have to solve the same problem over and over in different code | 21:55 |
| notmorgan | aye | 21:56 |
| * notmorgan finishes a series of keystone cleanups and then eyes more occ/ksa work | 21:56 | |
| notmorgan | novaclient and swift are top of my list to knock out | 21:56 |
| *** erlon has quit IRC | 21:56 | |
| *** sdake has joined #openstack-infra | 21:57 | |
| clarkb | hrm if we ride out the memory issues for a couple months we could upgrade gerrit to newer ubuntu lts | 21:57 |
| clarkb | depending on how much lead time we think we need to give our users that might be worth considering | 21:57 |
| anteaya | clarkb: when is it out? | 21:57 |
| clarkb | anteaya: mid april | 21:57 |
| notmorgan | anteaya: sometime around when we release ;) | 21:58 |
| jeblair | clarkb: last time i think we were told 3 week minimum but 2 months ideal. | 21:58 |
| anteaya | perfect | 21:58 |
| *** DuaneDeC7 has quit IRC | 21:58 | |
| *** flwang1 has joined #openstack-infra | 21:58 | |
| anteaya | April 21st is Xenial's release | 21:59 |
| clarkb | notmorgan: out of curiousity, couldn't keystone do its own logging rather than rely on apache? and if so would that be preferable for consistency with python and the rest of openstack? | 21:59 |
| anteaya | Mitaka is the week of April 4-8 | 21:59 |
| *** achanda has joined #openstack-infra | 21:59 | |
| jeblair | clarkb: we could do weekly restarts on saturday morning or something | 22:00 |
| *** shashank_hegde has quit IRC | 22:00 | |
| notmorgan | clarkb: we could, in general we prefer to let apache handle anything it can esp. things like time on requests. | 22:00 |
| clarkb | notmorgan: or are there issues with arbitrary threads all writing to the same location? | 22:00 |
| jeblair | (then we will be Real Java Admins (TM)) | 22:00 |
| anteaya | and austin is april 25-29 | 22:00 |
| notmorgan | clarkb: it still needs to be all written to STD...Err.oroutorone of those. we wouldn't write to a file | 22:00 |
| anteaya | would a weekly restart get us through until April? | 22:00 |
| clarkb | jeblair: yes we could, we could also possibly redirect /p/ to git.o.o and reclaim all that memory | 22:00 |
| notmorgan | clarkb: we could just make keystone log the timestamp instead of apache too. | 22:01 |
| *** esker has quit IRC | 22:01 | |
| *** Sukhdev has joined #openstack-infra | 22:01 | |
| clarkb | jeblair: we have 30GB total, 12GB are heap for gerrit, ~10GB is caches and the rest is apache serving git repos iirc | 22:02 |
| *** esker has joined #openstack-infra | 22:02 | |
| notmorgan | so the apache log would just not apply apache timestamp and keystone could add it, if milliseconds are really thar important it wouldn't be hard to swap that around | 22:02 |
| *** rossella_s has quit IRC | 22:02 | |
| fungi | i think the latency difference between gerrit's local and ssh replication is likely irrelevant to most uses of /p (where are we still hitting that in our automation?) | 22:02 |
| *** trown is now known as trown|outttypeww | 22:02 | |
| *** rossella_s has joined #openstack-infra | 22:03 | |
| notmorgan | i'd go to %M as the formatter in ErrorLogFormat instead of %{c}t %M | 22:03 |
| clarkb | notmorgan: I am not sure they are that important, just trying to figure out if we can be consistent | 22:03 |
| anteaya | and does 2.11 work on Xenial? | 22:03 |
| *** shashank_hegde has joined #openstack-infra | 22:03 | |
| fungi | jeblair: i like the binary wheel_keytab in hiera. that's convenient (also small!) | 22:04 |
| *** jsavak has quit IRC | 22:04 | |
| jeblair | fungi: yeah, and hieraedit just dtrt when you give it a binary file | 22:05 |
| *** edmondsw has quit IRC | 22:05 | |
| clarkb | anteaya: it should, we already install libs from not the distro when we have to | 22:05 |
| notmorgan | clarkb: i'll defer to you, sdague, and -qa folks on that front. i don't care either way. I prefer apache to do any lifting it can. but the next step is move to uwsgi - so we might want keystone to do the timestamp anyway | 22:05 |
| fungi | jeblair: hooray for pyyaml making sane choices? | 22:05 |
| jeblair | fungi: yep! | 22:05 |
| anteaya | clarkb: wonderful, I hope it does, the question just floated through my mind | 22:05 |
| openstackgerrit | Merged openstack-infra/system-config: Added conditional install statements for kernel header packages https://review.openstack.org/273205 | 22:06 |
| *** dkranz has joined #openstack-infra | 22:07 | |
| dims_ | fungi : when you get a chance a pypi upload failed for osprofiler. can you please peek to see how we could fix it? http://logs.openstack.org/7a/7a428885ddb138f3d3a388e72bca276cfa2a5299/release/osprofiler-pypi-both-upload/5c2c80a/console.html#_2016-02-02_22_01_26_314 | 22:07 |
| clarkb | dims_: that looks like pypi simply had an error. Our best bet is to re run it | 22:08 |
| clarkb | maybe check with dstufft to see if things aren't in an unhappy state pypi side | 22:09 |
| clarkb | https://status.python.org/ claims it is happy | 22:10 |
| *** piet has quit IRC | 22:10 | |
| openstackgerrit | Merged openstack-infra/system-config: Added afs to wheel mirror slaves https://review.openstack.org/273054 | 22:10 |
| clarkb | dims_: I can go ahead and requeue that job | 22:11 |
| fungi | clarkb: won't work | 22:11 |
| *** jsavak has joined #openstack-infra | 22:11 | |
| fungi | from what dstufft has described in the past, fastly sometimes returns an internal server error on the post even though the upload succeeded | 22:12 |
| fungi | which seems to be the case here | 22:12 |
| ianw | jeblair: i expect f23 to work | 22:12 |
| fungi | clarkb: the job will just fail again if you rerun it, but this time because pypi will refuse to let you replace the wheel that's already there | 22:12 |
| clarkb | does that mean we just need to upload the sdist? | 22:12 |
| fungi | yeah, i'm about to do that | 22:12 |
| clarkb | fun | 22:12 |
| fungi | was discussing it in -release | 22:13 |
| clarkb | we could try adding a if 500 then get and if get succeeds then continue step | 22:15 |
| clarkb | else fail | 22:15 |
| openstackgerrit | Ondřej Nový proposed openstack/requirements: Bump PyECLib version to 1.0.8 https://review.openstack.org/275181 | 22:15 |
| fungi | clarkb: yep, it's been suggested in the past. nobody has found the time to add it yet (or maybe they have and it's lost in our review backlog) | 22:16 |
| *** esker has quit IRC | 22:16 | |
| clarkb | though its not as simple as just checking name/version since you have to check the whl/sdist path directly | 22:17 |
| clarkb | but you can use pip for that :) | 22:17 |
| fungi | it may also get more ugly when we start uploading signatures along with wheels and tarballs | 22:17 |
| clarkb | I think pip can check all that for us but I am reading its help output | 22:18 |
| fungi | not sure if it's possible for the signature to get uploaded and not the thing that's signed, and what order twine puts them in, and which pypi will let you reupload and which it won't | 22:18 |
| clarkb | apparnelty --no-install is deprecated | 22:19 |
| clarkb | but can use --download | 22:20 |
| StevenK | pip 8 has 'pip download' too | 22:20 |
| clarkb | pip install --no-deps --download '$name==$version' check the return code and branch on that | 22:21 |
| fungi | heads up: i've approved 270979 just now and will try to keep an eye out for potential apache issues on our mirrors (note that the commit message isn't completely correct--it adds some apache rewrites but not a vhost) | 22:22 |
| *** rfolco has joined #openstack-infra | 22:22 | |
| *** sdake has quit IRC | 22:24 | |
| *** piet has joined #openstack-infra | 22:24 | |
| *** markmcclain has quit IRC | 22:25 | |
| openstackgerrit | Dina Belova proposed openstack/requirements: Change OSprofiler constraints to >=1.0.0 https://review.openstack.org/275426 | 22:25 |
| *** sc68cal has quit IRC | 22:25 | |
| *** sdake has joined #openstack-infra | 22:26 | |
| *** markmcclain has joined #openstack-infra | 22:26 | |
| dims_ | clarkb : thanks for looking into it with fungi | 22:28 |
| anteaya | fungi: ack | 22:29 |
| dims_ | jeblair : dhellmann and i were wondering if there's a way for that job to report failure on the -release irc channel and fungi suggested "irc reporter" for zuul (after zuul v3 gets real) | 22:29 |
| *** zul has joined #openstack-infra | 22:29 | |
| jeblair | AJaeger: when you have a moment, can you look at http://specs.openstack.org/openstack-infra/infra-specs/specs/doc-publishing.html starting at "The openstack-manuals repo builds multiple manuals" and tell me what the current situation looks like there? | 22:30 |
| anteaya | dims_: there is the openstackrecheck bot | 22:30 |
| anteaya | dims_: that is active in -qa, would it do what you want? | 22:30 |
| *** annegentle has quit IRC | 22:30 | |
| clarkb | wait --download runs setup.py | 22:30 |
| *** achanda has quit IRC | 22:30 | |
| clarkb | so that won't work | 22:30 |
| jeblair | dims_: i think it's a swell idea | 22:31 |
| *** e0ne has joined #openstack-infra | 22:31 | |
| *** hashar has quit IRC | 22:31 | |
| SergeyLukjanov | fungi hi, sorry, I've been unable to join todat | 22:31 |
| SergeyLukjanov | fungi hi, sorry, I've been unable to join today | 22:31 |
| fungi | anteaya: the recheck bot watches the gerrit event stream. release job results don't report into gerrit, so it would never see them | 22:32 |
| fungi | at least i think it's watching gerrit | 22:32 |
| anteaya | ah sorry | 22:32 |
| fungi | i need to confirm my recollection now | 22:32 |
| mtreinish | fungi: it is | 22:32 |
| *** samuelBartel has joined #openstack-infra | 22:32 | |
| fungi | confirmed! ;) | 22:32 |
| anteaya | :) | 22:32 |
| clarkb | time to check pip download | 22:33 |
| SergeyLukjanov | fungi, I can do renaming in PT time zone Mon-Fri | 22:33 |
| *** EricGonczer_ has quit IRC | 22:33 | |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: Test for gear disconnect with queued builds https://review.openstack.org/273703 | 22:33 |
| anteaya | SergeyLukjanov: what is PT time? | 22:33 |
| fungi | i'm guessing pst/pdt | 22:34 |
| anteaya | I was guessing the same | 22:34 |
| *** jsavak has quit IRC | 22:34 | |
| anteaya | wanted to confirm | 22:34 |
| fungi | SergeyLukjanov: sounds like next friday (the 12th?) would be good so that we have more root admins around, since this will be our first rename run after the gerrit 2.11 upgrade | 22:34 |
| *** shashank_hegde has quit IRC | 22:35 | |
| fungi | no idea what might need adjusting in our process for 2.11 | 22:35 |
| *** eharney has quit IRC | 22:35 | |
| clarkb | ha pip download has no no use wheel option | 22:36 |
| clarkb | pip install runs setup.py even with --download | 22:36 |
| SergeyLukjanov | (me calculating tz) | 22:36 |
| clarkb | so maybe using pip to hceck this isn't a good idea | 22:36 |
| SergeyLukjanov | fungi, yeah Friday Feb 12 works good for me | 22:36 |
| SergeyLukjanov | anteaya yeah, I mean PST (that is PT/PST) | 22:37 |
| *** zeih has quit IRC | 22:37 | |
| anteaya | SergeyLukjanov: oh okay yeah pacific time | 22:37 |
| SergeyLukjanov | yeah | 22:37 |
| fungi | awesome, i'll bring that up again on tuesday to confirm with everyone | 22:37 |
| SergeyLukjanov | fungi cool, looking forward for it | 22:38 |
| clarkb | fungi: dims_ I thougt pip would make this easy but it builds in too many assumptions around what you want to do with the downloaded files. Our best bet is likely an explicit curl/wget based on the path that things should end up at | 22:38 |
| fungi | which unfortunately is a non-guaranteed implementation detail of pypi | 22:39 |
| fungi | so could change at any time | 22:39 |
| dims_ | right | 22:39 |
| *** thorst has quit IRC | 22:39 | |
| fungi | but may be stable enough to use as a safety harness i guess | 22:39 |
| clarkb | I think the /simple index has been fairly consistent | 22:39 |
| fungi | worst case it falls back to the current state | 22:40 |
| anteaya | SergeyLukjanov: so are you in pacific time permanently now? | 22:40 |
| clarkb | let me whip up a thing for that | 22:40 |
| *** bexelbie_ has joined #openstack-infra | 22:40 | |
| fungi | so if we notice that we're back to running into this set of symptoms again, then we can surmise pypi has changed its /simple structure i guess | 22:41 |
| *** jamielennox|away is now known as jamielennox | 22:41 | |
| *** thorst has joined #openstack-infra | 22:41 | |
| fungi | oh, and you're right, enough stuff depends on /simple that there very well may be an api guarantee around its structure | 22:41 |
| *** zul has quit IRC | 22:42 | |
| *** e0ne has quit IRC | 22:43 | |
| anteaya | did anyone get a chance to check why so very many bluebox nodes are in error state? | 22:44 |
| *** armax_ has joined #openstack-infra | 22:44 | |
| *** annegentle has joined #openstack-infra | 22:44 | |
| *** armax has quit IRC | 22:45 | |
| *** armax_ is now known as armax | 22:45 | |
| fungi | i started looking at the graphs for it when you first mentioned it in here, but have been head-down on other stuff | 22:45 |
| fungi | it looks from grafana like stuff was still running there, but i may have misinterpreted | 22:46 |
| *** thorst has quit IRC | 22:46 | |
| anteaya | I understand | 22:46 |
| fungi | oh, the "error node launch attempts" graph | 22:47 |
| fungi | yeah, that's pretty gnarly | 22:47 |
| zaro | clarkb: was gerrit resarted? | 22:48 |
| clarkb | zaro: I don't think so | 22:49 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Check twine uploads for success https://review.openstack.org/275431 | 22:49 |
| clarkb | I havne't been in a good place to do that with babies and nodepool distracting me | 22:49 |
| zaro | clarkb: is the 503 issue happening now or has it died down? | 22:49 |
| anteaya | zaro: clarkb I haven't seen anyone announce that they did a restart | 22:49 |
| fungi | anteaya: it looks from http://grafana.openstack.org/dashboard/db/nodepool-blue-box like there was some sort of state change in bbox around 22:00 utc too | 22:50 |
| anteaya | zaro: it is pretty constant, if you look at the channel logs | 22:50 |
| dstufft | clarkb: fungi y'all know twine has a --skip-existing thing right? | 22:50 |
| fungi | dstufft: NO! | 22:50 |
| *** bhunter71 has joined #openstack-infra | 22:50 | |
| fungi | how did i miss that? | 22:50 |
| anteaya | fungi: looking at the test nodes graph, that looks interesting | 22:51 |
| *** amitgandhinz has quit IRC | 22:51 | |
| * fungi makes a note to rtfm more often | 22:51 | |
| anteaya | zaro: also people report 502 and 503 after a restart | 22:51 |
| pabelanger | 636693 | 22:51 |
| anteaya | zaro: so not all instances are due to GC | 22:51 |
| dstufft | fungi: it's new as of 1.6 (released 2015-09-14) | 22:52 |
| *** piet has quit IRC | 22:53 | |
| dims_ | fungi : dstufft is one step ahead :) | 22:53 |
| *** piet has joined #openstack-infra | 22:53 | |
| openstackgerrit | Samuel BARTEL proposed openstack-infra/project-config: add project poppy-ui a Horizon Plugin to work with Poppy's CDN API https://review.openstack.org/267806 | 22:53 |
| dstufft | wasn't me! I do very little with twine now adays, that's mostly sigmavirus24_awa :] | 22:53 |
| *** ayoung has joined #openstack-infra | 22:53 | |
| dstufft | fungi: you can't upload a .asc after the fact and you can't reupload a file with an already used name. | 22:54 |
| clarkb | neat, I think we still want to check if the upload succeeded indepdnently so that we can upload the sdist after whl | 22:54 |
| clarkb | but if we add that then we can rerun at will | 22:54 |
| dims_ | thanks sigmavirus24_awa :) | 22:54 |
| dstufft | I think the latest twine always uploads the wheel first if you pass it all in one invocation | 22:54 |
| clarkb | dstufft: we currently do it in two | 22:54 |
| dstufft | yea, also from 1.6.0 -> :feature:`106` Upload wheels first to PyPI | 22:55 |
| *** dimtruck is now known as zz_dimtruck | 22:56 | |
| dstufft | legacy pypi is not so good with transaction management :[ | 22:56 |
| prometheanfire | ianw: for some reason I thought that basename took the full path, not just a string | 22:56 |
| *** lucasagomes_ has quit IRC | 22:56 | |
| clarkb | dstufft: do you know if pypi 500s on the first upload if the second one will be attempted in one invocation? | 22:56 |
| anteaya | I'm still working on a cold and am calling it a night, thanks for the day, hope I was helpful, if not there's always tomorrow | 22:56 |
| clarkb | dstufft: it may be best for us to continue to use two simply to work around that | 22:56 |
| *** dingyichen has joined #openstack-infra | 22:57 | |
| dstufft | clarkb: it does not appear if the second one will be attempted | 22:57 |
| dstufft | https://github.com/pypa/twine/blob/master/twine/commands/upload.py#L102-L129 | 22:57 |
| *** shashank_hegde has joined #openstack-infra | 22:58 | |
| ianw | prometheanfire: yeah, it's just glorified str.split("/")[-1] :) | 22:58 |
| *** tiswanso has quit IRC | 22:59 | |
| mgagne | fungi our new default security group rules will be effective today for new users. thanks for the feedback =) | 23:00 |
| fungi | mgagne: thanks for listening and taking action! | 23:00 |
| *** thorst has joined #openstack-infra | 23:00 | |
| jeblair | this deserves a cookie! | 23:01 |
| jeblair | me gets cookies | 23:01 |
| *** lucasagomes has joined #openstack-infra | 23:03 | |
| smcginnis | anteaya: Thanks! ;) | 23:04 |
| *** dizquierdo has joined #openstack-infra | 23:05 | |
| *** dkranz has quit IRC | 23:05 | |
| *** jamielennox is now known as jamielennox|away | 23:05 | |
| *** thorst has quit IRC | 23:05 | |
| clarkb | I can restart gerrit now if we are ready to do that | 23:05 |
| jeblair | nodepool is very sad | 23:05 |
| zaro | anteaya: those errors indicate that gerrit is too slow to process requests. the slowness can be caused by many things, one of them being the GC. | 23:06 |
| jeblair | probably because of the label deletion | 23:06 |
| zaro | so yes, you might be right | 23:06 |
| prometheanfire | thought I did a git review | 23:07 |
| ianw | jeblair: sad like deleting everything under / sad? | 23:07 |
| prometheanfire | now I did | 23:07 |
| clarkb | jeblair: yes we removed ubuntu-trusty-check | 23:07 |
| clarkb | and the statsd key gen is exploding on it | 23:07 |
| clarkb | a restart should correct it | 23:07 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Add support for OpenRC to dib-init-system https://review.openstack.org/273772 | 23:07 |
| jeblair | i thought i wrote a patch to fix that | 23:07 |
| fungi | clarkb: gerrit restart now seems reasonable | 23:07 |
| clarkb | KeyError: 'nodepool.label.ubuntu-trusty-check.nodes.delete' | 23:07 |
| clarkb | will wait for nodepool to be happy so it is only one thing at a time | 23:08 |
| fungi | jeblair: you may have written a patch to fix that and it's just not reviewed :/ | 23:08 |
| jeblair | heh, i think it landed, however, it may not be running | 23:08 |
| fungi | oh! | 23:08 |
| fungi | easier solution then... nodepool restart time | 23:08 |
| jeblair | https://review.openstack.org/272742 | 23:08 |
| zaro | clarkb: is 'gerrit show-queue' showing a really full queue? | 23:09 |
| *** peristeri has quit IRC | 23:09 | |
| jeblair | rather than try to figure out if it's actually running, which is really hard because of all the local installs i did last week, i'm just going to restart it and if we see it again, take action | 23:09 |
| *** unicell1 has joined #openstack-infra | 23:09 | |
| clarkb | zaro: 5 tasks | 23:10 |
| *** unicell has quit IRC | 23:10 | |
| *** jpr has quit IRC | 23:10 | |
| *** thorst has joined #openstack-infra | 23:10 | |
| jeblair | nope still broken | 23:11 |
| zaro | clarkb: lots of discussion point to gerrit not being able to process queue and then return 503. but loooks like that's definately not case ehre. | 23:11 |
| *** thorst_ has joined #openstack-infra | 23:11 | |
| clarkb | zaro: I think what happens is gerrit goes to allocate more memory and that triggers long and slow gc while that happens apache times out and we 500 | 23:11 |
| clarkb | zaro: the more requests we have coming in the more things need memory the worse it gets | 23:11 |
| clarkb | zaro: and apache will cache that 500 error for an interval as well so once it happens many people get that result | 23:12 |
| *** salv-orlando has quit IRC | 23:13 | |
| *** salv-orlando has joined #openstack-infra | 23:13 | |
| prometheanfire | ianw: don't want to exit 1 though since that will break other OS's | 23:14 |
| clarkb | jeblair: you ok with me restarting gerrit with nodepool unhappy or should I wait? | 23:15 |
| *** thorst has quit IRC | 23:15 | |
| jeblair | clarkb: go for it | 23:15 |
| clarkb | ok restarting now | 23:16 |
| ianw | prometheanfire: hmm, won't that happen anyway? | 23:16 |
| jeblair | clarkb: i just did "nodepool delete 7616039 --now" to delete the one node in that image type | 23:16 |
| prometheanfire | ah, you are right, after the edit to remove the other init systems that will happen | 23:16 |
| zaro | clarkb: i think gerrit just died | 23:16 |
| prometheanfire | ianw: want me to add the other init's back and add a *) at the end? | 23:16 |
| clarkb | zaro: I just restarted it | 23:16 |
| prometheanfire | I think it could just be debian) ;; | 23:17 |
| prometheanfire | should be obvious as a noop I hope | 23:17 |
| ianw | prometheanfire: i think so, just have them as blanks. if we ever get another init system (god i hope not) it should break with a helpful message at that | 23:17 |
| *** thorst_ has quit IRC | 23:17 | |
| *** tzn has quit IRC | 23:18 | |
| prometheanfire | fedora/rh might want this functionality as they don't enable services by default iitv | 23:18 |
| ianw | no, they don't but i think most places we add the service, we enable it there | 23:18 |
| *** thorst has joined #openstack-infra | 23:18 | |
| clarkb | anteaya: the bluebox issues may be related to floating ip exhaustion | 23:18 |
| ianw | glean for example | 23:18 |
| *** e0ne has joined #openstack-infra | 23:19 | |
| mgagne | clarkb you can speed up recovery with "ProxyPass retry=3" in proxy config for gerrit | 23:19 |
| prometheanfire | ah, guess I just generified all those specifics | 23:19 |
| *** baoli has quit IRC | 23:20 | |
| *** dizquierdo has quit IRC | 23:20 | |
| mgagne | clarkb works when backend is down then back up, I'm not sure what Gerrit actually returns in the current use case | 23:20 |
| clarkb | mgagne: I am not sure we want to increase the number of requests once gerrit has decided it needs a break | 23:20 |
| *** rfolco has quit IRC | 23:20 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Add support for OpenRC to dib-init-system https://review.openstack.org/273772 | 23:20 |
| prometheanfire | ianw: try that one on for size | 23:20 |
| * mgagne hammers F5 | 23:20 | |
| clarkb | I worry we will stampede it with requests when it is least happy so the current behacior wile annoying is probably sane | 23:21 |
| *** regXboi has quit IRC | 23:21 | |
| *** ok_delta has joined #openstack-infra | 23:21 | |
| *** ok___delta has joined #openstack-infra | 23:21 | |
| *** shashank_hegde has quit IRC | 23:21 | |
| clarkb | we have 55 floating ips allocated in bluebox | 23:22 |
| clarkb | max servers is 39 | 23:22 |
| clarkb | we likely have a leak, I will work on cleaning that up | 23:23 |
| ianw | prometheanfire: before jenkins gets to far, want to just s/DEST=// | 23:23 |
| prometheanfire | thought I did that there | 23:23 |
| prometheanfire | oh, missed it | 23:23 |
| prometheanfire | done | 23:23 |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Add support for OpenRC to dib-init-system https://review.openstack.org/273772 | 23:23 |
| fungi | clarkb: thanks, that might explain the error conditions in the bbox nodepool graphs | 23:25 |
| *** xyang1 has quit IRC | 23:25 | |
| ianw | prometheanfire: ok, LGTM, let jenkins do its thing | 23:25 |
| prometheanfire | just verified ) ;; works as noop too | 23:26 |
| *** bexelbie has quit IRC | 23:26 | |
| *** bexelbie_ is now known as bexelbie | 23:26 | |
| fungi | amusing. i need to install bzr to be able to run pip's unit tests | 23:27 |
| *** thorst has quit IRC | 23:27 | |
| prometheanfire | fungi: try fossil out | 23:27 |
| fungi | the most amusing part is i'm not sure how i didn't already have bzr installed | 23:27 |
| prometheanfire | we were using cvs til about 5-6 months ago | 23:28 |
| *** e0ne has quit IRC | 23:28 | |
| fungi | prometheanfire: i don't actually need to interact with bzr. pip is maintained via git. but one of its unit (i use the term loosely) tests wants to create a test package and does something with bzr during the course of that | 23:29 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Drop I18N Translation test meeting https://review.openstack.org/275045 | 23:29 |
| prometheanfire | lol | 23:29 |
| openstackgerrit | Merged openstack-infra/system-config: Added AFS keytab to wheel slave https://review.openstack.org/273206 | 23:29 |
| fungi | so when running tox -e py27 on a commit for pip, it broke complaining tha it couldn't run bzr | 23:29 |
| *** annegentle has quit IRC | 23:30 | |
| dstufft | fungi: I don't even run the tests locally :[ | 23:30 |
| dstufft | unless something is failing | 23:30 |
| fungi | oh, heh | 23:30 |
| dstufft | then I'll do tox -e pywhatever -- -k <testname> | 23:30 |
| dstufft | to run just that test | 23:30 |
| dstufft | not gonna trick me into installing bzr that easily | 23:30 |
| fungi | well, in this case i'm trying to tweak my pull request and confirm tests actually pass | 23:30 |
| openstackgerrit | Merged openstack-infra/project-config: Make murano tempest job voting and add it to gate pipeline https://review.openstack.org/275191 | 23:31 |
| openstackgerrit | Merged openstack-infra/project-config: Add windmill-jobs for ansible-role-zuul https://review.openstack.org/275419 | 23:31 |
| fungi | oh, i get you/ you're saying just try rerunning the tests that didn't pass in the ci | 23:31 |
| openstackgerrit | Merged openstack-infra/project-config: Test translation imports https://review.openstack.org/274359 | 23:31 |
| fungi | meh, now i actually want to see what's involved in running them ;) | 23:31 |
| mgagne | any updates/news on image update random slowness? | 23:32 |
| openstackgerrit | Merged openstack-infra/system-config: Added wheel hosting for AFS mirror slave https://review.openstack.org/270979 | 23:32 |
| openstackgerrit | Merged openstack-infra/system-config: Add AFS and Admin keytab to release slave. https://review.openstack.org/273538 | 23:32 |
| *** krotscheck1 has joined #openstack-infra | 23:32 | |
| dstufft | fungi: you might want pytest-xdist and -n 8 or so | 23:32 |
| dstufft | otherwise they take a long time | 23:32 |
| *** gordc has quit IRC | 23:33 | |
| clarkb | floating IPs are deleting | 23:33 |
| dstufft | I think over an hour if you run the serially | 23:33 |
| dstufft | maybe less now adays, we started doing some things better | 23:33 |
| krotscheck1 | Wow, wheel things are merging! | 23:33 |
| fungi | dstufft: cool. i'll check the stats and find out. i have a lot of balls in the air anyway so no biggie if i work on something else while this runs | 23:33 |
| fungi | krotscheck1: yep, brace for impact! | 23:33 |
| clarkb | mgagne: I am not sure I know specificaly what you refer to but we need to get the change that detects gearman client disconnects in and we need to figure out rax uploads once our token expires | 23:34 |
| * krotscheck1 holds on tight! | 23:34 | |
| krotscheck1 | ok, so all the puppet things are there. has te slave been built yet? | 23:35 |
| mgagne | clarkb I think jeblair has details. It was a problem he had last week IIRC with random upload speed at OVH or some other providers. | 23:35 |
| *** lucasagomes has quit IRC | 23:35 | |
| *** shashank_hegde has joined #openstack-infra | 23:35 | |
| prometheanfire | hold onto your butts | 23:36 |
| mgagne | clarkb I'm interested in the details because we found similar symptoms on our side. I wish to check if it's caused by the same problem (which we don't know the source yet) | 23:36 |
| jroll | prometheanfire: http://butt.holdings | 23:38 |
| *** samuelBartel has quit IRC | 23:39 | |
| prometheanfire | jroll: this makes me sooo happy | 23:39 |
| jroll | :D | 23:39 |
| jroll | I'll show you my other favorite abuse of new TLDs on friday :P | 23:39 |
| *** lucasagomes has joined #openstack-infra | 23:40 | |
| clarkb | mgagne: for that I am not sure I must've been traveling when it was discussed | 23:41 |
| *** dizquierdo has joined #openstack-infra | 23:42 | |
| *** bexelbie- has joined #openstack-infra | 23:44 | |
| sridhar_ram1 | infra team - looking for you help. dvsm functional jobs are failing for the tacker project, see for e.g. https://review.openstack.org/#/c/231280/ | 23:44 |
| *** shashank_hegde has quit IRC | 23:44 | |
| *** tzn has joined #openstack-infra | 23:44 | |
| sridhar_ram1 | This seems to have started three days back after the merge of this project-config check-requirements patchset to tacker-horizon https://review.openstack.org/#/c/271980/ | 23:44 |
| *** bexelbie has quit IRC | 23:44 | |
| *** bexelbie- is now known as bexelbie | 23:44 | |
| *** tzn has quit IRC | 23:45 | |
| sridhar_ram1 | The actual error looks like "Could not satisfy constraints for 'horizon': installation from path or url cannot be constrained to a version" | 23:45 |
| *** Sukhdev has quit IRC | 23:45 | |
| *** ybathia has quit IRC | 23:45 | |
| clarkb | yup thats the problem | 23:46 |
| sridhar_ram1 | clarkb: can you parse this mesg for me please ? :) | 23:46 |
| clarkb | sridhar_ram1: https://git.openstack.org/cgit/openstack/tacker-horizon/tree/test-requirements.txt#n12 is what it is mad about | 23:48 |
| clarkb | pip is using constraints when installing and apparently lines like that are not allowed | 23:48 |
| *** sdake has quit IRC | 23:48 | |
| *** shashank_hegde has joined #openstack-infra | 23:49 | |
| sridhar_ram1 | clarkb: I see. So those two tarballs lines need to go ? | 23:49 |
| *** Sukhdev has joined #openstack-infra | 23:49 | |
| clarkb | yes | 23:49 |
| sridhar_ram1 | clarkb: cool.. will take care. Thanks a lot! | 23:49 |
| *** sridhar_ram1 is now known as sridhar_ram | 23:50 | |
| *** bpokorny_ has joined #openstack-infra | 23:50 | |
| fungi | krotscheck1: no slave yet, we're getting there though | 23:50 |
| armax | anteaya: ping | 23:50 |
| *** ok_delta has quit IRC | 23:50 | |
| fungi | armax: i think she retired for the night--wasn't feeling well | 23:51 |
| *** ok___delta has quit IRC | 23:51 | |
| armax | fungi: ah, thanks for letting me know | 23:51 |
| armax | I’ll try tomorrow | 23:51 |
| openstackgerrit | James E. Blair proposed openstack-infra/git-restack: Correct metadata URLs https://review.openstack.org/275444 | 23:51 |
| openstackgerrit | James E. Blair proposed openstack-infra/git-review: Correct metadata URLs https://review.openstack.org/275445 | 23:52 |
| *** bpokorny has quit IRC | 23:53 | |
| *** boris-42 has quit IRC | 23:53 | |
| fungi | dstufft: on a successful run, i got "613 passed, 10 skipped, 1 xfailed, 1 xpassed, 1 warnings in 1371.68 seconds" | 23:54 |
| fungi | dstufft: so really didn't take very long | 23:54 |
| dstufft | 22 minutes is pretty long tbh | 23:55 |
| *** jpr has joined #openstack-infra | 23:56 | |
| *** gildub has joined #openstack-infra | 23:58 | |
| *** dizquierdo has quit IRC | 23:59 | |
| fungi | faster than nova's unit tests | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!