| *** apoorvad has joined #openstack-infra | 00:00 | |
| mordred | it's also more efficient to just hit neutron than to hit nova and have nova hit neutron for you | 00:00 |
|---|---|---|
| clarkb | its the same number of calls I thought | 00:02 |
| clarkb | (I don't think efficiency changes) | 00:02 |
| clarkb | at least in the make me a server case | 00:02 |
| clarkb | for any pure proxied requests ya it will be an extra request as part of the proxy | 00:02 |
| mordred | right. the efficiency is becuase of the proxy call - it's not an extra client call | 00:03 |
| mordred | the reason the neutron fip is better is you have one less leak/race case | 00:03 |
| clarkb | (I'm just still gurmpy that to use hpcloud I had to make a subnet, a netowrk, configure dhcp, make a router, attach network to router, and boot vm on particular network) | 00:03 |
| clarkb | (instead of boot me a VM) | 00:03 |
| clarkb | (one of those is far more efficient to the user than the other) | 00:03 |
| mordred | since the first call is a query, followed by a create/attach - as opposed to a create then an attach | 00:03 |
| mordred | clarkb: right. but you're comparing apples and organges, as much as I agree with you about the router nonsense | 00:04 |
| clarkb | mordred: I think I am trying to say that we shouldn't force clouds to expose a broken user experience if they provide something that just works | 00:04 |
| mordred | clarkb: you don't do that on every boot - and to be fair to HP - that's not what your expeience is supposed to be, we got some accounts that were created wrong | 00:04 |
| *** otter768 has joined #openstack-infra | 00:05 | |
| mordred | clarkb: but I didn't say anything about exposing a floating-ip/router model | 00:06 |
| *** slogan621_ has joined #openstack-infra | 00:06 | |
| clarkb | mordred: thats the only reason you would want neutron api direct access right? | 00:06 |
| mordred | clarkb: I said "if they are runing neutron, which they darned well should be because it's a new cloud, then they should allow me, as an openstack user, to access the APIs if I choose" | 00:06 |
| *** rbrndt has quit IRC | 00:07 | |
| slogan621_ | Does anyone know if there are restrictions on who can present a session at OpenStack Summit - contributors or open stack "member" companies only, or just anyone with a good presentation? | 00:07 |
| mordred | slogan621_: anyone with a good presentation | 00:07 |
| *** sfinucan has quit IRC | 00:07 | |
| clarkb | and selection process is public voting which is fed into track specific selection groups | 00:07 |
| clarkb | having the most vote sdoesn't necessary mean your talk is accepted (the voting is advisory) | 00:08 |
| *** weshay_xchat has quit IRC | 00:08 | |
| mordred | clarkb: the thing that kills me in supporting clouds is them being different, not any of the specific API interaction qualities. a new cloud with neutron without API would be yet-another snowflake. I do not care at all what the reasons for it being a snowflake are - it would be a snowflake, and as such, it should be discouraged at all costs | 00:08 |
| slogan621_ | mordred: thanks! | 00:09 |
| mordred | clarkb: rackspace have "GREAT" reasons for every single way in which they are different that all of the rest of the clouds out there | 00:09 |
| clarkb | sure I just know as a use rI never ever ever use the neutron api | 00:09 |
| mordred | clarkb: and it does not chagne that fact that they are different in every conceivable way for every other openstack cloud that exists | 00:09 |
| clarkb | I think I used it once | 00:09 |
| clarkb | then I quickly went and used horizon | 00:09 |
| mordred | clarkb: I use it all the time | 00:09 |
| JayF | It's reasonable to say you're special, mordred :D | 00:09 |
| mordred | JayF: :) | 00:09 |
| mordred | JayF: does that make me a cloud feature? | 00:10 |
| clarkb | beause the router + subnet + network + dhcp + floating ip runaround was not documented and the client wasn't helping me | 00:10 |
| clarkb | thankfully horizon did help | 00:10 |
| clarkb | and actually makes you fill i nall the details that you need for workingness | 00:10 |
| JayF | mordred: no, you're just one of those dang snowflake users | 00:10 |
| mtreinish | I've used it, but that's kinda because I had to | 00:10 |
| JayF | mordred: why can't you want the same features as the masses? /s | 00:10 |
| mordred | JayF: I do want the same features as the masses | 00:10 |
| JayF | mordred: seriously just joking and have to go :) | 00:10 |
| mordred | JayF: :) | 00:10 |
| clarkb | and my only other exposure to clouds that use neutron is that they just work because they don't expose neutron to me | 00:11 |
| mtreinish | can't talk to whether it's documented in the client or not though, since I was writing a new one in the process | 00:11 |
| JayF | lets just say I'm personally very glad to be running a private API | 00:11 |
| clarkb | (ovh and rax) | 00:11 |
| JayF | (ironic api) | 00:11 |
| clarkb | so as a usre my prefernece is that model | 00:11 |
| clarkb | where Vms just have networking and nova things just work | 00:11 |
| mordred | clarkb: yes. I don't disagree with that | 00:11 |
| mordred | clarkb: again, I agree about direct attached IPs | 00:11 |
| mordred | clarkb: I'm just saying - expose the APIs becuase you are running them | 00:11 |
| mordred | clarkb: and not exposing them is rude | 00:11 |
| mordred | clarkb: that ovh exposes them does not break your experience of that cloud | 00:12 |
| mordred | clarkb: that rax hides them _does_ break mine | 00:12 |
| *** Hink has quit IRC | 00:12 | |
| mordred | so just expose them - it breaks literally no things | 00:13 |
| mtreinish | clarkb: the way I like to think about mordred's point is looking at rax trove. It's a cloud that only exposes the trove api | 00:13 |
| mtreinish | everything else is hidden and it makes a bunch of things difficult or impossible | 00:13 |
| mtreinish | which has annoyed me to know end with the subunit2sql stuff | 00:14 |
| clarkb | mtreinish: ya though in fairness to rax thats because trove has access to the rabbit bus iirc | 00:14 |
| clarkb | so thats more trove's fault | 00:14 |
| mordred | ++ | 00:14 |
| *** xyang1 has quit IRC | 00:15 | |
| *** sileht has quit IRC | 00:15 | |
| mtreinish | clarkb: heh, yep | 00:15 |
| lifeless | I still don't get troves value proposition | 00:15 |
| lifeless | its entirely identical to running your own mysql, just with less power and flexability | 00:16 |
| fungi | lifeless: it abstracts away all the inconvenience of apt-get install mysql-server | 00:16 |
| *** woodster_ has quit IRC | 00:16 | |
| lifeless | fungi: such a terrible inconvenience | 00:16 |
| * fungi is joking of course... a little | 00:16 | |
| mordred | the GCE model with container service is great | 00:16 |
| mordred | you click "I want a kubernets cluster" | 00:16 |
| clarkb | I think the theory is you would get backups, and performance tuning all somewhat done for you | 00:16 |
| lifeless | like, if it was cheaper | 00:16 |
| mordred | and itmakes you one | 00:17 |
| mordred | in your tenant | 00:17 |
| mordred | on vms | 00:17 |
| lifeless | or faster | 00:17 |
| mordred | that you can ssh in to | 00:17 |
| *** chlong has joined #openstack-infra | 00:17 | |
| mordred | but it manages the stuff for you | 00:17 |
| mordred | so what you did was "give me a cluster" | 00:17 |
| lifeless | or came with hard stuff - e.g. HA clustering built in | 00:17 |
| lifeless | that would compensate | 00:17 |
| clarkb | lifeless: yes that | 00:17 |
| clarkb | lifeless: which it does do | 00:17 |
| fungi | i think the idea is that you can do it cheaper (again as clark points out if you take advantage of enough of the optional features) than putting mysqld on a similarly sized nova instance | 00:17 |
| fungi | not sure if they price it that way though | 00:18 |
| lifeless | I thought it was strictly 1:1 though | 00:18 |
| lifeless | no shared tenants | 00:18 |
| lifeless | which implies no less resource use | 00:18 |
| lifeless | it could only be cheaper if nova is overpriced... | 00:18 |
| *** otter768 has quit IRC | 00:19 | |
| fungi | well, it gives them some insight into what that instance's usage patterns are, which in aggregate makes it cheaper to support for things like peak sizing and capacity planning | 00:19 |
| *** otter768 has joined #openstack-infra | 00:19 | |
| *** akshai has quit IRC | 00:19 | |
| fungi | because it's an instance which, by design, can only do one thing: handle database queries and store some very specific kinds of files | 00:20 |
| fungi | rather than a general-purpose nova instance where you could be doing just about anything on it | 00:20 |
| *** sridhar_ram has quit IRC | 00:21 | |
| fungi | again, that could be complete bunk, but these are the value propositions at least | 00:22 |
| *** sridhar_ram has joined #openstack-infra | 00:22 | |
| *** niedbalski has quit IRC | 00:23 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Separate concerns in pypi_mirror.pp https://review.openstack.org/253236 | 00:26 |
| *** sileht has joined #openstack-infra | 00:26 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 00:26 | |
| *** Swami has quit IRC | 00:27 | |
| *** devananda has quit IRC | 00:28 | |
| *** amotoki has joined #openstack-infra | 00:28 | |
| *** vilobhmm11 has quit IRC | 00:30 | |
| *** vilobhmm11 has joined #openstack-infra | 00:30 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Separate concerns in pypi_mirror.pp https://review.openstack.org/253236 | 00:31 |
| *** sdake has joined #openstack-infra | 00:31 | |
| *** ivar-laz_ has joined #openstack-infra | 00:34 | |
| *** ujuc has quit IRC | 00:35 | |
| *** yamamoto has quit IRC | 00:36 | |
| *** lykinsbd has joined #openstack-infra | 00:37 | |
| *** ivar-lazzaro has quit IRC | 00:38 | |
| openstackgerrit | Dong Ma proposed openstack-infra/jenkins-job-builder: Add support for cucumber reports plugin https://review.openstack.org/251489 | 00:38 |
| *** amuller has quit IRC | 00:40 | |
| *** amotoki has quit IRC | 00:44 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Separate concerns in pypi_mirror.pp https://review.openstack.org/253236 | 00:46 |
| *** lykinsbd_ has joined #openstack-infra | 00:46 | |
| *** rkukura has quit IRC | 00:47 | |
| *** rkukura has joined #openstack-infra | 00:48 | |
| *** lykinsbd has quit IRC | 00:49 | |
| *** vilobhmm11 has quit IRC | 00:51 | |
| openstackgerrit | Dong Ma proposed openstack-infra/jenkins-job-builder: Add support for cucumber reports plugin https://review.openstack.org/251489 | 00:52 |
| *** vilobhmm11 has joined #openstack-infra | 00:52 | |
| *** otter768 has quit IRC | 00:52 | |
| *** rhallisey has quit IRC | 00:53 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 00:53 | |
| *** devananda has joined #openstack-infra | 00:54 | |
| *** sridhar_ram has quit IRC | 00:54 | |
| *** Sukhdev has joined #openstack-infra | 00:55 | |
| *** rkukura has quit IRC | 00:58 | |
| *** hichihara has joined #openstack-infra | 00:59 | |
| *** Sukhdev has quit IRC | 00:59 | |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/puppet-ciwatch: Add initial implementation to deploy ciwatch https://review.openstack.org/238606 | 01:00 |
| *** slogan621_ has quit IRC | 01:02 | |
| *** sdake has quit IRC | 01:02 | |
| *** slogan621 has quit IRC | 01:02 | |
| *** eil397 has quit IRC | 01:02 | |
| *** apuimedo has quit IRC | 01:04 | |
| *** apuimedo has joined #openstack-infra | 01:05 | |
| *** sridhar_ram has joined #openstack-infra | 01:06 | |
| *** rkukura has joined #openstack-infra | 01:07 | |
| *** doug-fish has quit IRC | 01:08 | |
| *** barnaby has quit IRC | 01:11 | |
| *** spzala has joined #openstack-infra | 01:12 | |
| openstackgerrit | will soula proposed openstack-infra/jenkins-job-builder: Fix pollscm backwards compatiblity https://review.openstack.org/252069 | 01:12 |
| *** gildub has quit IRC | 01:14 | |
| *** eil397 has joined #openstack-infra | 01:14 | |
| *** spzala has quit IRC | 01:14 | |
| *** sridhar_ram has quit IRC | 01:15 | |
| *** rkukura has quit IRC | 01:17 | |
| *** pvaneck has quit IRC | 01:17 | |
| *** sputnik13 has quit IRC | 01:18 | |
| *** yamamoto has joined #openstack-infra | 01:21 | |
| openstackgerrit | Dong Ma proposed openstack-infra/jenkins-job-builder: Correct the entry points of cucumber-testresult https://review.openstack.org/253282 | 01:22 |
| *** apuimedo has quit IRC | 01:22 | |
| *** apuimedo has joined #openstack-infra | 01:23 | |
| *** barnaby has joined #openstack-infra | 01:27 | |
| *** apuimedo has quit IRC | 01:27 | |
| *** apuimedo has joined #openstack-infra | 01:28 | |
| *** mriedem has joined #openstack-infra | 01:29 | |
| *** gildub has joined #openstack-infra | 01:30 | |
| *** mtanino has quit IRC | 01:32 | |
| *** baoli_ has quit IRC | 01:34 | |
| *** hdd has quit IRC | 01:34 | |
| *** zhenguo has joined #openstack-infra | 01:37 | |
| *** iwamoto has joined #openstack-infra | 01:41 | |
| *** rkukura has joined #openstack-infra | 01:41 | |
| openstackgerrit | Ramy Asselin proposed openstack-infra/elastic-recheck: Make Elastic Recheck Watch more reusable https://review.openstack.org/251515 | 01:42 |
| openstackgerrit | Masayuki Igawa proposed openstack-infra/subunit2sql: Add docstrings of test_ids https://review.openstack.org/252223 | 01:42 |
| *** apuimedo has quit IRC | 01:42 | |
| *** apuimedo has joined #openstack-infra | 01:43 | |
| *** dangers_away has quit IRC | 01:46 | |
| *** apuimedo has quit IRC | 01:48 | |
| *** apuimedo has joined #openstack-infra | 01:48 | |
| *** EricGonczer_ has joined #openstack-infra | 01:49 | |
| *** gyee has quit IRC | 01:50 | |
| *** apuimedo has quit IRC | 01:53 | |
| *** apuimedo has joined #openstack-infra | 01:55 | |
| *** Daisy has joined #openstack-infra | 01:56 | |
| *** lykinsbd_ has quit IRC | 01:56 | |
| *** tiswanso has joined #openstack-infra | 01:57 | |
| *** julim has joined #openstack-infra | 01:57 | |
| *** tiswanso has quit IRC | 01:57 | |
| openstackgerrit | Ramy Asselin proposed openstack-infra/elastic-recheck: Make Elastic Recheck Watch more reusable https://review.openstack.org/251515 | 01:57 |
| *** tiswanso has joined #openstack-infra | 01:58 | |
| *** eil397 has quit IRC | 01:58 | |
| *** salv-orlando has joined #openstack-infra | 02:02 | |
| *** rkukura has quit IRC | 02:04 | |
| *** apuimedo has quit IRC | 02:04 | |
| *** apuimedo has joined #openstack-infra | 02:05 | |
| openstackgerrit | Sachi King proposed openstack-dev/pbr: WIP: Handle markers to support sdist on pip < 6 https://review.openstack.org/253289 | 02:06 |
| *** sdake has joined #openstack-infra | 02:06 | |
| *** salv-orlando has quit IRC | 02:07 | |
| *** apuimedo has quit IRC | 02:09 | |
| *** apuimedo has joined #openstack-infra | 02:10 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/requirements: add graphviz library https://review.openstack.org/253290 | 02:11 |
| *** amotoki has joined #openstack-infra | 02:13 | |
| *** apuimedo has quit IRC | 02:14 | |
| *** rkukura has joined #openstack-infra | 02:15 | |
| *** rkukura has quit IRC | 02:15 | |
| *** sabeen has joined #openstack-infra | 02:16 | |
| *** apuimedo has joined #openstack-infra | 02:16 | |
| *** EricGonczer_ has quit IRC | 02:20 | |
| *** darrenc is now known as darrenc_afk | 02:23 | |
| *** bpokorny_ has joined #openstack-infra | 02:23 | |
| *** rajinir_ has quit IRC | 02:25 | |
| *** gongysh_ has joined #openstack-infra | 02:25 | |
| *** sdake has quit IRC | 02:26 | |
| *** bpokorny has quit IRC | 02:27 | |
| *** bpokorny_ has quit IRC | 02:28 | |
| *** apuimedo has quit IRC | 02:28 | |
| *** apuimedo has joined #openstack-infra | 02:30 | |
| *** signed8bit_ZZZzz is now known as signed8bit | 02:33 | |
| *** sdake has joined #openstack-infra | 02:36 | |
| *** markvoelker has quit IRC | 02:40 | |
| *** apuimedo has quit IRC | 02:40 | |
| *** apuimedo has joined #openstack-infra | 02:41 | |
| *** yonglihe has quit IRC | 02:41 | |
| *** baoli has joined #openstack-infra | 02:44 | |
| *** apuimedo has quit IRC | 02:45 | |
| *** r1chardj0n3s has joined #openstack-infra | 02:46 | |
| openstackgerrit | Richard Jones proposed openstack-infra/infra-specs: Releasing xstatic packages https://review.openstack.org/253296 | 02:46 |
| *** apuimedo has joined #openstack-infra | 02:47 | |
| *** signed8bit has quit IRC | 02:47 | |
| *** apoorvad has quit IRC | 02:47 | |
| r1chardj0n3s | lifeless ^^ (also mordred, dhellmann, anyone else who cares about releases) | 02:48 |
| *** krabador has joined #openstack-infra | 02:48 | |
| *** unicell1 has quit IRC | 02:49 | |
| *** darrenc_afk is now known as darrenc | 02:50 | |
| *** otter768 has joined #openstack-infra | 02:53 | |
| *** apuimedo has quit IRC | 02:54 | |
| *** apuimedo has joined #openstack-infra | 02:55 | |
| *** otter768 has quit IRC | 02:58 | |
| *** akshai has joined #openstack-infra | 02:59 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add tests to check migrations and models match https://review.openstack.org/253301 | 02:59 |
| openstackgerrit | Masayuki Igawa proposed openstack-infra/subunit2sql: Make graph.py just a module https://review.openstack.org/253302 | 03:00 |
| *** annegentle has quit IRC | 03:01 | |
| *** annegentle has joined #openstack-infra | 03:02 | |
| *** akshai has quit IRC | 03:03 | |
| *** akshai has joined #openstack-infra | 03:04 | |
| *** annegentle has quit IRC | 03:07 | |
| *** gongysh_ has quit IRC | 03:10 | |
| *** apuimedo has quit IRC | 03:12 | |
| *** ivar-laz_ has quit IRC | 03:12 | |
| *** apuimedo has joined #openstack-infra | 03:12 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Enable db api tests on all backends https://review.openstack.org/253304 | 03:13 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Remove group regex from testr.conf https://review.openstack.org/253305 | 03:13 |
| *** ivar-lazzaro has joined #openstack-infra | 03:15 | |
| red_trela | Once I created review requests to add accessbot, gerritbot and eavesdrop to a channel, do I need to add specific reviewers or write an email or something? Or do you guys look at all the reviews eventually? Different teams seem to have different ways, just trying to make sure it's not overlooked :) | 03:15 |
| *** rlandy has quit IRC | 03:15 | |
| *** apuimedo has quit IRC | 03:17 | |
| *** rguillebert has quit IRC | 03:17 | |
| *** apuimedo has joined #openstack-infra | 03:18 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add tests to check migrations and models match https://review.openstack.org/253301 | 03:20 |
| *** rkukura has joined #openstack-infra | 03:20 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Remove group regex from testr.conf https://review.openstack.org/253305 | 03:20 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Enable db api tests on all backends https://review.openstack.org/253304 | 03:20 |
| *** vilobhmm11 has quit IRC | 03:22 | |
| *** vilobhmm11 has joined #openstack-infra | 03:22 | |
| *** rkukura has quit IRC | 03:23 | |
| *** vilobhmm11 has quit IRC | 03:23 | |
| *** apuimedo has quit IRC | 03:23 | |
| *** yamahata has quit IRC | 03:23 | |
| *** apuimedo has joined #openstack-infra | 03:24 | |
| *** hdd has joined #openstack-infra | 03:24 | |
| *** ivar-lazzaro has quit IRC | 03:25 | |
| *** btully has joined #openstack-infra | 03:28 | |
| *** apuimedo has quit IRC | 03:29 | |
| *** Piet has joined #openstack-infra | 03:31 | |
| *** apuimedo has joined #openstack-infra | 03:31 | |
| *** btully has quit IRC | 03:35 | |
| *** krabador has quit IRC | 03:36 | |
| *** hichihara has quit IRC | 03:36 | |
| *** btully has joined #openstack-infra | 03:36 | |
| *** apuimedo has quit IRC | 03:38 | |
| *** apuimedo has joined #openstack-infra | 03:38 | |
| *** moravec1 has joined #openstack-infra | 03:39 | |
| *** olaph has quit IRC | 03:41 | |
| *** Daisy has quit IRC | 03:44 | |
| *** apuimedo has quit IRC | 03:45 | |
| *** flwang1 has quit IRC | 03:46 | |
| *** apuimedo has joined #openstack-infra | 03:47 | |
| openstackgerrit | Richard Jones proposed openstack-infra/infra-specs: Releasing xstatic packages https://review.openstack.org/253296 | 03:48 |
| *** btully has quit IRC | 03:49 | |
| *** olaph has joined #openstack-infra | 03:51 | |
| *** e0ne_ has joined #openstack-infra | 03:52 | |
| *** bpokorny has joined #openstack-infra | 03:53 | |
| *** red_trela has quit IRC | 03:57 | |
| *** btully has joined #openstack-infra | 03:58 | |
| *** apuimedo has quit IRC | 03:58 | |
| *** apuimedo has joined #openstack-infra | 04:00 | |
| *** links has joined #openstack-infra | 04:00 | |
| *** e0ne_ has quit IRC | 04:00 | |
| *** red_trela| has joined #openstack-infra | 04:00 | |
| *** btully has quit IRC | 04:05 | |
| *** ramineni has joined #openstack-infra | 04:05 | |
| *** red_trela| is now known as red_trela | 04:06 | |
| openstackgerrit | Kanagaraj Manickam proposed openstack/requirements: Adds python-monascaclient in global requirements https://review.openstack.org/252859 | 04:07 |
| *** ramineni1 has joined #openstack-infra | 04:07 | |
| *** baoli has quit IRC | 04:08 | |
| *** hdd has quit IRC | 04:08 | |
| *** stevemar has quit IRC | 04:09 | |
| *** ramineni has quit IRC | 04:09 | |
| *** btully has joined #openstack-infra | 04:10 | |
| *** ramineni1 has quit IRC | 04:11 | |
| *** e0ne has joined #openstack-infra | 04:12 | |
| *** stevemar_znc has joined #openstack-infra | 04:12 | |
| *** sdake has quit IRC | 04:12 | |
| *** akshai has quit IRC | 04:16 | |
| *** akshai has joined #openstack-infra | 04:16 | |
| *** e0ne has quit IRC | 04:16 | |
| openstackgerrit | Merged openstack-infra/subunit2sql: Add docstrings of test_ids https://review.openstack.org/252223 | 04:19 |
| *** links has quit IRC | 04:23 | |
| openstackgerrit | Merged openstack-infra/subunit2sql: Make graph.py just a module https://review.openstack.org/253302 | 04:24 |
| *** akshai has quit IRC | 04:26 | |
| *** mriedem has quit IRC | 04:27 | |
| *** fawadkhaliq has joined #openstack-infra | 04:28 | |
| *** sdake has joined #openstack-infra | 04:32 | |
| *** hdd has joined #openstack-infra | 04:34 | |
| openstackgerrit | Anusha Ramineni proposed openstack-infra/project-config: Congress: Add different gate job for stable branch https://review.openstack.org/250598 | 04:35 |
| *** apuimedo has quit IRC | 04:40 | |
| *** apuimedo has joined #openstack-infra | 04:40 | |
| openstackgerrit | Richard Jones proposed openstack-infra/infra-specs: Releasing xstatic packages https://review.openstack.org/253296 | 04:41 |
| *** fawadkhaliq has quit IRC | 04:42 | |
| *** bpokorny has quit IRC | 04:47 | |
| *** apuimedo has quit IRC | 04:47 | |
| *** btully has quit IRC | 04:47 | |
| *** apuimedo has joined #openstack-infra | 04:48 | |
| *** ilyashakhat has quit IRC | 04:49 | |
| *** ilyashakhat has joined #openstack-infra | 04:49 | |
| *** salv-orlando has joined #openstack-infra | 04:51 | |
| *** filler has quit IRC | 04:51 | |
| *** f1ller has joined #openstack-infra | 04:53 | |
| *** otter768 has joined #openstack-infra | 04:54 | |
| *** hdd has quit IRC | 04:56 | |
| *** salv-orlando has quit IRC | 04:56 | |
| *** otter768 has quit IRC | 04:59 | |
| *** apuimedo has quit IRC | 05:00 | |
| *** apuimedo has joined #openstack-infra | 05:01 | |
| *** rfolco has quit IRC | 05:02 | |
| *** annegentle has joined #openstack-infra | 05:02 | |
| *** Daisy has joined #openstack-infra | 05:03 | |
| *** fawadkhaliq has joined #openstack-infra | 05:04 | |
| *** Ramanjaneya has joined #openstack-infra | 05:04 | |
| *** bpokorny has joined #openstack-infra | 05:05 | |
| *** mwagner_lap has joined #openstack-infra | 05:05 | |
| *** markvoelker has joined #openstack-infra | 05:05 | |
| *** annegentle has quit IRC | 05:07 | |
| *** amotoki_ has joined #openstack-infra | 05:09 | |
| *** links has joined #openstack-infra | 05:09 | |
| *** markvoelker has quit IRC | 05:09 | |
| *** markvoelker has joined #openstack-infra | 05:10 | |
| *** apuimedo has quit IRC | 05:10 | |
| *** apuimedo has joined #openstack-infra | 05:11 | |
| *** amotoki_ has quit IRC | 05:12 | |
| *** _denisra has quit IRC | 05:13 | |
| *** amotoki has quit IRC | 05:13 | |
| *** [2]Thelo has joined #openstack-infra | 05:15 | |
| *** deepakcs has joined #openstack-infra | 05:15 | |
| *** btully has joined #openstack-infra | 05:15 | |
| *** [1]Thelo has quit IRC | 05:17 | |
| *** amotoki has joined #openstack-infra | 05:17 | |
| *** amotoki has quit IRC | 05:17 | |
| deepakcs | Hello, can any cores pls +A https://review.openstack.org/251421 (its has everything else) | 05:19 |
| *** sdake has quit IRC | 05:19 | |
| deepakcs | fungi: anteaya SergeyLukjanov clarkb ^^ pls | 05:20 |
| *** apuimedo has quit IRC | 05:20 | |
| *** apuimedo has joined #openstack-infra | 05:22 | |
| oomichi | dhellmann: lifeless: hi, can you take a look at g-r patch? : https://review.openstack.org/#/c/252739/ | 05:23 |
| oomichi | dhellmann: lifeless: this is a topic of last day on tokyo summit :) | 05:24 |
| *** sabeen3 has joined #openstack-infra | 05:25 | |
| *** hdd has joined #openstack-infra | 05:26 | |
| *** sabeen has quit IRC | 05:28 | |
| *** apuimedo has quit IRC | 05:29 | |
| *** apuimedo has joined #openstack-infra | 05:30 | |
| *** apuimedo has quit IRC | 05:35 | |
| *** apuimedo has joined #openstack-infra | 05:35 | |
| *** maishsk has joined #openstack-infra | 05:38 | |
| *** maishsk has quit IRC | 05:39 | |
| *** Sukhdev has joined #openstack-infra | 05:39 | |
| *** apuimedo has quit IRC | 05:40 | |
| *** apuimedo has joined #openstack-infra | 05:41 | |
| *** tiswanso has quit IRC | 05:41 | |
| *** tiswanso has joined #openstack-infra | 05:41 | |
| *** maishsk has joined #openstack-infra | 05:42 | |
| *** maishsk has quit IRC | 05:42 | |
| *** Sukhdev has quit IRC | 05:43 | |
| *** changbl has quit IRC | 05:45 | |
| *** changbl has joined #openstack-infra | 05:45 | |
| *** markvoelker_ has joined #openstack-infra | 05:47 | |
| *** armax has quit IRC | 05:48 | |
| *** markvoelker has quit IRC | 05:50 | |
| *** apuimedo has quit IRC | 05:50 | |
| *** apuimedo has joined #openstack-infra | 05:50 | |
| *** maishsk has joined #openstack-infra | 05:51 | |
| *** sabeen3 has quit IRC | 05:51 | |
| *** markvoelker_ has quit IRC | 05:51 | |
| *** salv-orlando has joined #openstack-infra | 05:52 | |
| *** maishsk has quit IRC | 05:52 | |
| *** sabeen1 has joined #openstack-infra | 05:52 | |
| *** bpokorny has quit IRC | 05:52 | |
| *** mohankumar has joined #openstack-infra | 05:52 | |
| *** apuimedo has quit IRC | 05:55 | |
| *** salv-orl_ has joined #openstack-infra | 05:56 | |
| *** salv-orlando has quit IRC | 05:56 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Separate concerns in pypi_mirror.pp https://review.openstack.org/253236 | 05:56 |
| *** salv-orl_ has quit IRC | 05:56 | |
| *** apuimedo has joined #openstack-infra | 05:57 | |
| *** salv-orlando has joined #openstack-infra | 05:57 | |
| *** harlowja_at_home has joined #openstack-infra | 05:58 | |
| *** apuimedo has quit IRC | 06:06 | |
| *** apuimedo has joined #openstack-infra | 06:07 | |
| *** aeng has quit IRC | 06:07 | |
| oomichi | lifeless: thanks! | 06:07 |
| *** salv-orlando has quit IRC | 06:10 | |
| *** pc_m has quit IRC | 06:10 | |
| *** pc_m has joined #openstack-infra | 06:11 | |
| *** apuimedo has quit IRC | 06:11 | |
| *** apuimedo has joined #openstack-infra | 06:12 | |
| *** rcernin has joined #openstack-infra | 06:13 | |
| *** zhurong has joined #openstack-infra | 06:14 | |
| *** apuimedo has quit IRC | 06:17 | |
| *** apuimedo has joined #openstack-infra | 06:18 | |
| *** sabeen1 has quit IRC | 06:18 | |
| *** Sukhdev has joined #openstack-infra | 06:19 | |
| *** ParsectiX has joined #openstack-infra | 06:19 | |
| *** hdd has quit IRC | 06:20 | |
| *** changbl has quit IRC | 06:23 | |
| *** apuimedo has quit IRC | 06:25 | |
| *** apuimedo has joined #openstack-infra | 06:26 | |
| *** changbl has joined #openstack-infra | 06:26 | |
| *** harlowja_at_home has quit IRC | 06:29 | |
| *** amotoki has joined #openstack-infra | 06:30 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/253339 | 06:33 |
| *** jlk has quit IRC | 06:41 | |
| *** jlk has joined #openstack-infra | 06:41 | |
| *** oomichi has quit IRC | 06:46 | |
| *** jaosorior has joined #openstack-infra | 06:49 | |
| *** tjcocozz has quit IRC | 06:49 | |
| *** tjcocozz has joined #openstack-infra | 06:50 | |
| *** gildub has quit IRC | 06:52 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/requirements: add graphviz library https://review.openstack.org/253290 | 06:53 |
| *** otter768 has joined #openstack-infra | 06:55 | |
| *** rcernin has quit IRC | 06:59 | |
| *** otter768 has quit IRC | 07:00 | |
| *** annegentle has joined #openstack-infra | 07:03 | |
| *** rcernin has joined #openstack-infra | 07:07 | |
| *** vilobhmm11 has joined #openstack-infra | 07:08 | |
| *** annegentle has quit IRC | 07:08 | |
| *** scheuran has joined #openstack-infra | 07:15 | |
| *** chlong has quit IRC | 07:19 | |
| *** Sukhdev has quit IRC | 07:23 | |
| *** kozhukalov_ has joined #openstack-infra | 07:29 | |
| *** kozhukalov_ has quit IRC | 07:34 | |
| *** mohankumar has quit IRC | 07:35 | |
| *** pahuang has quit IRC | 07:37 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/system-config: WIP: Adding rebuild_phab_infra-dev.py https://review.openstack.org/240795 | 07:41 |
| deepakcs | Hello, can any cores pls +A https://review.openstack.org/251421 (its has everything else) | 07:42 |
| *** salv-orlando has joined #openstack-infra | 07:44 | |
| *** btully has quit IRC | 07:44 | |
| *** btully has joined #openstack-infra | 07:47 | |
| *** kozhukalov_ has joined #openstack-infra | 07:48 | |
| *** AJaeger has joined #openstack-infra | 07:50 | |
| *** yamahata has joined #openstack-infra | 07:55 | |
| *** unicell has joined #openstack-infra | 08:00 | |
| openstackgerrit | Merged openstack-infra/project-config: Add pep8 job to ironic-inspector-specs https://review.openstack.org/252903 | 08:02 |
| *** kozhukalov_ has quit IRC | 08:03 | |
| *** rossella_s has joined #openstack-infra | 08:04 | |
| *** ihrachys has joined #openstack-infra | 08:04 | |
| *** unicell1 has joined #openstack-infra | 08:05 | |
| *** unicell has quit IRC | 08:05 | |
| *** vilobhmm111 has joined #openstack-infra | 08:05 | |
| *** mpavone has joined #openstack-infra | 08:07 | |
| *** vilobhmm11 has quit IRC | 08:07 | |
| *** zhurong has quit IRC | 08:13 | |
| *** yamahata has quit IRC | 08:13 | |
| *** vilobhmm111 has quit IRC | 08:14 | |
| *** ihrachys_ has joined #openstack-infra | 08:19 | |
| *** kozhukalov_ has joined #openstack-infra | 08:19 | |
| *** ihrachys has quit IRC | 08:19 | |
| *** btully has quit IRC | 08:19 | |
| *** zhurong has joined #openstack-infra | 08:20 | |
| *** btully has joined #openstack-infra | 08:21 | |
| *** ildikov has quit IRC | 08:22 | |
| *** egonzalez has joined #openstack-infra | 08:24 | |
| *** zhurong has quit IRC | 08:24 | |
| *** u_glide1 has joined #openstack-infra | 08:30 | |
| u_glide1 | ianw: ping | 08:31 |
| *** fawadkhaliq has quit IRC | 08:31 | |
| *** katyafervent is now known as katyafervent_awa | 08:31 | |
| *** fawadkhaliq has joined #openstack-infra | 08:31 | |
| *** ihrachys has joined #openstack-infra | 08:32 | |
| *** btully has quit IRC | 08:32 | |
| *** ifarkas has joined #openstack-infra | 08:32 | |
| u_glide1 | ianw: Could you please review my patch to project-config #link https://review.openstack.org/233065/ | 08:32 |
| *** zeih has joined #openstack-infra | 08:34 | |
| *** btully has joined #openstack-infra | 08:34 | |
| *** aspiers has joined #openstack-infra | 08:34 | |
| *** arxcruz has joined #openstack-infra | 08:34 | |
| *** ihrachys_ has quit IRC | 08:35 | |
| *** _nadya_ has joined #openstack-infra | 08:37 | |
| *** matrohon has joined #openstack-infra | 08:38 | |
| *** jlanoux has joined #openstack-infra | 08:40 | |
| *** vgridnev has quit IRC | 08:40 | |
| *** salv-orlando has quit IRC | 08:42 | |
| *** btully has quit IRC | 08:44 | |
| *** mohankumar has joined #openstack-infra | 08:45 | |
| *** btully has joined #openstack-infra | 08:46 | |
| *** fawadkhaliq has quit IRC | 08:47 | |
| *** fawadkhaliq has joined #openstack-infra | 08:47 | |
| *** salv-orlando has joined #openstack-infra | 08:48 | |
| *** jcoufal has joined #openstack-infra | 08:49 | |
| *** hashar has joined #openstack-infra | 08:52 | |
| *** tsg has joined #openstack-infra | 08:52 | |
| *** apuimedo has quit IRC | 08:53 | |
| *** apuimedo has joined #openstack-infra | 08:54 | |
| *** ociuhandu has quit IRC | 08:54 | |
| *** oomichi has joined #openstack-infra | 08:55 | |
| *** ociuhandu has joined #openstack-infra | 08:55 | |
| *** otter768 has joined #openstack-infra | 08:56 | |
| *** btully has quit IRC | 08:58 | |
| *** ociuhandu has quit IRC | 08:59 | |
| *** btully has joined #openstack-infra | 09:00 | |
| AJaeger | yolanda, ianw: Any ideas what changed in our setup? I see a complete broken build for https://review.openstack.org/#/c/251103/ | 09:00 |
| AJaeger | None of the targets gets executed - and for checkbuild there's not even a log file | 09:00 |
| yolanda | AJaeger, i don't know, let me take a look at that change | 09:00 |
| AJaeger | I caught the build at https://jenkins03.openstack.org/job/gate-openstack-manuals-tox-doc-publish-checkbuild/815/console | 09:00 |
| *** otter768 has quit IRC | 09:01 | |
| AJaeger | yolanda: that is a change for the manuals - and it should call tox but does not... | 09:01 |
| *** markus_z has joined #openstack-infra | 09:01 | |
| yolanda | AJaeger, i'm not aware of any change, but i can help with debug | 09:01 |
| *** apuimedo has quit IRC | 09:01 | |
| yolanda | AJaeger, when did this behaviour start? yesterday? | 09:02 |
| AJaeger | yolanda: compare with https://review.openstack.org/#/c/253341/ - looks fine | 09:02 |
| AJaeger | yolanda: only saw it this morning | 09:02 |
| *** apuimedo has joined #openstack-infra | 09:03 | |
| *** acabot has quit IRC | 09:03 | |
| AJaeger | checkbuild on that is fine - but the checkdeletions is also broken ;( | 09:03 |
| *** ildikov has joined #openstack-infra | 09:03 | |
| *** annegentle has joined #openstack-infra | 09:04 | |
| AJaeger | this one from yesterday looks good https://review.openstack.org/#/c/253175/ | 09:04 |
| *** dims_ has quit IRC | 09:05 | |
| yolanda | so it just shall run tox with env checkbuild, and it's not calling it? | 09:05 |
| *** shardy has joined #openstack-infra | 09:05 | |
| AJaeger | exactly | 09:05 |
| red_trela | we'd like to add accessbot, gerritbot and logging to #midonet, could someone please review my patches? https://review.openstack.org/#/c/252173/ and https://review.openstack.org/#/c/252172/ | 09:06 |
| AJaeger | yolanda: have a look at https://jenkins03.openstack.org/job/gate-openstack-manuals-tox-doc-publish-checkbuild/configure | 09:07 |
| AJaeger | That's broken, isn't it? | 09:07 |
| AJaeger | did we break zuul or jjb? | 09:07 |
| yolanda | checking | 09:08 |
| yolanda | so there is no tox step there | 09:08 |
| yolanda | i can look at history | 09:08 |
| * AJaeger does not find the tox invocation there... | 09:08 | |
| AJaeger | yolanda: yes, please. I have no idea where to look... | 09:09 |
| *** annegentle has quit IRC | 09:09 | |
| yolanda | AJaeger, so you can see https://jenkins03.openstack.org/job/gate-openstack-manuals-tox-doc-publish-checkbuild/jobConfigHistory/showDiffFiles?histDir1=%2Fvar%2Flib%2Fjenkins%2Fjobs%2Fgate-openstack-manuals-tox-doc-publish-checkbuild%2Fconfig-history%2F2015-11-26_10-14-03&histDir2=%2Fvar%2Flib%2Fjenkins%2Fjobs%2Fgate-openstack-manuals-tox-doc-publish-checkbuild%2Fconfig-history%2F2015-12-04_08-11-54 ? | 09:10 |
| yolanda | toxbuilder has gone | 09:10 |
| AJaeger | I cannot see any output, seems I don't have those permissions | 09:11 |
| *** dims has joined #openstack-infra | 09:11 | |
| AJaeger | zuul has not seen an update for a few days | 09:11 |
| yolanda | ok so there is a diff, between 26/11 and 04/12. Something that landed today broke it | 09:12 |
| openstackgerrit | Zhiyuan Cai proposed openstack-infra/project-config: Add python jobs to tricircle https://review.openstack.org/253394 | 09:12 |
| yolanda | AJaeger, i don't see anything on project-config that may have done it | 09:12 |
| AJaeger | what about https://review.openstack.org/#/c/251249/ | 09:12 |
| AJaeger | yolanda: neither do I - that's why I'm looking at JJB | 09:13 |
| AJaeger | http://logs.openstack.org/49/251249/3/check/jenkins-job-builder-compare-xml/059089e/console.html | 09:13 |
| AJaeger | Let's revert that change, it's broken | 09:13 |
| *** ociuhandu has joined #openstack-infra | 09:13 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/jenkins-job-builder: Revert "Add support for "tox" builder" https://review.openstack.org/253396 | 09:14 |
| yolanda | oh yes, let's rollback it | 09:14 |
| AJaeger | infra-root, please approve https://review.openstack.org/#/c/253396/ | 09:14 |
| yolanda | so that broke all our tox tests?! | 09:14 |
| AJaeger | have a look at the compare-xml, it loooks like it | 09:14 |
| openstackgerrit | Merged openstack-infra/project-config: Enable devstack-plugin-ceph jobs https://review.openstack.org/251421 | 09:14 |
| yolanda | jhesketh, around for a +2? i'll do a quick +2 +a if not | 09:14 |
| *** fbo_ is now known as fbo | 09:15 | |
| AJaeger | perhaps only after we merged project-config this morning... | 09:15 |
| yolanda | approved the revert, it's a blocker | 09:16 |
| *** shardy has quit IRC | 09:17 | |
| AJaeger | thanks, yolanda ! | 09:17 |
| oomichi | yolanda: hello, can you take a look at https://review.openstack.org/#/c/252739/ ? it gets +2 and +1 from the PTL | 09:18 |
| AJaeger | yolanda: https://review.openstack.org/#/c/251421/ - the gate was broken as well... | 09:18 |
| AJaeger | so, it affects other repos for sure - looks like all tox targets | 09:19 |
| yolanda | AJaeger, yes, i saw... every test related with tox | 09:19 |
| AJaeger | oomichi: infra team only looks in emergencies at requirements, plesae ask the cores for it | 09:19 |
| yolanda | oomichi, for requirements, i'd prefer if someone on that group approves it | 09:19 |
| AJaeger | yolanda: once that change is in: Will you update all jobs on all Jenkinses? | 09:20 |
| oomichi | AJaeger: yolanda: I see, thanks. will wait for the cores :) | 09:20 |
| yolanda | AJaeger, wow yes, it can take long | 09:20 |
| yolanda | in theory puppet shall do it, but seems that when there is a massive job reconfig, it can fail | 09:21 |
| AJaeger | you can also watch out that it does not run in timeouts... | 09:21 |
| AJaeger | Let me send an email to #openstack-dev | 09:21 |
| yolanda | AJaeger yes please | 09:21 |
| ttx | AJaeger: can you unblock https://review.openstack.org/#/c/252572/ ? Deps merged | 09:21 |
| * AJaeger meant openstack-dev mailing list | 09:22 | |
| *** dguitarbite has joined #openstack-infra | 09:22 | |
| *** yuanying has quit IRC | 09:22 | |
| AJaeger | yolanda: what about a warning on IRC that tox is currently broken? | 09:22 |
| yolanda | in openstack-dev ? | 09:22 |
| AJaeger | ttx, let's wait a bit - tox jobs are broken at the moment | 09:22 |
| *** acabot has joined #openstack-infra | 09:22 | |
| ttx | ah. | 09:22 |
| AJaeger | yolanda: global notification | 09:22 |
| ttx | Good to know :) | 09:22 |
| AJaeger | ttx, all will succeed ;) | 09:22 |
| ttx | PERFECT | 09:23 |
| AJaeger | and they're really fast - tox has become a noop ;( | 09:23 |
| AJaeger | ttx, yolanda approved the revert of the broken change, now we need to wait for everything to merge and update... | 09:23 |
| *** dims has quit IRC | 09:23 | |
| yolanda | AJaeger, i'm not experienced with setting topics on channels, i have never done it. Can you guide me? i may have perms but i lack the knowledge | 09:24 |
| *** jordanP has joined #openstack-infra | 09:24 | |
| AJaeger | ttx, can you help, please? I suggest a notification, not a topic change... | 09:24 |
| *** shardy has joined #openstack-infra | 09:25 | |
| ttx | let me se | 09:25 |
| ttx | e | 09:25 |
| ttx | yolanda: http://docs.openstack.org/infra/system-config/irc.html#statusbot | 09:25 |
| ttx | you want "#status notice ..." | 09:25 |
| ttx | yolanda: try it and if you don't have teh rights I can push it | 09:26 |
| yolanda | thanks | 09:26 |
| ttx | something like "#status notice Things are broken at the mmoment. Do Foo until BAR." | 09:26 |
| AJaeger | what about "Do not approve changes until we give the all clear" ? | 09:27 |
| yolanda | #status notice Tox tests are broken at the moment. From openstack-infra we are working to fix them. Please don't approve changes until we notify that tox tests work again. | 09:28 |
| yolanda | mm nothing happened? | 09:28 |
| AJaeger | yolanda: wait a bit... | 09:28 |
| yolanda | ok :) | 09:28 |
| ttx | could be that you don't have the rights though | 09:29 |
| ttx | in which case the bot politely ignores you | 09:29 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Revert "Add support for "tox" builder" https://review.openstack.org/253396 | 09:29 |
| AJaeger | but that is slow indeed | 09:29 |
| ttx | Let me try | 09:29 |
| ttx | #status notice Tox tests are broken at the moment. From openstack-infra we are working to fix them. Please don't approve changes until we notify that tox tests work again. | 09:29 |
| openstackstatus | ttx: sending notice | 09:29 |
| yolanda | ttx, worked better for you | 09:30 |
| ttx | yeah, you probably aren't in th ebot whitelist yet | 09:30 |
| AJaeger | yolanda: indeed, you're not on accessbot/channels.yaml - please add yourself as infra-root. | 09:30 |
| yolanda | AJaeger, will do | 09:30 |
| AJaeger | thanks ,ttx | 09:30 |
| -openstackstatus- NOTICE: Tox tests are broken at the moment. From openstack-infra we are working to fix them. Please don't approve changes until we notify that tox tests work again. | 09:31 | |
| openstackstatus | ttx: finished sending notice | 09:32 |
| openstackgerrit | Masayuki Igawa proposed openstack/os-testr: Fix coverage section in tox.ini https://review.openstack.org/253407 | 09:32 |
| *** ihrachys_ has joined #openstack-infra | 09:33 | |
| AJaeger | yolanda: change has merged. | 09:33 |
| yolanda | AJaeger, ok , i will monitor it | 09:33 |
| *** dims has joined #openstack-infra | 09:34 | |
| AJaeger | yolanda: how fast is jjb updated? Do we ahve to wait for anything? Or can I approve ttx' change and that one updates everything? | 09:34 |
| AJaeger | We can just try - the barbican change looks perfect ;) | 09:34 |
| *** shardy has quit IRC | 09:34 | |
| *** dtantsur|afk is now known as dtantsur | 09:35 | |
| *** ihrachys has quit IRC | 09:35 | |
| yolanda | AJaeger, so puppet is run over an interval of time , so first we need to give time for change to be replicated to git mirrors | 09:35 |
| yolanda | once that is there, puppet will consume it, and apply to all jenkins masters | 09:35 |
| yolanda | going to check in puppetmaster | 09:35 |
| openstackgerrit | Masayuki Igawa proposed openstack/os-testr: [WIP] Add unit test for colorizer https://review.openstack.org/253409 | 09:36 |
| yolanda | interval is each 15 minutes | 09:36 |
| *** shardy has joined #openstack-infra | 09:37 | |
| AJaeger | change merged at 10:29 (local time), let's wait until 10:45 and I'll approve ttx' change... | 09:37 |
| yolanda | AJaeger, so not as easy... in 15 minutes puppet will trigger the updates. But jjb can take long to reconfigure all jobs, in case of a massive reconfig like that, it can take more than 1 hour | 09:38 |
| *** derekh has joined #openstack-infra | 09:38 | |
| yolanda | i'm tailing logs so i can tell you | 09:38 |
| yolanda | unfortunately, i don't know about a better way to speed up jjb | 09:39 |
| AJaeger | yolanda: since you're tailing - please approve https://review.openstack.org/#/c/252572 when appropriate | 09:39 |
| yolanda | sure | 09:40 |
| yolanda | i see that jjb code was applied to jenkins01 , 3 minutes ago. I can trigger a jjb reconfig | 09:40 |
| *** apuimedo has quit IRC | 09:41 | |
| *** apuimedo has joined #openstack-infra | 09:42 | |
| AJaeger | AFAIR this is already the second time that a tox plugin was added to JJB - and everything failed ;( | 09:43 |
| *** dizquierdo has joined #openstack-infra | 09:43 | |
| *** shardy has quit IRC | 09:43 | |
| yolanda | yes i know... problem is that there is that compare xml job, that shows the output changed | 09:43 |
| yolanda | but it's non-voting right? | 09:43 |
| AJaeger | it'S non-voting - since some diffs are fine... | 09:44 |
| yolanda | AJaeger, weird, i saw an install from jjb applied on jenkins01. But i triggered jjb reconfig and nothing happened :( | 09:45 |
| yolanda | going to check if jjb code was really applied | 09:45 |
| * AJaeger will be back later | 09:45 | |
| AJaeger | yolanda: no idea;( | 09:45 |
| AJaeger | and thanks a lot, yolanda ! | 09:46 |
| yolanda | AJaeger, no problem, and thanks for noticing the problem... | 09:46 |
| *** shardy has joined #openstack-infra | 09:46 | |
| *** mdbooth has quit IRC | 09:47 | |
| *** hashar has quit IRC | 09:48 | |
| *** hashar has joined #openstack-infra | 09:48 | |
| *** iwamoto has quit IRC | 09:48 | |
| *** ihrachys has joined #openstack-infra | 09:48 | |
| AJaeger | perhaps we should push the project-config change through - so that we have new content... | 09:49 |
| yolanda | mm, we may need to ignore caching on jjb... | 09:49 |
| yolanda | because it doesn't see these jobs as changed | 09:49 |
| yolanda | so that's not even trying to reconfigure | 09:49 |
| AJaeger | so, +A for 252572? | 09:49 |
| yolanda | mm, no | 09:49 |
| yolanda | i'll do a jjb reconfig without caching on jenkins01 | 09:50 |
| AJaeger | yolanda: I have no experience on the operational side, so don't trust me in this ;) | 09:50 |
| *** ihrachys_ has quit IRC | 09:50 | |
| yolanda | i've done that on downstream, so it shall be the same, i want to try on one jenkins first to be sure | 09:51 |
| *** mdbooth has joined #openstack-infra | 09:51 | |
| *** migi has quit IRC | 09:52 | |
| *** salv-orlando has quit IRC | 09:52 | |
| yolanda | AJaeger, it will take long... | 09:52 |
| openstackgerrit | Masayuki Igawa proposed openstack/os-testr: [WIP] Add unit test for colorizer https://review.openstack.org/253409 | 09:55 |
| yolanda | AJaeger, it's reconfiguring all jobs on jenkins01 now, but it will be slow | 09:58 |
| AJaeger | yeah, that takes ages ;( | 09:58 |
| AJaeger | can we do the other in parallel? | 09:59 |
| *** zhenguo has quit IRC | 09:59 | |
| yolanda | AJaeger we can, but also that jjb massive reconfig affects jenkins performance, i'm afraid of leaving all jenkins without capacity | 10:00 |
| yolanda | although it's a major problem now, and people won't be approving changes anyway so... | 10:00 |
| *** ParsectiX has quit IRC | 10:01 | |
| *** nijaba has quit IRC | 10:01 | |
| *** kevinbenton has quit IRC | 10:01 | |
| *** rguillebert has joined #openstack-infra | 10:02 | |
| AJaeger | and the tox jobs running on it are broken | 10:03 |
| *** kevinbenton has joined #openstack-infra | 10:04 | |
| *** dizquierdo has quit IRC | 10:04 | |
| yolanda | ok i'll trigger on parallel on all jenkins, as soon as we recover , better | 10:04 |
| *** bauzas is now known as bauwser | 10:05 | |
| *** ParsectiX has joined #openstack-infra | 10:05 | |
| *** nijaba has joined #openstack-infra | 10:06 | |
| *** nijaba has quit IRC | 10:06 | |
| *** nijaba has joined #openstack-infra | 10:06 | |
| sileht | Hi, I have noticed that #openstack-telemetry doesn't have the openstackstatus bot but only the openstackgerrit one. How can I fix that ? | 10:06 |
| *** _skolekonov is now known as skolekonov | 10:08 | |
| *** salv-orlando has joined #openstack-infra | 10:10 | |
| sileht | found it ! | 10:11 |
| red_trela | sileht: now I'm wondering as well...how? :) | 10:12 |
| *** btully has quit IRC | 10:12 | |
| u_glide1 | AJaeger: I've answered to ianw question in review #link https://review.openstack.org/233065/ . Is it enough to get +2 from you? :) | 10:14 |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack-infra/system-config: Rename ceilometer to telemetry https://review.openstack.org/253420 | 10:16 |
| sileht | red_trela, ^^ | 10:16 |
| *** btully has joined #openstack-infra | 10:16 | |
| red_trela | sileht: ah, right...now that I see the code, I remember ;) thanks :) | 10:19 |
| openstackgerrit | Merged openstack-infra/project-config: Add (access,gerrit)bot for networking-bgpvpn https://review.openstack.org/252513 | 10:21 |
| *** maishsk has joined #openstack-infra | 10:26 | |
| *** migi has joined #openstack-infra | 10:27 | |
| *** yuanying has joined #openstack-infra | 10:29 | |
| openstackgerrit | Lingxian Kong proposed openstack/requirements: Bump python-mistralclient to 1.2.0 https://review.openstack.org/253423 | 10:29 |
| *** Daisy has quit IRC | 10:30 | |
| *** openstackgerrit has quit IRC | 10:32 | |
| *** yuanying has quit IRC | 10:32 | |
| *** openstackgerrit has joined #openstack-infra | 10:32 | |
| *** yuanying has joined #openstack-infra | 10:33 | |
| *** yuanying has quit IRC | 10:33 | |
| *** yuanying has joined #openstack-infra | 10:34 | |
| *** markmcd has joined #openstack-infra | 10:34 | |
| openstackgerrit | Igor Malinovskiy proposed openstack-infra/project-config: Use Manila specific job templates for building images https://review.openstack.org/233065 | 10:35 |
| *** electrofelix has joined #openstack-infra | 10:35 | |
| *** dzamboni has joined #openstack-infra | 10:39 | |
| *** sambetts-afk is now known as sambetts | 10:46 | |
| u_glide1 | AJaeger, ianw: Could you please review latest patch-set in #link https://review.openstack.org/#/c/233065/ | 10:46 |
| u_glide1 | AJaeger, ianw: sorry for spamming, but manila project really need this change to improve gate jobs speed and stability (we need to release image and cache it to speedup devstack jobs) | 10:46 |
| u_glide1 | AJaeger, ianw: thanks in advance! | 10:46 |
| *** katyafervent_awa is now known as katyafervent | 10:50 | |
| *** sdake has joined #openstack-infra | 10:54 | |
| *** ldnunes has joined #openstack-infra | 10:56 | |
| *** otter768 has joined #openstack-infra | 10:57 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Tidy up project groups states https://review.openstack.org/253049 | 10:57 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Add an "Unlimited" option to the paging controls https://review.openstack.org/235993 | 10:58 |
| *** otter768 has quit IRC | 11:01 | |
| *** dguitarbite has quit IRC | 11:07 | |
| *** salv-orlando has quit IRC | 11:07 | |
| *** btully has quit IRC | 11:10 | |
| *** btully has joined #openstack-infra | 11:13 | |
| *** yamamoto has quit IRC | 11:14 | |
| openstackgerrit | Victor Stinner proposed openstack-infra/project-config: Make gate-swift-python34 voting and add gate job https://review.openstack.org/253445 | 11:14 |
| *** aysyd has joined #openstack-infra | 11:17 | |
| *** salv-orlando has joined #openstack-infra | 11:18 | |
| *** zeih has quit IRC | 11:23 | |
| *** ParsectiX has quit IRC | 11:23 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/requirements: Bump pyudev minimum to 0.18 https://review.openstack.org/253448 | 11:26 |
| dtantsur | folks, may I get your attention to this ^^ please | 11:26 |
| openstackgerrit | Julien Danjou proposed openstack-infra/system-config: Install etcd on nodes for tooz unit tests https://review.openstack.org/253450 | 11:26 |
| dtantsur | ironic-python-agent gate is broken | 11:26 |
| *** abhi has quit IRC | 11:30 | |
| *** claudiub has joined #openstack-infra | 11:34 | |
| *** salv-orlando has quit IRC | 11:36 | |
| *** btully has quit IRC | 11:37 | |
| *** olaph has quit IRC | 11:39 | |
| *** olaph has joined #openstack-infra | 11:39 | |
| *** btully has joined #openstack-infra | 11:39 | |
| dims | AJaeger : tox jobs back on their feet? | 11:40 |
| yolanda | dims, i'm executing a massive reconfig of jjb, we had to revert a change | 11:42 |
| yolanda | they will be recovered soon | 11:42 |
| dims | yolanda : ah thanks | 11:42 |
| yolanda | i'm seeing some of them starting to be recovered | 11:43 |
| *** erlon has joined #openstack-infra | 11:44 | |
| *** chlong has joined #openstack-infra | 11:44 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/requirements: Cap pydev at <0.18 https://review.openstack.org/253448 | 11:45 |
| electrofelix | so it seems we missed https://review.openstack.org/251249 affecting the openstack JJB jobs, we really do need something to highlight no-voting jobs failing | 11:46 |
| yolanda | AJaeger, i triggered a recheck on the patch you needed to be approved, as i saw most of jobs have been properly reconfigured now | 11:47 |
| *** fawadkhaliq has quit IRC | 11:53 | |
| *** ParsectiX has joined #openstack-infra | 11:53 | |
| *** ParsectiX has quit IRC | 11:54 | |
| *** fhubik has joined #openstack-infra | 11:55 | |
| *** mnencia has quit IRC | 11:55 | |
| *** mnencia has joined #openstack-infra | 11:55 | |
| *** fhubik is now known as fhubik_brb | 11:56 | |
| *** ParsectiX has joined #openstack-infra | 11:56 | |
| *** zeih has joined #openstack-infra | 11:58 | |
| AJaeger | yolanda: that job looks fine now, will approve | 11:58 |
| *** noqa_v_g1ovnie is now known as noqa_v_qoovnie | 11:58 | |
| AJaeger | yolanda: did all jenkins finish? | 11:58 |
| yolanda | AJaeger, not yet | 11:58 |
| yolanda | reconfiguring gate-t* jobs now | 11:58 |
| yolanda | it will take a while... | 11:59 |
| AJaeger | oh... | 11:59 |
| AJaeger | and I thought I was gone for a long time ;/ | 11:59 |
| yolanda | it can take several hours | 11:59 |
| AJaeger | did you see my email? Once everything is green, could you give the go ahead if I'm not around, please? | 12:00 |
| yolanda | sure | 12:00 |
| AJaeger | yolanda: https://review.openstack.org/#/c/248898/ should be approved soon as well... | 12:00 |
| yolanda | by the time they finish, i'm sure US infra-roots will be here | 12:01 |
| AJaeger | u_glide1: Sorry, I'm not feeling confident enough to review that change further ;( | 12:01 |
| yolanda | AJaeger, that change has a depends that has not landed | 12:02 |
| *** abregman has joined #openstack-infra | 12:03 | |
| *** btully has quit IRC | 12:03 | |
| *** yamamoto has joined #openstack-infra | 12:04 | |
| *** abregman is now known as abregman|afk | 12:04 | |
| *** yuanying_ has joined #openstack-infra | 12:04 | |
| AJaeger | johnthetubaguy: ^ | 12:05 |
| AJaeger | johnthetubaguy: you first need to approve https://review.openstack.org/248895 - and please wait with approval until our CI is green again... | 12:05 |
| AJaeger | johnthetubaguy: Once that is in, we can approve 248898... | 12:05 |
| AJaeger | yolanda: you're right, thanks! | 12:06 |
| openstackgerrit | Merged openstack-infra/project-config: Add reno jobs for barbican https://review.openstack.org/252572 | 12:06 |
| *** lucasagomes has joined #openstack-infra | 12:06 | |
| *** btully has joined #openstack-infra | 12:06 | |
| openstackgerrit | Darragh Bailey proposed openstack-infra/system-config: Add result_WARNING style class for failing non-voting jobs https://review.openstack.org/253462 | 12:07 |
| *** yamamoto has quit IRC | 12:08 | |
| *** yuanying has quit IRC | 12:08 | |
| *** dtantsur is now known as dtantsur|brb | 12:09 | |
| *** kozhukalov_ has quit IRC | 12:13 | |
| openstackgerrit | Darragh Bailey proposed openstack-infra/system-config: Add result_WARNING style class for failing non-voting jobs https://review.openstack.org/253462 | 12:18 |
| *** aspiers has quit IRC | 12:19 | |
| *** zul has joined #openstack-infra | 12:22 | |
| *** yuanying_ has quit IRC | 12:23 | |
| *** zul has quit IRC | 12:24 | |
| *** zul has joined #openstack-infra | 12:25 | |
| openstackgerrit | Darragh Bailey proposed openstack-infra/project-config: Use WARNING style for JJB compare XML job failure https://review.openstack.org/253468 | 12:26 |
| *** kozhukalov_ has joined #openstack-infra | 12:26 | |
| electrofelix | yolanda: combination of https://review.openstack.org/253462 and https://review.openstack.org/253468 should make sure that a change in the XML output from the JJB jobs now results in 'WARNING' appearing in yellow. Hopefully that will stop us from causing problems in the future. | 12:26 |
| *** yuanying has joined #openstack-infra | 12:27 | |
| *** yuanying has quit IRC | 12:27 | |
| *** fhubik_brb is now known as fhubik | 12:27 | |
| openstackgerrit | Darragh Bailey proposed openstack-infra/project-config: Use WARNING style for JJB compare XML job failure https://review.openstack.org/253468 | 12:28 |
| *** btully has quit IRC | 12:28 | |
| *** ihrachys has quit IRC | 12:28 | |
| *** mnaser has quit IRC | 12:30 | |
| *** deepakcs has quit IRC | 12:32 | |
| *** btully has joined #openstack-infra | 12:33 | |
| *** trown|outttypeww is now known as trown | 12:36 | |
| *** dizquierdo has joined #openstack-infra | 12:37 | |
| *** zeih_ has joined #openstack-infra | 12:38 | |
| *** mnaser has joined #openstack-infra | 12:39 | |
| *** abregman|afk is now known as abregman | 12:40 | |
| *** tsg has quit IRC | 12:40 | |
| *** zeih has quit IRC | 12:42 | |
| johnthetubaguy | AJaeger: ah, good point, thank you! | 12:43 |
| *** btully has quit IRC | 12:46 | |
| *** _nadya_ has quit IRC | 12:46 | |
| *** dtantsur|brb is now known as dtantsur | 12:46 | |
| *** maishsk has quit IRC | 12:46 | |
| *** btully has joined #openstack-infra | 12:47 | |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: Adds an API migration and docs generation tool, fairy-slipper https://review.openstack.org/245352 | 12:49 |
| *** jaosorior has quit IRC | 12:49 | |
| AJaeger | electrofelix: nice, thanks! | 12:49 |
| *** jaosorior has joined #openstack-infra | 12:50 | |
| yolanda | mm, jenkins02 reconfig was aborted with proxy error | 12:50 |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config: [sahara] gate-grenade-dsvm-sahara to merge check https://review.openstack.org/253473 | 12:50 |
| *** salv-orlando has joined #openstack-infra | 12:54 | |
| *** weshay_xchat has joined #openstack-infra | 12:55 | |
| *** dingyichen has quit IRC | 12:57 | |
| *** otter768 has joined #openstack-infra | 12:58 | |
| *** btully has quit IRC | 12:58 | |
| *** salv-orlando has quit IRC | 12:58 | |
| *** markus_z is now known as markus_z_meeting | 12:59 | |
| *** btully has joined #openstack-infra | 13:00 | |
| *** tiswanso has quit IRC | 13:00 | |
| *** tiswanso has joined #openstack-infra | 13:01 | |
| yolanda | AJaeger, jenkins05 was reconfigured properly | 13:02 |
| *** otter768 has quit IRC | 13:03 | |
| yolanda | i had to relaunch reconfiguration on jenkins02 | 13:03 |
| yolanda | and the others are still executing | 13:03 |
| *** dprince has joined #openstack-infra | 13:07 | |
| *** ryansb has joined #openstack-infra | 13:07 | |
| *** btully has quit IRC | 13:11 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Allow puppet to enabling or disabling Gerrit drafts feature https://review.openstack.org/253110 | 13:12 |
| *** annegentle has joined #openstack-infra | 13:13 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Allow puppet to enabling or disabling Gerrit drafts feature https://review.openstack.org/253110 | 13:13 |
| *** btully has joined #openstack-infra | 13:13 | |
| *** fhubik has quit IRC | 13:14 | |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: remove gate-devstack-dsvm-cells https://review.openstack.org/248270 | 13:14 |
| sdague | AJaeger: fixed | 13:14 |
| *** fhubik has joined #openstack-infra | 13:15 | |
| *** hdd has joined #openstack-infra | 13:18 | |
| openstackgerrit | Xi Yang proposed openstack/requirements: Remove redundant space in the README.rst https://review.openstack.org/253482 | 13:21 |
| *** rhallisey has joined #openstack-infra | 13:22 | |
| *** yamamoto has joined #openstack-infra | 13:22 | |
| *** olaph has quit IRC | 13:24 | |
| *** sambetts has quit IRC | 13:25 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update barbican to correct repo https://review.openstack.org/253484 | 13:26 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update fuel to correct repo https://review.openstack.org/253485 | 13:26 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update mistral to correct repo https://review.openstack.org/253486 | 13:26 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update murano to correct repo https://review.openstack.org/253487 | 13:26 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update puppet-sahara to correct repo https://review.openstack.org/253488 | 13:26 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update solum to correct repo and add missing https://review.openstack.org/253489 | 13:26 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Remove projects not in OpenStack repo https://review.openstack.org/253490 | 13:26 |
| *** sambetts has joined #openstack-infra | 13:26 | |
| *** yamamoto has quit IRC | 13:26 | |
| *** rfolco has joined #openstack-infra | 13:28 | |
| *** jaosorior has quit IRC | 13:30 | |
| *** jaosorior has joined #openstack-infra | 13:30 | |
| *** btully has quit IRC | 13:30 | |
| *** tiswanso has quit IRC | 13:32 | |
| *** tiswanso has joined #openstack-infra | 13:32 | |
| *** btully has joined #openstack-infra | 13:32 | |
| *** abregman is now known as abregman|afk | 13:33 | |
| *** tlian has quit IRC | 13:34 | |
| *** regXboi has joined #openstack-infra | 13:35 | |
| *** Piet has quit IRC | 13:36 | |
| *** fhubik is now known as fhubik_brb | 13:37 | |
| *** fhubik_brb is now known as fhubik | 13:37 | |
| *** olaph has joined #openstack-infra | 13:38 | |
| yolanda | AJaeger, ttx, only reconfig on jenkins1 and jenkins2 is still ongoing, it failed on these two due to proxy errors | 13:38 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update solum to correct repo and add missing https://review.openstack.org/253489 | 13:39 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Make the recent events list collapsible https://review.openstack.org/253497 | 13:40 |
| *** tlian has joined #openstack-infra | 13:40 | |
| *** david-lyle has quit IRC | 13:43 | |
| *** ayoung has quit IRC | 13:46 | |
| openstackgerrit | Denis Laxalde proposed openstack-infra/jenkins-job-builder: Add support for "tox-builder" builder https://review.openstack.org/253500 | 13:46 |
| *** tiswanso has quit IRC | 13:46 | |
| *** markvoelker has joined #openstack-infra | 13:47 | |
| *** olaph has quit IRC | 13:48 | |
| *** markus_z_meeting is now known as markus_z | 13:48 | |
| *** olaph has joined #openstack-infra | 13:49 | |
| BobBall | patrickeast: amazon scoreboard down? | 13:50 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update fuel to correct repo https://review.openstack.org/253485 | 13:51 |
| *** dguitarbite has joined #openstack-infra | 13:52 | |
| *** barnaby has quit IRC | 13:53 | |
| *** ParsectiX has quit IRC | 13:55 | |
| *** scheuran has quit IRC | 13:56 | |
| *** ayoung has joined #openstack-infra | 13:58 | |
| *** baoli_ has joined #openstack-infra | 13:58 | |
| *** mriedem has joined #openstack-infra | 13:59 | |
| *** mohankumar has quit IRC | 14:00 | |
| AJaeger | tahnks, yolanda ! | 14:02 |
| AJaeger | I need to sign off now, thanks a lot for babysitting this! | 14:02 |
| *** links has quit IRC | 14:02 | |
| *** AJaeger has quit IRC | 14:03 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Created mirror.<region>.openstack.org https://review.openstack.org/238754 | 14:04 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Update pypi_trusted_hosts https://review.openstack.org/253504 | 14:04 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Point nodepool slaves at new pypi mirror location. https://review.openstack.org/238756 | 14:04 |
| *** alaski is now known as lascii | 14:04 | |
| *** fhubik has quit IRC | 14:05 | |
| *** EricGonczer_ has joined #openstack-infra | 14:05 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack-infra/reviewstats: Update murano to correct repo https://review.openstack.org/253487 | 14:05 |
| *** fhubik has joined #openstack-infra | 14:05 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Removed manifests for legacy mirrors https://review.openstack.org/253506 | 14:06 |
| *** alivigni has joined #openstack-infra | 14:08 | |
| openstackgerrit | Major Hayden proposed openstack-infra/project-config: Add functional/check tests for osa-security https://review.openstack.org/252978 | 14:08 |
| *** barnaby has joined #openstack-infra | 14:08 | |
| *** gordc has joined #openstack-infra | 14:09 | |
| *** ericksonsantos has quit IRC | 14:10 | |
| yolanda | AJaeger, have a nice weekend | 14:10 |
| *** peristeri has joined #openstack-infra | 14:12 | |
| *** dkranz has joined #openstack-infra | 14:12 | |
| *** tiswanso has joined #openstack-infra | 14:12 | |
| *** zhenguo has joined #openstack-infra | 14:13 | |
| *** btully has quit IRC | 14:15 | |
| *** salv-orlando has joined #openstack-infra | 14:16 | |
| *** btully has joined #openstack-infra | 14:16 | |
| *** egonzalez has quit IRC | 14:17 | |
| *** hdd has quit IRC | 14:19 | |
| *** placko has joined #openstack-infra | 14:20 | |
| *** markvoelker has quit IRC | 14:21 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Improve test coverage: server secgroup API https://review.openstack.org/253516 | 14:21 |
| *** rlandy has joined #openstack-infra | 14:22 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Add wheel mirror directory to unified mirrors https://review.openstack.org/253521 | 14:26 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Don't close the "add item" modal until the items are added https://review.openstack.org/253526 | 14:28 |
| *** bhunter71 has joined #openstack-infra | 14:28 | |
| fungi | i could have sworn we already reverted the tox builder in jjb once | 14:29 |
| fungi | did someone else try to add it again? | 14:29 |
| *** kgiusti has joined #openstack-infra | 14:29 | |
| yolanda | hi fungi | 14:29 |
| yolanda | so yes, seems it was approved yesterday | 14:29 |
| yolanda | we reverted this morning | 14:29 |
| yolanda | fungi, i'm still running a massive jjb reconfig on jenkins01 and jenkins02, they failed twice because of proxy errors | 14:29 |
| yolanda | do you know how to avoid that? | 14:30 |
| fungi | restart jenkins | 14:30 |
| *** EricGonc_ has joined #openstack-infra | 14:30 | |
| yolanda | jenkins01 is reconfiguring periodic-* now | 14:30 |
| yolanda | fingers crossed | 14:30 |
| yolanda | jenkins02 still on gate-m* | 14:30 |
| krotscheck | EmilienM, greghaynes: I've created a series of patches for the pypi rename, which should make room for rubygems and wheel mirrors on mirror.<region>.openstack.org. The topic is 'unified_mirror' | 14:30 |
| yolanda | fungi, the others were reconfigured successfully | 14:31 |
| fungi | yolanda: see https://review.openstack.org/230438 for the _last_ time this happened | 14:31 |
| krotscheck | greghaynes: I've also created a patch that creates an rsync target for you, once you figure out how to get the wheel builds to create the correct file structure -> https://review.openstack.org/#/c/253521/ | 14:31 |
| *** EricGonczer_ has quit IRC | 14:32 | |
| yolanda | fungi yep, same problem happened today | 14:32 |
| krotscheck | EmilienM: ^^ is a good example of how to hook into the mirror manifest, though I suggest you rebase on https://review.openstack.org/#/c/253506/ instead (so that you're not dependent on wheel mirror things landing) | 14:32 |
| fungi | it's been barely two months. surprising how short some people's memories can be | 14:32 |
| yolanda | it was like a dejavu | 14:32 |
| krotscheck | EmilienM: Or you can wait for my nodejs mirror patches to come in :) | 14:32 |
| * krotscheck goes and finishes those. | 14:32 | |
| *** amitgandhinz has joined #openstack-infra | 14:32 | |
| yolanda | fungi, that's for yesterday's change https://review.openstack.org/#/c/251249/ | 14:33 |
| yolanda | and even it the revert, a new change was added https://review.openstack.org/#/c/253500/ | 14:33 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Separate concerns in pypi_mirror.pp https://review.openstack.org/253236 | 14:33 |
| fungi | yeah, the same addition proposed by yet a second contributor after the first one got emergency reverted | 14:36 |
| yolanda | fungi, once it is fixed, an announce need to be sent to notify people can continue with their jobs. ttx sent and announce with statusbot this morning | 14:37 |
| yolanda | i need to add myself to the channels, as i don't have perms, i hope to do it today | 14:37 |
| fungi | yolanda: is it just jenkins02 that still needs job updates? | 14:39 |
| fungi | oh, you said 01 is still updating as well | 14:39 |
| yolanda | jenkins01 is still on process as well yes | 14:39 |
| fungi | if either or both of those fail to update again, we should gracefully restart them | 14:39 |
| fungi | in fact, putting them in prepare for shutdown now will cause all new jobs to run on working jenkins masters. we can live without 2 of the 7 for a few hours if need be | 14:40 |
| EmilienM | krotscheck: ok I'll rebase on top of it, thanks | 14:40 |
| yolanda | fungi, i ran jjb with --ignore-cache, but looking at that failures, it should have been better to wipe cache, then run again. Because at least, if the cache was emptied totally, i could start jjb reconfig at the point where it failed first time | 14:41 |
| yolanda | with --ignore-cache i had to start from scratch again | 14:41 |
| krotscheck | EmilienM: Sweet | 14:41 |
| yolanda | fungi, are jenkins01 and jenkins02 receiving more load than others? | 14:41 |
| fungi | yolanda: no, they perform worse and worse the longer they go without a restart, but those two may also be on older nova flavors so could be slightly slower anyway | 14:42 |
| fungi | yolanda: you've been using sudo -H not just sudo, right? jjb uses the homedir for caching so if you forget -H you're not using the same cache puppet is | 14:42 |
| yolanda | fungi, i just used root user. I forgot that! :( | 14:43 |
| *** annegentle has quit IRC | 14:43 | |
| fungi | oh, well if you're in an interactive shell as root then that's sufficient | 14:44 |
| *** akshai has joined #openstack-infra | 14:44 | |
| *** tongli has joined #openstack-infra | 14:44 | |
| yolanda | yes, luckily i did that way | 14:44 |
| fungi | also, why would you want to wipe (or ignore) the cache? | 14:45 |
| fungi | is it corrupt or something? | 14:45 |
| fungi | oh, right, there were no changes to the config | 14:46 |
| fungi | i understand | 14:46 |
| *** fawadkhaliq has joined #openstack-infra | 14:46 | |
| fungi | anyway, it will be safer and speed things up if you set jenkins01 and jenkins02 in prepare for shutdown mode now | 14:46 |
| fungi | so that we'll stop using them to run new jobs | 14:47 |
| yolanda | fungi ok | 14:47 |
| yolanda | yes, i had to ignore cache, because it's not a job change problem | 14:47 |
| yolanda | fungi, so i just throw a jenkins restart on both? it won't cause jjb to stop reconfiguring, and leave things in a bad state? | 14:48 |
| *** ildikov has quit IRC | 14:51 | |
| fungi | just go to the manage jenkins page and use the prepare for shutdown button | 14:51 |
| fungi | so that it will stop running any new jobs | 14:51 |
| fungi | jjb updating will be unaffected by that | 14:52 |
| yolanda | ok | 14:52 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Add NodeJS and NPM to single-use-slave https://review.openstack.org/242161 | 14:52 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Added NPM Mirror to Infra https://review.openstack.org/235617 | 14:52 |
| *** amuller has joined #openstack-infra | 14:52 | |
| *** annegentle has joined #openstack-infra | 14:52 | |
| *** burgerk has joined #openstack-infra | 14:53 | |
| *** dansmith is now known as superdan | 14:53 | |
| *** hdd has joined #openstack-infra | 14:53 | |
| *** fhubik is now known as fhubik_brb | 14:54 | |
| yolanda | fungi, done | 14:54 |
| *** alex_xu is now known as alexus | 14:56 | |
| fungi | thanks. i've noticed that also seems to make jjb hit api failures less often if you're running it on a server which is in prepare for shutdown | 14:57 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Set npm mirror location https://review.openstack.org/242191 | 14:57 |
| *** salv-orlando has quit IRC | 14:57 | |
| *** zz_dimtruck is now known as dimtruck | 14:57 | |
| *** fhubik_brb is now known as fhubik | 14:58 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Use non-versioned cinderclient constructor https://review.openstack.org/253547 | 14:59 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Change the client imports to stop shadowing https://review.openstack.org/253548 | 14:59 |
| *** otter768 has joined #openstack-infra | 14:59 | |
| *** annegentle has quit IRC | 14:59 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Removed npm-install macro https://review.openstack.org/242193 | 15:00 |
| *** fhubik is now known as fhubik_brb | 15:01 | |
| krotscheck | The problem with doing mirror work, is that inevitably the VM you're working with runs out of harddrive space and you have to start over. | 15:01 |
| *** mohankumar has joined #openstack-infra | 15:01 | |
| krotscheck | #drivespaceischeapproblems | 15:02 |
| EmilienM | krotscheck: I'll rebase on top of https://review.openstack.org/#/c/242161/ | 15:03 |
| *** otter768 has quit IRC | 15:03 | |
| krotscheck | EmilienM: Why that one? | 15:04 |
| krotscheck | Just to keep the chain clean? | 15:04 |
| *** annegentle has joined #openstack-infra | 15:04 | |
| *** fhubik_brb is now known as fhubik | 15:05 | |
| *** annegentle has quit IRC | 15:05 | |
| EmilienM | krotscheck: yeah | 15:05 |
| EmilienM | krotscheck: mine is not "urgent" so I prefer rebase on top of your work so we're sure we're consistent | 15:06 |
| krotscheck | EmilienM: You got it. | 15:06 |
| *** tongli_ has joined #openstack-infra | 15:07 | |
| *** patrickeast has quit IRC | 15:07 | |
| *** patrickeast has joined #openstack-infra | 15:08 | |
| *** olaph has quit IRC | 15:08 | |
| *** tongli has quit IRC | 15:08 | |
| *** shardy has quit IRC | 15:08 | |
| *** salv-orlando has joined #openstack-infra | 15:12 | |
| *** thorst_ has joined #openstack-infra | 15:13 | |
| *** zeih_ has quit IRC | 15:13 | |
| *** btully has quit IRC | 15:14 | |
| mordred | infra-root: tagging shade 1.3.0 - contains no changes that should effect nodepool- it's mostly all boot-from-volume related stuff | 15:14 |
| ttx | yolanda: what's the status on the tox jobs issue ? Still in lockdown ? | 15:14 |
| yolanda | ttx, waiting for reconfig on jenkins01 and jenkins02 | 15:14 |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config: Add yolanda as accessbot operator https://review.openstack.org/253554 | 15:14 |
| ttx | ok | 15:15 |
| *** btully has joined #openstack-infra | 15:15 | |
| yolanda | jenkins01 is processing xstatic* jobs, so it's nearly finishing | 15:15 |
| yolanda | but jenkins02 is still on gate jobs | 15:15 |
| ttx | been merging one openstack/releases thing for mitaka-1 but that should not really affect anything | 15:15 |
| *** [1]Thelo has joined #openstack-infra | 15:16 | |
| yolanda | if it doesn't rely on tox tests, you are fine | 15:16 |
| *** pradk_ has joined #openstack-infra | 15:18 | |
| yolanda | jenkins01 finished reconfig | 15:18 |
| *** [2]Thelo has quit IRC | 15:18 | |
| *** FallenPegasus has joined #openstack-infra | 15:19 | |
| *** olaph has joined #openstack-infra | 15:20 | |
| *** kozhukalov_ has quit IRC | 15:20 | |
| *** dprince has quit IRC | 15:21 | |
| sdague | fedora 21 is eol | 15:23 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 15:24 | |
| sdague | did anyone actually get f23 working? | 15:24 |
| *** yamahata has joined #openstack-infra | 15:24 | |
| *** tsg has joined #openstack-infra | 15:25 | |
| *** rbrndt has joined #openstack-infra | 15:26 | |
| *** btully has quit IRC | 15:26 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: the timeout query isn't helping anything, it's just masking failures https://review.openstack.org/252985 | 15:27 |
| mordred | sdague: ianw is working on it | 15:27 |
| openstackgerrit | Dmitry Tantsur proposed openstack/requirements: Cap pydev at <0.18 https://review.openstack.org/253448 | 15:27 |
| *** markus_z is now known as markus_z_meeting | 15:27 | |
| *** btully has joined #openstack-infra | 15:27 | |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: switch to f23 https://review.openstack.org/253561 | 15:28 |
| openstackgerrit | Merged openstack-infra/project-config: add release notes jobs for novaclient https://review.openstack.org/248898 | 15:28 |
| sdague | ok, so we should just flip all the jobs | 15:29 |
| sdague | I just looked at an f21 job fail, and it's not going to be fixable sanely, as we've got additional packages that are needed which aren't in it | 15:29 |
| *** nadya has joined #openstack-infra | 15:29 | |
| sdague | at least f23 fails would be fixable | 15:29 |
| *** mriedem has quit IRC | 15:30 | |
| *** nadya is now known as Guest26736 | 15:30 | |
| *** Guest26736 is now known as _nadya_ | 15:31 | |
| *** mtanino has joined #openstack-infra | 15:31 | |
| rcarrillocruz | ttx: clouseau is brilliant, ++ :D | 15:37 |
| yolanda | fungi, i was thinking in creating an "infra runbook" section, that could document the most common error, tasks... that an infra-root needs to perform, in order for all of us to proceed the same way, and share the knowledge. What do you think? Where could i place something like that? | 15:39 |
| yolanda | we have the same in downstream, and proved to be very useful | 15:39 |
| *** abregman|afk is now known as abregman | 15:40 | |
| *** dmsimard is now known as dmsimard|afk | 15:40 | |
| *** btully has quit IRC | 15:40 | |
| mordred | yolanda: I tink in the infra-manual in system-config | 15:41 |
| *** fawadkhaliq has quit IRC | 15:41 | |
| *** jcoufal has quit IRC | 15:41 | |
| *** lucasagomes is now known as lucas-hungry | 15:41 | |
| * fungi notes the infra-manual is not in system-config, but presumably mordred meant putting it somewhwere in system-config | 15:42 | |
| fungi | not in infra-manual | 15:42 |
| *** david-lyle has joined #openstack-infra | 15:42 | |
| yolanda | fungi, i understood the opposite :) | 15:42 |
| yolanda | to put on infra-manual, wherever it is | 15:42 |
| mordred | oh | 15:42 |
| *** btully has joined #openstack-infra | 15:42 | |
| fungi | not in infra-manual | 15:42 |
| mordred | no, I meant the thing in system-config | 15:43 |
| fungi | our jenkins operating documentation in system-config | 15:43 |
| mordred | since that's the docs for us | 15:43 |
| *** tjones has joined #openstack-infra | 15:43 | |
| yolanda | http://docs.openstack.org/infra/system-config/ ? | 15:43 |
| *** tiswanso has quit IRC | 15:43 | |
| fungi | infra-manual is for people who use our infrastructure, system-config documentation is for people who run our infrastructure | 15:43 |
| mordred | yah | 15:43 |
| *** tiswanso has joined #openstack-infra | 15:43 | |
| yolanda | ok makes sense | 15:44 |
| *** jcoufal has joined #openstack-infra | 15:44 | |
| *** salv-orlando has quit IRC | 15:44 | |
| yolanda | some "troubleshoot" section, or add more sections to system administration if needed | 15:44 |
| *** yonglihe has joined #openstack-infra | 15:44 | |
| *** jamesmcarthur has joined #openstack-infra | 15:45 | |
| *** markvoelker has joined #openstack-infra | 15:45 | |
| openstackgerrit | Dan Moravec proposed openstack-infra/puppet-openstackci: Clarifications to the getting started notes in contrib/README https://review.openstack.org/253153 | 15:45 |
| *** shardy has joined #openstack-infra | 15:46 | |
| *** tmorin has joined #openstack-infra | 15:46 | |
| fungi | well, we have a whole section on systems administration of our jenkins servers, why put other information about managing our jenkins servers somewhere else? | 15:46 |
| *** rhallisey has quit IRC | 15:47 | |
| *** rhallisey has joined #openstack-infra | 15:47 | |
| fungi | yolanda: http://docs.openstack.org/infra/system-config/jenkins.html | 15:47 |
| yolanda | fungi ok, in that particular case for jenkins. But i wanted to add more sections, for example on debug manage-projects, restart gerrit, etc | 15:48 |
| fungi | system-config docs _is_ our operations manual and it's already organized by system | 15:48 |
| openstackgerrit | Dan Moravec proposed openstack-infra/puppet-openstackci: Clarifications to the getting started notes in contrib/README https://review.openstack.org/253153 | 15:49 |
| *** EricGonc_ has quit IRC | 15:49 | |
| fungi | so, yes, add more documents about systems that aren't represented there, or improve the documents which are there if they're just lacking good information like that | 15:49 |
| *** eharney has joined #openstack-infra | 15:49 | |
| *** jaosorior has quit IRC | 15:49 | |
| *** f1ller has quit IRC | 15:50 | |
| *** dprince has joined #openstack-infra | 15:50 | |
| purp | Hi, smart folks like @all-of-you. Was looking at the unified_mirror spec and had a question to help address my ignorance. | 15:50 |
| *** tmorin has left #openstack-infra | 15:50 | |
| fungi | i'll attempt to address it with my own ignorance ;) | 15:51 |
| purp | Oh, wait, mordred's here. @most-of-you. ;D | 15:51 |
| purp | Awesome! I believe my ignorance is superior to yours. =D | 15:51 |
| fungi | we shall see... we shall see | 15:51 |
| purp | So, in reading it, it seems clear we're setting up bare-metal servers for this. | 15:51 |
| purp | You can see it coming: why not VMs? | 15:51 |
| fungi | i must have missed the bare metal part | 15:52 |
| * purp thinks he knows the answer. | 15:52 | |
| *** matrohon has quit IRC | 15:52 | |
| purp | fungi: I could be mistaken. When folks talk about the pains of upgrading OSes on existing servers, that sounds like bare metal. | 15:52 |
| *** f1ller has joined #openstack-infra | 15:52 | |
| fungi | ahh, nope | 15:52 |
| fungi | even just upgrading from one ubuntu lts to another implies at least some substantial downtime for services like apache unless you carefully stage the order of packages you're upgrading (which can be tricky to work out because of interdependencies) | 15:53 |
| *** ryansb has left #openstack-infra | 15:53 | |
| * mordred throws bucket of fish at purp | 15:53 | |
| * purp catches the bucket. mmm, fish. | 15:54 | |
| fungi | talking about operating system major release upgrades, not just regular package updates | 15:54 |
| mordred | purp: we do not treat VMs as flitting ephemeral things round here | 15:54 |
| *** skolekonov is now known as _skolekonov | 15:54 | |
| mgagne | mordred heresy! | 15:55 |
| mordred | mgagne: inorite? | 15:55 |
| fungi | right, virtual machine doesn't necessarily mean cattle | 15:55 |
| *** markus_z_meeting is now known as markus_z | 15:55 | |
| purp | fungi: that makes sense; have been through many painful OS upgrades. Would have expected that a virtualized mirror would mean fire up another, put in proper state, place in service behind LB, and switch off the old. | 15:55 |
| fungi | purp: yep, that's one option | 15:55 |
| mordred | purp: yah - except for a couple of things: | 15:55 |
| persia | state | 15:55 |
| mgagne | 6TB of data transfer later... | 15:55 |
| mordred | that | 15:55 |
| purp | persia mordred: say more? | 15:56 |
| fungi | or shared network filesystem | 15:56 |
| *** EricGonczer_ has joined #openstack-infra | 15:56 | |
| purp | fungi: okay, so it's lack of shared persistence? | 15:56 |
| mordred | purp: state. also, each of our mirrors is single per region, so a LB would be weird | 15:56 |
| fungi | not 6tb (that's a bit exaggeraty) | 15:56 |
| persia | purp: Not all the services used in infra are entirely stateless, so simply swapping them behind an LB doesn't always work cleanly (not that they are all behind LBs, but ...) | 15:56 |
| jeblair | fungi: i have a thing about cattle -- you know what you do when one of your cattle gets sick? you call the friggin vet, because it's worth $5000 and if it just dies you don't get to feed your family. | 15:56 |
| mordred | jeblair: ++ | 15:57 |
| fungi | indeed | 15:57 |
| fungi | olaph can fill us in on cattle expenses | 15:57 |
| * purp applauds jeblair as only a farm kid can. | 15:57 | |
| mgagne | fungi let me live in my own ignorance :O | 15:57 |
| purp | I've had cattle that were pets. | 15:57 |
| * mordred has witnessed olaph giving antibiotics to his cow's udder | 15:57 | |
| jeblair | tasty pets | 15:57 |
| purp | But leaving that analogy for a moment. | 15:57 |
| *** markus_z has quit IRC | 15:57 | |
| purp | jeblair: Oh yeah, I've had Prime Rib of Ol' Hateful. | 15:57 |
| mordred | purp: we have _some_ load-balanced farms of things where the approach of create another / delete first works really well | 15:58 |
| mordred | purp: but we have many things that we treat as long-lived servers | 15:58 |
| fungi | our git server mirrors being the primary example | 15:58 |
| mordred | some of those because we have to (gerrit) | 15:58 |
| fungi | though part of the problem with load balancing is that the load balancers become new pets | 15:59 |
| mordred | others because the effort to make them not that isn't terribly worth it | 15:59 |
| * purp listens and thinks for a moment. | 15:59 | |
| mgagne | fungi fortunately, LB don't have much state into it, easy to transfer elsewhere unlike a mirror | 15:59 |
| fungi | mgagne: this is very true | 15:59 |
| fungi | and if your load balancers have shared state then it can be done with no downtime | 16:00 |
| *** placko has quit IRC | 16:00 | |
| persia | fungi: I'm curious about that: I thought LB with routing rules was a commonly-solved problem, so that LB servers could use shared consensual backing stores. | 16:00 |
| *** placko has joined #openstack-infra | 16:00 | |
| * olaph hands fungi an udder | 16:00 | |
| mordred | persia: then the shared backing store becomes a long-lived server | 16:01 |
| mordred | the prolem with the 12-factor paradigm | 16:01 |
| persia | mordred: consensual stores don't map to servers, but I see the point. | 16:01 |
| mordred | is that it ignores the fact that SOMEWHERE something is providing the persistence services to all the 12-factor apps | 16:01 |
| fungi | well, your network becomes a pet... i think the sorts of lb persia is talking about are injecting routing protocol updates into your network for you | 16:01 |
| mordred | yah | 16:01 |
| persia | Right. Usually with OSPF | 16:02 |
| mordred | and I'm not sure we have the ability to do such things with ours currenty | 16:02 |
| persia | But yes, it does make network a pet. I retract my comments :) | 16:02 |
| fungi | indeed. now where did i put our igp? | 16:02 |
| *** sabeen1 has joined #openstack-infra | 16:02 | |
| mgagne | I'm not sure the cloud providers allow you to manage network traffic with OSPF. | 16:02 |
| fungi | yep, my point | 16:03 |
| *** ihrachys has joined #openstack-infra | 16:03 | |
| sdague | hmmmm so we've got a failure on multinode that is only on ovh | 16:03 |
| fungi | or ibgp hopefully! | 16:03 |
| mordred | so I generally find that just managing the precious services as if tey are precious is less work than trying to think through allowing an external framework to manage some of the precious for you | 16:03 |
| openstackgerrit | Merged openstack-infra/shade: Improve test coverage: container API https://review.openstack.org/253156 | 16:03 |
| * purp will go draw some drawings and think more, then come back with more questions. | 16:03 | |
| mordred | sdague: oh yeah | 16:03 |
| fungi | sdague: connectivity related, storage related, can you tell? | 16:03 |
| mordred | sdague: oh yay | 16:03 |
| mordred | purp: woot! | 16:03 |
| jeblair | https://twitter.com/OpenStack/status/672797858257608704 https://www.openstack.org/blog/2015/12/announcing-a-new-cloud-provider-for-openstacks-ci-system-ovh/ | 16:03 |
| mordred | purp: it's a good question | 16:03 |
| sdague | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message:%5C%22libvirtError:%20cannot%20read%20header%5C%22%20%20%20AND%20message:%5C%22'/dev/disk/by-path%5C%22%20%20%20AND%20tags:%5C%22screen-n-cpu.txt%5C%22 | 16:03 |
| jeblair | that's probably because we just announced it's in production ^ :) | 16:03 |
| sdague | it's libvirt + volumes | 16:04 |
| sdague | on grenade multinode | 16:04 |
| mordred | purp: and one where we have divergent opinions often here than some of our brethren in the cloud space | 16:04 |
| mgagne | it's difficult to come up with a proposed solution when you don't know the baseline feature each providers offer: LB, floating IP, IP reassignment, etc. | 16:04 |
| fungi | i wonder if the 80gb rootfs is still just slightly too small for grenade multinode? | 16:04 |
| purp | mordred: glad I'm not inflicting ignorance on folks this early. It's been known to happen. | 16:04 |
| purp | mordred fungi: Totally get that some things aren't cattle, and not advocating for that. | 16:04 |
| mordred | purp: nope, it's great to dive into it and poke at things | 16:05 |
| fungi | i love having our assumptions challenged. helps avoid tunnel vision | 16:05 |
| jeblair | sdague: i think i see a bluebox in there too? | 16:05 |
| jeblair | sdague, fungi: bb has a similarly sized fs | 16:06 |
| purp | mordred fungi: I'm realizing that drawings will help me explain, so I'll make some. Is there a standard spot to put an image you want to discuss, or should I just use what's easy for me? | 16:06 |
| fungi | docaedo: i have reports that apps.o.o is showing django debug details. i'm about to start looking into it but you have a better grip on that than anyone so help appreciated if you're around | 16:06 |
| sdague | jeblair: yeh, I guess in the 7 day there is also a bb | 16:07 |
| fungi | purp: use whatever's easy for you. we don't have an image paste service | 16:07 |
| mgagne | is the git load-balancer redundant in any way? how hard would it be to switch traffic to an other lb? | 16:07 |
| purp | fungi: oteydotey. Thanks! | 16:07 |
| openstackgerrit | Merged openstack-infra/shade: Improve test coverage: server secgroup API https://review.openstack.org/253516 | 16:07 |
| jeblair | mgagne: when we change it out, we just switch DNS. | 16:08 |
| fungi | mgagne: it was "redundant" for a while (via dns round-robin) but that wasn't for failover purposes it was to deal with additional volume | 16:08 |
| jeblair | (which didn't actually work that well for our backends because the 2 frontends didn't share state) | 16:08 |
| fungi | but yeah, we boot a replacement lb, then switch dns with a reasonably short ttl, and wait for connections to move and then tear down the old one once it's safe | 16:08 |
| mgagne | right. so having a load-balancer "could" allow you to upgrade the mirror by switching backend servers (assuming data is on a shared fs) when the new one is ready. as for load-balancer upgrade, it should be a matter of DNS. | 16:09 |
| fungi | keep in mind that the git.o.o load balancer isn't configured to do failover and health checks--it's purely for balancing load not for failure mitigation | 16:09 |
| mgagne | yea, it's more or less status-quo regarding HA | 16:10 |
| clarkb | (it does do simple tcp syn ack checks though) | 16:10 |
| mgagne | just more tool to ease OS upgrades | 16:10 |
| fungi | we do manually down backend servers in it before replacing them, sure | 16:10 |
| *** AJaeger has joined #openstack-infra | 16:10 | |
| tsufiev | hello, guys! | 16:10 |
| tsufiev | are there some known network issues today? | 16:10 |
| fungi | but again, the issue is that replacing the pypi mirrors is more about getting the rather large amount of data available to the replacement | 16:11 |
| tsufiev | e.g. https://review.openstack.org/#/c/252961/ jobs involving devstack install are failing again and again | 16:11 |
| jeblair | sdague: http://logs.openstack.org/09/253109/1/check/gate-grenade-dsvm-multinode/f951ff8/logs/worlddump-2015-12-04-081251.txt.gz | 16:11 |
| mgagne | fungi shared file system, I don't see much alternative | 16:11 |
| jeblair | sdague: that says filesystem is not the cause, right? | 16:11 |
| fungi | mgagne: right, but having a load balancer in front of the old and new server doesn't help much either way | 16:12 |
| fungi | certainly doesn't solve the shared data problem | 16:12 |
| mgagne | fungi how? | 16:12 |
| mgagne | fungi you mean there is no solution to the shared data problem? | 16:12 |
| fungi | mgagne: the problem is one of shared data between the old and new servers. a load balancer does nothing to solve _that_ challenge | 16:12 |
| mgagne | fungi I agree | 16:12 |
| AJaeger | fungi, yes, you reverted the tox bulider on 2nd of October already ;( I986261dc2233d59383b0f9e35bb74ba81c95253f | 16:13 |
| *** u_glide1 has left #openstack-infra | 16:13 | |
| mgagne | fungi especially if you have access to DNS and TTL | 16:13 |
| fungi | right | 16:13 |
| fungi | moving connections between servers is already a solved problem for us | 16:14 |
| rcarrillocruz | SpamapS: i summon you! | 16:14 |
| rcarrillocruz | please have a look when you get a chance | 16:14 |
| rcarrillocruz | https://review.openstack.org/#/c/252037/ | 16:14 |
| rcarrillocruz | i need you confirm something i'm seeing from our friend upstart | 16:14 |
| *** annegentle has joined #openstack-infra | 16:14 | |
| fungi | AJaeger: i can understand how it happens, it's just disappointing almost to the point of irony | 16:15 |
| *** xyang1 has joined #openstack-infra | 16:15 | |
| AJaeger | fungi: yeah... | 16:16 |
| AJaeger | and reading backscroll you referenced it the change already, sorry for the noise... | 16:16 |
| AJaeger | yolanda, fungi: Is everything green now? I'd like to send an email to openstack-dev... | 16:16 |
| yolanda | AJaeger, not yet | 16:17 |
| yolanda | jenkins02 still reconfiguring :( | 16:17 |
| yolanda | it's incredibly slow | 16:17 |
| jeblair | yolanda: https://review.openstack.org/253554 approved, but you'll want to add yourself to statusbot in system-config manifests/site.pp | 16:17 |
| *** armax has joined #openstack-infra | 16:18 | |
| AJaeger | yolanda: What fun ;( | 16:18 |
| yolanda | jeblair ok, thanks | 16:18 |
| fungi | yolanda: but with it set to prepare for shutdown it shouldn't run any more jobs, so i think it's safe to sound the all clear to the rest of the community while we quietly work on jenkins02 | 16:18 |
| jeblair | fungi: ++ | 16:18 |
| AJaeger | yolanda: but you shutdown jenkins02, correct? So it's fine... | 16:18 |
| AJaeger | fungi is typing too fast for me ;) | 16:18 |
| *** lucas-hungry is now known as lucasagomes | 16:19 | |
| yolanda | AJaeger yes, you are right | 16:19 |
| AJaeger | who wants to write the email? I can but if somebody else prefers... | 16:19 |
| fungi | AJaeger: i will attenuate my typing speed on your behalf ;) | 16:19 |
| AJaeger | fungi: don't! I'm just tired. As long as you forgive me for echoing you... ;) | 16:19 |
| *** bpokorny has joined #openstack-infra | 16:20 | |
| fungi | trying to figure out what's going on with the django debug stuff on apps.o.o so i can get back to juno eol stuff | 16:20 |
| * AJaeger sends the email then... | 16:22 | |
| AJaeger | Regarding Juno EOL: clarkb, are you going to review the python 2.6 removal? https://review.openstack.org/#/c/252448/ | 16:22 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add a query for libvirt error when 'TestBootVolumePattern' fails https://review.openstack.org/253134 | 16:23 |
| *** arxcruz has quit IRC | 16:23 | |
| *** dimtruck is now known as zz_dimtruck | 16:26 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/system-config: Add yolanda to statusbot nicks https://review.openstack.org/253586 | 16:26 |
| *** sputnik13 has joined #openstack-infra | 16:27 | |
| pedroalvarez | Is that patch ^ what comes before saying that CI is ok again?? :) | 16:28 |
| *** placko has quit IRC | 16:28 | |
| *** ifarkas has quit IRC | 16:28 | |
| *** agentle_ has joined #openstack-infra | 16:28 | |
| *** dmsimard|afk is now known as dmsimard | 16:29 | |
| yolanda | ttx, fungi, can you send a notice to status, telling that everything should be back again?¿ | 16:29 |
| yolanda | i may still not have access | 16:29 |
| fungi | docaedo: i'm not able to find whatever error fifieldt described, so maybe it was corrected when https://review.openstack.org/250061 merged? | 16:30 |
| AJaeger | yolanda: wait 5minutes until your patch mergs ;) | 16:30 |
| yolanda | so i have the honors :) | 16:30 |
| fungi | yeah, that'll be a fine test of it ;) | 16:30 |
| * AJaeger leaves again.. | 16:31 | |
| *** AJaeger has quit IRC | 16:32 | |
| *** annegentle has quit IRC | 16:32 | |
| clarkb | should we rename our macro? | 16:32 |
| yolanda | so it should be something like "status ok we fixed" ? | 16:32 |
| clarkb | this-is-tox | 16:32 |
| clarkb | and avoid this common collision in the future? | 16:32 |
| yolanda | clarkb sounds like a good idea | 16:33 |
| *** hashar has quit IRC | 16:36 | |
| fungi | clarkb: mordred: looking at the git log on puppetmaster.o.o, it appears the hiera keys i added back in september for smarcet's 214344 got removed when the fqdn refactoring took place | 16:36 |
| fungi | how was the refactoring of hiera keys accomplished? | 16:36 |
| fungi | did we maybe lose any that weren't referenced in our global site manifest? | 16:37 |
| docaedo | fungi: will read scroll-back and see what's up :) | 16:37 |
| fungi | docaedo: all the detail i have was a message from fifieldt at 05:10 utc stating "apps.openstack.org http://apps.openstack.org has django debug on..." | 16:38 |
| fungi | he doesn't seem to be responding at the moment, so i don't know any more than that | 16:38 |
| mordred | fungi: yes. that is correct. I believe if we did not refernece them in the manifest the script may not have put them anywhere | 16:38 |
| mordred | fungi: because it did not know which servers should have access to them | 16:39 |
| fungi | mordred: got it. so any we added for changes that were still in review need to be readded | 16:39 |
| mordred | yah | 16:39 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard: Document manually setting up notifications https://review.openstack.org/253591 | 16:39 |
| mordred | sorry bout that - bug in the migration | 16:39 |
| fungi | infra-root: ^ heads up to double-check hiera keys for open changes that predate the fqdn refactor, as they may now be gone | 16:39 |
| mordred | they WILL be in the git history | 16:39 |
| docaedo | fungi: ok - debug is still on but I believe this is because no changes to app-catalog repo have happened (which will trigger all the things from the manifest) | 16:40 |
| *** ericksonsantos has joined #openstack-infra | 16:40 | |
| *** zz_dimtruck is now known as dimtruck | 16:40 | |
| docaedo | Later today when there's not too much exciting stuff happening and a rooter is around, I'll merge something minor to trigger everything (which should turn off debug) | 16:40 |
| fungi | mordred: yep, that's where i managed to first confirm i hadn't gone insane and imagined i'd added them | 16:40 |
| fungi | docaedo: got it, that makes sense. thanks! | 16:41 |
| *** FallenPegasus is now known as MarkAtwood | 16:41 | |
| *** _nadya_ has quit IRC | 16:42 | |
| *** dramalho has joined #openstack-infra | 16:43 | |
| *** othermordred has quit IRC | 16:44 | |
| *** warpc has quit IRC | 16:44 | |
| *** samyewell has quit IRC | 16:44 | |
| *** warpc has joined #openstack-infra | 16:44 | |
| *** samyewell has joined #openstack-infra | 16:44 | |
| *** othermordred has joined #openstack-infra | 16:44 | |
| fungi | infra-root: there is a staged change to production/fqdn/puppetmaster.openstack.org.yaml on puppetmaster.openstack.org which hasn't been committed yet. looks like someone was renaming openstackci_rax_ keys to openstackjenkins_rax_ | 16:45 |
| jeblair | mordred: ^ probably? | 16:46 |
| *** kozhukalov_ has joined #openstack-infra | 16:47 | |
| *** rfolco has quit IRC | 16:48 | |
| mordred | yes | 16:48 |
| mordred | that was me | 16:48 |
| mordred | pleaes commit it | 16:48 |
| openstackgerrit | Yan Ning proposed openstack-infra/project-config: add seperate job for monasca plugin in devstack install https://review.openstack.org/244893 | 16:48 |
| *** david-lyle has quit IRC | 16:48 | |
| mordred | the other thing was a typo | 16:48 |
| fungi | mordred: will do, thanks | 16:49 |
| mordred | sorry | 16:49 |
| fungi | just making sure i'm not stepping on anyone's toes | 16:49 |
| *** Ramanjaneya has quit IRC | 16:49 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Use reno for release notes https://review.openstack.org/253597 | 16:49 |
| *** david-lyle has joined #openstack-infra | 16:49 | |
| yolanda | fungi, my change hasn't landed , it even didn't pass gate and i need to leave. Are you ok to send the announce? i can test in another moment :) | 16:50 |
| fungi | yolanda: sure, someone else with access can send a notice. have a good evening! | 16:51 |
| openstackgerrit | Vladimir Kozhukalov proposed openstack-infra/project-config: Create bareon project https://review.openstack.org/231677 | 16:51 |
| *** rcernin has quit IRC | 16:52 | |
| fungi | #status notice The earlier JJB bug which disrupted tox-based job configurations has been reverted and applied; jobs seem to be running successfully for the past two hours. | 16:52 |
| openstackstatus | fungi: sending notice | 16:52 |
| *** gyee has joined #openstack-infra | 16:53 | |
| *** sridhar_ram has joined #openstack-infra | 16:53 | |
| -openstackstatus- NOTICE: The earlier JJB bug which disrupted tox-based job configurations has been reverted and applied; jobs seem to be running successfully for the past two hours. | 16:53 | |
| *** dizquierdo has quit IRC | 16:54 | |
| *** dtantsur is now known as dtantsur|afk | 16:54 | |
| openstackgerrit | Yan Ning proposed openstack-infra/project-config: add seperate job for monasca plugin in devstack install https://review.openstack.org/244893 | 16:54 |
| *** rfolco has joined #openstack-infra | 16:55 | |
| openstackstatus | fungi: finished sending notice | 16:55 |
| *** Piet__ has joined #openstack-infra | 16:55 | |
| *** tsg has quit IRC | 16:55 | |
| *** jlanoux has quit IRC | 16:59 | |
| *** otter768 has joined #openstack-infra | 16:59 | |
| pleia2 | good morning | 16:59 |
| openstackgerrit | Merged openstack-infra/shade: Use non-versioned cinderclient constructor https://review.openstack.org/253547 | 17:00 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: elaborate on how to check out releases repo to run releases by hand https://review.openstack.org/253599 | 17:00 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: elaborate on requirements handling in the scripts https://review.openstack.org/253600 | 17:00 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: add instructions for using announce.sh https://review.openstack.org/253601 | 17:00 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: remove release_postversion.sh https://review.openstack.org/253602 | 17:00 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: cosmetic changes to command examples https://review.openstack.org/253603 | 17:00 |
| openstackgerrit | Merged openstack-infra/shade: Change the client imports to stop shadowing https://review.openstack.org/253548 | 17:00 |
| openstackgerrit | Dmitry Tantsur proposed openstack/requirements: Cap pydev at <0.18 https://review.openstack.org/253448 | 17:01 |
| *** salv-orlando has joined #openstack-infra | 17:01 | |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder: Add dib element to generate logical volumes https://review.openstack.org/252041 | 17:01 |
| *** tqtran has joined #openstack-infra | 17:02 | |
| openstackgerrit | Merged openstack-infra/storyboard: Work around "duplicate preferences" issue https://review.openstack.org/251970 | 17:02 |
| eantyshev | Hello, guys! Does anyone want to review a small imrovement for system-config: https://review.openstack.org/239479 ? | 17:03 |
| *** otter768 has quit IRC | 17:04 | |
| nibalizer | pleia2: good morning | 17:07 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Improve test coverage: list_router_interfaces API https://review.openstack.org/253610 | 17:07 |
| *** mpavone has quit IRC | 17:09 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/system-config: Added Gem Mirror to Infra https://review.openstack.org/253616 | 17:10 |
| EmilienM | krotscheck: ^ | 17:11 |
| krotscheck | EmilienM: Sweet, looking | 17:11 |
| *** david-lyle has quit IRC | 17:13 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/requirements: Exclude pyudev 0.18 https://review.openstack.org/253448 | 17:13 |
| krotscheck | EmilienM: My only comment is that the robots.txt doesn't need to be there, it's already provided at the root http level. | 17:14 |
| EmilienM | krotscheck: ack, dropping it! | 17:14 |
| krotscheck | EmilienM: Sweet! | 17:15 |
| EmilienM | krotscheck: your work on that is awesome - big thank you | 17:15 |
| openstackgerrit | Merged openstack-infra/project-config: [sahara] gate-grenade-dsvm-sahara to merge check https://review.openstack.org/253473 | 17:15 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/system-config: Added Gem Mirror to Infra https://review.openstack.org/253616 | 17:15 |
| krotscheck | EmilienM: Likewise! Your assistance reduces the load on me just a bit :) | 17:15 |
| openstackgerrit | Evgeny Antyshev proposed openstack-infra/system-config: Use httpd::mod instead of httpd_mod https://review.openstack.org/253619 | 17:16 |
| ociuhandu | hello, on a gerrit CI account created using the old system (mail request), how can we manage the ssh keys? As back then there was no launchpad account created, from what we have seen | 17:18 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Make the recent events list collapsible https://review.openstack.org/253497 | 17:18 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Don't close the "add item" modal until the items are added https://review.openstack.org/253526 | 17:19 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Update $resource wrappers to use new permissions API https://review.openstack.org/235475 | 17:19 |
| openstackgerrit | Evgeny Antyshev proposed openstack-infra/system-config: Use httpd::mod instead of httpd_mod https://review.openstack.org/253619 | 17:19 |
| *** sambetts is now known as sambetts-afk | 17:19 | |
| fungi | codesearch.o.o is throwing an error page. anyone have time to look into what's happened to hound? | 17:21 |
| ociuhandu | can we use directly the gerrit set-account —add-ssh-key cmd? | 17:21 |
| jeblair | fungi: will do | 17:21 |
| jeblair | (i'm waiting for nodepool) | 17:22 |
| fungi | ociuhandu: you can just create a new account in gerrit and let us know which old account to disable | 17:22 |
| jeblair | fungi: wfm | 17:22 |
| fungi | ociuhandu: that's what everyone else has done. it gets you an account under your control. otherwise the next time you need a change made to the old account you end up having to ask us to do it again (and again, and again) | 17:23 |
| fungi | thanks jeblair! | 17:23 |
| jeblair | fungi: (i mean, i didn't do anything, it just wfm) | 17:23 |
| fungi | oh weird, it's up now | 17:23 |
| fungi | i wonder if it goes dormant somehow and needs to wake up a little | 17:24 |
| ociuhandu | fungi: thank you | 17:24 |
| fungi | jeblair: i definitely had an error from apache there the first time i tried to hit it | 17:24 |
| fungi | (mere moments ago) | 17:24 |
| fungi | perhaps i caught it in the middle of puppet restarting something | 17:25 |
| pabelanger | clarkb: I'm free for the next 2 hours or so, then stepping out for an hour | 17:26 |
| *** dramalho has quit IRC | 17:27 | |
| *** katyafervent has quit IRC | 17:27 | |
| *** hashar has joined #openstack-infra | 17:28 | |
| *** kzaitsev_ws has quit IRC | 17:28 | |
| *** kzaitsev_ws has joined #openstack-infra | 17:29 | |
| *** yamamoto has joined #openstack-infra | 17:29 | |
| *** kozhukalov_ has quit IRC | 17:29 | |
| *** dramalho has joined #openstack-infra | 17:30 | |
| *** tlian has quit IRC | 17:30 | |
| *** abregman is now known as abregman|afk | 17:30 | |
| *** tlian has joined #openstack-infra | 17:32 | |
| SpamapS | rcarrillocruz: ACK | 17:33 |
| krotscheck | Anyone around to assuage my concerns? As soon as I started the unified mirror effort, people are coming out of the woodwork wanting there to be more mirrors. I'm a little worried that putting all the things on the same mirror will cause things to become less stable, because those machines will get hammered. | 17:33 |
| jeblair | krotscheck: check cacti | 17:33 |
| *** yamamoto has quit IRC | 17:34 | |
| jeblair | krotscheck: look at the mirrors and see if they show signs of being under heavy load | 17:34 |
| mordred | krotscheck: also, I have a hunch that the amount we hammer the python mirror with all the openstack tests will dwarf the amounts of load added by additional things for quite some time | 17:34 |
| *** spzala has joined #openstack-infra | 17:34 | |
| krotscheck | mordred: You sure? mrunge just asked for an ubuntu mirror. | 17:35 |
| mordred | krotscheck: not beause of any particular quality, but because of the giant number of python jobs that download things | 17:35 |
| mordred | krotscheck: we'd still be doing many many more python mirrror hits | 17:35 |
| krotscheck | jeblair: Yeah, those don't look like they're under heavy load. | 17:35 |
| mordred | krotscheck: since we mostly pre-cache ubuntu packages on hosts and we don't install packages on unittest jobs | 17:35 |
| SpamapS | An ubuntu mirror for server packages is hard to "hammer" | 17:35 |
| jeblair | krotscheck: yeah, this is probably our busiest mirror: http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=2482&rra_id=all | 17:35 |
| mordred | krotscheck: bbut we DO install python packages for everything from mirrors - even for pep8 jos | 17:36 |
| SpamapS | You don't have any openoffice or GB of images.. | 17:36 |
| mordred | jobs | 17:36 |
| krotscheck | Ok, I am less worried now. | 17:36 |
| mordred | yay! | 17:36 |
| SpamapS | average package size for server stuff is probably 150k | 17:36 |
| SpamapS | With a few 10MB'ers.. things like mysql and kernels being the exception. | 17:36 |
| *** katyafervent has joined #openstack-infra | 17:36 | |
| mordred | this is also easy-ish to scale if it does start to be hamered - but I don't expect that to happen for a while | 17:36 |
| mordred | SpamapS: yup - which are things we pre-cache anyway | 17:36 |
| krotscheck | Especially since we're split by region. | 17:36 |
| *** Guest93374 has quit IRC | 17:37 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/infra-specs: Unified Mirrors https://review.openstack.org/252678 | 17:37 |
| SpamapS | mordred: right, because if we didn't have a mirror to hammer, we'd be hammering external bandwidth anyway. | 17:37 |
| *** asselin__ has quit IRC | 17:37 | |
| SpamapS | so, I say, HAMMER AWAY | 17:37 |
| mordred | krotscheck: btw - ubuntu mirror is I believe a different effort that we've got some thoughts on | 17:37 |
| mordred | krotscheck: the problem we have with those is atomic updates typically | 17:38 |
| mordred | due to the way ubuntu archive mirroring works | 17:38 |
| mordred | so if we ran one normally, it would suffer from essentially the same problems as the existing per-cloud mirrors | 17:38 |
| jeblair | i have a filesystem for that | 17:38 |
| mordred | yup | 17:38 |
| jeblair | (maybe even two) | 17:39 |
| krotscheck | jeblair: Does the filesystem have a name? | 17:39 |
| krotscheck | Is the name Andrew? ;) | 17:39 |
| mordred | mainly mentioning it as a note that I don't think we need to try to solve that problem with this effort just right now | 17:39 |
| mordred | krotscheck: :) | 17:39 |
| jeblair | krotscheck: i see you've met our friend andrew! | 17:39 |
| krotscheck | jeblair: Oh, andrew and I go Waaaaay back. | 17:39 |
| * krotscheck has a CMU degree collecting dust around here somewhere... | 17:39 | |
| krotscheck | mordred: Alright- think you can come up with a "todo list"section on https://review.openstack.org/252678 for ubuntu? If so, I can see if I can convince mrunge to pick up the work for it :) | 17:40 |
| *** jordanP has quit IRC | 17:40 | |
| SpamapS | Note that apt 1.1 fixes the mirror problem | 17:40 |
| fungi | jeblair: mordred: clarkb: infrastructure donors are now included at http://www.openstack.org/foundation/companies/ | 17:40 |
| SpamapS | and will ship with 16.04 | 17:40 |
| jeblair | fungi: rock on! | 17:41 |
| SpamapS | and presumably the Ubuntu mirrors will be built the correct way with the archive bits that 1.1 uses to be atomic. | 17:41 |
| SpamapS | So, in about 6 months.. we can MAYBE start caring less about that. :) | 17:41 |
| *** asselin_ has joined #openstack-infra | 17:41 | |
| fungi | jeblair: i can already see i need to suggest a tweak to the wording, since not all our infrastructure donors will necessarily be running public clouds | 17:42 |
| krotscheck | SpamapS: There are a lot of neat things that will hopefully be in 10.04 | 17:42 |
| jeblair | fungi: yeah, fortunately that works for those 3 :) | 17:42 |
| krotscheck | I mean 16.04 | 17:42 |
| SpamapS | rcarrillocruz: hm | 17:42 |
| SpamapS | rcarrillocruz: so.. the glean thing.. | 17:42 |
| mordred | SpamapS: oh, we were just goign to fix the archive at the archive level | 17:43 |
| *** unicell1 has quit IRC | 17:43 | |
| mordred | SpamapS: since it's 100% not a hard thing to do | 17:43 |
| SpamapS | rcarrillocruz: I kind of wonder if we shouldn't change glean to just always write vlan interfaces that are hosted on the interface being processed. | 17:43 |
| jeblair | 10.04 was the best. not kidding. | 17:43 |
| mordred | ++ | 17:43 |
| SpamapS | 10.04 had the _worst_ upstart server files ever tho | 17:43 |
| *** unicell has joined #openstack-infra | 17:43 | |
| SpamapS | mysql _often_ failed | 17:43 |
| mordred | SpamapS: atomic filesystem snapshot/release after archive integrity check | 17:43 |
| SpamapS | and the shutdown would corrupt your database | 17:43 |
| SpamapS | but yeah.. best ever. ;) | 17:44 |
| fungi | jeblair: well, at least for the next month until hpe is no longer running a public cloud ;) | 17:44 |
| mordred | SpamapS: so that there are no in-flight updates | 17:44 |
| SpamapS | mordred: I understand that you can fix it without apt's help. I don't think that is actually a solution that works for everyone though. | 17:44 |
| mordred | SpamapS: I'm not sure I follow you? | 17:44 |
| SpamapS | mordred: and it's also 100% unnecessary if the metadata is updated in an atomic way. | 17:44 |
| mordred | SpamapS: I mena, apt fixing it too is awesome | 17:44 |
| jeblair | you are risking violent agreement :) | 17:45 |
| krotscheck | So, with that in mind, the mirror move I'm proposing is actually infra-core intensive, and I'm headed on vacation next friday. Is this something we could do next week, or shall we postpone to 21st-and-after? | 17:45 |
| mordred | SpamapS: I'm just saying that for the period of time that we still have things that are not running 16.04, fixing it server seide for us will be a benefit | 17:45 |
| SpamapS | You wouldn't have to care about snapshotting if apt-ftparchive actually was able to change things atomically. | 17:45 |
| mordred | SpamapS: yes | 17:45 |
| jeblair | oh, the fix is in apt-ftparchive? | 17:45 |
| *** yamahata has quit IRC | 17:45 | |
| jeblair | so it's also a server-side fix? | 17:45 |
| *** derekh has quit IRC | 17:45 | |
| SpamapS | jeblair: it has to be in apt too | 17:45 |
| jeblair | oh | 17:45 |
| mordred | SpamapS: but since it doesn't - we will just do our things on the server side | 17:45 |
| SpamapS | jeblair: the problem was the format of the metadata | 17:46 |
| mordred | and then at some point it will become not-necessary | 17:46 |
| mordred | and that'll be neat too | 17:46 |
| clarkb | pabelanger will have time in about 15-20 minutes | 17:46 |
| SpamapS | mordred: sure, if the urgency is high, we should just do it. | 17:46 |
| SpamapS | because I suspect 14.04 will be a thing for a long time. :) | 17:46 |
| *** fawadkhaliq has joined #openstack-infra | 17:46 | |
| mordred | yup | 17:46 |
| pabelanger | clarkb: sounds good | 17:46 |
| mordred | SpamapS: we have it designed, it just isnt' quite urgent enough that we've done the work on it yet :) | 17:46 |
| *** stevemar_znc is now known as stevemar | 17:47 | |
| mordred | krotscheck: I'm travelling next week, so I'm less useful then | 17:47 |
| mordred | krotscheck: wait - wasn't I supposed to help you root something today? | 17:47 |
| jeblair | (fyi, i have rudely aborted some jenkins jobs with no regard to the consequences in order to debug something, sorry) | 17:48 |
| *** unicell has quit IRC | 17:48 | |
| *** sridhar_ram has quit IRC | 17:48 | |
| mordred | oops | 17:48 |
| krotscheck | mordred: Yes, at 10AM, and then we realized that greghaynes's patches made more sense if we did the rename first. | 17:48 |
| krotscheck | So I followed up by pinging you that rooting things today would probably not happen. | 17:49 |
| mordred | awesome | 17:49 |
| zaro | morning | 17:49 |
| mordred | krotscheck: sweet | 17:49 |
| krotscheck | But, hey, if you want to help me root things, go review this and the following 3 patches https://review.openstack.org/#/c/253236/ :) | 17:50 |
| jeblair | krotscheck: when do you think the rename/move patches will be ready? | 17:50 |
| krotscheck | jeblair: I rebuilt the entire patch chain this morning. | 17:50 |
| *** Swami has joined #openstack-infra | 17:50 | |
| krotscheck | See ^^ | 17:50 |
| krotscheck | So the answer is "now" | 17:51 |
| krotscheck | Well. | 17:51 |
| krotscheck | Now + review time | 17:51 |
| krotscheck | Because I like extra eyeballs. | 17:51 |
| krotscheck | (They pop in my mouth like candy) | 17:51 |
| *** amitgandhinz has quit IRC | 17:51 | |
| jeblair | i'd like to assume any nodepool node held for more than 30 days is probably not actually in use. | 17:51 |
| jeblair | anyone object to me deleting those? | 17:51 |
| jeblair | good | 17:52 |
| mordred | jeblair: do it | 17:52 |
| onovy | hi, can someone help me with https://review.openstack.org/#/c/245603/ pls? | 17:52 |
| mordred | krotscheck: mmm. candy | 17:52 |
| *** jamesmcarthur has quit IRC | 17:53 | |
| jeblair | sdague, clarkb: i have 2 devstack-trusty-2-node sets, one each on ovh and rax | 17:53 |
| *** dizquierdo has joined #openstack-infra | 17:53 | |
| *** ihrachys has quit IRC | 17:53 | |
| jeblair | sdague, clarkb: i'm prepared to re-run a failing grenade job on both | 17:53 |
| krotscheck | Incidentally, if anyone wants the vagrantfile and local.pp patch that I used to step through my patches, I'd be more than happy to provide. | 17:53 |
| *** ihrachys_ has joined #openstack-infra | 17:54 | |
| jeblair | sdague, clarkb: anything you would lke me to do before i start? | 17:54 |
| mordred | krotscheck, jeblair, fungi: does a cinder volume snapshot cause things to freeze like a nova one does? | 17:55 |
| krotscheck | mordred: I do not konw. | 17:55 |
| jeblair | mordred: no idea :) | 17:55 |
| jeblair | if we're worried, let's start with rax-dfw and quiesce it if there's a problem | 17:56 |
| openstackgerrit | Evgeny Antyshev proposed openstack-infra/puppet-openstackci: Allow logserver to run on CI server https://review.openstack.org/252508 | 17:56 |
| jeblair | that's our least-used old-mirror region right now | 17:56 |
| electrofelix | jeblair zaro: so I'm happy to go with exclude combining or individual styles for the result_WARNING change https://review.openstack.org/253462 is there a preferred style? | 17:56 |
| jeblair | electrofelix: no preference | 17:57 |
| krotscheck | mordred: A brief google says that it's driver specific. | 17:57 |
| krotscheck | mordred, jeblair: rsync may just work better. | 17:57 |
| krotscheck | more better. | 17:57 |
| mordred | yah. it might take longer - but it's intra-cloud traffic and should not cause the mirror to become unusable | 17:58 |
| jeblair | erm | 17:58 |
| sdague | jeblair: honestly, I don't have a full handle on the failures, I think kashyap was looking at that one a little. But even the theory about what's going on is a little weird. | 17:58 |
| jeblair | the two hosts i've logged into look different | 17:58 |
| jeblair | oh... rax is probably still a snapshot build | 17:59 |
| openstackgerrit | Yan Ning proposed openstack-infra/project-config: add seperate job for monasca plugin in devstack install https://review.openstack.org/244893 | 17:59 |
| jeblair | sdague: think it's useful for me to run reproduce so people can poke around? | 17:59 |
| mordred | krotscheck: commented on the spec with the thing about rsync - other than that looks great! | 18:00 |
| sdague | jeblair: I'm not sure, because I think the useful postmortem might get destroyed by nova cleaning things up | 18:00 |
| sdague | before you could catch it | 18:00 |
| zaro | electrofelix: replied in the change | 18:01 |
| *** lezbar has quit IRC | 18:01 | |
| fungi | mordred: jeblair: in situations where we can use a cinder volume, i expect just turning off anything which is writing to that and then switching the filesystem read-only ought (in theory) to make it possible to continue to read from the volume while it's being snapshotted | 18:01 |
| fungi | but of course i have no evidence to back this up | 18:01 |
| *** lezbar has joined #openstack-infra | 18:01 | |
| jeblair | sdague: hrm. well, good, cause apparently reproduce.sh doesn't work with multinode grenade jobs :) | 18:01 |
| *** dizquierdo has quit IRC | 18:02 | |
| mordred | fungi: yah - seems like another branching condition to think about that's unlikely to be significantly better? | 18:02 |
| jeblair | sdague: i'll keep these nodes around though for a bit in case you or someone wants to run something manually on them | 18:02 |
| fungi | mordred: indeed | 18:02 |
| krotscheck | mordred: I will ask the cinder people. They might know. | 18:03 |
| sdague | I think for right now, we're just going to keep an eye on the fail rates in ER, and see where it hits a threshold | 18:03 |
| zaro | fungi, jeblair : have you guys got email for gerrit upgrade? i haven't seen it. | 18:03 |
| fungi | also with dfw, we're already so low utilization there after the most recent nodepool config tweak, we could consider dialing it down to 0 before we work on that mirror to eliminate disruption | 18:03 |
| *** sdake has quit IRC | 18:03 | |
| nibalizer | zaro: i sent it | 18:03 |
| fungi | zaro: i saw the one nibalizer sent to the -dev ml | 18:03 |
| jeblair | sdague: we are likely to run more jobs on ovh over time and less jobs elsewhere | 18:03 |
| *** sdake has joined #openstack-infra | 18:04 | |
| fungi | zaro: was either tuesday after the meeting or maybe sometime wednesday | 18:04 |
| *** dizquierdo has joined #openstack-infra | 18:04 | |
| zaro | ok, found it. sorry about that. | 18:04 |
| nibalizer | unfortunately I forgot to [topic] it | 18:05 |
| jeblair | i never topic those | 18:05 |
| clarkb | pabelanger: ok here now | 18:05 |
| *** lezbar has quit IRC | 18:06 | |
| clarkb | pabelanger: I should start booting trusty asterisk host, stop centos6 asterisk service when that is puppeted? | 18:06 |
| *** Piet__ has quit IRC | 18:06 | |
| *** lxsli has quit IRC | 18:06 | |
| *** dzamboni has quit IRC | 18:07 | |
| pabelanger | clarkb: I'd stop centos6 first, before puppet. Otherwise, we might need to way 5/10mins for the registration of asterisk and voip.ms | 18:08 |
| clarkb | pabelanger: ok | 18:08 |
| krotscheck | mordred: Response from jgriffith - it's apparently just an LVM copy, so the only concern is that it might look werid if bandersnatch is still writing to the volume during the snapshot. | 18:08 |
| *** jcoufal has quit IRC | 18:08 | |
| *** fhubik has quit IRC | 18:08 | |
| krotscheck | mordred: So- disable bandersnatch, make copy, start bandersnatch. | 18:08 |
| clarkb | pabelanger: current asterisk server is a 2GB standard rax flavor | 18:09 |
| jgriffith | krotscheck: lvm snap just to be precise | 18:09 |
| *** spzala has quit IRC | 18:09 | |
| mordred | jgriffith: but that's only the LVM backend, yeah? | 18:09 |
| jgriffith | krotscheck: the way it works is it just creates a cow table | 18:09 |
| clarkb | pabelanger: iirc jeblair did a bunch of testing across flavors and found that was fine so I am going to stick with that unless there is objections | 18:09 |
| pabelanger | clarkb: Works for me | 18:10 |
| jgriffith | mordred: yeah.. perhaps I don't have all the info here ;) | 18:10 |
| krotscheck | Cow? | 18:10 |
| mordred | jgriffith: I'm guessing that other drivers, such as say solidfire, might do something different? | 18:10 |
| jgriffith | krotscheck: copy on write | 18:10 |
| mordred | jgriffith: we're wondering about doing volume snaps on our public cloud accounts | 18:10 |
| mordred | jgriffith: and what effect that would have on the service using those volumes | 18:10 |
| mordred | jgriffith: for clouds where we know nothing about the implementation of the cloud | 18:10 |
| jgriffith | mordred: if it's a descent backend NONE | 18:10 |
| *** yamahata has joined #openstack-infra | 18:10 | |
| mordred | k | 18:10 |
| clarkb | pabelanger: jeblair actually I am tempted to boot it on a performance 2GB | 18:10 |
| jgriffith | mordred: and by Cinder standards that's how it's designed as well | 18:11 |
| clarkb | it has a better rx/tx factor and only 40GB of disk instead of 80GB | 18:11 |
| mordred | kk | 18:11 |
| jeblair | clarkb: it probably predates performance | 18:11 |
| jgriffith | mordred: if you hit a snag, you discovered somebody that sucks and isn't testing | 18:11 |
| clarkb | jeblair: should I make new one performance? | 18:11 |
| mordred | jgriffith: so we should assume that snaps are safe then | 18:11 |
| mordred | jgriffith: well ... | 18:11 |
| jeblair | clarkb: yeah, worth a try | 18:11 |
| mordred | jgriffith: now you just described my normal week :) | 18:11 |
| jgriffith | mordred: absolutely... again assuming I understand everything correctly. Cinder snapshots absolutely better be safe | 18:11 |
| jeblair | clarkb: i'd say all the old testing is probably not relevant at this point anyway :) | 18:11 |
| mordred | cool | 18:11 |
| mordred | jgriffith: thanks! | 18:11 |
| jgriffith | mordred: no problem | 18:11 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Tidy up project groups states https://review.openstack.org/253049 | 18:12 |
| fungi | clarkb: agreed, i'm in favor of consistency of what we're using for other new servers at this point, and then if we discover performance-related regressions we can consider alternatives | 18:13 |
| clarkb | ok new server is building | 18:13 |
| clarkb | will stop asterisk on old as soon as new server starts puppeting | 18:14 |
| clarkb | done | 18:14 |
| clarkb | now we wait | 18:14 |
| *** hdd has quit IRC | 18:14 | |
| pabelanger | dun dun dun | 18:14 |
| nibalizer | not something that gets done every day | 18:15 |
| fungi | did anyone send out an e-mail to the -dev list about taking pbx.o.o down briefly? just in case someone is trying to use it for a call | 18:15 |
| clarkb | fungi: woops I did not | 18:15 |
| pabelanger | I did not either | 18:15 |
| fungi | fingers crossed that it's not particularly popular ;) | 18:15 |
| clarkb | pabelanger: any chance you can do that really quickly? | 18:15 |
| pabelanger | I can | 18:15 |
| clarkb | pabelanger: you can blame me for any dropped service :) | 18:15 |
| fungi | heh | 18:15 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder: Infer and build entry points from component modules https://review.openstack.org/252335 | 18:15 |
| krotscheck | This sounds like a "What could possibly go wrong" situation. | 18:16 |
| nibalizer | fungi: now you have me thinking about how to graphite asterisk | 18:16 |
| *** erlon has quit IRC | 18:16 | |
| clarkb | oh derp | 18:16 |
| fungi | nibalizer: there's almost certainly a custom snmp mib for asterisk | 18:16 |
| clarkb | I forgot to use an env for the puppet stuff | 18:17 |
| clarkb | BUT | 18:17 |
| *** unicell has joined #openstack-infra | 18:17 | |
| clarkb | I think it iwll just work | 18:17 |
| fungi | nibalizer: so probably not hard at all | 18:17 |
| clarkb | pabelanger: ^ | 18:17 |
| krotscheck | mordred: Ok, so am I updating the spec, or no? | 18:17 |
| clarkb | pabelanger: the puppet change is 99% cleanup so should be fine without it being in right? | 18:17 |
| fungi | nibalizer: worth checking, the distro packages might even include a suggested net-snmp config to enable it | 18:17 |
| clarkb | selinux won't affect ubuntu since its red hat only | 18:18 |
| clarkb | the only thing is that config file that is deleted will go in | 18:18 |
| clarkb | 23.253.242.172 | 18:19 |
| clarkb | should be up now | 18:19 |
| mordred | krotscheck: nah - i can remove my -1 | 18:19 |
| mordred | krotscheck: jgriffith tells me it will work | 18:19 |
| mordred | krotscheck: I can just kill him if he's wrong | 18:19 |
| clarkb | oh nope asterisk install failed to find some packages | 18:19 |
| krotscheck | mordred: So we can blame jgriffith if the world explodes :D | 18:20 |
| clarkb | pabelanger: E: Unable to locate package asterisk-sounds-moh-opsound-wav etc | 18:20 |
| krotscheck | mordred: So, are we doing this now? My patches don't have enough reviews yet. | 18:20 |
| clarkb | I can get a full paste if it will help | 18:20 |
| clarkb | my guess is going to be different package names/sets on ubuntu vs centos | 18:21 |
| pabelanger | clarkb: wait | 18:21 |
| pabelanger | did we merge puppet-asterisk changes? | 18:21 |
| clarkb | pabelanger: no, I was going to run it out of my env but didn't | 18:21 |
| clarkb | pabelanger: if thats a problem I can delete and rerun | 18:21 |
| pabelanger | Ya | 18:21 |
| pabelanger | we need to land that | 18:21 |
| *** hrubi has quit IRC | 18:21 | |
| clarkb | kk | 18:21 |
| clarkb | will redo with env properly setup | 18:21 |
| pabelanger | https://review.openstack.org/#/c/252618/ | 18:21 |
| clarkb | gotcha | 18:22 |
| *** lxsli has joined #openstack-infra | 18:22 | |
| clarkb | and sorry for the noise will be just a few minutes to make the puppet env and rebuild | 18:22 |
| pabelanger | np | 18:22 |
| jeblair | krotscheck: link to your mirror patches? | 18:23 |
| *** hashar has quit IRC | 18:23 | |
| *** trown is now known as trown|lunch | 18:24 | |
| *** sputnik13 has quit IRC | 18:24 | |
| krotscheck | jeblair: You can start here -> https://review.openstack.org/#/c/253236/ , the entire topic is here -> https://review.openstack.org/#/q/status:open+branch:master+topic:unified_mirror,n,z , everything is dependent on the spec merging. | 18:24 |
| krotscheck | (The spec being here: https://review.openstack.org/#/c/252678/ ) | 18:24 |
| *** olaph has quit IRC | 18:25 | |
| *** olaph has joined #openstack-infra | 18:25 | |
| *** sputnik13 has joined #openstack-infra | 18:26 | |
| greghaynes | wow, many patches | 18:26 |
| *** tsg has joined #openstack-infra | 18:27 | |
| *** vilobhmm11 has joined #openstack-infra | 18:27 | |
| krotscheck | greghaynes: Yep. Most of them are the "Hey lets not break things during the rename" patches. | 18:27 |
| *** hrubi has joined #openstack-infra | 18:27 | |
| clarkb | pabelanger: rebuilding now with env set correctly | 18:27 |
| * krotscheck is pretty glad he already had most of this code ready to go. | 18:27 | |
| clarkb | pabelanger: then if this works I will disable puppet on old host, merge patches, move forward with new host | 18:28 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder: Order component retrieval to favour user defined https://review.openstack.org/253640 | 18:28 |
| *** pradk_ has quit IRC | 18:28 | |
| pabelanger | clarkb: roger | 18:28 |
| clarkb | pabelanger: you failed to blame me in that email :) | 18:28 |
| clarkb | "direct all complaints to clarkb" | 18:28 |
| krotscheck | Oh, bloody.... I fatfingered something, just a minute | 18:29 |
| pabelanger | clarkb: I still have time in the follow up email | 18:29 |
| clarkb | pabelanger: haha | 18:29 |
| electrofelix | zaro: https://review.openstack.org/253640 will ensure macros are used ahead of modules added to JJB, so it doesn't matter any more if we add new modules that have name space collisions with macros in jobs | 18:29 |
| pabelanger | nibalizer: graphite which parts of asterisk? | 18:29 |
| nibalizer | pabelanger: use, for one | 18:30 |
| nibalizer | some simple googling implies that on squeeze, asterisk needs to be re-compiled to enable snmp listening | 18:30 |
| openstackgerrit | Evgeny Antyshev proposed openstack-infra/puppet-openstackci: Allow logserver to run on CI server (part 1) https://review.openstack.org/253642 | 18:30 |
| pabelanger | nibalizer: I actually wrote a ceilometer plugin 18months ago to help monitor all my call centre stats. Still working in production | 18:30 |
| nibalizer | but maybe something like asterisk show channels | nc graphite.openstack.org could be functional | 18:30 |
| pabelanger | nibalizer: Pretty easy, could break out my old toolbox :) | 18:31 |
| pabelanger | using AMI is another opton | 18:31 |
| pabelanger | option* | 18:31 |
| nibalizer | ok, when the upgrade is done we can work on that | 18:31 |
| pabelanger | even CLI | 18:31 |
| pabelanger | asterisk -rx "core show channels" | 18:31 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Created mirror.<region>.openstack.org https://review.openstack.org/238754 | 18:32 |
| nibalizer | im betting that the use of asterisk is between zero and five calls per month | 18:32 |
| greghaynes | krotscheck: spec LGTM aside from the thing mordred pointed out | 18:32 |
| krotscheck | greghaynes: Yeah, he was going to remove that -1 | 18:32 |
| krotscheck | greghaynes: The cinder thing turns out to be safe. | 18:32 |
| krotscheck | according to jgriffith | 18:33 |
| greghaynes | krotscheck: ah, ok | 18:33 |
| mordred | dhellmann: any idea why release notes don't show up for https://review.openstack.org/#/c/253597/ here http://docs-draft.openstack.org/97/253597/1/check/gate-shade-docs/b50a504//doc/build/html/ ? | 18:33 |
| clarkb | pabelanger: 23.253.226.32 should be up now and asterisk is running | 18:33 |
| mordred | dhellmann: it worked for me locally - did I do something silly? | 18:33 |
| clarkb | and everyone else that wants to try SIP | 18:33 |
| pabelanger | nibalizer: some stats I was rendering with grafana: https://twitter.com/pabelanger/status/489231700682366976 | 18:33 |
| clarkb | I am going to try POTS | 18:33 |
| pabelanger | clarkb: checking | 18:33 |
| clarkb | conf room 6000? | 18:33 |
| pabelanger | sure | 18:33 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Created mirror.<region>.openstack.org https://review.openstack.org/238754 | 18:35 |
| pabelanger | clarkb: asterisk is still running on pbx.o.o, it will need to be shutdown | 18:36 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Enable db api tests on all backends https://review.openstack.org/253304 | 18:36 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add tests to check migrations and models match https://review.openstack.org/253301 | 18:36 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Remove group regex from testr.conf https://review.openstack.org/253305 | 18:36 |
| *** sabeen1 has quit IRC | 18:36 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Update pypi_trusted_hosts https://review.openstack.org/253504 | 18:37 |
| clarkb | gah ok I think puppet restarted it | 18:37 |
| clarkb | mordred: whic hfile do I edit to kill puppet? | 18:37 |
| jeblair | ensure=>running is evil | 18:37 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Removed manifests for legacy mirrors https://review.openstack.org/253506 | 18:37 |
| jeblair | clarkb: i think the file is called emergency? | 18:38 |
| krotscheck | Ok, that does it | 18:38 |
| *** mohankumar has quit IRC | 18:38 | |
| jeblair | clarkb: http://docs.openstack.org/infra/system-config/sysadmin.html#disable-enable-puppet | 18:38 |
| clarkb | jeblair: thanks | 18:38 |
| jeblair | clarkb: /etc/ansible/hosts/emergency | 18:38 |
| pabelanger | okay both sip:echo@23.253.226.32 and sip:6000@23.253.226.32 appear to be working | 18:39 |
| fungi | yeah, in this case you could just proceed deleting the ssh known hosts key for it on puppetmaster.o.o in preparation for dns updates | 18:39 |
| *** eil397 has joined #openstack-infra | 18:39 | |
| fungi | then ansible won't be able to run puppet again on the old server anyway | 18:39 |
| clarkb | jeblair: that doc is severly lacking in content :? | 18:39 |
| pabelanger | sip:conference@23.253.226.32 works too | 18:39 |
| fungi | it's something you'd do anyway after updating the dns entry | 18:39 |
| *** hrubi has quit IRC | 18:39 | |
| jeblair | clarkb: huh? | 18:39 |
| clarkb | emergerncy is an empty file | 18:39 |
| jeblair | clarkb: that's not documentation, that's the file you add the host to | 18:40 |
| jeblair | clarkb: the documentation is here http://docs.openstack.org/infra/system-config/sysadmin.html#disable-enable-puppet | 18:40 |
| clarkb | jeblair: yes I know | 18:40 |
| clarkb | the documentation does not have enough info for me to do this | 18:40 |
| nibalizer | look at the static file | 18:40 |
| nibalizer | /etc/ansible/hosts/static | 18:40 |
| nibalizer | crib from that | 18:40 |
| *** hashar has joined #openstack-infra | 18:40 | |
| fungi | i though i agree that a commented example /etc/ansible/hosts/emergency would be useful | 18:40 |
| mordred | sorry - I'll add that to the patch | 18:41 |
| mordred | put an entry for the host in a [disabled:children] section in that file | 18:41 |
| *** kozhukalov_ has joined #openstack-infra | 18:41 | |
| jeblair | right, i think all of that is covered by words, but quick copy/pasta is probably a good idea for a file named 'emergency' :) | 18:41 |
| clarkb | jeblair: its not actually covered by workds | 18:42 |
| clarkb | you need a disabled section too from what I can tell | 18:42 |
| clarkb | and entries for each disabled host? | 18:42 |
| mordred | you do not | 18:42 |
| mordred | oh - do you? | 18:42 |
| clarkb | well static implies you do | 18:42 |
| mordred | nah - i just put both in there so that there was a place to put both kinds of entries | 18:42 |
| clarkb | I think the docs needs less words and more examples then :) | 18:43 |
| mordred | disabled is for host entries, children is for group entries - openstack hosts by name are actually groups, because names are not unique | 18:43 |
| mordred | yah | 18:43 |
| mordred | working on it | 18:43 |
| krotscheck | greghaynes: Will https://review.openstack.org/#/c/253521/ let you move forward on the wheel filestructure thingst hat were pointed out? | 18:43 |
| jeblair | i'm okay with more examples, but not less words. it's actually covered pretty clearly in the next paragraph. | 18:43 |
| clarkb | jeblair: as someon that just tried to write the changes by hand I disagree | 18:44 |
| greghaynes | krotscheck: I think so. | 18:44 |
| clarkb | it assumes vast knowledge of ansible | 18:44 |
| clarkb | (and there are 3 sections of static not 2) | 18:45 |
| greghaynes | krotscheck: clarkb one other question I have about wheel mirrors - now that we are doing pip wheel rather than pypi-mirror we wont have an index file, that is a requirement for pip mirrors, correct? Does our apache setup use autoindex which will make that 'just work'? | 18:45 |
| clarkb | anyways it should be disabled now | 18:46 |
| clarkb | pabelanger: ^ should we be trying POTS again before updating DNS? | 18:46 |
| *** hrubi has joined #openstack-infra | 18:46 | |
| *** rossella_s has quit IRC | 18:47 | |
| clarkb | mordred: why is [ci-backup-rs-ord.openstack.org] its own group in static? | 18:47 |
| pabelanger | clarkb: let me reforce registration | 18:47 |
| openstackgerrit | Evgeny Antyshev proposed openstack-infra/system-config: Use httpd::mod instead of httpd_mod https://review.openstack.org/253619 | 18:47 |
| *** amuller is now known as amuller_afk | 18:47 | |
| *** pradk_ has joined #openstack-infra | 18:47 | |
| pabelanger | clarkb: okay, give pots a try | 18:48 |
| pabelanger | I'm connected over SIP | 18:48 |
| *** abregman|afk is now known as abregman | 18:48 | |
| mordred | Because of the way static and dynamic inventories get merged by ansible, the | 18:50 |
| mordred | static file needs to stand alone. If you need to disable a dynamic host from | 18:50 |
| mordred | OpenStack (pretty much all of our hosts) you need to not only add it to | 18:50 |
| mordred | dynamic:children, you need to add an emtpy group into the static file too. | 18:50 |
| mordred | clarkb: ^^ | 18:50 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Add examples to make the disabling prose clearer https://review.openstack.org/253646 | 18:51 |
| mordred | clarkb: ^^ hopefully that helps? | 18:51 |
| *** spzala has joined #openstack-infra | 18:51 | |
| jeblair | mordred: that applies to the emergency file as well, right? | 18:52 |
| mordred | jeblair: yes | 18:52 |
| *** flwang1 has joined #openstack-infra | 18:52 | |
| clarkb | just a minute updating dns | 18:52 |
| clarkb | ok ~5 minute sfor that to update | 18:55 |
| *** tsg is now known as tsg_away | 18:55 | |
| * krotscheck is switching to diaper duties. Contact impossible until naptime. | 18:55 | |
| fungi | i.e., from nappytime to naptime? | 18:57 |
| *** tsg_away has quit IRC | 18:57 | |
| clarkb | mordred: maybe a link to the ansible inventory docs too? | 18:57 |
| mordred | clarkb: kk | 18:57 |
| clarkb | mordred: because for me the major missing piece is what is the actual format of the thing | 18:57 |
| clarkb | the raw data is there but an understanding of how that ranslates is helpful (the examples are a big part of that) | 18:57 |
| pabelanger | clarkb: I take it you didn't have to update any of the patches on review.o.o to make it work? | 18:58 |
| clarkb | pabelanger: nope | 18:58 |
| pabelanger | cool | 18:58 |
| clarkb | pabelanger: I pulled them into my puppet env as is and it ran just fine | 18:58 |
| pabelanger | so we can land them when ready I guess | 18:58 |
| clarkb | pabelanger: yup | 18:59 |
| clarkb | pabelanger: you can remove the WIPs | 18:59 |
| mordred | clarkb: hrm. so - question there | 18:59 |
| pabelanger | clarkb: think I did for all | 18:59 |
| mordred | clarkb: that page is just about disabling, we talk about ansible-to-run-puppet on a different page | 18:59 |
| krotscheck | fungi: Yep, basically | 18:59 |
| mordred | clarkb: nevemind - I think I've got an idea | 18:59 |
| openstackgerrit | Adam Gandelman proposed openstack-infra/project-config: Add functional test job to astara-horizon https://review.openstack.org/253648 | 18:59 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add examples to make the disabling prose clearer https://review.openstack.org/253646 | 18:59 |
| jeblair | typo ^ | 19:00 |
| *** electrofelix has quit IRC | 19:00 | |
| *** ihrachys_ has quit IRC | 19:00 | |
| *** amitgandhinz has joined #openstack-infra | 19:00 | |
| clarkb | and maybe make it clear that the third section belongs to that one paragraph? | 19:00 |
| *** otter768 has joined #openstack-infra | 19:00 | |
| * clarkb is trying to think how to phrase that and can comment in change | 19:01 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Clarify section on ansible inventory file https://review.openstack.org/253650 | 19:01 |
| pabelanger | okay, sip:echo@pbx.openstack.org is routing to the new server for me | 19:02 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Add examples to make the disabling prose clearer https://review.openstack.org/253646 | 19:02 |
| mordred | oh. crap | 19:03 |
| *** bpokorny_ has joined #openstack-infra | 19:03 | |
| mordred | jeblair: we may be stepping on each other | 19:03 |
| jeblair | mordred: just spell emergency right and we'll be fine :) | 19:03 |
| pabelanger | and sip:6000@pbx.openstack.org is working | 19:03 |
| pabelanger | cool | 19:03 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Add examples to make the disabling prose clearer https://review.openstack.org/253646 | 19:03 |
| mordred | jeblair: done! | 19:03 |
| * jeblair waves at pabelanger | 19:03 | |
| clarkb | mordred: I am still confused about the fqdn groupname thing do I need an entry for that in static and emergency? | 19:04 |
| mordred | no - you only need a fake group in the file that you are disabling a thing in | 19:04 |
| jeblair | clarkb: read https://review.openstack.org/253650 and tell me if you're still confused | 19:04 |
| clarkb | jeblair: ah yup much better, you also need to s/dynamic:/disabled:/ | 19:05 |
| *** woodster_ has joined #openstack-infra | 19:05 | |
| *** otter768 has quit IRC | 19:05 | |
| *** bpokorny has quit IRC | 19:05 | |
| jeblair | oh, yeah, i'll fix that in my patch | 19:05 |
| pabelanger | jeblair: Going to see how asterisk holds up on the raspberry pi zero. IIRC you were working on something to use at the summit | 19:06 |
| mordred | I think a problem is that the file named 'static' is a 'static inventory file' - as is the file named 'emergency' | 19:06 |
| clarkb | now I think I am much less confused | 19:06 |
| mordred | perhaps we should rename the static file to something else so that the word static isn't overloaded? | 19:06 |
| *** david-lyle has joined #openstack-infra | 19:06 | |
| *** david-ly_ has joined #openstack-infra | 19:06 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Clarify section on ansible inventory file https://review.openstack.org/253650 | 19:07 |
| clarkb | mordred: maybe disabled_inventory ? | 19:07 |
| clarkb | mordred: though that may cause confusion with people editing that instead of emergency :) | 19:07 |
| mordred | clarkb: well - I also expect that we will put the two by-hand hosts in the infra cloud in that file | 19:07 |
| jeblair | we can put comments in these? | 19:07 |
| mordred | clarkb: since they will not come from a nova inventory anywhere | 19:07 |
| clarkb | jeblair: oh! good idea | 19:07 |
| clarkb | jeblair: I think so as its ini format | 19:07 |
| *** claudiub has quit IRC | 19:07 | |
| clarkb | er python configparser format | 19:07 |
| clarkb | mordred: gothca | 19:08 |
| jeblair | right, what windows ini? ;) | 19:08 |
| clarkb | mordred: "statically_managed_inventory" and "hand_edited_inventory" (I am having a hard time capturing this in filenames | 19:09 |
| mordred | wait - are we runing windows on puppetmaster now? and that's where we run ansible? | 19:09 |
| pabelanger | clarkb: stepping away for a few minutes. Should I send the email to ML before or after puppet patches land? | 19:09 |
| clarkb | probably the idea of using comments would be good | 19:09 |
| clarkb | pabelanger: I think now is fine, its up and those patches will merge soon | 19:09 |
| *** hashar has quit IRC | 19:10 | |
| mordred | (speaking of, should we start bikeshedding now on what we should rename puppetmaster to once it is no-longer a puppetmaster?) | 19:10 |
| *** dizquierdo has quit IRC | 19:11 | |
| clarkb | mordred: I don't think puppetmaster would run on windows but now that its not actually a puppetmaster we can totally do that | 19:11 |
| nibalizer | haha | 19:11 |
| nibalizer | bastion.openstack.org | 19:11 |
| nibalizer | would be my suggestion | 19:11 |
| clarkb | borg-queen? | 19:11 |
| clarkb | brain-bug | 19:11 |
| notmorgan | clarkb: brain-bug | 19:12 |
| notmorgan | clarkb: all the way | 19:12 |
| *** ihrachys has joined #openstack-infra | 19:13 | |
| *** trown|lunch is now known as trown | 19:14 | |
| mordred | brain-bug | 19:17 |
| *** dizquierdo has joined #openstack-infra | 19:19 | |
| openstackgerrit | Kyle Mestery proposed openstack-infra/project-config: Add release notes job for Octavia https://review.openstack.org/253657 | 19:20 |
| *** mancdaz has quit IRC | 19:21 | |
| *** dizquierdo has quit IRC | 19:21 | |
| *** hdd has joined #openstack-infra | 19:21 | |
| *** mancdaz has joined #openstack-infra | 19:22 | |
| pabelanger | unpuppetmaster.o.o | 19:23 |
| *** abregman is now known as abregman|afk | 19:24 | |
| *** rhallisey has quit IRC | 19:24 | |
| *** dkranz has quit IRC | 19:25 | |
| *** dramalho has quit IRC | 19:26 | |
| *** dkranz has joined #openstack-infra | 19:27 | |
| openstackgerrit | Merged openstack-infra/system-config: Add examples to make the disabling prose clearer https://review.openstack.org/253646 | 19:27 |
| mordred | anybody seen this: http://logs.openstack.org/47/252447/2/check/gate-requirements-integration-dsvm/3d80433/console.html#_2015-12-04_18_53_44_610 ? | 19:27 |
| mordred | it's failing constantly for a requirements change | 19:27 |
| mordred | on | 19:27 |
| mordred | "error: can't copy 'etc/neutron/plugins/ml2/linuxbridge_agent.ini': doesn't exist or not a regular file" | 19:27 |
| *** f1ller is now known as filler | 19:28 | |
| *** rhallisey has joined #openstack-infra | 19:28 | |
| mordred | clarkb: logstash query for that owuld be message:"error: can't copy 'etc/neutron/plugins/ml2/linuxbridge_agent.ini': doesn't exist or not a regular file" yeah? | 19:29 |
| *** fawadkhaliq has quit IRC | 19:29 | |
| clarkb | mordred: yes, you should restrict to filename:"console.html" as well | 19:29 |
| clarkb | mordred: use AND between the predicates | 19:29 |
| clarkb | mordred: that way it won't look at all the other log files | 19:29 |
| openstackgerrit | Merged openstack-infra/system-config: Clarify section on ansible inventory file https://review.openstack.org/253650 | 19:29 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/infra-specs: add "complete the reviewable release automation" https://review.openstack.org/245907 | 19:30 |
| mordred | yup. looks like a thing, looks recent | 19:31 |
| *** sridhar_ram has joined #openstack-infra | 19:31 | |
| notmorgan | is this the root of the error in the gate job? http://logs.openstack.org/47/252447/2/check/gate-requirements-integration-dsvm-resolver/6e3a8af/console.html#_2015-12-04_18_22_52_122 I've not seen "hit step limit" before. | 19:31 |
| notmorgan | (same changeset, different gate job) | 19:31 |
| notmorgan | that is also failing (cc mordred clarkb ^) | 19:31 |
| mordred | http://bit.ly/1O8cf3p | 19:31 |
| clarkb | mordred: I am gonna guess that its a symlink or similar | 19:31 |
| clarkb | mordred: since I want to say this has hapened before | 19:31 |
| mordred | clarkb: any idea why it's only broken on requirements and not on other changes? | 19:32 |
| clarkb | mordred: that I do not know | 19:32 |
| *** Sukhdev has joined #openstack-infra | 19:33 | |
| mordred | armax: ^^ | 19:33 |
| clarkb | mordred: I am wrong https://git.openstack.org/cgit/openstack/neutron/tree/etc/neutron/plugins/ml2 file is actually just gone | 19:33 |
| clarkb | so error is correct | 19:33 |
| mordred | awesome | 19:33 |
| clarkb | still doesn't explain why damage is limited | 19:33 |
| *** ildikov has joined #openstack-infra | 19:34 | |
| clarkb | https://git.openstack.org/cgit/openstack/neutron/tree/setup.cfg#n50 that needs cleaning up | 19:34 |
| clarkb | hrm I wonder | 19:34 |
| clarkb | neutron may be dynamically generating that on some tox targets to get by? | 19:34 |
| notmorgan | icky? | 19:34 |
| mordred | there's a thing with ./etc/oslo-config-generator/linuxbridge_agent.ini | 19:35 |
| * mordred goes over to neutron channel | 19:35 | |
| clarkb | though I don't see that happening in tox.ini | 19:35 |
| openstackgerrit | Ryan Brandt proposed openstack-infra/project-config: Move Monasca tempest check job to gate https://review.openstack.org/253663 | 19:35 |
| *** flwang1 has quit IRC | 19:36 | |
| mordred | clarkb: tox -egenconfig does something | 19:37 |
| openstackgerrit | Merged openstack-infra/puppet-jenkins: Properly setup of jenkins keys https://review.openstack.org/215197 | 19:38 |
| *** david-lyle has quit IRC | 19:39 | |
| *** david-ly_ is now known as david-lyle | 19:39 | |
| armax | clarkb, mordred: it’s because some data files were kept in the setup.cfg | 19:41 |
| openstackgerrit | Monty Taylor proposed openstack/requirements: Bump Keystoneauth minimum https://review.openstack.org/252447 | 19:41 |
| mordred | armax: nod- regXboi has told me about https://review.openstack.org/#/c/253395/2 | 19:42 |
| armax | mordred: that’s the one | 19:42 |
| mordred | clarkb, lifeless, dhellmann: ^^ at some point we should figure out why this only breaks the requirements gate and not devstack and friends | 19:43 |
| notmorgan | mordred: we might still hit that step limit error | 19:43 |
| notmorgan | mordred: but lets see | 19:43 |
| mordred | notmorgan: sure. we might | 19:43 |
| mordred | but that's a whole other thing | 19:43 |
| notmorgan | can chase that if it continues | 19:43 |
| clarkb | to finish up the pbx work can https://review.openstack.org/#/c/252618/ and https://review.openstack.org/#/c/252626/ get reviews | 19:44 |
| armax | mordred: we do prevent symlinks to cause breakage by checking for its existence | 19:44 |
| clarkb | super simple puppet changes and they work (I used them to deploy) | 19:44 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/infra-specs: add "complete the reviewable release automation" https://review.openstack.org/245907 | 19:44 |
| *** markvoelker has quit IRC | 19:44 | |
| *** olaph has quit IRC | 19:45 | |
| armax | mordred: we should probably do the same for data files | 19:45 |
| * armax wears the thinking hat | 19:45 | |
| clarkb | mordred: I know | 19:46 |
| mordred | armax: well, I mean - it's a file that's referenced in setup.cfg that does not exist - why does that not break devstack from a pip/setup.py angle | 19:46 |
| mordred | clarkb: oh? sweet | 19:46 |
| clarkb | mordred: well I think I do, its going to be sdist vs develop | 19:46 |
| clarkb | since develop isn't copying taring stuff | 19:46 |
| mordred | oh for the love of | 19:46 |
| clarkb | it just assumes its all there | 19:46 |
| mordred | yup | 19:46 |
| mordred | you're right | 19:46 |
| zaro | asselin: trying to get jenkins_dev.o.o working with puppet again. I'm wondering what happened to the jenkins_api_key, it's not in jenkins.pp anymore. | 19:47 |
| mordred | clarkb: maybe we should add a "python setup.py sdist" line to the pep8 job | 19:47 |
| armax | yeah, today everyone replies faster than me :( | 19:47 |
| mordred | clarkb: just to prevent people from setup.cfg breaking | 19:47 |
| mordred | armax: TYPE FASTERER | 19:47 |
| armax | clarkb: that’s what I was going to toy around | 19:47 |
| clarkb | mordred: or make tox sdist in that job | 19:47 |
| armax | mordred: it’s not the typing that’s slow | 19:48 |
| armax | it’s my brain | 19:48 |
| clarkb | mordred: but seems like a good way to prevent this | 19:48 |
| armax | that is | 19:48 |
| mordred | clarkb: I'm not sure we can override that on a tox invocation basis, can we? | 19:48 |
| armax | clarkb: in neutron we have this: | 19:48 |
| armax | https://github.com/openstack/neutron/blob/master/tox.ini#L105 | 19:48 |
| clarkb | mordred: I think yo ucan override in on a per testenv basis | 19:49 |
| *** olaph has joined #openstack-infra | 19:49 | |
| armax | pep8 stopped being just about pep8 long time ago | 19:49 |
| asselin_ | zaro, was it ever? | 19:49 |
| clarkb | mordred: though maybe you can't unset back to default | 19:49 |
| mordred | armax: yah - but I mean we should put at infra level a global check to make sure all projects can run python setup.py sdist | 19:49 |
| clarkb | armax: ya, though I wouldn't put it in a shell script like that | 19:49 |
| mordred | since that's important | 19:49 |
| *** rbrndt has quit IRC | 19:49 | |
| armax | clarkb: the sdist no | 19:49 |
| mordred | and we can piggyback that on pep8 job since it's a quick job and everyone runs it | 19:50 |
| notmorgan | def. a global thing not just a neutron thing. | 19:50 |
| notmorgan | mordred: ++ on pep8 | 19:50 |
| armax | so shall I take my thinking hat off? | 19:51 |
| notmorgan | armax: nah don't do that ;) | 19:51 |
| armax | I could have a stab at it and see if I can enhance the global pep8 thingy | 19:51 |
| armax | but I’d need pointers to where I need to start breaking the world | 19:52 |
| zaro | asselin_: hmm, looking at past changes. looks like that was only used in jenkins_dev.pp | 19:52 |
| armax | notmorgan: :) | 19:52 |
| clarkb | once those puppet changes are merged I will undisable puppet on new pbx | 19:53 |
| zaro | asselin_: i'm probably testing it incorrectly then | 19:54 |
| asselin_ | zaro, what are you trying to test? | 19:54 |
| *** regXboi has quit IRC | 19:55 | |
| zaro | asselin_: i'm running puppet apply --noop on jenkins-dev to test the manifests but getting an error. | 19:55 |
| armax | clarkb: the thing with infra level global check is that it wouldn’t work locally, would it? | 19:55 |
| zaro | asselin_: ran this command "'puppet apply --noop --modulepath='/opt/system-config/production/modules:/etc/puppet/modules' -e 'include openstack_project::puppetmaster'" | 19:56 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Add sdist check to pep8 buil script https://review.openstack.org/253676 | 19:56 |
| clarkb | armax: I think the idea is global by putting it in every tox.ini | 19:56 |
| mordred | clarkb: ^^ | 19:56 |
| clarkb | oh maybe not :) | 19:56 |
| armax | well, then it’s same difference | 19:56 |
| mordred | we don't want it in tox - people will bitch about local pep8 checks taking forever | 19:56 |
| clarkb | mordred: so armax makes a good point, maybe we start with 253676 then move to sdist in tox.ini? | 19:56 |
| clarkb | mordred: hrm | 19:56 |
| armax | right | 19:56 |
| zaro | asselin_: get this msg "Error: Must pass jenkins_api_key to Class[Openstack_project::Puppetmaster] at line 1 on node jenkins-dev.openstack.org" | 19:56 |
| clarkb | I just worry about people being confused when it doesn't work in jenkins | 19:57 |
| mordred | that's why I added an error message | 19:57 |
| armax | mordred: ok, I am even slower at coming up with patche | 19:57 |
| armax | s | 19:57 |
| armax | I am terrible today | 19:57 |
| zaro | asselin_: jenkins_api_key comes from heira. | 19:57 |
| clarkb | mordred: to avoid potential confusion maybe add reproductio nsteps to the error message? | 19:57 |
| mordred | clarkb: yah. will | 19:57 |
| mordred | do | 19:57 |
| clarkb | mordred: and will that need a virtualenv? | 19:57 |
| mordred | nope | 19:58 |
| armax | btw, how often has this type of issues happened in the past? | 19:58 |
| mordred | rarely | 19:58 |
| mordred | which is why I dont want to burden all of the project | 19:58 |
| armax | ok, so it’s just us :) | 19:58 |
| mordred | this should basically just always work | 19:58 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Add sdist check to pep8 buil script https://review.openstack.org/253676 | 19:58 |
| mordred | gah | 19:58 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Add sdist check to pep8 build script https://review.openstack.org/253676 | 19:58 |
| *** sabeen has joined #openstack-infra | 19:59 | |
| *** dahoo has joined #openstack-infra | 19:59 | |
| *** mhayden has joined #openstack-infra | 19:59 | |
| mordred | jeblair, sdague: ^^ you're the types of humans who might have strong opinions | 19:59 |
| clarkb | mordred: lgtm | 20:00 |
| mordred | should I add a note to the commit message about the failure vector? yeah - maybe so | 20:00 |
| clarkb | can't hurt | 20:00 |
| pleia2 | coolsvap: is there a reason your reviewstats changes are individual changes rather than doing them all in one? a single one would be much easier for all of us [4~:) | 20:00 |
| armax | mordred: we’d still want to allow this to be caught early | 20:00 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Add sdist check to pep8 build script https://review.openstack.org/253676 | 20:01 |
| mordred | clarkb: updated commit message | 20:01 |
| armax | mordred: if a developer cares enough to run pep8 locally before submitting (e.g. has pep8 command in a git hook) | 20:01 |
| armax | mordred: but 253676 seems reasonable to me | 20:02 |
| armax | it’s the ultimate safeguard | 20:02 |
| jeblair | mordred: i thought tox did that; what am i missing? | 20:02 |
| mordred | armax: honestly, it's a really rare thing, I hate to add the overhead to dev day to day - the chances of a negative are uncommon | 20:02 |
| notmorgan | armax: it is probably unlikely someone will break it this way on regular basis | 20:02 |
| mordred | jeblair: we use "usedevelop" in tox | 20:02 |
| notmorgan | hah mordred types faster than me today | 20:02 |
| mordred | jeblair: because speed | 20:02 |
| *** dimtruck is now known as zz_dimtruck | 20:02 | |
| notmorgan | (probably most days) | 20:02 |
| openstackgerrit | Merged openstack-infra/project-config: add release notes jobs for manila https://review.openstack.org/242961 | 20:02 |
| armax | notmorgan: you’re underestimate us | 20:02 |
| jeblair | mordred: ah, oops. :) | 20:02 |
| armax | :) | 20:03 |
| mordred | jeblair: yah, rihgt? | 20:03 |
| jeblair | mordred: i do recall that was one of the reasons we ran everything in tox | 20:03 |
| jeblair | but i guess we missed that tradeoff along the way | 20:03 |
| notmorgan | armax: no i don't :P this is something that a fast fail in pep8 test should be fine for. i really don't see it being tooooooooo common a breakage | 20:03 |
| clarkb | devstack also uses develop | 20:03 |
| mordred | to be fair, this has been the case for like 2 years and this is the first time this has broken | 20:03 |
| clarkb | for the same reason I think | 20:03 |
| mordred | so the track record is good | 20:03 |
| notmorgan | mordred: ++ | 20:03 |
| clarkb | mordred: I think it has broken on the symlink thing once before | 20:04 |
| mordred | clarkb: devstack uses develop because developers use devstack to hack and like develop installs | 20:04 |
| clarkb | which is why I thought symlinks were at fault | 20:04 |
| mordred | clarkb: yah | 20:04 |
| mordred | but still - it's a really low hit rate | 20:04 |
| clarkb | which is similar since setuptools can't copy a symlink but develop just works because its there | 20:04 |
| armax | notmorgan: well, the specific instance that led to the failure is not unrealistic, someone forgot to remove a data file from setup.cfg | 20:04 |
| *** tjones has quit IRC | 20:04 | |
| mordred | yup | 20:04 |
| *** ihrachys has quit IRC | 20:04 | |
| clarkb | yup definitely low incidence | 20:04 |
| notmorgan | armax: and we'll catch it now ;) | 20:04 |
| jeblair | well, the first time we've noticed -- thoroughness of requirements checking has waxed and waned | 20:04 |
| armax | clarkb: we don’t go about deleting files every day, but forgetting is not that rare :) | 20:05 |
| jeblair | we might be about to find some more | 20:05 |
| jeblair | mordred: approved!!! :) | 20:05 |
| zaro | help with disabling puppet on nodes, http://docs.openstack.org/infra/system-config/puppet.html#running-puppet-on-nodes, how does one add an entry to the ansible inventory “disabled” group? | 20:05 |
| armax | cool | 20:06 |
| armax | thanks guys and sorry for the brief glitch | 20:06 |
| fungi | zaro: https://review.openstack.org/253646 | 20:08 |
| openstackgerrit | Merged openstack-infra/puppet-openstackci: Clarifications to the getting started notes in contrib/README https://review.openstack.org/253153 | 20:08 |
| fungi | oh, that's already merged | 20:08 |
| *** baoli_ has quit IRC | 20:09 | |
| *** tjones has joined #openstack-infra | 20:09 | |
| clarkb | ok, now that we are basically not using centos6 anymore is the last thing pbr? I saw lifeless' ops htread didn't really get any bites | 20:09 |
| *** sabeen has quit IRC | 20:09 | |
| fungi | and is about to fire the post jobs | 20:09 |
| *** ericksonsantos has quit IRC | 20:09 | |
| clarkb | which may imply its not a problem | 20:09 |
| zaro | fungi: thanks | 20:09 |
| fungi | zaro: okay, now http://docs.openstack.org/infra/system-config/sysadmin.html#disable-enable-puppet has updated with the new example | 20:09 |
| *** ericksonsantos has joined #openstack-infra | 20:10 | |
| *** dkranz has quit IRC | 20:10 | |
| *** lucasagomes is now known as lucas-afk | 20:10 | |
| openstackgerrit | Merged openstack-infra/system-config: Migrate pbx.openstack.org to Ubuntu Trusty https://review.openstack.org/252626 | 20:10 |
| openstackgerrit | Merged openstack-infra/system-config: Break out openstack_project::server from pbx.o.o https://review.openstack.org/252661 | 20:10 |
| openstackgerrit | sebastian marcet proposed openstack-infra/system-config: Updated OpenStackId configuration https://review.openstack.org/214344 | 20:10 |
| openstackgerrit | Merged openstack-infra/puppet-asterisk: Add support for Ubuntu https://review.openstack.org/252618 | 20:10 |
| openstackgerrit | Merged openstack-infra/project-config: Add sdist check to pep8 build script https://review.openstack.org/253676 | 20:10 |
| fungi | clarkb: yeah, i'd be inclined to press forward with centos 6 teardown everywhere now, but seeing as how it's friday, it can certainly wait for a new week | 20:11 |
| *** dkranz has joined #openstack-infra | 20:12 | |
| fungi | worth replying to his e-mail noting that the pbr python 2.6 job is now the _only_ think still using centos 6 anywhere in our infrastructure | 20:12 |
| *** AJaeger has joined #openstack-infra | 20:12 | |
| fungi | s/think/thing/ | 20:12 |
| *** bpokorny_ has quit IRC | 20:12 | |
| clarkb | and maybe we should start a stop using precise crusade next | 20:13 |
| clarkb | clean up those image types too | 20:13 |
| *** bpokorny has joined #openstack-infra | 20:13 | |
| mordred | clarkb: yes1 | 20:13 |
| *** bpokorny has quit IRC | 20:13 | |
| mordred | fungi: which email? | 20:13 |
| *** gordc has quit IRC | 20:13 | |
| *** bpokorny has joined #openstack-infra | 20:14 | |
| fungi | clarkb: absolutely, it's been hanging out there a while but i think we're getting a lot closer to where we can just push forward on the remaining bits with some renewed vigor | 20:14 |
| clarkb | mordred: its on the ops list asking about python2.6 pbr support needs | 20:14 |
| clarkb | mordred: the idea was people using older releases can't easily cap pbr so if on python2.6 and old release pbr should continue to support them | 20:14 |
| clarkb | but no one said they need the support so I am inclined to give it a few more days and decide we don't need it | 20:14 |
| mordred | I am inclined to not do that | 20:15 |
| mordred | and to remove 2.6 support | 20:15 |
| fungi | mordred: http://lists.openstack.org/pipermail/openstack-operators/2015-December/009046.html | 20:15 |
| mordred | in exactly the way we've been saying we're going to | 20:15 |
| openstackgerrit | Merged openstack-infra/project-config: remove gate-devstack-dsvm-cells https://review.openstack.org/248270 | 20:15 |
| clarkb | mordred: ya thats what I mean | 20:15 |
| *** bpokorny_ has joined #openstack-infra | 20:15 | |
| clarkb | basically no one screamed, lifeless was being nice now we continue plan as planned | 20:15 |
| mordred | clarkb: ya. I'm just saying I don't think we need to wait a few more days | 20:15 |
| clarkb | oj gotcha | 20:15 |
| fungi | however, i'm suggesting that if it's not an exciting way to spend your friday, you're certainly _welcome_ to wait a few more days | 20:16 |
| clarkb | after lunch I can make time to uncentos6 our world :) | 20:17 |
| mordred | yeah - nobody responded to that | 20:17 |
| mordred | however, honest to god - even if they did, I do not think that there is a mitigation | 20:17 |
| notmorgan | woo no cent6 too!? yaay! | 20:17 |
| mordred | keeping EOL'd servers around for one library is too much | 20:17 |
| mordred | pbr is a low volume project | 20:17 |
| fungi | notmorgan: yeah, deleting stuff always feels rewarding | 20:17 |
| clarkb | mordred: one suggestion was to use the deadsnakes ppa on ubuntu | 20:17 |
| mordred | if we release a pbr that breaks 2.6 some how, someone can file a bug | 20:17 |
| clarkb | mordred: but ya one offing pbr is more work than its worth probably | 20:18 |
| mordred | I do not think that is a viable option | 20:18 |
| *** ldnunes has quit IRC | 20:18 | |
| mordred | and I think we've decided it's not a viable option many times before | 20:18 |
| mordred | and nothing has really changed | 20:18 |
| fungi | it was suggested as a custom non-voting job, since it would need to add the ppa and install packages from it at runtime, which we won't be able to cache | 20:18 |
| mordred | and 2.6 is not a supported python | 20:18 |
| mordred | and the distros have promised to support their users | 20:18 |
| mordred | who are not our users | 20:18 |
| *** bpokorny has quit IRC | 20:18 | |
| clarkb | ++ | 20:19 |
| *** matrohon has joined #openstack-infra | 20:19 | |
| fungi | and i expect that people who have a strong need for some sort of python 2.6 testing would write and support that job on their own because we have plenty of other things to work on | 20:19 |
| *** dtroyer has quit IRC | 20:19 | |
| mordred | yup | 20:19 |
| mordred | it's entirely possible for someone to come in later and do the wor | 20:20 |
| mordred | work | 20:20 |
| mordred | we don't need to wait on them | 20:20 |
| fungi | agreed | 20:20 |
| *** olaph has quit IRC | 20:20 | |
| mordred | is there a "remove 2.6" patch already up? | 20:20 |
| fungi | it's already merged | 20:20 |
| fungi | pbr is the only remaining holdout | 20:20 |
| mordred | git grep does not agree with you | 20:20 |
| fungi | oh, i thought it had? | 20:20 |
| fungi | there is definitely a change series up. AJaeger was working on it earlier in the week | 20:21 |
| clarkb | ya AJaeger wrote it not sure if merged yet | 20:22 |
| mordred | ah. I see it | 20:22 |
| *** olaph has joined #openstack-infra | 20:22 | |
| clarkb | I kept meaning t oreview it but have been pulled al lthe directions this week | 20:22 |
| mordred | https://review.openstack.org/#/c/252621/ | 20:22 |
| clarkb | opened for review after lunch but don't wait on me | 20:22 |
| *** larainema has quit IRC | 20:22 | |
| * mordred is going to land them - becuase he wants to | 20:23 | |
| *** sabeen has joined #openstack-infra | 20:23 | |
| AJaeger | clarkb: https://review.openstack.org/#/q/status:open+project:openstack-infra/project-config+branch:master+topic:juno-eol,n,z | 20:24 |
| AJaeger | not merged yet | 20:24 |
| AJaeger | and there's https://review.openstack.org/#/c/252596/ - can I remove the -2 there now? | 20:24 |
| mordred | AJaeger: yes please | 20:25 |
| AJaeger | Or update it to include asterisk? | 20:25 |
| mordred | AJaeger: also, did you make one to remove the templates and stuff from zuul? | 20:25 |
| openstackgerrit | Elizabeth K. Joseph proposed openstack-infra/infra-manual: Remove references to StackForge in Creator's Guide https://review.openstack.org/253687 | 20:25 |
| jeblair | also, i can not think of a better way to friday than to remove an image type :) | 20:26 |
| AJaeger | mordred: My patch series removes the templates and stuff as well | 20:26 |
| AJaeger | let me update 252596 now... | 20:26 |
| mordred | AJaeger: I disagree with you | 20:26 |
| clarkb | AJaeger: asterisk is on trusty no | 20:27 |
| clarkb | *now | 20:27 |
| AJaeger | I'll update the patch... | 20:27 |
| pleia2 | someone asked me about stackforge today, should we be a bit more aggressive about saying it's over on http://docs.openstack.org/infra/system-config/stackforge.html ? | 20:27 |
| jeblair | pleia2: see I4f50a00767ae04c6a917d1486ef55865908d3840 | 20:27 |
| pleia2 | jeblair: I saw that, but it didn't remove all references | 20:28 |
| jeblair | pleia2: it didn't intend to | 20:28 |
| *** slogan621 has joined #openstack-infra | 20:28 | |
| *** dtroyer has joined #openstack-infra | 20:28 | |
| jeblair | people wanted to keep the name | 20:28 |
| pleia2 | jeblair: so is it still being used then? | 20:28 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Remove the last little 2.6 bits https://review.openstack.org/253688 | 20:28 |
| mordred | AJaeger: ^^ | 20:28 |
| fungi | stackforge the namespace is dead, long live stackforge the concept? | 20:28 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove gate-infra-puppet-apply-bare-centos6 https://review.openstack.org/252596 | 20:29 |
| jeblair | fungi: that's the idea as i understand it. who am i to argue? | 20:29 |
| jeblair | fungi, pleia2: i grant it's confusing. | 20:29 |
| zaro | i'm working to make puppet happy on jenkins-dev again so i tried to run a puppet apply --noop to verify that the manifest still works. can this still be done due to the introduction of ansible? | 20:29 |
| AJaeger | mordred: 253688 is too early - we first need to remove it from pbr | 20:30 |
| fungi | "stackforge" now basically means "git repository hosting on openstack's infrastructure without being an official openstack project" | 20:30 |
| AJaeger | mordred: ah, you best it on top of mine ;) | 20:30 |
| AJaeger | mordred: Yes, that was missing - thanks! | 20:30 |
| mordred | AJaeger: :) | 20:30 |
| AJaeger | lifeless: are you around? | 20:30 |
| *** thorst_ has quit IRC | 20:30 | |
| pleia2 | jeblair, fungi ok thanks (though it is confusing), can we at least agree we shouldn't have to do anymore stackforge to openstack renames? | 20:31 |
| mordred | AJaeger: you are still -2 on https://review.openstack.org/#/c/252596/ | 20:31 |
| AJaeger | Regarding pbr and python 2.6, lifeless wrote this email http://lists.openstack.org/pipermail/openstack-operators/2015-December/009046.html | 20:31 |
| fungi | AJaeger: yep, i linked to that in here just a few minutes ago | 20:31 |
| zaro | i'm seeing in an error where the manifests expects aren't getting values from heira | 20:31 |
| AJaeger | mordred, fungi: Sorry, to much going on right now ;) | 20:31 |
| AJaeger | mordred: removed -2 | 20:31 |
| *** ociuhandu has quit IRC | 20:32 | |
| slogan621 | back when I was looking at Neutron closely (icehouse release I think) - I recall that project required contributors of plugins to set up a local test system that could be integrated with jenkins (zuul?) - iirc notifications would be sent about reviewable commits to gerrit, the system would pull code, pass or fail, post logs to dropbox or similar, and report back. | 20:32 |
| *** jbernard has quit IRC | 20:32 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove python 2.6 jobs https://review.openstack.org/252448 | 20:32 |
| slogan621 | Is such a system a universal requirement, or is all the testing I would require run on openstack foundation managed hosts? I'm not a neutron plugin, btw | 20:33 |
| openstackgerrit | Elizabeth K. Joseph proposed openstack-infra/infra-manual: Remove StackForge rename details Creator's Guide https://review.openstack.org/253687 | 20:33 |
| pleia2 | ^^ I think that's more accurate then | 20:33 |
| AJaeger | since there was not a single reply to lifeless' pbr email, should I remove my -2 from https://review.openstack.org/#/c/252621/ ? | 20:33 |
| clarkb | AJaeger: that was how we were treating it | 20:33 |
| clarkb | AJaeger: no one complained or screamed so must not be a problem | 20:33 |
| clarkb | and if it is an issue it can be addressed in the future if/when it happens | 20:34 |
| jeblair | pleia2: absolutely! | 20:34 |
| mordred | AJaeger: yah | 20:34 |
| * AJaeger removed the -2, so feel free to approve ;) | 20:34 | |
| mordred | clarkb, jeblair, pleia2: I did the honors on 2 of hte others - y'all wanna do pbr and https://review.openstack.org/#/c/252596/ ? | 20:35 |
| jeblair | pleia2, fungi: if we want to remove it entirely, i'm not opposed to it, just more or less saying that in the tc discussion, people found value in referring to 'unofficial openstack projects' as 'stackforge', and the current manual wording around that is an intentional attempt to align with that. | 20:35 |
| fungi | mordred: see zaro's question, do we have a solution under the new puppet apply model whereby someone with root access to a dev server but not root access to the puppetmaster can puppet apply and get the correct hiera bits? | 20:35 |
| mordred | yes | 20:35 |
| jeblair | pleia2, fungi: if it ends up just confusing people in the long run and no one actually uses it that way, we should remove it | 20:36 |
| AJaeger | also, https://review.openstack.org/#/c/252455/ is ready | 20:36 |
| jeblair | pleia2, fungi: i think maybe it's still too early to say though? | 20:36 |
| pleia2 | jeblair: that's fair, I just hadn't kep up with those discussions | 20:36 |
| mordred | fungi: as long as ansible has copied the hiera bits to the server once, they're always there | 20:36 |
| pleia2 | kept | 20:36 |
| openstackgerrit | Merged openstack-infra/project-config: Remove python 2.6 job from python-jobs job group https://review.openstack.org/252620 | 20:36 |
| mordred | fungi, zaro: so "puppet apply /opt/system-config/production/manifests/site.pp" should just work on that host | 20:36 |
| mordred | and if you need to locally edit/test things, you can edit the local hieradata | 20:36 |
| fungi | thanks! | 20:37 |
| mordred | (just remember to get a root person to add those to the real hiera before going live again) | 20:37 |
| AJaeger | https://review.openstack.org/#/c/252455/ disables nodepool for centos6... | 20:37 |
| pabelanger | fungi: looping back to the trystack.org SSL, did we figure out how to make that work? | 20:38 |
| mordred | pabelanger, fungi: what's the trystack.org problem? | 20:38 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Remove centos6 node references from site.pp https://review.openstack.org/253690 | 20:38 |
| fungi | slogan621: policies are somewhat dependent on the state of governance of the project/repository in question. without more specifics about your situation it's hard to point you to somewhere you can find an authoritative answer | 20:38 |
| mordred | also - everyone knows letsencrypt is now live, yeah? | 20:38 |
| *** abregman has joined #openstack-infra | 20:39 | |
| *** abregman|afk has quit IRC | 20:39 | |
| pabelanger | mordred: we didn't redirects for trystack.org over SSL working. So I added: https://review.openstack.org/#/c/251454/ | 20:39 |
| pleia2 | know about letsencrypt, haven't looked at it closely yet | 20:39 |
| pabelanger | mordred: but, now we need a SSL cert for static.o.o | 20:39 |
| mordred | pleia2: it moved from private to public beta | 20:39 |
| mordred | ah | 20:39 |
| pabelanger | mordred: Ya, I'm actually giving it a go today | 20:40 |
| mordred | pabelanger: neat! | 20:40 |
| fungi | pabelanger: mordred: if we're going to host an https://trystack.org/ redirect on static.openstack.org then we'll either need to update the openstack_project::static class to handle additional cert files, or replace the current san-based multi-domain cert we're using for that | 20:40 |
| pabelanger | seeing how to automate it | 20:40 |
| pabelanger | since they expire every 3 months | 20:40 |
| *** amuller_afk is now known as amuller | 20:40 | |
| clarkb | hrm I could ssl my weechat with a real cert | 20:40 |
| pleia2 | I'm using cacert personally, may work on switching | 20:40 |
| *** rbrndt has joined #openstack-infra | 20:40 | |
| pleia2 | in my free time \o/ | 20:40 |
| *** thorst_ has joined #openstack-infra | 20:41 | |
| AJaeger | mordred's final cleanup is the last change to be approved for 2.6 removal: https://review.openstack.org/#/c/253688/ . Anybody wanting to +2A? | 20:41 |
| fungi | pabelanger: mordred: dstufft suggested startssl.com might make more financial sense at our current scale of subdomains since they don't charge per cert or per subdomain, just a flat subscription fee for basically unlimited use | 20:41 |
| *** rhallisey has quit IRC | 20:41 | |
| slogan621 | fungi: it would be my own, unofficial project but you answered my question, it's not universal | 20:41 |
| slogan621 | thanks! | 20:42 |
| pleia2 | AJaeger: lgtm | 20:42 |
| clarkb | pabelanger: their docs seem to make automation easy | 20:42 |
| fungi | pleia2: yeah, i'm still using cacert.org for my personal stuff, and i have some philosophical struggle with how things played out with letsencrypt in light of the mofo pushback on cacert | 20:42 |
| clarkb | pabelanger: since its all toolified | 20:42 |
| zaro | mordred: thanks, that worked. | 20:42 |
| *** hashar has joined #openstack-infra | 20:42 | |
| AJaeger | So, is python 2.6 and CentOS6 retirement worth a #success? | 20:42 |
| mordred | any takers on https://review.openstack.org/253688 and https://review.openstack.org/252621 ? | 20:43 |
| fungi | AJaeger: it's worth many successes | 20:43 |
| mordred | we need those 2 to land | 20:43 |
| mordred | then yes | 20:43 |
| *** kozhukalov_ has quit IRC | 20:43 | |
| *** thorst_ has quit IRC | 20:43 | |
| pabelanger | clarkb: Ya, would be hotness to get that imported into puppet some how | 20:43 |
| clarkb | mordred: ya finishing pizza then can look | 20:43 |
| AJaeger | mordred: 253688 is approved... | 20:43 |
| jeblair | and so is the other :) | 20:43 |
| mordred | woot! there they go ... | 20:44 |
| AJaeger | woot! | 20:44 |
| * mordred waits for openstackgerrit to announce the happy | 20:44 | |
| * AJaeger joins mordred in waiting | 20:44 | |
| zaro | mordred: so if i wanted to do a manual puppet apply should i execute 'puppet apply..' or '/opt/system-config/production/run_all.sh' | 20:45 |
| openstackgerrit | Merged openstack-infra/project-config: Disable CentOS6 nodes in nodepool https://review.openstack.org/252455 | 20:46 |
| mordred | zaro: hrm. that's a good question. puppet apply is fine if you don't need updates to dependent puppet modules | 20:46 |
| *** ildikov has quit IRC | 20:46 | |
| mordred | zaro: you never want to run run_all.sh on the node | 20:46 |
| * fungi notes that we're only about 2^11 away from 2^18 gerrit changes... another milestone drawing close! | 20:47 | |
| mordred | zaro: but you could also run run_puppet.sh | 20:47 |
| lifeless | AJaeger: that was only yesterdat | 20:47 |
| lifeless | AJaeger: I think giving folk at least a *few* days would be good | 20:47 |
| *** rhallisey has joined #openstack-infra | 20:48 | |
| lifeless | AJaeger: *plus*, I've personally got a large stack of projects that still CI on 2.6 | 20:48 |
| *** pradk_ has quit IRC | 20:48 | |
| fungi | sorry, we're still about 2^13 away from 2^18, but still quite soon | 20:48 |
| lifeless | AJaeger: and until we figure out what we're doing, I was going to put together a deadsnaes patch | 20:48 |
| openstackgerrit | Merged openstack-infra/project-config: Remove gate-infra-puppet-apply-bare-centos6 https://review.openstack.org/252596 | 20:48 |
| clarkb | lifeless: as mordred pointed out pbr is low traffic and we can address any problems with 2.6 if/when they happen | 20:48 |
| openstackgerrit | Merged openstack-infra/project-config: Remove python 2.6 from pbr https://review.openstack.org/252621 | 20:48 |
| openstackgerrit | Merged openstack-infra/project-config: Remove the last little 2.6 bits https://review.openstack.org/253688 | 20:48 |
| AJaeger | lifeless: your mail is from Wednesday and today is Friday | 20:48 |
| lifeless | clarkb: well actually - I pointed that out in my email | 20:48 |
| clarkb | so I don't think a deadsnakes job is a requirement before moving forward | 20:49 |
| lifeless | I have to go, its saturday | 20:49 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove centos6 from openstack_functions https://review.openstack.org/253693 | 20:52 |
| AJaeger | mordred: another cleanup ^ | 20:52 |
| *** tjcocozz has quit IRC | 20:53 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove centos6 handling from cache_devstack https://review.openstack.org/253694 | 20:53 |
| AJaeger | and the last bit ^ | 20:53 |
| clarkb | did we get the intsall_puppet.sh script too? | 20:54 |
| AJaeger | clarkb: I only checked project-config... | 20:54 |
| clarkb | ok | 20:54 |
| clarkb | AJaeger: you can go weekend :) | 20:54 |
| AJaeger | thanks, clarkb ! | 20:55 |
| AJaeger | #success With Juno End-of-Life, Python 2.6 testing has been removed from OpenStack CI. | 20:55 |
| openstackstatus | AJaeger: Added success to Success page | 20:55 |
| zaro | mordred: thanks. | 20:56 |
| AJaeger | #success pbx.openstack.org has been put on a newer server | 20:56 |
| openstackstatus | AJaeger: Added success to Success page | 20:56 |
| *** tjcocozz has joined #openstack-infra | 20:56 | |
| pleia2 | exciting day :) | 20:56 |
| mordred | AJaeger: ++ | 20:56 |
| mordred | WOOT | 20:57 |
| AJaeger | #sucess OpenStack CI is not using CentOS6 testing anymore, all previous jobs on it - mainly Python 2.6 have been removed or migrated to newer OSes. | 20:57 |
| AJaeger | #success OpenStack CI is not using CentOS6 testing anymore, all previous jobs on it - mainly Python 2.6 have been removed or migrated to newer OSes. | 20:57 |
| openstackstatus | AJaeger: Added success to Success page | 20:57 |
| AJaeger | clarkb: NOw I'm enjoying my weekend. If there's more success to celebrate, go ahead and add it... | 20:58 |
| * AJaeger wishes everybody a great weekend... | 20:58 | |
| clarkb | I am working on the system-config change | 20:59 |
| mordred | clarkb: I look forward to approving your system-config change | 21:00 |
| *** ajmiller_ has joined #openstack-infra | 21:00 | |
| nibalizer | mordred: not if i beat you to it | 21:00 |
| AJaeger | clarkb: when can we remove it completely from nodepool/nodepool.yaml ? | 21:01 |
| purp | nibalizer: Or just beat him. | 21:01 |
| AJaeger | would be nice if somebody could do that as well... | 21:01 |
| * AJaeger is now really gone... | 21:01 | |
| *** AJaeger has quit IRC | 21:01 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove centos6 from openstack_functions https://review.openstack.org/253693 | 21:01 |
| clarkb | the change to set nodepool max servers to -1 merged right? | 21:01 |
| clarkb | I think monday we can delete the config for it | 21:01 |
| *** otter768 has joined #openstack-infra | 21:02 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Remove centos6 support https://review.openstack.org/253698 | 21:03 |
| pabelanger | Where is this success page? | 21:03 |
| *** pvaneck has joined #openstack-infra | 21:03 | |
| pabelanger | first time seeing that | 21:03 |
| clarkb | pabelanger: https://wiki.openstack.org/Successes | 21:03 |
| *** ajmiller has quit IRC | 21:03 | |
| jeblair | clarkb: https://review.openstack.org/253690 | 21:03 |
| pabelanger | Oh, neat | 21:03 |
| clarkb | jeblair: oh I can rebase | 21:03 |
| jeblair | clarkb: or i can abandon | 21:04 |
| clarkb | jeblair: rebase is fine | 21:04 |
| *** Sukhdev has quit IRC | 21:04 | |
| pabelanger | #success I found the successes page | 21:04 |
| openstackstatus | pabelanger: Added success to Success page | 21:04 |
| jeblair | clarkb: well, i broke abandon in unmerged gertty patches, so i guess we'll go with rebase :) | 21:04 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Remove centos6 support https://review.openstack.org/253698 | 21:04 |
| clarkb | oh commit message needed an update | 21:05 |
| *** weshay_xchat is now known as weshay | 21:05 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Remove centos6 support https://review.openstack.org/253698 | 21:05 |
| mordred | we should update the successes page to have anchor links to each success | 21:05 |
| jeblair | mordred: nice | 21:06 |
| mordred | hhahaha | 21:06 |
| mordred | https://review.openstack.org/#/c/253690/1 | 21:06 |
| clarkb | mordred: I think there is functionality to link back to the irc logs | 21:06 |
| mordred | jeblair: ^^ we need the centos6 job removal to land and puppet before we can land your patch :) | 21:06 |
| *** otter768 has quit IRC | 21:06 | |
| mordred | clarkb: yah - but I can't easily tweet a link to the success | 21:06 |
| clarkb | oh you mean anchor in the wiki | 21:07 |
| jeblair | mordred: yep | 21:07 |
| clarkb | gotcha | 21:07 |
| mordred | yah | 21:07 |
| *** baoli has joined #openstack-infra | 21:07 | |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add navigation to interactive search https://review.openstack.org/238263 | 21:07 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add interactive search to diff view https://review.openstack.org/238262 | 21:07 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add a simple kill ring https://review.openstack.org/238261 | 21:07 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Support multiple key input https://review.openstack.org/242885 | 21:07 |
| openstackgerrit | James E. Blair proposed openstack/gertty: Add help entries for kill, yank, isearch https://review.openstack.org/242884 | 21:07 |
| jeblair | mordred: git restack! ^ | 21:07 |
| mordred | jeblair: woot! | 21:07 |
| *** tongli_ has quit IRC | 21:09 | |
| *** agentle_ has quit IRC | 21:11 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove CentOS6 from nodepool scripts https://review.openstack.org/253701 | 21:12 |
| *** rlandy has quit IRC | 21:12 | |
| *** akshai has quit IRC | 21:12 | |
| openstackgerrit | Ramy Asselin proposed openstack-infra/puppet-openstackci: Add nodepool log and configuration location https://review.openstack.org/253703 | 21:12 |
| *** cp16net has quit IRC | 21:14 | |
| *** cp16net has joined #openstack-infra | 21:16 | |
| mordred | jeblair, clarkb: I have rechecked https://review.openstack.org/#/c/253690/ | 21:16 |
| *** sdake has quit IRC | 21:16 | |
| *** annegentle has joined #openstack-infra | 21:17 | |
| *** sdake has joined #openstack-infra | 21:19 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove centos6 handling from cache_devstack https://review.openstack.org/253694 | 21:19 |
| *** kgiusti has quit IRC | 21:21 | |
| *** sridhar_ram has quit IRC | 21:21 | |
| jeblair | krotscheck: in 238754 why are you symlinking /pypi -> /web ? | 21:21 |
| *** rhallisey has quit IRC | 21:22 | |
| *** ildikov has joined #openstack-infra | 21:23 | |
| *** tsg has joined #openstack-infra | 21:23 | |
| mordred | jeblair: krotscheck is with child | 21:24 |
| jeblair | mordred: good thing this is async :) | 21:24 |
| *** zz_dimtruck is now known as dimtruck | 21:24 | |
| jeblair | yeah, i don't see anything in the spec that would imply that either, so i'll leave a comment on the review | 21:26 |
| *** tsg has quit IRC | 21:28 | |
| jeblair | oh, i think that's about getting apache to point at the bandersnatch web subdirectory... | 21:28 |
| *** boris-42_ has joined #openstack-infra | 21:28 | |
| jeblair | but why don't we just have the apache vhost documentroot there? | 21:28 |
| jeblair | oh, aha, now i get it | 21:29 |
| dstufft | fungi: Lets Encrypt is publically available now btw | 21:30 |
| dstufft | I think | 21:30 |
| jeblair | it's so that the index page for the hostname lists all the mirrors, even if the docroots of the mirrors themselves are note docrooted there (which is necessary in the case of pypi to skip the non-web root dir of bandersnatch) | 21:30 |
| dstufft | also I'm only here for a minute so :D | 21:30 |
| jeblair | krotscheck: nm, i grok ^ | 21:31 |
| *** rfolco has quit IRC | 21:31 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: OIDC - OpenId Connect Implementation https://review.openstack.org/198458 | 21:33 |
| jeblair | krotscheck: i think you want to pull https://review.openstack.org/253504 out of sequence and merge asap. | 21:33 |
| *** sabeen has quit IRC | 21:34 | |
| jeblair | mordred: https://review.openstack.org/247084 has not landed yet | 21:35 |
| *** bpokorny_ has quit IRC | 21:36 | |
| *** bpokorny has joined #openstack-infra | 21:37 | |
| *** Hal1 has joined #openstack-infra | 21:38 | |
| *** afazekas has quit IRC | 21:39 | |
| mordred | zomg | 21:39 |
| jeblair | mordred: i'm updating | 21:39 |
| *** afazekas has joined #openstack-infra | 21:40 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool: Write the cloud name into the provider info https://review.openstack.org/247084 | 21:40 |
| *** matrohon has quit IRC | 21:40 | |
| jeblair | mordred: ^ | 21:40 |
| jeblair | greghaynes: ^ | 21:40 |
| *** Hal1 has quit IRC | 21:41 | |
| jeblair | mordred, krotscheck: i hesitated to suggest this earlier to avoid potentially delaying the mirror work, but considering the lengths we're going to now to get it done right all at once, we may want to go ahead and incorporate the mirror system rename in there too. | 21:42 |
| jeblair | i will comment on the spec. | 21:42 |
| greghaynes | LGTM | 21:44 |
| jeblair | did we want mirror.<region>.<cloud>.openstack.org or <region>.<cloud>.mirror.openstack.org or something else? | 21:44 |
| clarkb | I like mirror.region.cloud as it does the tree in the right direction | 21:45 |
| greghaynes | ++ on mirror.region.cloud | 21:45 |
| mordred | mirror.region.cloud | 21:45 |
| rbrndt | looking to make some changes to the monasca gate jobs, if anyone has a moment https://review.openstack.org/#/c/253663/1 | 21:45 |
| clarkb | what is the difference between provider and cloud? | 21:46 |
| jeblair | mirror.dfw.rax.openstack.org; mirror.bhs1.ovh.openstack.org; mirror.regionone.bluebox-sjc.openstack.org | 21:46 |
| clarkb | is it just hte bits in the string? | 21:46 |
| jeblair | clarkb: provider=rax-dfw; cloud=rax; region=dfw; | 21:46 |
| openstackgerrit | Merged openstack-dev/cookiecutter: Discover is not needed anywhere anymore. https://review.openstack.org/253261 | 21:46 |
| clarkb | jeblair: thanks | 21:46 |
| jeblair | clarkb: i left out the example of why that's important, actually -- it's mostly hp | 21:47 |
| jeblair | clarkb: provider=hpcloud-b1; cloud=hpcloud | 21:47 |
| clarkb | looking at the code ya that | 21:47 |
| *** olaph has quit IRC | 21:47 | |
| jeblair | that way we end up with a single mirror.region-b.hpcloud.openstack.org | 21:47 |
| clarkb | providers are logical but cloud is singular and is mapped onto | 21:47 |
| jeblair | rather than having mirror.region-b.hpcloud-b[12345].o.o | 21:48 |
| jeblair | clarkb: yep | 21:48 |
| greghaynes | We require os-cloud-config now, correct? | 21:48 |
| greghaynes | in nodepool | 21:48 |
| jeblair | os-client-config | 21:48 |
| *** olaph has joined #openstack-infra | 21:48 | |
| greghaynes | right, that | 21:48 |
| jeblair | i believe so | 21:48 |
| greghaynes | there are too many os-c-c's | 21:48 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 21:49 | |
| jeblair | i always type the wrong one searching | 21:49 |
| clarkb | yes client config | 21:49 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 21:49 | |
| jeblair | it's esp confusing since os_client_config returns something called a 'cloud' :) | 21:49 |
| mordred | jeblair: provider=rackspace | 21:50 |
| jeblair | mordred: provider=rax-dfw; cloud=rackspace | 21:50 |
| mordred | nope. that's backwards fromos-client-config perspective | 21:51 |
| mordred | oh - wait | 21:51 |
| mordred | sorry | 21:51 |
| greghaynes | cloud is just whatever it is named in os-c-c | 21:51 |
| mordred | profile is occ speak | 21:51 |
| jeblair | mordred: yeah, we're talking nodepool | 21:51 |
| mordred | sorry | 21:51 |
| clarkb | do we need to update that change to use profile form cloud_config? | 21:51 |
| * mordred shuts up | 21:51 | |
| greghaynes | Just to make it even more confusing :) | 21:51 |
| mordred | nope - nodepool should reference named clouds from occ | 21:51 |
| jeblair | greghaynes: actually i believe using clouds.yaml is not required | 21:52 |
| mordred | profile should only show up in clouds.yaml files | 21:52 |
| jeblair | greghaynes: i assume you're getting at the case when we don't provide a 'cloud' entry... | 21:52 |
| mordred | I do not believe passing profile in as an argument to get_one_cloud() is supported | 21:52 |
| greghaynes | jeblair: Yep. Trying to figure out how well that is supported now | 21:52 |
| jeblair | mordred: why are we talking about profiles? | 21:52 |
| jeblair | oh because clarkb asked about that | 21:53 |
| clarkb | ya just curous if self.provider.cloud_config.name is the most correct value | 21:53 |
| clarkb | I am guessing yes since its name not cloud | 21:53 |
| *** olaph has quit IRC | 21:53 | |
| *** olaph has joined #openstack-infra | 21:54 | |
| mordred | jeblair: only becuase it was what I was mis-reading when you were saying provider earlier | 21:54 |
| mordred | and then yeah, that led to that question | 21:54 |
| jeblair | greghaynes: i think name is the empty string if you don't specify it | 21:54 |
| jeblair | mordred: ^ true? | 21:55 |
| mordred | yes | 21:55 |
| *** Sukhdev has joined #openstack-infra | 21:55 | |
| greghaynes | ah, perfect | 21:55 |
| jeblair | (which seems like good/expected behavior to me) | 21:55 |
| mordred | I was considering that perhaps we should default cloud to equal provider if cloud is not given in nodepool.yaml | 21:55 |
| mordred | but then no, nevermind | 21:55 |
| mordred | that makes no sense | 21:55 |
| * mordred shuts up again | 21:55 | |
| jeblair | mordred: yeah, maybe leave that logic to shell scripts if needed | 21:56 |
| krotscheck | jeblair: adding <cloud> things into the mirror thing doesn't seem like a huge deal. I'd need to know the names of those clouds though for that cacti/etc patch. | 21:56 |
| jeblair | krotscheck: https://etherpad.openstack.org/p/unified-mirrors | 21:57 |
| jeblair | wow i really like region-b.geo-1 | 21:58 |
| *** alivigni has quit IRC | 21:59 | |
| *** mwagner_lap has quit IRC | 21:59 | |
| krotscheck | jeblair: Thankfully it's going away | 21:59 |
| krotscheck | ....for a very broad defenition of "thankfully" | 21:59 |
| openstackgerrit | Merged openstack-infra/system-config: Remove centos6 node references from site.pp https://review.openstack.org/253690 | 22:00 |
| jeblair | krotscheck, mordred: i think that list is complete now | 22:01 |
| *** annegentle has quit IRC | 22:01 | |
| krotscheck | Coo | 22:01 |
| jeblair | we should be aware of the extra hostname component of region-b.geo-1. we already have that problem now and the regex accounts for it -- but just something to keep in mind as we're changing it | 22:02 |
| *** trown is now known as trown|outttypeww | 22:02 | |
| *** annegentle has joined #openstack-infra | 22:04 | |
| clarkb | ok I am reenabling puppet for pbx | 22:05 |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder: Add openssh-server package-install to local-config https://review.openstack.org/253718 | 22:07 |
| *** peristeri has quit IRC | 22:08 | |
| *** vilobhmm11 has quit IRC | 22:08 | |
| *** eharney has quit IRC | 22:08 | |
| *** abregman has quit IRC | 22:09 | |
| *** amitgandhinz has quit IRC | 22:10 | |
| *** tsg has joined #openstack-infra | 22:14 | |
| *** pradk has quit IRC | 22:15 | |
| *** tiswanso has quit IRC | 22:15 | |
| *** spzala has quit IRC | 22:16 | |
| *** vilobhmm11 has joined #openstack-infra | 22:16 | |
| *** barnaby has quit IRC | 22:17 | |
| *** barnaby has joined #openstack-infra | 22:18 | |
| fungi | heading out to grab something to eat. back later | 22:18 |
| pleia2 | later, fungi | 22:19 |
| *** sdake has quit IRC | 22:22 | |
| *** sdake_ has joined #openstack-infra | 22:22 | |
| *** hashar has quit IRC | 22:23 | |
| *** mancdaz has quit IRC | 22:26 | |
| *** mancdaz has joined #openstack-infra | 22:27 | |
| *** unicell has quit IRC | 22:29 | |
| *** david-ly_ has joined #openstack-infra | 22:29 | |
| *** lascii is now known as alaski | 22:29 | |
| *** bauwser is now known as bauzas | 22:30 | |
| *** david-ly_ is now known as david-lyle_ | 22:31 | |
| *** slogan621 has quit IRC | 22:31 | |
| *** david-lyle has quit IRC | 22:32 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove CentOS6 from nodepool scripts https://review.openstack.org/253701 | 22:32 |
| *** jamesmcarthur has joined #openstack-infra | 22:33 | |
| openstackgerrit | Merged openstack-infra/nodepool: Write the cloud name into the provider info https://review.openstack.org/247084 | 22:35 |
| *** thorst has joined #openstack-infra | 22:36 | |
| *** dimtruck is now known as zz_dimtruck | 22:36 | |
| openstackgerrit | Merged openstack-infra/system-config: Remove centos6 support https://review.openstack.org/253698 | 22:37 |
| *** dprince has quit IRC | 22:37 | |
| *** xyang1 has quit IRC | 22:40 | |
| *** ociuhandu has joined #openstack-infra | 22:41 | |
| *** ildikov_ has joined #openstack-infra | 22:46 | |
| *** sdake_ has quit IRC | 22:48 | |
| *** unicell has joined #openstack-infra | 22:48 | |
| *** ildikov has quit IRC | 22:49 | |
| *** olaph has quit IRC | 22:50 | |
| *** eil397 has quit IRC | 22:50 | |
| rcarrillocruz | SpamapS: glean already writes vlan interfaces associated to physical interfaces. I had to put the post-script cos the issue is that the vlan interface is not brought up on boot, looks like the networking script does not bring it up | 22:51 |
| rcarrillocruz | so to recap: | 22:51 |
| SpamapS | rcarrillocruz: does glean mark the vlan as 'auto' ? | 22:51 |
| rcarrillocruz | my patch does: 1. create all .cfg files, physical and vlan 2. bring up the vlan interface | 22:52 |
| SpamapS | rcarrillocruz: because 'networking' is supposed to do an 'ifup -a' | 22:52 |
| rcarrillocruz | yeah, it has auto | 22:52 |
| SpamapS | ok so you need to block networking from starting | 22:52 |
| rcarrillocruz | that's why i asked you about the 'local-filesystems' event | 22:52 |
| SpamapS | until glean is done | 22:52 |
| *** olaph has joined #openstack-infra | 22:52 | |
| rcarrillocruz | you said if it gets triggered, the service consuming it makes it disappear | 22:52 |
| SpamapS | right you can't do that one. :) | 22:52 |
| SpamapS | so. | 22:52 |
| mordred | WORST TWEET EVER: https://twitter.com/RyverApp/status/637407979457740800 | 22:53 |
| SpamapS | a second job file.. | 22:53 |
| rcarrillocruz | so my assumption is maybe glean consumes 'local-filesystems' | 22:53 |
| SpamapS | start on starting networking | 22:53 |
| rcarrillocruz | thus 'networking' doesn't see it and doesn't run? | 22:53 |
| *** unicell has quit IRC | 22:53 | |
| SpamapS | actually it's even easier than that | 22:53 |
| SpamapS | start on starting networking and static-network-up | 22:53 |
| SpamapS | thats it, one line, nothing else would be needed | 22:54 |
| SpamapS | in theory :) | 22:54 |
| SpamapS | because that will block networking until 'static-network-up' is emitted | 22:54 |
| SpamapS | oh hm | 22:54 |
| *** salv-orl_ has joined #openstack-infra | 22:54 | |
| rcarrillocruz | just 'start on starting networking' suffices to me... | 22:54 |
| SpamapS | that might deadlock things | 22:54 |
| rcarrillocruz | i mean, just that | 22:54 |
| rcarrillocruz | makes the whole use case work | 22:54 |
| rcarrillocruz | create interfaces and bring up everything | 22:54 |
| *** tsg has quit IRC | 22:54 | |
| rcarrillocruz | (i've tested) | 22:54 |
| SpamapS | rcarrillocruz: so that's just a different race. | 22:55 |
| SpamapS | one that your test shows has a marked edge for the car you want to win. :) | 22:55 |
| SpamapS | It's probably fine that way | 22:55 |
| rcarrillocruz | so, you good with the patch? | 22:56 |
| SpamapS | Until we have some other thing that affects glean that happens before 'starting networking'. :-P | 22:56 |
| rcarrillocruz | cos quite frankly, the post-script thing is ugly | 22:56 |
| SpamapS | rcarrillocruz: well the whole situation is ugly | 22:56 |
| mordred | SpamapS: I don't suppose there is a "disks exist but networks don't" ? | 22:56 |
| rcarrillocruz | heh | 22:56 |
| SpamapS | but this may be the least ugly | 22:56 |
| rcarrillocruz | yeah | 22:56 |
| rcarrillocruz | :-) | 22:56 |
| SpamapS | mordred: there are no compound events like that | 22:56 |
| rcarrillocruz | so, +1 as 'least worst case' ? | 22:56 |
| mordred | darn | 22:56 |
| mordred | k. I have one more suggestion ... | 22:56 |
| SpamapS | mordred: what you want is a dependency based solution... | 22:56 |
| SpamapS | mordred: -> http://systemd.org/ | 22:56 |
| mordred | nope. what I _want_ is a linear solution | 22:57 |
| SpamapS | -> 2009 | 22:57 |
| mordred | becuase these machines are doing like 3 things when starting | 22:57 |
| mordred | so, and I really do mean this non-trolling/non-snark ... | 22:57 |
| SpamapS | rcarrillocruz: yeah, this has been a fun exercise :-P | 22:57 |
| *** salv-orlando has quit IRC | 22:57 | |
| SpamapS | rcarrillocruz: NOT ;) | 22:57 |
| *** abregman has joined #openstack-infra | 22:57 | |
| rcarrillocruz | SpamapS: so ,do you know why without the 'post-script' the interface doesn't come up on boot? is it because glean and 'networking' share the same event local-filesystems? | 22:58 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config: Update subunit job name selection regex https://review.openstack.org/253729 | 22:58 |
| mordred | why don't we delete all of the network setup upstart things so that _nothing_ starts networking based on any of the events | 22:58 |
| mordred | and do a single linear shell script | 22:58 |
| mordred | we know we're not expecting hot-plug network events | 22:58 |
| rcarrillocruz | ++ | 22:58 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: copy_puppet_logs: manage testrepository.subunit https://review.openstack.org/253730 | 22:59 |
| rcarrillocruz | i just want the bloody thing to behave on boot | 22:59 |
| mordred | like, is there anything that would _actual_ break? | 22:59 |
| rcarrillocruz | i don't expect things to happen post-boot | 22:59 |
| SpamapS | mordred: only if people start hotplugging things into kvm | 22:59 |
| SpamapS | :) | 22:59 |
| rcarrillocruz | i vote for removing 'networking' form glean once all is booted, so it cannot mess up with it :D | 22:59 |
| SpamapS | does glean keep track of the intance ID yet? IIRC it didn't at first. | 23:00 |
| mordred | SpamapS: what do you mean? | 23:00 |
| rcarrillocruz | ok | 23:02 |
| SpamapS | mordred: glean should do nothing if it has already run successfully for the current instance ID | 23:02 |
| *** otter768 has joined #openstack-infra | 23:02 | |
| rcarrillocruz | hmm | 23:02 |
| rcarrillocruz | glean having state | 23:02 |
| rcarrillocruz | ? | 23:02 |
| SpamapS | if not, you will likely get in the way of post-first-boot tooling | 23:03 |
| *** jamesmcarthur has quit IRC | 23:03 | |
| rcarrillocruz | as in | 23:03 |
| rcarrillocruz | /var/run/glean/eth0 => hey, glean already did a thing on eth0 | 23:03 |
| rcarrillocruz | ? | 23:03 |
| SpamapS | cloud-init does this by keeping the config data it read in /var/lib/cloud/data/$INSTANCE_ID | 23:03 |
| rcarrillocruz | along those lines | 23:03 |
| clarkb | pbx is successfully puppeting again | 23:03 |
| clarkb | centos6 is gone | 23:03 |
| clarkb | what a day | 23:03 |
| SpamapS | ding dong the six is dead | 23:03 |
| clarkb | also the kids are out of the house | 23:04 |
| mordred | clarkb: woot! | 23:05 |
| clarkb | I'm just amazed it actually happend, its been so long | 23:05 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Update pypi_trusted_hosts https://review.openstack.org/253504 | 23:05 |
| *** unicell has joined #openstack-infra | 23:05 | |
| *** unicell has quit IRC | 23:05 | |
| rcarrillocruz | anyway | 23:06 |
| rcarrillocruz | off to bed | 23:06 |
| rcarrillocruz | too many beers | 23:06 |
| rcarrillocruz | SpamapS: i await for your vote! | 23:06 |
| rcarrillocruz | have a good one folks | 23:06 |
| rcarrillocruz | nice weekend | 23:06 |
| *** shardy has quit IRC | 23:07 | |
| *** otter768 has quit IRC | 23:07 | |
| *** burgerk has quit IRC | 23:08 | |
| SpamapS | rcarrillocruz: I +1'd | 23:09 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/infra-specs: Unified Mirrors https://review.openstack.org/252678 | 23:09 |
| *** unicell has joined #openstack-infra | 23:09 | |
| *** dims has quit IRC | 23:10 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Created mirror.<region>.<cloud>.openstack.org https://review.openstack.org/238754 | 23:11 |
| *** amuller has quit IRC | 23:11 | |
| *** Sukhdev has quit IRC | 23:12 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Removed manifests for legacy mirrors https://review.openstack.org/253506 | 23:13 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Add NodeJS and NPM to single-use-slave https://review.openstack.org/242161 | 23:14 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Added NPM Mirror to Infra https://review.openstack.org/235617 | 23:14 |
| *** unicell has quit IRC | 23:14 | |
| *** tlian has quit IRC | 23:14 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Added Gem Mirror to Infra https://review.openstack.org/253616 | 23:15 |
| *** hdd has quit IRC | 23:16 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Point nodepool slaves at new pypi mirror location. https://review.openstack.org/238756 | 23:20 |
| *** unicell has joined #openstack-infra | 23:21 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Point nodepool slaves at new pypi mirror location. https://review.openstack.org/238756 | 23:23 |
| *** annegentle has quit IRC | 23:23 | |
| clarkb | sdague: jeblair did we get anywhere on those ovh multinode failures? | 23:24 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Point nodepool slaves at new pypi mirror location. https://review.openstack.org/238756 | 23:25 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Set npm mirror location https://review.openstack.org/242191 | 23:28 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Point nodepool slaves at new pypi mirror location. https://review.openstack.org/238756 | 23:28 |
| openstackgerrit | Merged openstack-infra/system-config: Add #openstack-smaug to channel logging https://review.openstack.org/252939 | 23:28 |
| jeblair | clarkb: sdague mentioned someone was looking into it and baffled so far. i have 2 pairs held, one in rax and one in ovh if needed. | 23:28 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Removed npm-install macro https://review.openstack.org/242193 | 23:29 |
| krotscheck | ALright, I think that's the last of the requested updates | 23:29 |
| *** thorst has quit IRC | 23:30 | |
| *** vilobhmm11 has quit IRC | 23:30 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Set npm mirror location https://review.openstack.org/242191 | 23:32 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Removed npm-install macro https://review.openstack.org/242193 | 23:32 |
| krotscheck | And that's the last of the copy/pasta :) | 23:32 |
| nibalizer | krotscheck: wowza | 23:32 |
| openstackgerrit | Merged openstack-infra/project-config: Add python jobs to tricircle https://review.openstack.org/253394 | 23:32 |
| krotscheck | nibalizer: This humble student requests your review, oh great puppet master. | 23:33 |
| * krotscheck rings a large ceremonial gong | 23:33 | |
| *** weshay has quit IRC | 23:33 | |
| nibalizer | ya i'll start working through it | 23:33 |
| mordred | dude | 23:35 |
| *** openstack has joined #openstack-infra | 23:39 | |
| krotscheck | clarkb: It is | 23:39 |
| clarkb | krotscheck: ya I see it, I am not sure 500GB will be enough though. gems by itself is over 300GB iirc | 23:40 |
| *** ajmiller_ has quit IRC | 23:40 | |
| krotscheck | clarkb: I thought I said 500GB per mirror type? | 23:40 |
| *** tlian has quit IRC | 23:40 | |
| *** tlian2 has quit IRC | 23:40 | |
| clarkb | oh that sentence reads weird for me | 23:40 |
| clarkb | it says 300GB for the mirrors then 500GB per mirror type | 23:41 |
| clarkb | I read that as 300GB total needed so use 500GB for overhead | 23:41 |
| *** abregman has quit IRC | 23:41 | |
| krotscheck | Right. Is it the use of a semicolon to separate clauses that's the issue there? Ican modify it. | 23:41 |
| clarkb | ya I would make estimate and planned size as two separate thoughts/sentences | 23:42 |
| krotscheck | As it is, right now I'm asking for 2TB of disk space for each host | 23:42 |
| clarkb | We estimate 200-300GB per mirror type. We should provision 500GB per mirror type to provide overhead for growth over time. Or something | 23:42 |
| clarkb | anyways plan looks good | 23:42 |
| clarkb | hehe requirements "enough space to store the gems" but what is that number? :/ | 23:44 |
| clarkb | (this is not from the spec) | 23:44 |
| *** Sukhdev has joined #openstack-infra | 23:45 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/infra-specs: Unified Mirrors https://review.openstack.org/252678 | 23:46 |
| krotscheck | clarkb: Done. | 23:46 |
| krotscheck | clarkb: I think Scrooge McDuck knows how to estimate volume requirements for gem storage. | 23:46 |
| clarkb | krotscheck: thats a good point he needs to know how big to build his swimming pools | 23:47 |
| krotscheck | Think we can hire him as a consultant? | 23:47 |
| *** dims has joined #openstack-infra | 23:47 | |
| clarkb | we probably can't afford his rates | 23:47 |
| krotscheck | Yeah, let's just see if we can get his nephews to intern. | 23:48 |
| * krotscheck wanders off to play with letsencrypt | 23:48 | |
| clarkb | krotscheck: I think you mean fallout4 | 23:51 |
| krotscheck | clarkb: I wish. munchkin is sleeping on my chest right now, so I've only got access to my laptop. | 23:52 |
| *** thorst has joined #openstack-infra | 23:52 | |
| krotscheck | clarkb: Are your girls crawling yet? | 23:52 |
| clarkb | krotscheck: no just sort of derping on the floor | 23:52 |
| clarkb | one will spin in circles | 23:52 |
| SpamapS | HAHAHAH | 23:52 |
| clarkb | the other flails about | 23:53 |
| krotscheck | wheeeeee | 23:53 |
| SpamapS | I think they stop derping around 24 | 23:53 |
| krotscheck | SpamapS: Years? | 23:53 |
| SpamapS | krotscheck: indeed | 23:53 |
| Clint | SpamapS: definitely not before 25 | 23:53 |
| * krotscheck leaves this channel. Giggling is waking up the munchkin | 23:53 | |
| SpamapS | Clint: I never really stopped actually | 23:53 |
| Clint | take your time | 23:53 |
| clarkb | ooh humble bundle maker book bundle | 23:56 |
| *** rbrndt has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!