| *** dims has quit IRC | 00:00 | |
| *** dims has joined #openstack-infra | 00:01 | |
| clarkb | jeblair: I am going to disable puppet on old git.o.o, remove localhost from the haproxy config and reload haproxy. That way we don't possibly end up with weirdness once gerrit replication stops | 00:04 |
|---|---|---|
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Fix graceful exit https://review.openstack.org/43803 | 00:05 |
| *** mikal has quit IRC | 00:06 | |
| *** mikal has joined #openstack-infra | 00:06 | |
| jeblair | clarkb: sounds good. | 00:06 |
| clarkb | and all done | 00:06 |
| jeblair | clarkb: reviewing https://review.openstack.org/#/c/43743/ now | 00:06 |
| clarkb | the old DNS TTL was for one hour and I change it about an hour ago. I think I am ready to move onto switching over when you are | 00:07 |
| jeblair | clarkb: ok, aiui the process should be to change dns, wait a bit, then kill the old server, yeah? | 00:07 |
| jeblair | clarkb: there shouldn't really be a downtime since the two servers behave the same and can remain up | 00:08 |
| jeblair | clarkb: the new one will just take over from the old one as dns propogates | 00:08 |
| clarkb | change dns, merge my puppet change, make sure puppet runs everywhere, restart gerrit, kill old server | 00:08 |
| jeblair | clarkb: *nod( | 00:09 |
| *** michchap has joined #openstack-infra | 00:10 | |
| jeblair | clarkb: ready for me to merge? | 00:11 |
| clarkb | jeblair: sure, do you want me to update DNS now? | 00:11 |
| jeblair | clarkb: sounds good | 00:11 |
| openstackgerrit | A change was merged to openstack-infra/config: Split git.o.o haproxy out of cgit module. https://review.openstack.org/43743 | 00:13 |
| clarkb | DNS records modified | 00:14 |
| *** UtahDave has quit IRC | 00:14 | |
| clarkb | er only the A and AAAA records are done. Doing reverse PTRs now | 00:15 |
| jeblair | clarkb: oh, in the future you can go ahead and do those when creating the server | 00:15 |
| *** weshay has joined #openstack-infra | 00:20 | |
| clarkb | jeblair: which env do I need to have sourced to do reverse dns records? | 00:20 |
| jeblair | clarkb: do you have the output from when you created the server? | 00:21 |
| clarkb | running dns.py git.openstack.org complains about OS_USERNAME and running the commands that launch node spat out gives me unauthorized errors | 00:21 |
| clarkb | jeblair: I do | 00:21 |
| *** michchap has quit IRC | 00:21 | |
| jeblair | clarkb: you'll want: | 00:21 |
| jeblair | . ~root/rackdns-venv/bin/activate | 00:21 |
| jeblair | and... | 00:22 |
| jeblair | . ~root/ci-launch/openstackci-rs-nova.sh | 00:22 |
| *** michchap has joined #openstack-infra | 00:22 | |
| clarkb | that is what I have | 00:22 |
| clarkb | or I think that is what I have. I will start with a claen env and do that | 00:22 |
| jeblair | clarkb: you may want to start with a new shell then | 00:22 |
| clarkb | that seems to have worked. new shell was needed | 00:23 |
| *** dolphm has joined #openstack-infra | 00:24 | |
| *** sarob_ has joined #openstack-infra | 00:24 | |
| clarkb | according to haproxy.log on the new server we have connections going through it | 00:24 |
| jeblair | clarkb: new is 192.237.223.224 ? | 00:25 |
| clarkb | jeblair: yes | 00:25 |
| clarkb | old is 166.78.182.231 | 00:25 |
| jeblair | restarted snmpd | 00:26 |
| clarkb | I am going to make sure that puppet is running everywhere and picking up the change | 00:27 |
| *** sarob has quit IRC | 00:28 | |
| *** sarob_ has quit IRC | 00:29 | |
| clarkb | git nodes look good and review.o.o's replication config is up to date | 00:30 |
| pleia2 | \o/ | 00:30 |
| clarkb | jeblair: should we restart gerrit now? | 00:30 |
| jeblair | clarkb: yep | 00:31 |
| clarkb | doing that now | 00:31 |
| harlowja | oh no 503 | 00:32 |
| *** markmcclain has joined #openstack-infra | 00:32 | |
| harlowja | oh, nm, | 00:32 |
| jeblair | harlowja: ;| | 00:32 |
| clarkb | jeblair: gerrit should be back | 00:32 |
| morganfainberg | harlowja: restarting gerrit (in my experience) tends to do that. | 00:32 |
| harlowja | ;) | 00:32 |
| *** dina_belova has joined #openstack-infra | 00:33 | |
| jeblair | clarkb: quite a lot of tests have started since the ttl expired and none have blown up | 00:33 |
| clarkb | jeblair: cool | 00:34 |
| clarkb | jeblair: I don't see any unexpected errors in the gerrit error log | 00:34 |
| clarkb | jeblair: it is complaining about not being able to replicate All-Projects and API-Projects to gitXX | 00:34 |
| jeblair | clarkb: yep; someday we should do something about that. :) | 00:35 |
| jeblair | clarkb: but it's a perfectly normal error for now. | 00:35 |
| clarkb | jeblair: I started a tail on the haproxy log on old git.o.o and haven't seen any new connections for over 10 minutes | 00:35 |
| pleia2 | thinking tomorrow I'll take a little time to update https://wiki.openstack.org/wiki/Gerrit_Workflow & https://wiki.openstack.org/wiki/GerritJenkinsGithub (maybe rename+redirect too) | 00:36 |
| clarkb | jeblair: I am going to stop haproxy there and if we continue to see no breakage we can delete the server | 00:36 |
| jeblair | clarkb: sounds good | 00:36 |
| pleia2 | to point to git.o.o and mention it instead of github for things | 00:36 |
| clarkb | haproxy is stopped. keep your eyes open | 00:36 |
| jeblair | pleia2: yeah, i've been thinking that we should split up the gerritjenkinsgithub article into a series of 'reference guides' targeted to specific users (ptls, core, devs, release managers) | 00:37 |
| *** dina_belova has quit IRC | 00:37 | |
| *** portante|afk is now known as portante | 00:37 | |
| pleia2 | jeblair: that's a good idea, it has gotten a bit out of hand length-wise | 00:37 |
| jeblair | pleia2: that may be more than what you're thinking of doing; just throwing that out there. | 00:37 |
| jeblair | clarkb: i am curious about some changes at the bottom of the gate queue | 00:38 |
| clarkb | jeblair: because they have passed but are still in the pipeline? | 00:38 |
| jeblair | yep | 00:38 |
| anteaya | pleia2: I can help you with that if you want | 00:38 |
| jeblair | clarkb: and the nova change has failed and is still there | 00:39 |
| anteaya | new contributors get rather lost | 00:39 |
| clarkb | perhaps my gerrit restart caught them at the wrong time? | 00:39 |
| jeblair | clarkb: i'm starting to think so | 00:39 |
| pleia2 | anteaya: thanks, I'm going out for some fresh air now, but tomorrow I'll get to work on it and let you know :) | 00:39 |
| anteaya | pleia2: yeah, I am close to closing out too, just watching the git change over unfold | 00:40 |
| anteaya | tomorrow is good for me | 00:40 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Remove testr-full job https://review.openstack.org/43797 | 00:41 |
| jeblair | restarting zuul | 00:44 |
| clarkb | ok | 00:45 |
| jeblair | though it's a shame that change didn't make it in first | 00:45 |
| clarkb | jeblair: did you grab a stack trace first? | 00:45 |
| jeblair | clarkb: no, i have a handle on the bug from the logs | 00:46 |
| jeblair | it marked the change as reported despite it having failed to report, so it wouldn't touch it again | 00:46 |
| clarkb | gotcha | 00:46 |
| jeblair | it wasn't wedged | 00:46 |
| *** dolphm has quit IRC | 00:47 | |
| clarkb | jeblair: should I go ahead and delete the old git.o.o? | 00:49 |
| jeblair | clarkb: i think so. want to delete the old static.o.o while you're there? | 00:49 |
| jeblair | clarkb: should be in SHUTDOWN state | 00:49 |
| clarkb | jeblair: sure | 00:49 |
| *** dkliban has quit IRC | 00:50 | |
| clarkb | both are in the process of being deleted | 00:51 |
| *** jhesketh has quit IRC | 00:53 | |
| *** jhesketh has joined #openstack-infra | 00:54 | |
| *** alexpilotti has quit IRC | 00:55 | |
| clarkb | old git.o.o doesn't show up in nova list, but old static.o.o still has a status of DELETED | 00:56 |
| jeblair | clarkb: i wonder if i'll have to patch nodepool to deal with that too (i just did that for images) | 00:57 |
| clarkb | that? | 00:58 |
| clarkb | I am going to increase the DNS TTLs back up to an hour | 00:58 |
| clarkb | hmm I seem to have two git.o.o AAAA records. Oh I know, it says it modified the existing records with a TTL of 5 minutes but I bet it didn't | 01:01 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/gear: Fix ordering of state change in server https://review.openstack.org/43809 | 01:02 |
| clarkb | jeblair: I am just going to remove the record with a TTL of 5 | 01:02 |
| jeblair | that == servers sticking around in 'DELETED' state | 01:02 |
| jeblair | clarkb: sounds good, i think the AAAA handling is not right | 01:02 |
| jeblair | i've gotten in similar situations (though i think i removed both and added a new one) | 01:03 |
| jeblair | clarkb: ^ that gear change is, i believe, what caused the zuul 26 test failures | 01:03 |
| jeblair | er, i mean it fixes what caused them | 01:03 |
| clarkb | I think git.o.o should be good now | 01:04 |
| clarkb | removing the extra AAAA record left the existing one behind best I can tell | 01:04 |
| clarkb | static is still "DELETED" | 01:04 |
| *** weshay has quit IRC | 01:07 | |
| clarkb | jeblair: everything look good to you? | 01:12 |
| clarkb | jeblair: we seem to have survived a gate restart | 01:12 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Report item stats when removing a change https://review.openstack.org/43812 | 01:12 |
| jeblair | clarkb: new cacti graphs seem reasonable | 01:13 |
| jeblair | clarkb: and i'm kinda counting on all those jobs to turn red if they have problems with git | 01:13 |
| jeblair | clarkb: so i'd say lgtm | 01:13 |
| *** sandywalsh has quit IRC | 01:14 | |
| *** markmcclain has quit IRC | 01:15 | |
| *** rcleere has quit IRC | 01:16 | |
| clarkb | jeblair: what is the difference between change.ref and change.refspec in zuul? | 01:22 |
| clarkb | jeblair: seems to manifest as event.ref and event.refspec as well | 01:22 |
| *** nati_ueno has joined #openstack-infra | 01:22 | |
| *** jhesketh has quit IRC | 01:23 | |
| clarkb | jeblair: looking at https://review.openstack.org/#/c/43775/2/zuul/scheduler.py | 01:24 |
| clarkb | jeblair: and merger.py _mergeChange uses change.refspec | 01:24 |
| *** nati_ueno has joined #openstack-infra | 01:24 | |
| jeblair | clarkb: a Changeish only has a refspec if it is a Change (and has a ref only if it is a Ref) | 01:28 |
| jeblair | clarkb: Refs are created for ref-updated events | 01:28 |
| jeblair | clarkb: but that is referring to an event | 01:29 |
| jeblair | which is an Event | 01:29 |
| anteaya | that was fast, the speed at which all jobs below the gate failure cancelled was impressive | 01:29 |
| jeblair | which always has both attributes (though one may be None) | 01:29 |
| jeblair | which makes that code really really wrong | 01:29 |
| clarkb | jeblair: does https://review.openstack.org/#/c/43775/2/zuul/scheduler.py fix a bug then? | 01:29 |
| jeblair | clarkb: an inefficiency -- that's why the commit message says it always updates the repo on every change or ref-updated event | 01:30 |
| jeblair | clarkb: because the if was always true | 01:30 |
| clarkb | jeblair: or is it adding a bug? trying to parse your statement at 01:30:07 | 01:30 |
| jeblair | clarkb: the old behavior was inefficient because the code was incorrect (it made no sense and did not match its comments) but it was harmless (other than sucking up lots and lots of time) | 01:31 |
| *** sjing has joined #openstack-infra | 01:32 | |
| clarkb | so now we only fetch individual refs when they update | 01:33 |
| *** xchu has joined #openstack-infra | 01:33 | |
| *** jhesketh_ has joined #openstack-infra | 01:33 | |
| clarkb | which is more efficient | 01:33 |
| *** jhesketh has joined #openstack-infra | 01:33 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Set merge result correctly on exceptions https://review.openstack.org/43815 | 01:33 |
| *** dina_belova has joined #openstack-infra | 01:33 | |
| jeblair | clarkb: yep; that should be about half of 1 minute spent dealing with a nova event. the next change in that series deals with the other half. | 01:34 |
| jeblair | clarkb: i think 43815 fixes what we just saw | 01:34 |
| *** sandywalsh has joined #openstack-infra | 01:34 | |
| jeblair | jhesketh: you pinged earlier? | 01:36 |
| jeblair | jhesketh: er, like yesterday from your pov? :) | 01:36 |
| jhesketh | heh, yep | 01:36 |
| jhesketh | jeblair: in regards to the reporter separation if you have a few minutes? | 01:36 |
| jeblair | jhesketh: yep | 01:36 |
| Alex_Gaynor | mtreinish: So the tmpest-testr builds still say non-voting, is that change waiting on a <something> restart? | 01:37 |
| jeblair | Alex_Gaynor: the non-testr builds are now testr | 01:37 |
| jeblair | Alex_Gaynor: and the testr jobs are being deleted | 01:37 |
| Alex_Gaynor | oh. | 01:37 |
| Alex_Gaynor | Righton. | 01:37 |
| Alex_Gaynor | I imagine I didn't read the original email closely enough? | 01:37 |
| Alex_Gaynor | Does this apply to the postgresql ones as well? | 01:37 |
| jhesketh | so I've nearly finished the refactoring to your suggestion. I had actually initially started towards that model but ended up doing what was easier at the time. | 01:38 |
| openstackgerrit | A change was merged to openstack-infra/gear: Fix ordering of state change in server https://review.openstack.org/43809 | 01:38 |
| jeblair | Alex_Gaynor: yeah, i think it's everything except neutron | 01:38 |
| *** dina_belova has quit IRC | 01:38 | |
| Alex_Gaynor | jeblair: righton! | 01:38 |
| jhesketh | jeblair: anyway, I just have to deal with BasePipelineManager.getSubmitAllowNeeds() | 01:38 |
| jeblair | jhesketh: yeah, isn't that ugly? :) | 01:38 |
| jhesketh | heh, yep | 01:38 |
| clarkb | jeblair: your zuul changes lgtm | 01:38 |
| clarkb | I am going to AFK now | 01:38 |
| clarkb | dinner needs to be found | 01:39 |
| jeblair | jhesketh: ok, so how is that causing a problem? | 01:39 |
| jhesketh | (sorry my typing is a bit slow, I have one hand at the moment - silly rsi) | 01:40 |
| jhesketh | work out how I want to explain/tackle it | 01:40 |
| jeblair | jhesketh: because it's not clear which reporter should provide the list of allow-needs that should be given to the trigger... yeah? | 01:40 |
| jhesketh | yep :-) | 01:40 |
| jhesketh | so we can either scrape the gerrit reporters or we may need to change the layout a bit more | 01:41 |
| jeblair | jhesketh: so what if you asked the reporter if it can merge? | 01:41 |
| jhesketh | or rethink how it determines wen can merge | 01:41 |
| jeblair | jhesketh: instead of the trigger | 01:41 |
| *** nati_ueno has quit IRC | 01:42 | |
| *** nati_ueno has joined #openstack-infra | 01:42 | |
| jhesketh | hmm, I don't think a reporter should care | 01:42 |
| jeblair | that seems to make a bit of sense... | 01:42 |
| jeblair | jhesketh: but the reporter is the one doing the actual merge | 01:42 |
| jhesketh | jeblair: doesn't it just report to gerrit that it is okay to merge? | 01:43 |
| jeblair | jhesketh: so reporters get the canMerge method, and by default return True; the gerrit reporter does what its canMerge method does now (being passed the getsubmitallowneeds from the configuration of that reporter) | 01:43 |
| jeblair | jhesketh: if 'submit: True' is in the list, then that's the thing that actually tells gerrit to merge the change | 01:44 |
| jhesketh | okay, your suggestion makes sense | 01:44 |
| jeblair | jhesketh: (that gets turned into a 'gerrit review ... --submit' command which does the deed) | 01:44 |
| jhesketh | but I'm wondering if it would be too crazy to watch for another gerrit event of the --submit/merge and add that into the scheduler elsehow | 01:44 |
| openstackgerrit | A change was merged to openstack-infra/config: Remove testr-full job https://review.openstack.org/43797 | 01:45 |
| jeblair | jhesketh: as opposed to the busy-wait (is this discussion moving on to the isMerged method?) | 01:45 |
| jeblair | Alex_Gaynor: ^ https://review.openstack.org/43797 | 01:45 |
| Alex_Gaynor | cool | 01:46 |
| jhesketh | jeblair: not sure what you mean by busy-wait sorry.. | 01:46 |
| jhesketh | but kinda related to the isMerged, but more 'is going to merge' (if that makes sense) | 01:47 |
| jeblair | jhesketh: oh... well, uh, don't worry about that then; definitely don't look at how that method works. :) | 01:47 |
| jhesketh | lol | 01:47 |
| jeblair | jhesketh: so what canMerge is trying to do is determine whether zuul should even pay attention to a change before enqueuing it... | 01:48 |
| jeblair | jhesketh: as a specific example, if someone on openstack's gerrit sets 'approved +1' without a code-review +2, the change can't actually merge | 01:48 |
| jeblair | jhesketh: but our trigger is approved +1, so zuul uses the canMerge method to determine that it should actually ignore that change for now | 01:49 |
| jeblair | jhesketh: so this is all happening on the enqueueing side of things | 01:49 |
| jhesketh | right, okay | 01:49 |
| jeblair | jhesketh: (isMerged happens on the dequeuing side - where it has to find out if a change really did merge to determine if following changes which had assumed that the change had merged can proceed, or if it needs to reset) | 01:49 |
| *** jinkoo has joined #openstack-infra | 01:52 | |
| jeblair | jhesketh: (your suggestion of looking for the event from gerrit is applicable to isMerged, but that's almost certainly a separate change to zuul, and i'm not sure if it would be worth the trouble) | 01:52 |
| jhesketh | yeah, agreed | 01:52 |
| jeblair | jhesketh: got what you need? | 01:52 |
| jhesketh | I think so, thanks for that | 01:52 |
| jhesketh | just wrapping my head around it (looking through code a bit deeper) | 01:52 |
| jeblair | jhesketh: no prob; thanks for doing this :) i'm going to go try to find some food myself now... | 01:53 |
| jhesketh | jeblair: so what happens if somebody (other then zuul) posts a 'verified: +1'? | 01:53 |
| jhesketh | no worries :-) | 01:53 |
| anteaya | it looks to me like testing in the gate is taking a third of the time it took yesterday | 01:53 |
| jeblair | jhesketh: ah, so that's where get allow needs comes in | 01:54 |
| jeblair | jhesketh: because verified is in the config as something that zuul can set, it ignores it in canMerge | 01:54 |
| jeblair | jhesketh: and a +/-1 vote has no impact in gerrit, only a +/-2 vote | 01:54 |
| jeblair | jhesketh: and zuul is the only user permitted to leave those. | 01:54 |
| *** svarnau has quit IRC | 01:55 | |
| *** yaguang has joined #openstack-infra | 01:55 | |
| anteaya | a third to a half | 01:56 |
| *** dkliban has joined #openstack-infra | 01:56 | |
| jhesketh | jeblair: my reading of canMerge is that it requires zuul to be the one who set it | 01:57 |
| jhesketh | so I'm not sure what you mean by ignore? | 01:57 |
| *** anteaya has quit IRC | 02:00 | |
| *** jfriedly has quit IRC | 02:05 | |
| *** nati_ueno has quit IRC | 02:08 | |
| *** nati_ueno has joined #openstack-infra | 02:09 | |
| *** jamiem has quit IRC | 02:11 | |
| *** emagana has quit IRC | 02:13 | |
| *** nati_ueno has quit IRC | 02:13 | |
| jeblair | jhesketh: right, zuul knows (due to the configuration that says that this pipeline reports 'verified'; that's in getSubmitAllowNeeds) that it can ignore verified, so it passes that to canMerge as one of the things canMerge can ignore | 02:17 |
| jeblair | jhesketh: canMerge gets its information (other than what it should ignore) from gerrit, which says things like "this needs verified to merge" | 02:17 |
| jhesketh | oh I see, I read the logic inversely. Sorry! | 02:18 |
| *** melwitt has quit IRC | 02:29 | |
| *** dina_belova has joined #openstack-infra | 02:34 | |
| *** rcleere has joined #openstack-infra | 02:35 | |
| *** dina_belova has quit IRC | 02:38 | |
| *** mikal has quit IRC | 02:39 | |
| *** nati_ueno has joined #openstack-infra | 02:47 | |
| pabelanger | So, aside from vagrant, any other methods -infra is leaning towards for launch cloud instance? I know nodepool has something embedded and there is https://github.com/openstack-infra/config/tree/master/launch. I'd be curious to see if launch could / would move to another project. | 02:51 |
| pabelanger | I've been playing with it locally, and see like a good tool to have | 02:52 |
| *** zul has quit IRC | 03:00 | |
| *** zul has joined #openstack-infra | 03:02 | |
| *** ^d has quit IRC | 03:04 | |
| *** ^d has joined #openstack-infra | 03:05 | |
| *** ^d has joined #openstack-infra | 03:05 | |
| jhesketh | jeblair: next time you're around we should have a quick chat about backwards incompatibility with zuul | 03:06 |
| *** HenryG has quit IRC | 03:07 | |
| *** ^d has quit IRC | 03:09 | |
| *** nati_ueno has quit IRC | 03:12 | |
| *** nati_ueno has joined #openstack-infra | 03:13 | |
| *** nati_ueno has quit IRC | 03:17 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Add support for emailing results via SMTP https://review.openstack.org/42645 | 03:21 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/zuul: Separate reporters from triggers https://review.openstack.org/42643 | 03:21 |
| *** DennyZhang has joined #openstack-infra | 03:22 | |
| *** emagana has joined #openstack-infra | 03:23 | |
| *** vipul has quit IRC | 03:27 | |
| *** vipul has joined #openstack-infra | 03:27 | |
| *** emagana has quit IRC | 03:28 | |
| *** DennyZhang has quit IRC | 03:29 | |
| *** dina_belova has joined #openstack-infra | 03:34 | |
| *** changbl has quit IRC | 03:35 | |
| *** fifieldt has joined #openstack-infra | 03:35 | |
| *** dina_belova has quit IRC | 03:39 | |
| *** dguitarbite has quit IRC | 03:40 | |
| *** michchap_ has joined #openstack-infra | 03:44 | |
| *** michchap has quit IRC | 03:45 | |
| *** vogxn has joined #openstack-infra | 03:47 | |
| *** morganfainberg is now known as morganfainberg|a | 03:50 | |
| *** morganfainberg|a is now known as morganfainberg | 03:52 | |
| lifeless | can regular novaclient talk to rackspace? I'm getting nova list | 04:00 |
| lifeless | ERROR: AuthSystemNotFound: 'rackspace' | 04:00 |
| clarkb | it can but you either have to use tokens or passwords. I don't remember. Or install the rackspace auth plugin | 04:01 |
| *** nati_ueno has joined #openstack-infra | 04:06 | |
| lifeless | clarkb: sigh, why can't upstream be enough :( | 04:12 |
| lifeless | ahha, got it | 04:14 |
| jhesketh | is there any way to change your email address with gerrit? | 04:19 |
| clarkb | jhesketh: https://review.openstack.org/#/settings/contact I think it needs to be one of the emails you have configured with the openstack foundation | 04:20 |
| jhesketh | clarkb: clearly I'm very blind. Thanks for that | 04:21 |
| *** michchap_ has quit IRC | 04:24 | |
| *** michchap has joined #openstack-infra | 04:24 | |
| *** vipul is now known as vipul-away | 04:31 | |
| *** dina_belova has joined #openstack-infra | 04:35 | |
| *** dina_belova has quit IRC | 04:39 | |
| *** morganfainberg is now known as morganfainberg|a | 04:40 | |
| *** dstufft has quit IRC | 04:40 | |
| *** nati_ueno has quit IRC | 04:48 | |
| *** Ryan_Lane has quit IRC | 04:48 | |
| *** nati_ueno has joined #openstack-infra | 04:48 | |
| *** nati_ueno has joined #openstack-infra | 04:50 | |
| *** emagana has joined #openstack-infra | 05:00 | |
| *** afazekas has quit IRC | 05:00 | |
| *** morganfainberg|a is now known as morganfainberg | 05:02 | |
| *** boris-42 has joined #openstack-infra | 05:04 | |
| *** vogxn has quit IRC | 05:05 | |
| *** michchap_ has joined #openstack-infra | 05:10 | |
| *** nati_ueno has quit IRC | 05:13 | |
| *** michchap has quit IRC | 05:13 | |
| *** nati_ueno has joined #openstack-infra | 05:13 | |
| *** vogxn has joined #openstack-infra | 05:14 | |
| *** nicedice_ has quit IRC | 05:16 | |
| *** xBsd has joined #openstack-infra | 05:17 | |
| *** SergeyLukjanov has joined #openstack-infra | 05:17 | |
| *** nati_ueno has quit IRC | 05:18 | |
| *** Ryan_Lane has joined #openstack-infra | 05:18 | |
| *** afazekas has joined #openstack-infra | 05:18 | |
| *** dims has quit IRC | 05:21 | |
| *** xBsd has quit IRC | 05:23 | |
| *** vipul-away is now known as vipul | 05:29 | |
| *** dina_belova has joined #openstack-infra | 05:35 | |
| *** dina_belova has quit IRC | 05:40 | |
| *** dina_belova has joined #openstack-infra | 06:00 | |
| mkoderer | Hi folks, is there an overview picture about the full CI workflow somewhere? | 06:04 |
| mkoderer | maybe in your documentation? | 06:04 |
| jhesketh | mkoderer: Is this kinda what you're looking for: https://wiki.openstack.org/wiki/Gerrit_Workflow | 06:10 |
| mkoderer | jhesketh: great thank you! | 06:11 |
| jhesketh | no worries | 06:11 |
| *** gareth_kun has joined #openstack-infra | 06:14 | |
| *** odyssey4me has joined #openstack-infra | 06:16 | |
| *** dstufft has joined #openstack-infra | 06:16 | |
| *** vipul is now known as vipul-away | 06:24 | |
| *** emagana has quit IRC | 06:29 | |
| *** jpich has joined #openstack-infra | 06:34 | |
| *** michchap_ has quit IRC | 06:41 | |
| *** michchap has joined #openstack-infra | 06:42 | |
| *** dina_belova has quit IRC | 06:47 | |
| *** dina_belova has joined #openstack-infra | 06:47 | |
| *** dolphm has joined #openstack-infra | 06:48 | |
| *** ruhe has joined #openstack-infra | 06:49 | |
| *** dina_belova has quit IRC | 06:52 | |
| *** dolphm has quit IRC | 06:53 | |
| *** ruhe has quit IRC | 06:57 | |
| *** sjing has quit IRC | 07:02 | |
| *** sjing has joined #openstack-infra | 07:03 | |
| *** hashar has joined #openstack-infra | 07:14 | |
| *** jinkoo has quit IRC | 07:15 | |
| *** jinkoo has joined #openstack-infra | 07:16 | |
| *** sileht has quit IRC | 07:19 | |
| *** sileht has joined #openstack-infra | 07:20 | |
| *** thomasbiege has joined #openstack-infra | 07:20 | |
| *** jinkoo has quit IRC | 07:20 | |
| *** thomasbiege has quit IRC | 07:20 | |
| *** SergeyLukjanov has quit IRC | 07:27 | |
| *** Bada has joined #openstack-infra | 07:38 | |
| *** emagana has joined #openstack-infra | 07:39 | |
| *** emagana has quit IRC | 07:44 | |
| *** shardy_afk is now known as shardy | 07:45 | |
| *** jinkoo has joined #openstack-infra | 07:48 | |
| *** jinkoo has quit IRC | 07:48 | |
| *** amotoki_ has joined #openstack-infra | 07:54 | |
| *** amotoki has quit IRC | 07:56 | |
| *** thomasbiege1 has joined #openstack-infra | 07:57 | |
| *** dina_belova has joined #openstack-infra | 07:58 | |
| *** thomasbiege1 has quit IRC | 08:00 | |
| *** dina_belova has quit IRC | 08:02 | |
| *** xBsd has joined #openstack-infra | 08:04 | |
| *** morganfainberg is now known as morganfainberg|a | 08:05 | |
| *** ruhe has joined #openstack-infra | 08:05 | |
| *** fbo_away is now known as fbo | 08:10 | |
| *** BobBallAway is now known as BobBall | 08:18 | |
| *** thomasbiege1 has joined #openstack-infra | 08:20 | |
| *** thomasbiege1 has quit IRC | 08:24 | |
| *** derekh has joined #openstack-infra | 08:26 | |
| *** derekh has quit IRC | 08:26 | |
| *** derekh has joined #openstack-infra | 08:27 | |
| *** vogxn has quit IRC | 08:42 | |
| *** xBsd has quit IRC | 08:44 | |
| *** xBsd has joined #openstack-infra | 08:46 | |
| *** sjing has quit IRC | 08:52 | |
| *** dina_belova has joined #openstack-infra | 08:59 | |
| *** johnthetubaguy has joined #openstack-infra | 08:59 | |
| *** hashar has quit IRC | 08:59 | |
| *** dina_belova has quit IRC | 09:03 | |
| *** xBsd has quit IRC | 09:04 | |
| *** xBsd has joined #openstack-infra | 09:07 | |
| *** saper has quit IRC | 09:09 | |
| *** saper has joined #openstack-infra | 09:13 | |
| *** hashar has joined #openstack-infra | 09:16 | |
| EmilienM | Hi, someone from core team could have a look at this blueprint ? https://blueprints.launchpad.net/openstack-ci/+spec/jenkins-job-builder-plot | 09:23 |
| EmilienM | and if it's approved, review also this patch : https://review.openstack.org/#/c/43685/ | 09:23 |
| *** Dr01d has joined #openstack-infra | 09:24 | |
| hashar | EmilienM: hey :-] I am not sure what blueprints are but I can confirm you that patches in Gerrit are reviewed :-] | 09:33 |
| hashar | voilà :-] | 09:34 |
| EmilienM | hashar: thx | 09:34 |
| *** xBsd has quit IRC | 09:44 | |
| *** xBsd has joined #openstack-infra | 09:45 | |
| *** vogxn has joined #openstack-infra | 09:46 | |
| *** dolphm has joined #openstack-infra | 09:49 | |
| *** dolphm has quit IRC | 09:53 | |
| *** dina_belova has joined #openstack-infra | 09:59 | |
| *** dina_belova has quit IRC | 10:04 | |
| *** xchu has quit IRC | 10:04 | |
| yaguang | hi, all may be something wrong with jenkins check, I see https://review.openstack.org/#/c/35365/ fails | 10:12 |
| openstackgerrit | Emilien Macchi proposed a change to openstack-infra/jenkins-job-builder: Add SLOCCount plugin support https://review.openstack.org/43852 | 10:12 |
| *** ruhe has quit IRC | 10:12 | |
| *** DennyZhang has joined #openstack-infra | 10:14 | |
| *** dstufft_ has joined #openstack-infra | 10:16 | |
| *** gareth_kun has left #openstack-infra | 10:16 | |
| *** dstufft has quit IRC | 10:18 | |
| *** vogxn has quit IRC | 10:19 | |
| *** xBsd has quit IRC | 10:19 | |
| *** xBsd has joined #openstack-infra | 10:20 | |
| *** vogxn has joined #openstack-infra | 10:22 | |
| jd__ | FWIW something changed and now causes failure on devstack http://logs.openstack.org/51/43851/1/check/gate-tempest-devstack-vm-full/352a390/ | 10:29 |
| jd__ | I won't have time to look into it today, so if anybody's up for the task.. | 10:29 |
| *** fifieldt has quit IRC | 10:35 | |
| *** dstufft_ is now known as dstufft | 10:39 | |
| *** yaguang has quit IRC | 10:40 | |
| *** ruhe has joined #openstack-infra | 10:44 | |
| *** Bada has quit IRC | 10:45 | |
| *** DennyZhang has quit IRC | 10:46 | |
| *** dhellmann_ is now known as dhellmann | 10:54 | |
| *** dina_belova has joined #openstack-infra | 11:00 | |
| *** jamiem has joined #openstack-infra | 11:03 | |
| *** dina_belova has quit IRC | 11:04 | |
| *** dolphm has joined #openstack-infra | 11:06 | |
| *** weshay has joined #openstack-infra | 11:21 | |
| *** jamiem has quit IRC | 11:24 | |
| *** dolphm has quit IRC | 11:27 | |
| *** lcestari has joined #openstack-infra | 11:28 | |
| *** sjing has joined #openstack-infra | 11:30 | |
| *** vogxn has quit IRC | 11:33 | |
| *** pcm_ has joined #openstack-infra | 11:34 | |
| *** lcestari_ has joined #openstack-infra | 11:35 | |
| *** HenryG has joined #openstack-infra | 11:37 | |
| *** lcestari has quit IRC | 11:38 | |
| *** Bada has joined #openstack-infra | 11:41 | |
| *** sjing has quit IRC | 11:44 | |
| *** sjing has joined #openstack-infra | 11:44 | |
| *** sjing has quit IRC | 11:46 | |
| *** hashar has quit IRC | 11:46 | |
| *** david-lyle has quit IRC | 11:46 | |
| *** xBsd has quit IRC | 11:50 | |
| *** yaguang has joined #openstack-infra | 11:50 | |
| *** xBsd has joined #openstack-infra | 11:51 | |
| *** yaguang has quit IRC | 11:52 | |
| *** yaguang has joined #openstack-infra | 11:52 | |
| *** jhesketh_ has quit IRC | 11:53 | |
| *** david-lyle has joined #openstack-infra | 11:53 | |
| *** dolphm has joined #openstack-infra | 11:55 | |
| *** jamiem has joined #openstack-infra | 12:00 | |
| *** dina_belova has joined #openstack-infra | 12:00 | |
| *** dina_belova has quit IRC | 12:04 | |
| *** dina_belova has joined #openstack-infra | 12:14 | |
| *** hashar has joined #openstack-infra | 12:16 | |
| *** dina_belova has quit IRC | 12:18 | |
| *** alexpilotti has joined #openstack-infra | 12:21 | |
| *** redbeard2 has quit IRC | 12:24 | |
| *** rfolco has joined #openstack-infra | 12:26 | |
| lifeless | hmm | 12:29 |
| lifeless | Puppet::Parser::AST::Resource failed with error ArgumentError: Could not find declared class ntp at /root/config/modules/openstack_project/manifests/template.pp:24 on node gerrit | 12:29 |
| lifeless | trying the etherpad demo on http://ci.openstack.org/sysadmin.html | 12:29 |
| lifeless | where is the source code for the docs? | 12:30 |
| *** dprince has joined #openstack-infra | 12:31 | |
| hashar | lifeless: in puppet repo : openstack-infra/config | 12:32 |
| hashar | lifeless: the relevant file is ./doc/source/sysadmin.rst | 12:33 |
| lifeless | hashar: thanks; any thoughts on the error I am seeing about the ntp class ? | 12:33 |
| hashar | there is no ntp module ? :D | 12:35 |
| lifeless | hashar: I don't know. | 12:35 |
| openstackgerrit | lifeless proposed a change to openstack-infra/config: Fix copy-n-paste example to be clearer. https://review.openstack.org/43870 | 12:36 |
| *** dkranz has joined #openstack-infra | 12:39 | |
| *** hashar_ has joined #openstack-infra | 12:43 | |
| *** thomasbiege has joined #openstack-infra | 12:45 | |
| *** thomasbiege has quit IRC | 12:45 | |
| *** hashar has quit IRC | 12:46 | |
| *** hashar_ is now known as hashar | 12:46 | |
| *** ArxCruz has joined #openstack-infra | 12:52 | |
| *** michchap has quit IRC | 12:56 | |
| *** michchap has joined #openstack-infra | 12:56 | |
| *** dkranz has quit IRC | 12:57 | |
| *** xBsd has quit IRC | 13:00 | |
| *** vogxn has joined #openstack-infra | 13:01 | |
| *** xBsd has joined #openstack-infra | 13:03 | |
| *** vogxn has quit IRC | 13:04 | |
| *** anteaya has joined #openstack-infra | 13:06 | |
| *** dkliban has quit IRC | 13:06 | |
| *** changbl has joined #openstack-infra | 13:07 | |
| *** vogxn has joined #openstack-infra | 13:12 | |
| *** dina_belova has joined #openstack-infra | 13:14 | |
| *** dina_belova has quit IRC | 13:19 | |
| *** ruhe has quit IRC | 13:19 | |
| *** ruhe has joined #openstack-infra | 13:23 | |
| *** changbl has quit IRC | 13:23 | |
| *** rpodolyaka has joined #openstack-infra | 13:25 | |
| *** sgviking has quit IRC | 13:27 | |
| *** mriedem has joined #openstack-infra | 13:27 | |
| *** rcleere has quit IRC | 13:36 | |
| *** ruhe has quit IRC | 13:42 | |
| openstackgerrit | A change was merged to openstack/requirements: pin keyring to < 2.0 https://review.openstack.org/43690 | 13:44 |
| *** sandywalsh has quit IRC | 13:44 | |
| ArxCruz | ALL: I'm trying to replicate openstack infrastructure (jenkins, gerrit, zuul) in the company I work, does anyone knows where can I start? I read ci.openstack.org but isn't helping… | 13:44 |
| *** ruhe has joined #openstack-infra | 13:45 | |
| *** dkliban has joined #openstack-infra | 13:47 | |
| *** _TheDodd_ has joined #openstack-infra | 13:48 | |
| *** sgviking has joined #openstack-infra | 13:48 | |
| *** cppcabrera has joined #openstack-infra | 13:50 | |
| *** cppcabrera has left #openstack-infra | 13:50 | |
| *** xBsd has quit IRC | 13:54 | |
| *** xBsd has joined #openstack-infra | 13:55 | |
| *** sandywalsh has joined #openstack-infra | 13:57 | |
| anteaya | hi ArxCruz well replicating all of openstack infra is a bit of a task, have you selected a starting point? something you would like to launch first? | 14:00 |
| ArxCruz | anteaya: yes, Gerrit, I was able to install Gerrit in one of my machines manually, but I would like to use puppet could you please help me ? | 14:00 |
| ArxCruz | it's hard to find a how to | 14:01 |
| ArxCruz | :/ | 14:01 |
| anteaya | well I haven't done that myself, but recognizing that Gerrit is the first point of departure can help me dig up somethings that may help you | 14:01 |
| anteaya | what have you got in place so far? | 14:01 |
| *** burt has joined #openstack-infra | 14:01 | |
| ArxCruz | anteaya: well, not much, I have a machine prepared to install stuff | 14:02 |
| ArxCruz | I also have played with puppet | 14:02 |
| ArxCruz | but no success so far | 14:02 |
| *** michchap has quit IRC | 14:02 | |
| *** _TheDodd_ has quit IRC | 14:03 | |
| anteaya | ArxCruz: I'm assuming you are familiar with this page: http://ci.openstack.org/gerrit.html | 14:03 |
| *** michchap has joined #openstack-infra | 14:03 | |
| ArxCruz | anteaya: yes! | 14:03 |
| anteaya | and here: http://ci.openstack.org/sysadmin.html#sysadmin | 14:03 |
| ttx | anteaya: https://review.openstack.org/#/c/42999/ and https://review.openstack.org/#/c/42340/ have been approved yesterday and not on the gate queue yet | 14:03 |
| *** _TheDodd_ has joined #openstack-infra | 14:03 | |
| *** dkranz has joined #openstack-infra | 14:03 | |
| anteaya | ttx will look | 14:03 |
| ttx | anteaya: I "reverified no bug" them but that doesn't seem to work either | 14:03 |
| anteaya | ArxCruz: and you have a copy of: http://git.openstack.org/cgit/openstack-infra/gerrit/tree/ | 14:04 |
| ttx | although queue was done to 0 | 14:04 |
| *** dkranz has quit IRC | 14:04 | |
| anteaya | ttx odd | 14:04 |
| ttx | down | 14:04 |
| burt | ttx: do patches make it into the gate even if the patches they depend on are not approved? | 14:04 |
| ArxCruz | anteaya: now I have :) I've downloaded gerrit from gerrit web page | 14:05 |
| ttx | anteaya: that's why I ask :) | 14:05 |
| ttx | burt: oh, nice catch | 14:05 |
| ttx | no they don't | 14:05 |
| ttx | anteaya: ignore me | 14:06 |
| *** ruhe has quit IRC | 14:06 | |
| ttx | patch queue for that blueprint is pretty huge | 14:07 |
| anteaya | ttx no worries - I just found the verification at timestamp 2013-08-27T01:49:03 in last night's log: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2013-08-27.log | 14:08 |
| anteaya | only patches that can merge make it into the gate pipeline | 14:08 |
| anteaya | glad I was paying attention last night | 14:08 |
| anteaya | :D | 14:08 |
| anteaya | ArxCruz: might this be of use? http://git.openstack.org/cgit/openstack-infra/gerrit/plain/Documentation/install.txt | 14:10 |
| ArxCruz | anteaya: I will try it | 14:11 |
| *** olaph has quit IRC | 14:11 | |
| *** ruhe has joined #openstack-infra | 14:11 | |
| ArxCruz | and let you know If I got in trouble :) thanks! | 14:11 |
| anteaya | ArxCruz: give it a whirl - in an isolated environment of course and then yes, ask about your blockages | 14:11 |
| *** rcleere has joined #openstack-infra | 14:11 | |
| anteaya | np | 14:12 |
| krtaylor | anteaya, we have been trying to get puppet up and working, I feel that may be our first hurdle | 14:12 |
| krtaylor | anteaya, but ArxCruz and I will try working though a real example with gerrit | 14:13 |
| anteaya | ah okay | 14:13 |
| anteaya | yes, try standing up a gerrit first | 14:13 |
| anteaya | and see how far you get | 14:13 |
| anteaya | then stand up puppet separately | 14:13 |
| anteaya | then when you understand each in isolation try them both together | 14:13 |
| anteaya | the together part will have its own fun | 14:14 |
| krtaylor | anteaya, yes, I bet so, thanks | 14:14 |
| *** hashar has quit IRC | 14:14 | |
| anteaya | np | 14:14 |
| *** dina_belova has joined #openstack-infra | 14:15 | |
| anteaya | krtaylor: regarding puppet - I assume you have seen this: http://docs.puppetlabs.com/guides/introduction.html | 14:15 |
| *** redbeard2 has joined #openstack-infra | 14:16 | |
| *** ruhe has quit IRC | 14:16 | |
| *** vogxn has quit IRC | 14:17 | |
| mriedem | question about jenkins.openstack.org - how to get that fancy overview page? is that something in the latest jenkins b/c i'm not seeing a plugin. i'm using 1.514 and community is using 1.525. | 14:17 |
| anteaya | mriedem: is this the page you want? https://jenkins.openstack.org/ | 14:18 |
| mriedem | anteaya: yeah, i'm just wondering how they got the overview page | 14:18 |
| anteaya | I go to ci.openstack.org and in the top menu select the jenkins link | 14:18 |
| anteaya | ah when setting up their jenkins? | 14:19 |
| mriedem | anteaya: correct | 14:19 |
| anteaya | so in your jenkins you don't see this | 14:19 |
| *** dina_belova has quit IRC | 14:19 | |
| mriedem | anteaya: nope, but i'm backlevel | 14:19 |
| anteaya | hmmm, I don't know the answer to this question | 14:19 |
| mriedem | been thinking about updating but updating jenkins always scares me when i'm stable | 14:19 |
| *** jpich has quit IRC | 14:19 | |
| anteaya | yeah, I hear that | 14:19 |
| * anteaya goes looking | 14:20 | |
| *** dkranz has joined #openstack-infra | 14:20 | |
| mriedem | anteaya: this looks like it: https://wiki.jenkins-ci.org/display/JENKINS/Dashboard+View | 14:20 |
| openstackgerrit | David Kranz proposed a change to openstack-infra/config: WIP test of slow heat gate job https://review.openstack.org/43891 | 14:20 |
| anteaya | mriedem: does that plugin work with your version? | 14:21 |
| mriedem | anteaya: yeah, requires jenkins >= 1.509.1 | 14:21 |
| *** jpich has joined #openstack-infra | 14:21 | |
| mriedem | anyway, this doesn't really have anything to do with openstack, i just like to see what the pros are doing :) | 14:22 |
| anteaya | okay, I am looking at that page and the screenshots don't look the same to me, but if it gives you what you want, then good | 14:22 |
| anteaya | mriedem: sure, happy to help when we can | 14:22 |
| anteaya | sometimes it is only to show support and encourage you to try something yourself | 14:22 |
| anteaya | at least for me, I know the least, I am just the only one on the East coast right now | 14:23 |
| *** michchap has quit IRC | 14:23 | |
| ttx | anteaya: do we have early feedback on the impact of parallel-test on queue health ? | 14:25 |
| *** olaph has joined #openstack-infra | 14:25 | |
| anteaya | ttx my early assessment is that it cuts the gate pipeline tests in half - needs about 30 minutes to test a patch rather than 1 hour+ | 14:26 |
| ttx | anteaya: without a significant raise in the gate resets so far ? | 14:26 |
| anteaya | I also believe this means we require fewer test nodes up at the same time, since tests finish faster and then go back to the pool | 14:26 |
| *** jkt_ has joined #openstack-infra | 14:26 | |
| *** jkt_ has joined #openstack-infra | 14:26 | |
| ttx | in the end the metric should be number of patches merged per hour | 14:26 |
| *** jkt has quit IRC | 14:26 | |
| anteaya | ttx I don't have access to any data on number of gate resets since the change to parallel-testing | 14:27 |
| anteaya | if it is publicly consumable, I don't yet know where to look | 14:27 |
| *** jkt_ is now known as jkt | 14:27 | |
| anteaya | ttx the merge sparkline (next to post) seems to be doing pretty good business | 14:27 |
| anteaya | but I also don't have the number of patches merged per hour | 14:28 |
| dkliban | I submitted a patch to Nova, Jenkins marked it as a failure, however, it did not provide any test results. Is this normal? https://review.openstack.org/#/c/43513/ | 14:28 |
| anteaya | if it is on cacti or graphite, I hope to learn where so I can answer that question in future | 14:28 |
| anteaya | dkliban: I'll look | 14:28 |
| anteaya | dkliban: I see test logs attached to the patch: http://logs.openstack.org/13/43513/2/check/gate-nova-python26/c73210e/console.html | 14:29 |
| krtaylor | anteaya, re: puppet, yes, ArxCruz and I have been doing a lot of reading, thanks | 14:29 |
| anteaya | perhaps they arrived in the time between your question and me looking | 14:29 |
| anteaya | krtaylor: great | 14:29 |
| *** changbl has joined #openstack-infra | 14:30 | |
| ttx | anteaya: I guess we could graph change-merged events but my graphite-foo is a bit lacking | 14:31 |
| anteaya | ttx mine too | 14:31 |
| anteaya | let me see if I can figure it out | 14:31 |
| ttx | I have a graph but it's a bit too discrete to make any sense | 14:31 |
| *** adalbas has quit IRC | 14:32 | |
| openstackgerrit | David Kranz proposed a change to openstack-infra/config: Add experimental slow heat job https://review.openstack.org/43891 | 14:32 |
| anteaya | ttx does this help? http://tinyurl.com/llpf8fv | 14:32 |
| dkliban | anteaya, thanks for checking | 14:32 |
| *** dina_belova has joined #openstack-infra | 14:32 | |
| anteaya | dkliban: np | 14:33 |
| anteaya | and yes, if Jenkins reported a failure and failed to provide logs that would be a bug and we would want to know | 14:33 |
| *** ruhe has joined #openstack-infra | 14:33 | |
| ttx | anteaya: htat's what I got too. We need some aggregation over a significant time period like "one day" | 14:34 |
| anteaya | ttx okay, let me change the time frame | 14:34 |
| ttx | but then it's probably too early to have enough data | 14:34 |
| *** vogxn has joined #openstack-infra | 14:36 | |
| anteaya | this is from 8pm Eastern time yesterday to 8am Eastern this morning: http://tinyurl.com/mt7mv6j | 14:38 |
| *** hashar has joined #openstack-infra | 14:38 | |
| anteaya | it takes time in Eastern (my time zone) and then converts to UTC (I think) for the graphalot graph | 14:38 |
| pabelanger | lifeless, ntp is pulled in when install_modules.sh is run | 14:39 |
| *** pentameter has joined #openstack-infra | 14:39 | |
| anteaya | sorry the last one was out by an hour, try this one: http://tinyurl.com/mhd3g2v | 14:39 |
| anteaya | graphalot doesn't do daylight savings time apparently | 14:40 |
| *** datsun180b has joined #openstack-infra | 14:42 | |
| anteaya | for instance the vertical scale for that graph has a lot of 1.8 changes merged, no idea how you have 0.8 of a change merged. I don't know how the vertical scale is set | 14:46 |
| *** adalbas has joined #openstack-infra | 14:48 | |
| *** xBsd has quit IRC | 14:50 | |
| ttx | anteaya: you have an equivalent counter in stats_counter | 14:53 |
| ttx | which will give integer values :) | 14:53 |
| *** xBsd has joined #openstack-infra | 14:53 | |
| *** reed has joined #openstack-infra | 14:54 | |
| anteaya | ttx ah ha, I will look | 14:55 |
| anteaya | look at that, thanks ttx | 14:56 |
| *** rcleere_ has joined #openstack-infra | 15:03 | |
| *** dkranz has quit IRC | 15:03 | |
| *** rcleere has quit IRC | 15:04 | |
| *** rcleere_ is now known as rcleere | 15:04 | |
| *** xBsd has quit IRC | 15:04 | |
| *** xBsd has joined #openstack-infra | 15:08 | |
| *** nati_ueno has joined #openstack-infra | 15:11 | |
| *** ruhe has quit IRC | 15:13 | |
| *** yaguang has quit IRC | 15:13 | |
| *** nati_ueno has quit IRC | 15:16 | |
| *** dkranz has joined #openstack-infra | 15:17 | |
| *** nati_ueno has joined #openstack-infra | 15:17 | |
| rpodolyaka | Hey, guys! I've been wondering, is there a way to test changes to openstack-infra/config (e. g. a few new jobs have been added) before the corresponding change is actually merged? | 15:20 |
| anteaya | rpodolyaka: all patches submitted to gerrit are testing in our zuul and jenkins infrastructure | 15:21 |
| anteaya | what kind of testing do you mean, rpodolyaka? | 15:21 |
| *** nati_ueno has quit IRC | 15:21 | |
| anteaya | do you mean local testing before the patch is submitted to gerrit? | 15:21 |
| rpodolyaka | anteya: ok, I made a change to openstack-infra/config, which adds two new jobs for sqlalchemy-migrate | 15:21 |
| rpodolyaka | anteya: this change was merged, and then I made a "recheck" comment to my patch to sqlalchemy-migrate | 15:22 |
| rpodolyaka | anteya: it failed due to a mistake in the corresponding change to oopenstack-infra/config (I use run-tox.sh incorrectly...) | 15:22 |
| *** ruhe has joined #openstack-infra | 15:23 | |
| rpodolyaka | anteya: I want to fix my mistake, but this time I want to be sure my patch is actually OK before it's merged | 15:23 |
| *** branen_ has quit IRC | 15:23 | |
| anteaya | I question your use of the word merged in this context | 15:23 |
| anteaya | do you mean submitted to gerrit? | 15:23 |
| anteaya | rpodolyaka: have you a url for the patch? | 15:23 |
| rpodolyaka | anteya: submitted and approved in openstack-infra/config | 15:23 |
| anteaya | do you have a url? | 15:24 |
| *** branen has joined #openstack-infra | 15:24 | |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/reviewday: Update the URL's in the reviewday report for SSL https://review.openstack.org/43901 | 15:24 |
| anteaya | you can run tests locally, here is a way to get started: http://git.openstack.org/cgit/openstack/oslo-incubator/plain/TESTING.rst | 15:24 |
| rpodolyaka | anteya: here is patch to openstack-infra/config https://review.openstack.org/#/c/39304/ | 15:24 |
| anteaya | thanks, I'll look | 15:24 |
| rpodolyaka | anteya: and here is one to sqlalchemy-migrate https://review.openstack.org/#/c/40436/ which now has two new jobs that always fail... | 15:25 |
| anteaya | you are correct, the patch was merged | 15:25 |
| branen | jeblair, ping | 15:25 |
| anteaya | rpodolyaka: so you want to run the tests locally, on the 40436 patch is that correct? | 15:27 |
| anteaya | does the TESTING.rst file I shared with you help answer your question? | 15:27 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/reviewday: Update the URL's in the reviewday report for SSL https://review.openstack.org/43901 | 15:27 |
| anteaya | rpodolyaka: it is the tests to the tox file that are failing, correct? | 15:29 |
| openstackgerrit | A change was merged to openstack-infra/reviewday: Update the URL's in the reviewday report for SSL https://review.openstack.org/43901 | 15:29 |
| *** Bada has quit IRC | 15:32 | |
| *** UtahDave has joined #openstack-infra | 15:33 | |
| anteaya | rpodolyaka: if I understand this correctly, the tests that are failing for 40436 are the tests introduced by patch https://review.openstack.org/#/c/38459/4/ | 15:34 |
| anteaya | if that is the case, I think you need to talk to jeblair to get this sorted out | 15:34 |
| *** emagana has joined #openstack-infra | 15:34 | |
| anteaya | since some tests may need to be over-ridden to get the correct code in place | 15:34 |
| anteaya | rpodolyaka: how crucial are these changes? do you have to meet the code freeze deadline with this? | 15:35 |
| anteaya | we are sorting things on a time based priority right now | 15:36 |
| *** shadower is now known as shadower|ragequi | 15:40 | |
| *** hashar has quit IRC | 15:40 | |
| *** shadower|ragequi is now known as shadower | 15:40 | |
| jeblair | branen: pong | 15:41 |
| *** odyssey4me has quit IRC | 15:41 | |
| *** jruzicka has joined #openstack-infra | 15:42 | |
| jeblair | clarkb: https://review.openstack.org/#/c/42608/ i'll spin that up when it lgty | 15:43 |
| ArxCruz | anteaya: gerrit installed, is there a way to import the openstack projects automatically to gerrit, or need to do this manually ? | 15:43 |
| branen | jeblair, Hi, can you help me get a very simple version change approved? https://review.openstack.org/#/c/42991/ This library is only used by the HP 3PAR cinder drivers. Thanks | 15:43 |
| *** dkranz has quit IRC | 15:44 | |
| jeblair | branen: done (i don't believe gr-base needed to be changed) | 15:46 |
| anteaya | ArxCruz: congratulations on getting gerrit up | 15:46 |
| anteaya | that was fast | 15:46 |
| anteaya | I suggest you next get gerrit up with puppet | 15:46 |
| branen | jeblair, Thank you! | 15:46 |
| anteaya | once you understand how to get gerrit up with puppet it should be easier for you to get the other pieces | 15:47 |
| anteaya | what is the next piece you want, after gerrit is up with puppet? | 15:47 |
| *** afazekas has quit IRC | 15:47 | |
| rpodolyaka | anteaya: yes, you are absolutely correct, the new Jenkins jobs fail, though I can run tests locally. They aren't crucial at all, so I can wait, but it would be nice to have this fixed... | 15:51 |
| anteaya | rpodolyaka: great thanks, jeblair is here I will let him prioritize the work accordingly and he will fix or ping you with questions when he can get to it | 15:52 |
| jeblair | rpodolyaka: have you proposed a fix to infra/config? | 15:52 |
| rpodolyaka | anteaya: cool, thanks! | 15:52 |
| *** gyee has joined #openstack-infra | 15:53 | |
| rpodolyaka | jeblair: not yet, I wanted to test them first, but wasn't sure there was a way to do this, until the change is actually merged to openstack-infra/config | 15:53 |
| jeblair | rpodolyaka: well, you can run the 'run-tox' script locally | 15:54 |
| jeblair | rpodolyaka: but it seems like a pretty straightforward error and easy to fix | 15:54 |
| *** odyssey4me has joined #openstack-infra | 15:54 | |
| rpodolyaka | jeblair: sure. I'm just a little bit worried on all these magical job template substitution like 'github-org' or 'name' | 15:54 |
| jeblair | rpodolyaka: you can also put new jobs into the experimental or silent pipelines in zuul , or make them non-voting to have jenkins run them and make sure they work. | 15:55 |
| jeblair | rpodolyaka: you can see the output of that by running jenkins-job-builder on your change locally | 15:55 |
| jeblair | rpodolyaka: it will produce xml files that you can inspect | 15:55 |
| rpodolyaka | jeblair: cool! that's what I'm looking for! Thanks! | 15:55 |
| *** dkranz has joined #openstack-infra | 15:58 | |
| *** nati_ueno has joined #openstack-infra | 15:58 | |
| jeblair | tagged gear 0.4.0 | 15:58 |
| anteaya | jeblair clarkb I have been asked to officiate the Tuskar PTL elections, I am going with the notion I can get the mailing list of eligible voters from gerrit somehow | 15:59 |
| *** rpodolyaka has left #openstack-infra | 16:01 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Require gear 0.4.0 https://review.openstack.org/43912 | 16:01 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Use the 'resolve' merge strategy by default https://review.openstack.org/43629 | 16:01 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Fix graceful exit https://review.openstack.org/43803 | 16:01 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Set merge result correctly on exceptions https://review.openstack.org/43815 | 16:01 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Report item stats when removing a change https://review.openstack.org/43812 | 16:01 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Don't reset the local repo https://review.openstack.org/43792 | 16:01 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Fetch specific refs on ref-updated events https://review.openstack.org/43775 | 16:01 |
| jeblair | anteaya: i'd use git | 16:04 |
| jeblair | anteaya: there's an email address for everyone who has written a patch merged to the repo | 16:04 |
| anteaya | ah okay, thanks | 16:04 |
| anteaya | is there currently a script or known command for extracting email addresses from contributors? | 16:05 |
| jeblair | anteaya: we use a different script for openstack that does use gerrit, but it requires admin access, and we have particular rules about when contributions were merged, etc. | 16:06 |
| jeblair | anteaya: you should be able to construct something with 'git log'. | 16:06 |
| anteaya | okay, let me try some things with git log | 16:06 |
| anteaya | as I write up the wiki page for "how to run an election" shall I mention both gerrit and git for getting a list of voters? | 16:07 |
| *** rnirmal has joined #openstack-infra | 16:09 | |
| clarkb | morning | 16:11 |
| *** nicedice_ has joined #openstack-infra | 16:11 | |
| *** odyssey4me has quit IRC | 16:11 | |
| clarkb | anteaya: probably just git as gerrit needs special permissions | 16:12 |
| anteaya | morning clarkb | 16:12 |
| anteaya | okay | 16:12 |
| clarkb | mornings are so busy. jeblair I will get to that review asap | 16:13 |
| *** ^d has joined #openstack-infra | 16:13 | |
| *** ^d has joined #openstack-infra | 16:13 | |
| *** pcrews has quit IRC | 16:15 | |
| jeblair | reed: sure you don't want to have a generic 'conference-track-chairs' list and just repopulate it every 6 months? | 16:15 |
| jeblair | reed: or do you plan on deleting it after the summit? | 16:16 |
| reed | jeblair, I like to delete | 16:16 |
| reed | btw, how do you delete it with puppet? | 16:17 |
| jeblair | reed: ok. we can probably do that with puppet too (ensure => absent or something) it will be fun to test. :) | 16:17 |
| reed | cool | 16:17 |
| *** odyssey4me has joined #openstack-infra | 16:18 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Revert "Adding AnsiColor Support" https://review.openstack.org/43771 | 16:19 |
| clarkb | puppet should be able to delete lists. it can add and remove but not update | 16:22 |
| *** mattf has joined #openstack-infra | 16:27 | |
| mattf | ruhe, here? | 16:27 |
| ruhe | yes | 16:28 |
| ruhe | mattf, i've seen anteaya and jeblair discussing election process. so i thought you might help since you did the same just recently | 16:29 |
| mattf | ahh, gotcha. sure, is the discussion still going? | 16:29 |
| anteaya | ruhe mattf sure, mind if we go to -dev? | 16:29 |
| *** xBsd has quit IRC | 16:30 | |
| *** mattf has left #openstack-infra | 16:30 | |
| *** nati_ueno has quit IRC | 16:31 | |
| *** nati_ueno has joined #openstack-infra | 16:31 | |
| *** xBsd has joined #openstack-infra | 16:32 | |
| *** nati_ueno has quit IRC | 16:36 | |
| *** nati_ueno has joined #openstack-infra | 16:37 | |
| *** dina_belova has quit IRC | 16:39 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Not sending description or disabled by default https://review.openstack.org/39995 | 16:39 |
| *** dina_belova has joined #openstack-infra | 16:40 | |
| ArxCruz | anteaya: jenkins | 16:41 |
| *** jfriedly has joined #openstack-infra | 16:41 | |
| ArxCruz | anteaya: thanks :) | 16:41 |
| ArxCruz | anteaya: now with puppet, I don't know how to begin... | 16:42 |
| openstackgerrit | Steven Hardy proposed a change to openstack/requirements: bump keystoneclient to 0.3.2 https://review.openstack.org/43919 | 16:42 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add SSH user/key info to nodepool config https://review.openstack.org/43920 | 16:43 |
| *** dims has joined #openstack-infra | 16:43 | |
| *** dina_belova has quit IRC | 16:44 | |
| anteaya | ArxCruz: I suggest learning and standing up puppet on its own first, and then using it to stand up a new gerrit, since you have done gerrit once already | 16:48 |
| *** morganfainberg|a is now known as morganfainberg | 16:48 | |
| *** Dr01d has quit IRC | 16:49 | |
| anteaya | ArxCruz: there used to be a great puppet vm tutorial - you downloaded a vm with puppet already installed - but it seems to have disappeared - only option if for puppet enterprise | 16:49 |
| *** nati_ueno has quit IRC | 16:49 | |
| *** nati_ueno has joined #openstack-infra | 16:49 | |
| *** ruhe has quit IRC | 16:50 | |
| clarkb | ArxCruz: anteaya: http://ci.openstack.org/sysadmin.html#making-a-change-in-puppet has useful info on how to use puppet in our environment | 16:52 |
| *** fbo is now known as fbo_away | 16:52 | |
| *** nati_ueno_2 has joined #openstack-infra | 16:53 | |
| pleia2 | ArxCruz: and on http://ci.openstack.org/gerrit.html the "Puppet" section shows you the specific puppet modules + files we're using, it all comes from our config/ repo | 16:53 |
| pleia2 | review.pp is the manifest for our gerrit server, for instance | 16:54 |
| *** nati_ueno has quit IRC | 16:54 | |
| *** xBsd has quit IRC | 16:58 | |
| *** jpich has quit IRC | 16:58 | |
| *** mgagne has joined #openstack-infra | 16:59 | |
| *** dina_belova has joined #openstack-infra | 17:00 | |
| *** ruhe has joined #openstack-infra | 17:00 | |
| *** vipul-away is now known as vipul | 17:01 | |
| *** adalbas has quit IRC | 17:01 | |
| ArxCruz | pleia2: so, if I install puppet, and ran puppet apply review.pp I suposed to have gerrit installed? (removing all those hiera stuff of course) | 17:01 |
| *** vipul is now known as vipul-away | 17:03 | |
| pleia2 | ArxCruz: it's more complicated than that, you'll need to configure things, review.pp is specifically designed for openstack, you'll notice it's in the openstack_project module directory | 17:03 |
| pleia2 | ArxCruz: so have a browse through the gerrit/ module first, then look at our review.pp as an example, but you'll want to write your own with your own info and settings | 17:03 |
| ArxCruz | pleia2: that's the problem, it's "complicated" not so easy as ci.openstack.org/sysadmin.html shows | 17:03 |
| *** Bada has joined #openstack-infra | 17:04 | |
| pleia2 | ArxCruz: right, the sysadmin.html page shows just shows the super basics of spinning up any system in our infrastructure, beyond that you need to dig into the specific application to do the application-level configuration | 17:04 |
| ArxCruz | i believe it's should be cool if infra had a directory in git called puppet_generic for example | 17:04 |
| pleia2 | ArxCruz: well, everything that is not in the openstack_project/ module *should* be usable by other projects | 17:05 |
| pleia2 | openstack_project/ module applies our specific things | 17:05 |
| ArxCruz | pleia2: okay, I will digg into puppet now, if I got in trouble I will yell here :) | 17:06 |
| ArxCruz | pleia2: thanks! | 17:06 |
| pleia2 | ArxCruz: sure, good luck :) | 17:06 |
| clarkb | jeblair: I have reviewed the groups-dev patch and I think there are a couple things that need fixing | 17:06 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Use an OrderedDict for pipelines https://review.openstack.org/43922 | 17:06 |
| *** derekh has quit IRC | 17:07 | |
| *** thomasbiege has joined #openstack-infra | 17:07 | |
| clarkb | jgriffith: you are awesome thank you | 17:08 |
| jeblair | reed: ^ clark found some things on the groups-dev patch https://review.openstack.org/#/c/42608/4 | 17:08 |
| jeblair | clarkb: nice catches | 17:08 |
| *** thomasbiege has quit IRC | 17:09 | |
| *** pcrews has joined #openstack-infra | 17:10 | |
| jeblair | clarkb: https://review.openstack.org/#/c/43920/ should get us back to puppet-controlled nodepool | 17:10 |
| jeblair | clarkb: and when you review https://review.openstack.org/#/c/43912/ and https://review.openstack.org/#/c/43922/ i'll upgrade zuul | 17:10 |
| clarkb | jeblair: ok I will look at those now. Seems like a reasonable place to continue the review | 17:11 |
| dkranz | clarkb: So I'm good to go on what we discussed yesterday and just need +A on https://review.openstack.org/#/c/43891/ | 17:12 |
| clarkb | jeblair: does the nodepool change not use jenkins credential store? | 17:12 |
| clarkb | jeblair: is there a reason for that? | 17:13 |
| dkranz | clarkb: After that works I'll submit non-voting to all projects. | 17:13 |
| jeblair | clarkb: it does use it; see the bottom in the targets section (credentials-id) | 17:13 |
| jeblair | clarkb: what's being added there is info for nodepool itself to be able to log into hosts | 17:13 |
| clarkb | jeblair: and for dkranz's change are you still concerned that all of the exerpimental jobs are nonvoting? | 17:13 |
| jeblair | clarkb: (which it has always done, just with hardcoded values) | 17:14 |
| clarkb | jeblair: should we make them voting first? | 17:14 |
| jeblair | clarkb, dkranz: I'm looking into current behavior of the experimental pipeline now | 17:14 |
| *** nati_ueno_2 has quit IRC | 17:14 | |
| openstackgerrit | Mathieu Gagné proposed a change to openstack-infra/jenkins-job-builder: Sort entry_points in setup.py for maintainability https://review.openstack.org/43924 | 17:14 |
| clarkb | jeblair: dkranz's change lgtm. Can I +2 it and let you +A if you are happy with the non voting behavior? | 17:14 |
| *** nati_ueno has joined #openstack-infra | 17:14 | |
| jeblair | clarkb: yes | 17:15 |
| jeblair | clarkb: it looks like it works, but i lose my nickel bet; it reports overall 'success' regardless of outcome. | 17:15 |
| jeblair | https://review.openstack.org/#/c/43791/ | 17:15 |
| jeblair | clarkb, dkranz, jog0: ^ | 17:16 |
| clarkb | because they are not voting. I see | 17:16 |
| dkranz | The plan is to make it voting asap but I don't see how it can avoid going through a non-voting prelim. | 17:16 |
| clarkb | dkranz: this is non voting in the experimental pipeline | 17:17 |
| clarkb | dkranz: independent of the process of making it gating. Notice that even when a job fails jenkins says the experimental jobs succeed because they are both non voting | 17:17 |
| dkranz | clarkb: For experimental, what does it mean to be voting? | 17:17 |
| jeblair | dkranz: it doesn't matter in the experimental pipeline; all things being equal, i'd like everything there to be voting; however, if the penultimate stage is non-voting in the check and gate pipelines, then, well, it's less typing to leave it as-is. :) | 17:17 |
| *** adalbas has joined #openstack-infra | 17:18 | |
| dkranz | jeblair: Yes, thank you :) | 17:18 |
| clarkb | dkranz: not much, you just end up with slightly confusing report comments when a test fails | 17:18 |
| clarkb | dkranz: but everything else works as expected | 17:18 |
| *** nati_ueno has quit IRC | 17:19 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Use an OrderedDict for pipelines https://review.openstack.org/43922 | 17:19 |
| openstackgerrit | A change was merged to openstack-infra/config: Add experimental slow heat job https://review.openstack.org/43891 | 17:19 |
| *** branen has quit IRC | 17:19 | |
| jeblair | dkranz: ^ | 17:19 |
| dkranz | jeblair: Thanks! | 17:20 |
| *** ruhe has quit IRC | 17:20 | |
| clarkb | jeblair: nodepool change reviewed. It would be nice to manage that ssh key on the nodepool host as well | 17:20 |
| pabelanger | So, to continue with my question from this morning. Now playing with cloudenvy for provisioning | 17:21 |
| jeblair | clarkb: pretty sure it is | 17:21 |
| jeblair | clarkb: it should be the jenkins private key | 17:21 |
| jog0 | you guys see this thread on the | 17:21 |
| jog0 | ML http://lists.openstack.org/pipermail/openstack-dev/2013-August/014298.html | 17:21 |
| clarkb | jeblair: I skimmed through the puppet manifests and didn't see it. I may have missed it though | 17:21 |
| jog0 | ^ about rechecks | 17:22 |
| uvirtbot | jog0: Error: "about" is not a valid command. | 17:22 |
| *** branen has joined #openstack-infra | 17:22 | |
| clarkb | jog0: yes, that is potentially one of the things elasticsearch would be useful for | 17:22 |
| ArxCruz | pleia2: first question, Tue Aug 27 17:22:19 +0000 2013 Puppet (warning): Found multiple default providers for vcsrepo: cvs, svn, git, hg, bzr; using cvs | 17:22 |
| ArxCruz | why is always using cvs instead of git ? | 17:23 |
| clarkb | jog0: but it is still in its infancy and I don't want to contradict jgriffith's argument because it is really good | 17:23 |
| jog0 | clarkb: thats what I thought, but figured it would be better for you guys to respond then me | 17:23 |
| jog0 | clarkb: good point | 17:23 |
| jog0 | clarkb: perhaps you could say +1 to his and say we are working on the second idea ... but its a ways off | 17:24 |
| openstackgerrit | Mathieu Gagné proposed a change to openstack-infra/jenkins-job-builder: Sort entry_points in setup.py for maintainability https://review.openstack.org/43924 | 17:24 |
| jog0 | clarkb: or not | 17:24 |
| openstackgerrit | will soula proposed a change to openstack-infra/jenkins-job-builder: Adding validation to the ansicolor plugin https://review.openstack.org/43926 | 17:24 |
| clarkb | jog0: ya I can do that | 17:24 |
| jeblair | clarkb, jog0: I also read the following: | 17:24 |
| jeblair | I'd love to see our CI system enhanced to do all of the pattern | 17:24 |
| jeblair | matching to group failures by common patterns, and then when a technical | 17:24 |
| jeblair | contributor looks at these groups they have tons of data points to _fix_ | 17:24 |
| jeblair | the problem rather than just spending their precious time identifying it. | 17:24 |
| jeblair | as SpamapS volunteering to help out with that work | 17:25 |
| jeblair | SpamapS: so, thanks! | 17:25 |
| jeblair | SpamapS: clarkb will be following up with all the info you need to get started! :) | 17:25 |
| pleia2 | ArxCruz: afaik that's just the way the vcsrepo works (we didn't write it), you can configure your manifest to use the one you want | 17:25 |
| *** pcm_ has quit IRC | 17:25 | |
| *** HenryG has quit IRC | 17:25 | |
| clarkb | jog0: I think the point that maybe hasn't been stressed enough is that we are in the home stretch of havana when we need to be diligent and people are not | 17:26 |
| *** pcm_ has joined #openstack-infra | 17:26 | |
| *** vipul-away is now known as vipul | 17:26 | |
| clarkb | jog0: so while having automagic tomorrow would be super duper this is a problem today with a known work around | 17:26 |
| jog0 | clarkb: ++ | 17:26 |
| jog0 | clarkb: I do think being able to clean out and correct the recheck list would make that easier. yesterday I found a bug in recheck that was marked as a duplicate in launchpad | 17:27 |
| jog0 | so I did recheck the original bug not the dup | 17:27 |
| clarkb | jog0: yeah there is definitely work that can be done on the recheck tracking side too | 17:28 |
| jog0 | but that is neither here nor there | 17:28 |
| clarkb | that script is fairly naive and simple | 17:28 |
| jeblair | clarkb: https://etherpad.openstack.org/havana-ci-logging in case you want to mention that | 17:28 |
| *** vogxn has quit IRC | 17:29 | |
| jog0 | quick question: if I am resubmitting a reverted patch with proper fixes, do I use teh same commit iD? | 17:29 |
| jog0 | or a new one? | 17:29 |
| *** pcm__ has joined #openstack-infra | 17:29 | |
| jog0 | I am thinking a new one? | 17:29 |
| jeblair | jog0: change id? new one. | 17:29 |
| jog0 | jeblair: thanks | 17:29 |
| jeblair | jog0: (at least part of the reason is that you can't upload a new patchset to a closed (merged or abandoned) change. | 17:30 |
| jog0 | jeblair: good to know | 17:30 |
| *** pcm_ has quit IRC | 17:30 | |
| *** dina_belova has quit IRC | 17:30 | |
| *** dina_belova has joined #openstack-infra | 17:31 | |
| *** nati_ueno has joined #openstack-infra | 17:31 | |
| jog0 | one more random question: if I wanted to add a test to make sure nova's compute rpcapi is backwards compat like it is supposed to be (done by running tempest tests against trunk everything but nova-compute which would be grizz) where should that live | 17:33 |
| jog0 | grenade or devstack? | 17:33 |
| jeblair | jog0: nova-compute is part of nova? | 17:34 |
| *** thomasbiege has joined #openstack-infra | 17:34 | |
| clarkb | My first thought is grenade but grenade is really worried about upgrades and not mixed installs | 17:34 |
| jeblair | jog0: so _part_ of nova is one release, part is another? | 17:35 |
| clarkb | maybe a special devstack tempest test that deploys the desired environment with devstack? | 17:35 |
| *** dina_belova has quit IRC | 17:35 | |
| jog0 | jeblair: correct on both. I was thinking putting nova-compute in a venv. to make sure we can run nova-compute (controls libvirt) as grizzly | 17:37 |
| jog0 | jeblair: so this seems like a not great fit for devstck either | 17:38 |
| SpamapS | jeblair: any time! :) | 17:39 |
| reed | jeblair, clarkb: thanks, saw the comments | 17:39 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Add gate-tempest-devstack-vm-large-ops job https://review.openstack.org/43781 | 17:40 |
| jeblair | jog0: i'd talk to dtroyer and see if it fits with grenade | 17:40 |
| jeblair | clarkb: the re-run jobs change to zuul seems to be working | 17:40 |
| jeblair | clarkb: i found this example, which isn't great because the buildset was eventually canceled, but before that happened i think it worked as expected | 17:41 |
| jeblair | https://jenkins01.openstack.org/job/gate-grenade-devstack-vm/4151/console | 17:41 |
| jog0 | jeblair: cool | 17:41 |
| SpamapS | jog0: eventually I think tripleo's CI will handle this case btw | 17:42 |
| jog0 | SpamapS: even running last stable release nova-compute vs trunk rest? | 17:42 |
| SpamapS | jog0: thats just a different tag for the nova-compute element b[D[Duild. :) | 17:42 |
| jog0 | SpamapS: the triplo setup will make this much easier, right | 17:43 |
| jeblair | clarkb: follow the link to the description of the build, then follow one of the other links to another job-build eg https://jenkins02.openstack.org/job/gate-tempest-devstack-vm-testr-full/4280/ and then note the link to a different (newer) grenade test | 17:43 |
| SpamapS | jog0: though anything "stable release" is somewhat far down on the "tripleo cares about it" list. | 17:43 |
| jog0 | SpamapS: actually the fun test would be trunk everything and a second nova-compute on latest stable | 17:43 |
| jog0 | SpamapS: yeah but some folks in OpenStack care, and we don't test taht yet | 17:44 |
| *** dina_belova has joined #openstack-infra | 17:45 | |
| jog0 | SpamapS: but yeah Triplo will help this big time. perhaps the right answer for now is just test using upgrade_levels.compute=grizzly | 17:45 |
| jog0 | SpamapS: how far off do you think Triplo CI is? <6 months? | 17:45 |
| clarkb | jeblair: cool | 17:46 |
| *** branen has quit IRC | 17:46 | |
| *** branen has joined #openstack-infra | 17:47 | |
| clarkb | jog0: SpamapS jeblair response sent. | 17:47 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Use an OrderedDict for pipelines https://review.openstack.org/43922 | 17:48 |
| *** thomasbiege has quit IRC | 17:48 | |
| jog0 | clarkb: now that mroe people now about logstash it should be fun to see if it crashes | 17:51 |
| clarkb | jog0: hopefulyl it doesn't :) it was very crash happy in the past but it seems to be doing much better now that we index less data | 17:52 |
| clarkb | jog0: will be interesting to see. | 17:52 |
| clarkb | mtreinish: how is the tempest testr stuff doing? | 17:52 |
| mtreinish | clarkb: well we've got 2 big flaky fails that people seem to be hitting right now but no one has gone out of their way to yell at me | 17:53 |
| mtreinish | so I count it as a win :) | 17:53 |
| clarkb | hurray | 17:53 |
| *** fbo_away is now known as fbo | 17:54 | |
| mtreinish | https://bugs.launchpad.net/python-keystoneclient/+bug/1217159 and https://bugs.launchpad.net/tempest/+bug/1217163 in case anyone is interested | 17:54 |
| uvirtbot | Launchpad bug 1217159 in tempest "tempest SimpleReadOnlyKeystoneClientTest fails" [Critical,Confirmed] | 17:54 |
| jeblair | clarkb: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/manifests/nodepool.pp#n26 | 17:54 |
| *** sarob has joined #openstack-infra | 17:54 | |
| clarkb | jeblair: thanks. I was looking for use of the jenkins user manifest in the jenkins module | 17:55 |
| dkranz | clarkb: My first run of slow heat in experimental failed with LOST https://review.openstack.org/#/c/43927/ | 17:55 |
| dkranz | clarkb: Should I retry or is some other action needed? | 17:55 |
| clarkb | dkranz: I think you were faster than JJB. So the job hadn't been created in jenkins yet | 17:55 |
| *** branen_ has joined #openstack-infra | 17:56 | |
| clarkb | dkranz: if you go to https://jenkins01.openstack.org and search for the job do you see it? | 17:56 |
| clarkb | dkranz: if you do then go ahead and give it another try | 17:56 |
| *** ^d is now known as ^demon|away | 17:56 | |
| *** CaptTofu_ has quit IRC | 17:57 | |
| clarkb | jeblair: review on nodepool ssh key change updated | 17:57 |
| dkranz | clarkb: Don't see it. | 17:57 |
| clarkb | dkranz: the change merged long enough ago that it should have been applied. I will investigate | 17:58 |
| dkranz | clarkb: Thanks | 17:58 |
| *** branen has quit IRC | 17:58 | |
| *** emagana_ has joined #openstack-infra | 17:58 | |
| clarkb | dkranz: https://jenkins02.openstack.org/job/gate-tempest-devstack-vm-heat-slow/ you can change jenkins02 to jenkins01 to see it on the other server | 17:59 |
| clarkb | dkranz: let me check one more thing then you can try again | 18:00 |
| *** SergeyLukjanov has joined #openstack-infra | 18:00 | |
| clarkb | dkranz: I think it will work if run this time. Go ahead and try again | 18:00 |
| dkranz | clarkb: OK | 18:00 |
| *** markmcclain has joined #openstack-infra | 18:01 | |
| *** emagana has quit IRC | 18:01 | |
| SpamapS | jog0: I hope that we can have tripleo deploying and updating a whole cloud automatically/efficiently in 6 months yes. | 18:01 |
| SpamapS | jog0: but I won't make that promise :) | 18:02 |
| openstackgerrit | A change was merged to openstack-infra/config: Add SSH user/key info to nodepool config https://review.openstack.org/43920 | 18:02 |
| jog0 | SpamapS: thats good enough for me, I will focus on the easier test for now then | 18:06 |
| jog0 | thanks | 18:06 |
| mriedem | can anyone tell me why this hasn't run since 8/6? https://jenkins.openstack.org/view/All/job/nova-coverage/ | 18:06 |
| clarkb | mriedem: probably because we have three jenkins masters now, if you look on jenkins01 and jenkins02 you should see job runs | 18:07 |
| mriedem | clarkb: oh yeah, octojenkins | 18:07 |
| mriedem | checking | 18:07 |
| jeblair | clarkb: nodepool is running master and puppet is running on nodepool | 18:08 |
| clarkb | jeblair: awesome | 18:08 |
| *** ^demon|away is now known as ^d | 18:09 | |
| clarkb | jeblair: the two zuul changes you mentioned lgtm | 18:10 |
| mriedem | clarkb: so the coverage jobs must only keep a build history of a couple of days? | 18:11 |
| mriedem | clarkb: trying to find something like this but for another commit: http://logs.openstack.org/5c/5cc63c91d045f7a37136107053f71db1d8edf425/post/nova-coverage/e91683d/cover/ | 18:11 |
| clarkb | mriedem: you want http://logs.openstack.org/gitsha[:2]/gitsha/post/nova-coverage | 18:12 |
| clarkb | mriedem: make sure you use the gitsha that ended up being merged as that may be different than what you have locally | 18:13 |
| mriedem | clarkb: so is that only for merged commits? | 18:13 |
| clarkb | mriedem: yes, nova-coverage runs in the post queue which happens after merges | 18:14 |
| mriedem | clarkb: ah, ok. in #openstack-nova we were talking about how nice it would be if we had coverage reports for patches so we could know on a giant blueprint patch how much is actually covered in the tests, | 18:14 |
| mriedem | sort of going on about long review times in nova and what slows it down, one of which is giant patches | 18:15 |
| mriedem | and you have to dig into the tests to compare to the new code to make sure everything is reasonably covered | 18:15 |
| clarkb | mriedem: it would be nice :). The reason we don't run those tests prior to merging is they take forever to run and I think people want to see trends | 18:15 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Require gear 0.4.0 https://review.openstack.org/43912 | 18:15 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Use the 'resolve' merge strategy by default https://review.openstack.org/43629 | 18:15 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Fetch specific refs on ref-updated events https://review.openstack.org/43775 | 18:15 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Don't reset the local repo https://review.openstack.org/43792 | 18:15 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Fix graceful exit https://review.openstack.org/43803 | 18:15 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Report item stats when removing a change https://review.openstack.org/43812 | 18:15 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Set merge result correctly on exceptions https://review.openstack.org/43815 | 18:15 |
| clarkb | mriedem: we have fiddled with the idea of running them in the check queue instead for the reason you want them | 18:16 |
| mriedem | clarkb: ah, ok - cuz i was thinking just getting the coverage report for a patch set should be trivial | 18:16 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Use an OrderedDict for pipelines https://review.openstack.org/43922 | 18:16 |
| mriedem | clarkb: yup | 18:16 |
| clarkb | mriedem: it is trivial `tox -ecover` | 18:16 |
| mriedem | i didn't realize it took a long time | 18:16 |
| clarkb | mriedem: but instrumenting the code makes it much slower | 18:16 |
| jeblair | clarkb: are those run in parallel or serially? | 18:16 |
| clarkb | mriedem: you could `git review -d $CHANGE && tox -ecover` to generate it locally | 18:16 |
| mriedem | clarkb: ok, got it - thanks, glad you guys already looked into it (though i'm not surprised :) ) | 18:16 |
| clarkb | jeblair: parallel | 18:16 |
| mriedem | clarkb: yeah, generating locally for some guy that i don't know or care what his blueprint does isn't going to probably happen :) | 18:17 |
| mriedem | just brainstorming how to make reviews more efficient in nova | 18:17 |
| clarkb | jeblair: now that you mention it it isn't too slow with testr | 18:17 |
| jeblair | maybe worth a look; maybe after h3 | 18:18 |
| clarkb | jeblair: ++ | 18:18 |
| clarkb | mriedem: with testr we run the job with parallel test runners like the normal unittests do which makes the job quite a bit faster | 18:18 |
| jeblair | clarkb: what would be really nice is a return message from gearman saying "coverage: 98%" :) | 18:18 |
| clarkb | mriedem: we can look at shifting when that job is run after H3 | 18:18 |
| mriedem | clarkb: jeblair: ok, yeah, sounds great | 18:19 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Add a graphite key for all jobs in a pipeline https://review.openstack.org/43586 | 18:19 |
| clarkb | jeblair: or maybe abuse the experimental queue some more :) | 18:19 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Report pipeline queue length on startup to statsd https://review.openstack.org/43588 | 18:19 |
| jeblair | clarkb: i own 0 open changes in gerrit! | 18:20 |
| clarkb | wow | 18:20 |
| clarkb | I only have WIP and draft changes | 18:20 |
| jeblair | clarkb: i will write some more soon. :) | 18:20 |
| clarkb | after the meeting I hope to get some code review in and work on mysql backups a little more | 18:20 |
| clarkb | I think I am ready to start mysqldumping on review and review-dev | 18:21 |
| jeblair | clarkb: great! | 18:21 |
| clarkb | then will follow that up with a bup change (if necessary) | 18:21 |
| *** melwitt has joined #openstack-infra | 18:22 | |
| clarkb | going to find food before the meeting today back by noon | 18:22 |
| *** bodepd has quit IRC | 18:25 | |
| mtreinish | jeblair: do you guys have a raw recheck count somewhere? (probably graphite right?) | 18:25 |
| *** bodepd has joined #openstack-infra | 18:25 | |
| mtreinish | I'm just curious how much it's spiked since last night | 18:25 |
| jeblair | mtreinish: i don't think so | 18:26 |
| mtreinish | jeblair: ok | 18:27 |
| openstackgerrit | Khai Do proposed a change to openstack-infra/gear: Update gear docs to include gearman server daemon https://review.openstack.org/43780 | 18:27 |
| *** dina_belova has quit IRC | 18:27 | |
| * mtreinish starts working on a script to watch recheck count from gerrit stream | 18:28 | |
| jeblair | mtreinish: you can start with the recheckwatch script since that's what it does | 18:28 |
| *** dina_belova has joined #openstack-infra | 18:28 | |
| mtreinish | jeblair: ok cool | 18:28 |
| *** dina_belova has quit IRC | 18:32 | |
| *** thomasbiege has joined #openstack-infra | 18:32 | |
| *** anteaya has quit IRC | 18:33 | |
| clarkb | jeblair we could grep zuul logs or query gerrit db possibly | 18:35 |
| clarkb | (to get a number now) | 18:36 |
| *** thomasbiege has quit IRC | 18:37 | |
| *** ruhe has joined #openstack-infra | 18:39 | |
| jgriffith | jeblair: speaking of parallel testr etc | 18:39 |
| *** ruhe has quit IRC | 18:39 | |
| jgriffith | jeblair: what's up with the times reported by zuul now? | 18:39 |
| jgriffith | jeblair: are we getting faster just by reporting the wrong elapsed times? | 18:39 |
| jgriffith | jeblair: the countdown is really nice, don't get me wrong | 18:40 |
| jeblair | jgriffith: that sounds like a great strategy, but i don't think i'm following your question. :) | 18:40 |
| jeblair | jgriffith: (scotty: how else can i keep my reputation as a miracle worker?) | 18:40 |
| jgriffith | jeblair: :) | 18:41 |
| jgriffith | jeblair: so I'm looking again today (noticed this yesterday after the email went out that this was on) | 18:41 |
| jgriffith | jeblair: it's actually much better it seems | 18:41 |
| jeblair | jgriffith: yes, a tempest run is expected to take 20-30 minutes now | 18:41 |
| jgriffith | jeblair: what i noticed last night was like: on zuul, test kids off and the countdown says like 7 minutes | 18:41 |
| jgriffith | jeblair: right.. which is SWEET! | 18:42 |
| jeblair | jgriffith: oh, you're saying the "X mins" countdown for the whole change is reporting a value that's too low? | 18:42 |
| jgriffith | jeblair: did we always do a time remaining on the status page or is that new? | 18:42 |
| jgriffith | jeblair: correct | 18:42 |
| jgriffith | jeblair: but it seems MUCH more accurate today | 18:43 |
| jeblair | jgriffith: maybe a month old? | 18:43 |
| jgriffith | jeblair: so much for that excuse :) | 18:43 |
| jeblair | jgriffith: it's entirely possible it's wrong; i'll keep an eye on it | 18:43 |
| jgriffith | jeblair: maybe my math was just off... or my eyes were so blurry I was looking at the wrong jobs | 18:43 |
| mtreinish | jgriffith: I noticed it too. Last night when I pushed through the parallel change it was way off. | 18:43 |
| mtreinish | I think it's better now | 18:43 |
| jgriffith | mtreinish: phewwwww | 18:44 |
| jgriffith | mtreinish: I thought I was making myself look REALLY stupid | 18:44 |
| jeblair | jgriffith: it should be, roughly, max([all running jobs for this change] + time_for_change_ahead) | 18:44 |
| jgriffith | mtreinish: you're right, it is wayyyyy better today | 18:44 |
| jgriffith | jeblair: cool... sorry for the false alarm :) | 18:44 |
| jgriffith | jeblair: the new changes are pretty neat | 18:45 |
| jeblair | jgriffith: well, it may just be less wrong because the numbers are closer together. i'll keep an eye on it. | 18:45 |
| jgriffith | jeblair: cool, and if I"m completely wrong let's just say it was mtreinish that brought it up :) | 18:45 |
| *** alexpilotti has quit IRC | 18:46 | |
| mtreinish | jgriffith: well the thing I actually noticed last night was the little bars actually. The job would finish when it was at about 50%. | 18:46 |
| mtreinish | so you're on your own for the number | 18:47 |
| jgriffith | mtreinish: :) | 18:47 |
| jgriffith | mtreinish: actually we were seeing the same thing I believe, I was messing with it right after your email | 18:47 |
| jgriffith | mtreinish: nice job by the way! | 18:47 |
| mtreinish | jgriffith: thanks! I'm actually surprised that no one has tried to kill me yet because of the recheck rate going up... | 18:48 |
| jgriffith | mtreinish: nahh... | 18:49 |
| jeblair | mtreinish: oh, yeah, the bars being wrong would be due to the runtime of the existing job changing | 18:49 |
| jgriffith | mtreinish: IMO I haven't noticed anything signficant except for one test | 18:50 |
| jeblair | jenkins will adapt (i think it has) | 18:50 |
| jgriffith | mtreinish: which now has a patch submitted | 18:50 |
| jeblair | jgriffith: if the bars were off, then the overall time would be off too | 18:50 |
| mtreinish | jgriffith: the aggregates one? | 18:50 |
| jgriffith | jeblair: Ohhhh... thats interesting | 18:50 |
| jgriffith | mtreinish: yes | 18:50 |
| jeblair | so maybe that's what it was | 18:50 |
| mtreinish | yeah I pushed that patch I'm actually not sure that it's a legit fix. It was just my best guess based on looking at the error | 18:50 |
| jgriffith | mtreinish: yeah but it failed :( | 18:51 |
| mtreinish | I'm waiting for vishy to get back to me about it | 18:51 |
| mtreinish | jgriffith: pep8 :( | 18:51 |
| *** rnirmal has quit IRC | 18:51 | |
| jgriffith | indeed.. and pylint | 18:51 |
| jgriffith | silly things | 18:51 |
| jgriffith | mtreinish: I almost hit "recheck no bug" just to be a smart allec | 18:52 |
| jgriffith | mtreinish: but I don't think people would appreciate the humor | 18:52 |
| *** anteaya has joined #openstack-infra | 18:53 | |
| mtreinish | jgriffith: hahaha. Although the pylint error actually shows how it doesn't work. I missed a class name... | 18:53 |
| jgriffith | mtreinish: just saw that | 18:53 |
| jgriffith | mtreinish: kinda cool eh | 18:53 |
| *** svarnau has joined #openstack-infra | 18:54 | |
| mtreinish | jgriffith: yeah, it just teaches me that I should try to remember to run things before I push it out for review. | 18:57 |
| *** odyssey4me has quit IRC | 19:00 | |
| *** vipul is now known as vipul-away | 19:01 | |
| *** reed has quit IRC | 19:02 | |
| jeblair | meeting time! | 19:02 |
| *** HenryG has joined #openstack-infra | 19:11 | |
| *** sarob has quit IRC | 19:11 | |
| *** hashar has joined #openstack-infra | 19:16 | |
| *** mberwanger has joined #openstack-infra | 19:21 | |
| lifeless | mtreinish: hi? | 19:22 |
| mtreinish | lifeless: hi | 19:23 |
| lifeless | pabelanger: ah, thanks! | 19:24 |
| *** dina_belova has joined #openstack-infra | 19:28 | |
| *** lcestari_ has quit IRC | 19:31 | |
| *** lcestari has joined #openstack-infra | 19:31 | |
| *** dina_belova has quit IRC | 19:33 | |
| *** markmcclain has quit IRC | 19:34 | |
| *** markmc has joined #openstack-infra | 19:35 | |
| *** lcestari has quit IRC | 19:36 | |
| *** lcestari has joined #openstack-infra | 19:36 | |
| *** dina_belova has joined #openstack-infra | 19:39 | |
| *** lcestari has quit IRC | 19:39 | |
| *** mberwanger has quit IRC | 19:41 | |
| annegentle | oh. missed open discussion. -- was going to ask about moving clouddocs-maven-plugin to openstack,how's the pre-work? | 19:41 |
| clarkb | annegentle: mordred jumped the gone and made the switch | 19:42 |
| clarkb | s/gone/gun/ | 19:42 |
| clarkb | then we got really busy and I forgot to say anything ... | 19:42 |
| annegentle | heh | 19:42 |
| annegentle | so that patch merging wasn't a prereq completion | 19:42 |
| * annegentle talks really vaguely when unsure of real words | 19:42 | |
| clarkb | annegentle: http://git.openstack.org/cgit/stackforge/clouddocs-maven-plugin/ | 19:42 |
| *** gyee has quit IRC | 19:43 | |
| openstackgerrit | Ryan Petrello proposed a change to openstack-infra/config: Provide a more generic run-tox.sh. https://review.openstack.org/43145 | 19:43 |
| annegentle | clarkb: so. | 19:43 |
| clarkb | annegentle: I personally would've liked there to be a little more communication particularly with David but I didn't mark the change WIP so my bad | 19:43 |
| *** dina_belova has quit IRC | 19:43 | |
| annegentle | clarkb: if David's still producing releases not on stackforge (pretty sure he did this last week) what's the next step? | 19:43 |
| clarkb | annegentle: we will need to force push those changes into gerrit | 19:43 |
| annegentle | clarkb: ok | 19:44 |
| clarkb | annegentle: and we should do that when david is ready to switch over to using gerrit | 19:44 |
| clarkb | so that we only need to do it once | 19:44 |
| clarkb | I did send david email with a link to the change but didn't get any feedback from him | 19:44 |
| annegentle | clarkb: ok | 19:44 |
| annegentle | clarkb: IMing with David now | 19:44 |
| anteaya | just as an fyi, I requested an upgrade to my internet speed and the telecom monopoly that owns the lines downgraded me instead - I am pushing for a fix but if someone flips the wrong switch I might end up disconnected. If it happens that is why. | 19:44 |
| clarkb | so next step is getting david involved so that we can plan getting gerrit up to date and getting maven publishing jobs going for it | 19:45 |
| annegentle | clarkb: ok, so the gerrit updates are still pending? and with FF coming it might be a while? or? | 19:45 |
| clarkb | annegentle: the force push latest clouddocs into gerrit is pending. I only want to do that once | 19:46 |
| annegentle | clarkb: yeah makes sense | 19:46 |
| clarkb | annegentle: and that isn't affected by FF | 19:46 |
| clarkb | other than I may be busy if things fall over again | 19:46 |
| annegentle | clarkb: right | 19:46 |
| annegentle | clarkb: understandable | 19:47 |
| clarkb | anteaya: that wasn't very nice of them | 19:47 |
| anteaya | clarkb: I'll keep my response to arrrrrrrrggggghhhh | 19:47 |
| pleia2 | anteaya: ugh, that's awful | 19:48 |
| anteaya | clarkb: long boring story that hopefully has a happy ending soon | 19:48 |
| anteaya | pleia2: yeah, my ISP is stumped too | 19:48 |
| openstackgerrit | Ryan Petrello proposed a change to openstack-infra/config: Provide a more generic run-tox.sh. https://review.openstack.org/43145 | 19:49 |
| *** CaptTofu has joined #openstack-infra | 19:50 | |
| openstackgerrit | Ryan Petrello proposed a change to openstack-infra/config: Provide a more generic run-tox.sh. https://review.openstack.org/43145 | 19:50 |
| *** johnthetubaguy1 has joined #openstack-infra | 19:53 | |
| *** johnthetubaguy has quit IRC | 19:54 | |
| *** weshay has quit IRC | 19:57 | |
| *** weshay has joined #openstack-infra | 19:57 | |
| *** dolphm has left #openstack-infra | 19:59 | |
| jeblair | i'm going to restart zuul right after the next gate reset | 19:59 |
| jeblair | mtreinish: you should ignore "recheck" comments from me, btw. :) | 20:00 |
| clarkb | jeblair: ok | 20:00 |
| jeblair | mtreinish: in your stats | 20:00 |
| openstackgerrit | lifeless proposed a change to openstack-infra/reviewstats: Update TripleO projects. https://review.openstack.org/42511 | 20:00 |
| lifeless | russellb: ^ tada | 20:01 |
| mtreinish | jeblair: ok, yeah that's probably a good idea. | 20:01 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Update TripleO projects. https://review.openstack.org/42511 | 20:01 |
| openstackgerrit | A change was merged to openstack/requirements: Allow use of hp3parclient 2.0 https://review.openstack.org/42991 | 20:02 |
| *** jhesketh has quit IRC | 20:02 | |
| russellb | lifeless: tada ^ | 20:02 |
| jeblair | jgriffith: E: cinder.conf.sample is not up to date, please run tools/conf/generate_sample.sh | 20:02 |
| lifeless | \o/ | 20:02 |
| jeblair | jgriffith: there are a lot of cinder changes in the gate, and that's showing up | 20:02 |
| russellb | lifeless: just kicked it to re-run. it takes a bit though | 20:02 |
| clarkb | jeblair: I keep meaning to figure out what that hacking check is supposed to do and how to fix it when it breaks | 20:02 |
| clarkb | jog0: ^ are you in the know? | 20:03 |
| openstackgerrit | Emilien Macchi proposed a change to openstack-infra/jenkins-job-builder: Add SLOCCount plugin support https://review.openstack.org/43852 | 20:03 |
| clarkb | it might not even be run by hacking but it trips people up all the time | 20:03 |
| jgriffith | jeblair: I was afraid of that | 20:03 |
| jgriffith | jeblair: I just turned on auto-checking of conf updates | 20:03 |
| *** dprince has quit IRC | 20:03 | |
| jeblair | restarting zuul now | 20:03 |
| *** jfriedly has quit IRC | 20:03 | |
| jgriffith | jeblair: I suspect I should turn it off and let everything merge then turn it back on | 20:03 |
| *** morganfainberg is now known as morganfainberg|a | 20:03 | |
| *** ianw has quit IRC | 20:04 | |
| *** vipul-away is now known as vipul | 20:04 | |
| *** pcm__ has quit IRC | 20:04 | |
| *** ianw has joined #openstack-infra | 20:05 | |
| *** yjiang5 has joined #openstack-infra | 20:06 | |
| jeblair | stopped zuul again | 20:07 |
| jeblair | because it did not install | 20:07 |
| jeblair | because of pbr | 20:07 |
| clarkb | jeblair: :/ want an extra set of eyes? | 20:07 |
| jeblair | clarkb: sure | 20:07 |
| jeblair | pkg_resources.VersionConflict: (pbr 0.5.18 (/usr/local/lib/python2.7/dist-packages), Requirement.parse('pbr>=0.5.21,<1.0')) | 20:07 |
| *** nati_ueno has quit IRC | 20:08 | |
| jeblair | clarkb: should i pip -U install pbr ? | 20:08 |
| clarkb | jeblair: oh, manually upgrade pbr with pip first | 20:08 |
| clarkb | jeblair: yes | 20:08 |
| clarkb | jeblair: I believe that issue is caused by pbr being a setup_requires and how that is treated specially | 20:08 |
| *** nati_ueno has joined #openstack-infra | 20:08 | |
| jeblair | clarkb: oh, didn't read far enough back, i think this is caused by gear | 20:09 |
| jeblair | http://paste.openstack.org/ | 20:09 |
| jeblair | gcc | 20:09 |
| jeblair | grr | 20:09 |
| jeblair | http://paste.openstack.org/show/45236/ | 20:09 |
| clarkb | jeblair: I think the fix is to manually update pbr first | 20:09 |
| jeblair | root@zuul:/opt/zuul# pip | 20:09 |
| jeblair | -su: /usr/bin/pip: No such file or directory | 20:09 |
| * jeblair stabs mordred in the eyes with pbr | 20:10 | |
| jeblair | #status alert zuul is offline because of a pbr-related installation issue | 20:10 |
| openstackstatus | NOTICE: zuul is offline because of a pbr-related installation issue | 20:10 |
| *** ChanServ changes topic to "zuul is offline because of a pbr-related installation issue" | 20:10 | |
| clarkb | jeblair: I can find pip on zuul.o.o | 20:10 |
| *** HenryG has quit IRC | 20:10 | |
| clarkb | /usr/local/bin/pip | 20:10 |
| clarkb | I am guessing root doesn't have that in its path | 20:11 |
| jeblair | clarkb: i think it changed | 20:11 |
| *** morganfainberg|a is now known as morganfainberg | 20:12 | |
| jeblair | clarkb: ok, i think it's good now | 20:12 |
| *** hashar has quit IRC | 20:12 | |
| *** nati_ueno has quit IRC | 20:12 | |
| clarkb | jeblair: was the fix pip install -U pbr? | 20:13 |
| jeblair | clarkb: several times | 20:13 |
| jeblair | clarkb: and the pip install -U gear | 20:14 |
| jeblair | now i'm removing old versions of zuul | 20:14 |
| jog0 | clarkb: pong | 20:14 |
| jeblair | because pip freeze wasn't reporting the correct value | 20:14 |
| clarkb | jog0: the .conf check that runs in the pep8 jobs | 20:14 |
| jeblair | starting zuul | 20:14 |
| *** emagana_ has quit IRC | 20:15 | |
| clarkb | jog0: are you familiar with the problem that solves and how to fix failures? | 20:15 |
| jog0 | ohh I don't like that one | 20:15 |
| clarkb | jog0: it seems to happen fairly regularly and devs swear they are rebased on latest versions of the code but it still fails | 20:15 |
| jog0 | clarkb: that is to keep the nova.conf file in sync (and its not part of hacking FWIW). | 20:15 |
| jog0 | I swear at it too | 20:15 |
| clarkb | and I don't really know what to tell people other than to follow the directions in teh failure | 20:15 |
| *** rcleere has quit IRC | 20:15 | |
| jog0 | clarkb: for some reason it gets funny during rebases and stuff. | 20:16 |
| jog0 | to fix I usually just manually update the file | 20:16 |
| clarkb | which are minimal. So I thought I would see if you knew anything about it | 20:16 |
| jog0 | which is not the right way | 20:16 |
| annegentle | jog0: man I've got a patch where I want to update the nova.conf.sample too and it's just not working | 20:16 |
| *** hashar has joined #openstack-infra | 20:16 | |
| annegentle | jog0: at least I'm not alone | 20:16 |
| jog0 | annegentle: yeah I saw yours | 20:16 |
| jog0 | annegentle: whats the patch for yours again | 20:16 |
| annegentle | jog0: so what am I doing wrong? https://review.openstack.org/#/c/40271/5 | 20:17 |
| jog0 | you need to run the build new conf file code inside the venv | 20:17 |
| jog0 | .tox/py27/bin/activate | 20:17 |
| jog0 | source ^ | 20:18 |
| jog0 | or something like that | 20:18 |
| jog0 | but what I do, sometimes is just manaully update teh file until 'tox -epep8' shuts up | 20:18 |
| *** thomasm has joined #openstack-infra | 20:18 | |
| yjiang5 | annegentle: I didn't use venv to create the confif sample, simply "./tools/config/generate_sample.sh -b . -p nova -o etc/nova" | 20:19 |
| *** markmcclain has joined #openstack-infra | 20:19 | |
| anteaya | I was just getting used to experimental being in the middle but it makes more sense where it is now | 20:20 |
| clarkb | anteaya: that is a result of one of the many bugfixes jeblair added to zuul | 20:21 |
| annegentle | yjiang5: ok, that's good to know. That's what I did for the patch but it's not passing pep8 | 20:21 |
| anteaya | clarkb: yes | 20:21 |
| anteaya | makes sense | 20:21 |
| annegentle | jog0: ok. heh. Hmmm. | 20:21 |
| *** morganfainberg is now known as morganfainberg|a | 20:21 | |
| *** rnirmal has joined #openstack-infra | 20:21 | |
| jog0 | annegentle: 'git checkout HEAD^ etc/nova/nova.conf.sample' and fix by hand | 20:22 |
| jog0 | is the lazy wrong way | 20:22 |
| *** morganfainberg|a is now known as morganfainberg | 20:22 | |
| annegentle | jog0: LOL | 20:22 |
| clarkb | jog0: this discussion makes me think there is at least one major bug in that test :) | 20:22 |
| jog0 | clarkb: its the import module issue again | 20:22 |
| yjiang5 | annegentle: but your patch seems really strange with so many removed lines. | 20:23 |
| clarkb | jog0: where it has to actually import stuff? | 20:23 |
| jog0 | clarkb: if a pre-req is missing you cannot import it right now | 20:23 |
| clarkb | gotcha | 20:23 |
| jog0 | clarkb: I am just making things up right now | 20:23 |
| annegentle | yjiang5: yeah I don't know how that happens unless I need the venv? | 20:23 |
| jog0 | and hoping I am right | 20:23 |
| clarkb | :) | 20:23 |
| yjiang5 | annegentle: I cherry-pick your patch and regenerate the conf.sample and seems my result is correct, strange. | 20:25 |
| jog0 | clarkb: openstack.common.config.generator:82 | 20:26 |
| jog0 | and trace from there | 20:26 |
| jog0 | yjiang5: I bet you have more deps installed then annegentle on your box | 20:26 |
| jog0 | clarkb: yeah we try __import__ | 20:26 |
| yjiang5 | jog0: possibly that's the reason. | 20:26 |
| *** thomasbiege has joined #openstack-infra | 20:27 | |
| *** dina_belova has joined #openstack-infra | 20:28 | |
| jog0 | clarkb: if something exceptions out during an import we cannot generate the conf for that file | 20:28 |
| openstackgerrit | Emilien Macchi proposed a change to openstack-infra/jenkins-job-builder: Add Plot plugin support https://review.openstack.org/43685 | 20:29 |
| annegentle | jog0: yeah this is a rackspace cloud server running ubuntu 12.04 lts | 20:29 |
| annegentle | yjiang5: can you help me out and patch that patch ? :) I do want to figure out why my environment is bad, but. | 20:30 |
| jeblair | clarkb: i'm spot checking that zuul is creating correct patchsets | 20:30 |
| clarkb | jeblair: ok. This is to double check the new merger is working as expected? | 20:30 |
| jeblair | so far it looks good, though i noticed something that suggests we need more work on the zuul merge check -- | 20:30 |
| clarkb | I have just starred a bunch of changes I intend on reviewing | 20:30 |
| jeblair | clarkb: https://review.openstack.org/#/c/39890/ | 20:30 |
| dkranz | clarkb: I had a strange infrastructure failure on slow heat. I did another 'check experimental' almost two hours ago and nothing seems to have happened. | 20:30 |
| yjiang5 | annegentle: you mean I update the patch for you? | 20:31 |
| jog0 | annegentle: run 'source .tox/py27/bin/activate' and try again | 20:31 |
| dkranz | clarkb: I can't see anything in the zuul queue or jenkins https://review.openstack.org/#/c/43927/ | 20:31 |
| anteaya | dkranz: we did just restart zuul | 20:31 |
| jeblair | clarkb: ps27 of that change has a parent that is not a gerrit change, but zuul doesn't care. presumably gerrit will refuse to merge it. | 20:31 |
| clarkb | dkranz: 'check experimental' not 'recheck experimental' | 20:31 |
| dkranz | clarkb: Arghhhhh | 20:32 |
| clarkb | jeblair: yeah we ran into that with SpamapS in the past iirc | 20:32 |
| clarkb | jeblair: gerrit gets really cranky with that case | 20:32 |
| jeblair | clarkb: we'll want to take care of that to (a) make zuul kick that change out and (b) if zuul performs merges in the future, reject that case | 20:32 |
| SpamapS | clarkb: I'd say that SpamapS ran into "we" in the past.. but 6 -- 1/2 dozen.. ;) | 20:33 |
| dkranz | clarkb: Sorry about that. It's in zuul now. | 20:33 |
| clarkb | dkranz: no problem | 20:33 |
| clarkb | SpamapS: we should just put a crash test helmet on you and let you loose | 20:34 |
| openstackgerrit | Emilien Macchi proposed a change to openstack-infra/jenkins-job-builder: Add SLOCCount plugin support https://review.openstack.org/43852 | 20:34 |
| openstackgerrit | A change was merged to openstack-infra/config: Reduce state-management channel noise https://review.openstack.org/43613 | 20:34 |
| anteaya | jeblair: channel status update? | 20:35 |
| jeblair | #status ok Zuul has been restarted | 20:35 |
| openstackstatus | NOTICE: Zuul has been restarted | 20:35 |
| *** ChanServ changes topic to "Discussion of OpenStack Project Infrastructure | Docs http://ci.openstack.org/ | Bugs https://launchpad.net/openstack-ci | Code https://git.openstack.org/cgit/openstack-infra/" | 20:35 | |
| jeblair | anteaya: thx | 20:35 |
| anteaya | np | 20:35 |
| clarkb | jd__: still around? https://review.openstack.org/#/c/42692/6/modules/jenkins/manifests/slave.pp I think you also need to install the pypy package in your if include_pypy conditional | 20:35 |
| jd__ | clarkb: not guily for this one, I think you meant Alex_Gaynor? | 20:36 |
| clarkb | jd__: maybe | 20:36 |
| Alex_Gaynor | clarkb: yeah that PR is me, did I puppet badly? | 20:36 |
| clarkb | jd__: yes I meant Alex_Gaynor not sure why I thought that was your chnange. sorry for the noise | 20:36 |
| jd__ | but it's cool that Alex_Gaynor is doing this since I was thinking about doing so, starting with pbr | 20:36 |
| *** rnirmal has quit IRC | 20:36 | |
| clarkb | Alex_Gaynor: I think that file needs to also install pypy | 20:37 |
| clarkb | Alex_Gaynor: I will leave a comment with what it should look like | 20:37 |
| Alex_Gaynor | clarkb: So I'm bad at puppet, does apt::ppa not install? ok thanks! | 20:37 |
| SpamapS | clarkb: we did the let me lose part, its called "working on Heat and TripleO", but the crash test helmet.. crap.. that could have come in handy a few times ;) | 20:37 |
| *** jfriedly has joined #openstack-infra | 20:38 | |
| *** mberwanger has joined #openstack-infra | 20:38 | |
| clarkb | Alex_Gaynor: comment left | 20:39 |
| Alex_Gaynor | clarkb: thanks | 20:39 |
| clarkb | jeblair: I am approving https://review.openstack.org/#/c/42694/2 should only touch review-dev | 20:41 |
| clarkb | jeblair: nevermind just found an issue with it | 20:42 |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Add support for running tox with PyPy and use it for Marconi https://review.openstack.org/42692 | 20:42 |
| *** gyee has joined #openstack-infra | 20:42 | |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Add support for running tox with PyPy and use it for Marconi https://review.openstack.org/42692 | 20:44 |
| Alex_Gaynor | clarkb: I failed at aligning some columns, new patch updated :) | 20:44 |
| clarkb | thanks lgtm | 20:45 |
| clarkb | jeblair: ^ | 20:45 |
| jeblair | clarkb: done | 20:50 |
| *** mberwanger has quit IRC | 20:51 | |
| annegentle | yjiang5: yes that's what I meant... not that I really want to do that I'd rather get a good dev env. | 20:52 |
| annegentle | jog0: hm. I have .tox/venv/bin/python2.7 | 20:52 |
| *** mberwanger has joined #openstack-infra | 20:53 | |
| *** thomasm has quit IRC | 20:53 | |
| *** johnthetubaguy1 has quit IRC | 20:53 | |
| annegentle | jog0: file not found for source .tox/py27/bin/activate | 20:53 |
| clarkb | jeblair: https://review.openstack.org/#/c/42393/ LGTM but I am not +2 or +A because the js at the end of the js file is a bit obtuse to my not js friendly brain | 20:54 |
| clarkb | jeblair: I get that we are adding event handlers that do things like save and read the cookie but I am not sure when exactly they would fire | 20:54 |
| clarkb | jeblair: tl;dr feel free to approve if you are comfortable with the change | 20:54 |
| yjiang5 | annegentle: sure, let me know if you still need me to update the patch for you after you can't get a god dev env. | 20:55 |
| *** openstackgerrit has quit IRC | 20:55 | |
| *** openstackgerrit has joined #openstack-infra | 20:56 | |
| clarkb | annegentle: for https://review.openstack.org/#/c/43734/1 have you built the ja docs yourself? | 20:56 |
| annegentle | clarkb: nope | 20:56 |
| clarkb | annegentle: trying to figure out how we might tell if the line you have questions about is correct | 20:56 |
| annegentle | clarkb: I have an email in to Daisy about the pom.xml, whether there's a separate one for building ja | 20:57 |
| clarkb | annegentle: I have a feeling that you will end up generating english docs with that pom.xml | 20:57 |
| clarkb | generated/ja/openstack-ops/pom.xml fits into what I expect with what little I know of the process | 20:57 |
| clarkb | basically you end up with a separate set of source files generated from the english files | 20:57 |
| jog0 | annegentle: di you run 'tox -epy27'? | 21:00 |
| jog0 | what does 'ls .tox/' give you | 21:00 |
| *** ArxCruz has quit IRC | 21:01 | |
| *** johnthetubaguy has joined #openstack-infra | 21:03 | |
| *** johnthetubaguy has quit IRC | 21:04 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add support for running tox with PyPy and use it for Marconi https://review.openstack.org/42692 | 21:04 |
| Alex_Gaynor | clarkb: Does something somewhere need to be restarted before ^ takes effect? | 21:08 |
| clarkb | Alex_Gaynor: no, it should slowly happen (JJB updates jenkins, zuul reloads its configs, puppet install pypy) | 21:08 |
| Alex_Gaynor | clarkb: ok, but it's nto instant is the primary takeaway :) | 21:09 |
| clarkb | Alex_Gaynor: the central git repo updates in ~5 minutes then everything else can happen over the next 10-15 minutes | 21:09 |
| *** rfolco has quit IRC | 21:09 | |
| clarkb | Alex_Gaynor: correct. I would give it 20 minutes | 21:09 |
| Alex_Gaynor | clarkb: thanks | 21:09 |
| openstackgerrit | Emilien Macchi proposed a change to openstack-infra/jenkins-job-builder: Add Plot plugin support https://review.openstack.org/43685 | 21:10 |
| *** CaptTofu has quit IRC | 21:12 | |
| *** erfanian has joined #openstack-infra | 21:13 | |
| *** mberwanger has quit IRC | 21:13 | |
| *** erfanian has quit IRC | 21:13 | |
| *** erfanian has joined #openstack-infra | 21:13 | |
| *** jruzicka has quit IRC | 21:14 | |
| *** hashar has quit IRC | 21:14 | |
| yjiang5 | can anyone help me to have a look on https://review.openstack.org/#/c/39890/ . It's really strange that it has no dependency list, while it in fact depends on https://review.openstack.org/#/c/39606/ | 21:18 |
| *** cyeoh_ has joined #openstack-infra | 21:18 | |
| yjiang5 | is this the right challell asking for gerrit question? | 21:18 |
| *** cyeoh has quit IRC | 21:19 | |
| markmc | "This change was unable to be automatically merged with the current state of the repository and the following changes which were enqueued ahead of it" | 21:19 |
| markmc | this is new? | 21:19 |
| clarkb | yjiang5: yes this is the right channel | 21:19 |
| clarkb | markmc: semi new, I think it is at least a month old | 21:20 |
| markmc | what happens if an enqueued change caused the conflict and then fails to merge? | 21:20 |
| markmc | clarkb, ah, ok | 21:20 |
| clarkb | markmc: then you reverify | 21:20 |
| *** sandywalsh has quit IRC | 21:20 | |
| clarkb | markmc: but we think that happens in the minority of cases and that the majority of conflicts happen against $branch. Greater good and all | 21:20 |
| markmc | clarkb, hmm - why don't we hold off failing until the conflict merges ? | 21:20 |
| clarkb | markmc: because that stops everything in the gate until it merges | 21:21 |
| clarkb | markmc: which led to really shallow queueus | 21:21 |
| clarkb | yjiang5: is there a recent draft patchset on 39606? | 21:21 |
| yjiang5 | clarkb: I'm really surprised why it lost the dependency. After all, w/o the dependency it's sure to have conflict. | 21:21 |
| yjiang5 | clarkb: I don't think so. | 21:21 |
| jeblair | markmc: wasn't this your idea at the havana summit? | 21:22 |
| clarkb | yjiang5: what is the most recent patchset on 39606? | 21:22 |
| clarkb | yjiang5: I see 25 | 21:22 |
| markmc | jeblair, I doubt it, but who knows :) | 21:22 |
| yjiang5 | clarkb: I think it is. | 21:22 |
| yjiang5 | clarkb: even if it has, there should be 'outdated' in the 39890 | 21:22 |
| markmc | jeblair, I'm just looking at a failure now with 10 changes listed as enqueued ahead | 21:22 |
| markmc | jeblair, if the conflict is in one of those, I'm going to struggle to figure out which one | 21:23 |
| clarkb | yjiang5: not if there is a draft | 21:23 |
| clarkb | yjiang5: drafts make everything weird. Can you check 39606? | 21:23 |
| markmc | jeblair, and if it is, do I rebase onto it? what if it fails to merge? I've rebased myself behind a failing change | 21:23 |
| yjiang5 | clarkb: draft? No, that patch series has been there sooo long, I don't think any draft for it. | 21:23 |
| *** erfanian has quit IRC | 21:23 | |
| markmc | jeblair, I think I'd prefer not to be told my change failed to merge until the conflict change actually got merged | 21:24 |
| clarkb | yjiang5: is patchset 25 the latest patchset? | 21:24 |
| yjiang5 | clarkb: yes, it is. | 21:24 |
| markmc | jeblair, not a huge deal right now, just feedback :) | 21:24 |
| jeblair | markmc: you are saying the things i'm pretty sure i said to you or someone who looks like you at the havana summit | 21:24 |
| yjiang5 | there is a DRAFT in patchset 1, I think that's ok, right? | 21:24 |
| clarkb | jeblair: yes | 21:24 |
| markmc | jeblair, lies :) | 21:24 |
| clarkb | yjiang5: yeah that should be ok, what is the sha of that patchset? | 21:24 |
| clarkb | markmc: fwiw this has made zuul a lot more responsive | 21:24 |
| markmc | jeblair, honestly don't recall discussing this at all | 21:25 |
| markmc | clarkb, ok | 21:25 |
| clarkb | markmc: I think it is one of the more important zuul improvements that has gone in this cycle | 21:25 |
| yjiang5 | clarkb: you mean my local sha? | 21:25 |
| clarkb | yjiang5: no, the sha for the commit in patchset 1 | 21:25 |
| clarkb | yjiang5: I cannot see your draft which is why I am asking you | 21:25 |
| jeblair | markmc: another thing that may make that show up more often... | 21:25 |
| clarkb | markmc: we are currently running tests for 29 changes in the gate queue. Preivously we would never get above 10 because something would conflict | 21:26 |
| yjiang5 | clarkb: how can I see the sha in gerrit? | 21:26 |
| jeblair | markmc: is that after the discussion about your patch series the other day, i found that zuul was not mimicking gerrit's merge strategy correctly | 21:26 |
| jeblair | markmc: so it should be much closer now | 21:26 |
| markmc | jeblair, cool | 21:26 |
| yjiang5 | b95a461a4226607ae2506a2311c2edf09b0a1e87 | 21:26 |
| clarkb | yjiang5: ok so that is different than what 39890 depends on | 21:26 |
| *** ^d has left #openstack-infra | 21:27 | |
| markmc | clarkb, I don't really follow - we were leaving merge conflicts in the gate queue previously ? | 21:27 |
| clarkb | markmc: yes, until we knew that we could get rid of them nicely | 21:27 |
| clarkb | markmc: that takes a long time. So we decided to not be so nice at the last summit | 21:27 |
| clarkb | yjiang5: there is a phantom sha here. | 21:27 |
| markmc | clarkb, I guess I'm saying, if a gate reset happens then my change should be considered again | 21:27 |
| jeblair | markmc: the good side of that is that zuul will more correctly predict whether gerrit can merge something, the bad side is that when it predicts that it can't, you'll see that message (whereas before, zuul would have kept it in the queue and have been surprised) | 21:27 |
| markmc | clarkb, since the conflicting change ahead of me may have gotten kicked out | 21:27 |
| clarkb | markmc: I understand what you are saying | 21:28 |
| yjiang5 | clarkb: the strangest is, I saw it several success for it in zull, I can't imagine how can that work considering the dependency in the exception.py on previous patch. | 21:28 |
| clarkb | yjiang5: patchset 26 of 39890 depends on patchset 25 of 39606. patchset 27 of 39890 depends on a sha that is not present in gerrit | 21:29 |
| markmc | clarkb, well, I don't understand what you're saying :) | 21:29 |
| * markmc heads off to rebase his change | 21:29 | |
| clarkb | yjiang5: so somehow you pushed patchset 27 without updating 39606 | 21:29 |
| *** dina_belova has quit IRC | 21:29 | |
| mtreinish | jeblair, clarkb: you keep throwing me for a loop moving where the gate pipeline is on the status page. :) | 21:29 |
| clarkb | markmc: I am saying that these things were considered at the last summit | 21:29 |
| markmc | clarkb, so no further feedback/discussion welcomed ? | 21:29 |
| clarkb | markmc: but we collectively made a decision there that these tradeoffs were worth the increased throughput | 21:29 |
| *** dina_belova has joined #openstack-infra | 21:29 | |
| markmc | clarkb, don't mind the snark, joking honest :) | 21:29 |
| clarkb | markmc: I am fine with further input. I am justifying the "brokeness" | 21:30 |
| jeblair | clarkb: do you remember who was pushing for this? | 21:30 |
| clarkb | I think we all agree that this is not "correct" | 21:30 |
| markmc | clarkb, yeah, I just don't follow how keeping the change around for reconsideration if the gate resets decreases throughput | 21:30 |
| clarkb | jeblair: sdague was definitely there. It may have been sdague | 21:30 |
| clarkb | markmc: because then you potentially wait hours for feedback on your change | 21:30 |
| clarkb | markmc: or you can be told asap that an issue occurred | 21:31 |
| markmc | clarkb, ah, so it slows down feedback on my specific change, not slows down the gate as a whole? | 21:31 |
| *** HenryG has joined #openstack-infra | 21:31 | |
| clarkb | correct | 21:31 |
| markmc | clarkb, you mentioned shallow queues before this, though ? | 21:31 |
| jeblair | markmc: zuul doesn't have alternate scenarios -- it only tests one scenario at a time (keep in mind that testing the current scenario is using >150 machines) | 21:32 |
| jeblair | markmc: so within that framework, our only choices are keep a change in the queue or not; there's no facility for keeping an alternate branch prediction around | 21:32 |
| *** sandywalsh has joined #openstack-infra | 21:33 | |
| jeblair | markmc: i'm not opposed to that, it's just a bunch of complicated code that hasn't been written yet | 21:33 |
| yjiang5 | clarkb: hmm, I noticed the parents of 39890 is aa1c5ef, which is my local patch's sha. | 21:33 |
| clarkb | yjiang5: correct, but aa1c5ef is not in gerrit | 21:33 |
| markmc | jeblair, it's not really an alternate branch prediction though - say, you've commits A, B, C, D and C conflicts with A | 21:33 |
| clarkb | yjiang5: do you remember how you constructed that branch and pushed it to gerrit? | 21:33 |
| *** erfanian has joined #openstack-infra | 21:33 | |
| markmc | jeblair, we predict and test A, B, D | 21:33 |
| jeblair | markmc: so our immediate tradeoff is, keep it in the queue which means that by odds, we'll hit a change that can't merge, and only test, say, the top 5 commits out of a queue that's 20 deep | 21:33 |
| markmc | jeblair, A fails and we get a gate reset | 21:33 |
| anteaya | mtreinish: like a game of where's waldo, we need to keep status.o.o/zuul exciting :D | 21:33 |
| jeblair | markmc: or kick it out and test all 20 | 21:33 |
| markmc | jeblair, when not now predict B, C, D and test that? | 21:34 |
| *** dina_belova has quit IRC | 21:34 | |
| yjiang5 | clarkb: git checkcout, git cherry-pick, ...., git cherry-pick, git push gerrit HEAD:refs/for/master/pci-passthrough-enhancement. | 21:34 |
| markmc | jeblair, honestly, I'm just curious - if it's a waste of time explaining, nevermind. Over beers sometime :) | 21:35 |
| clarkb | yjiang5: does aa1c5ef have a change-id in your local branch? canyou `git log -2` and paste that? | 21:35 |
| clarkb | yjiang5: also is there a reason you are not using git-review? | 21:35 |
| jeblair | markmc: the two sconarios we can currently handle are ABCD (which includes BCD -- this is the old behavior where we were conservative about merge conflicts), and ABD (where we have already forgotten about C) | 21:36 |
| yjiang5 | clarkb: I was told by others that this can avoid upload un-needed patches. | 21:36 |
| *** dkliban has quit IRC | 21:36 | |
| clarkb | yjiang5: I think it has avoided uploading a needed patch | 21:36 |
| jeblair | markmc: getting from ABCD to BCD is an scenario that zuul can't handle atm; it involves branching the prediction of ABCD to include both BCD and ABD | 21:37 |
| anteaya | yjiang5: what others? | 21:37 |
| clarkb | yjiang5: git review checks bunch of stuff and makes sure things are in order before pushing. It is quite useful | 21:37 |
| *** thomasbiege has quit IRC | 21:37 | |
| markmc | jeblair, ok, thanks | 21:37 |
| *** thomasbiege has joined #openstack-infra | 21:37 | |
| jeblair | markmc: beers are the best way to talk about zuul | 21:37 |
| *** thomasbiege has quit IRC | 21:37 | |
| yjiang5 | clarkb: http://paste.openstack.org/show/45243/ | 21:38 |
| yjiang5 | anteaya: another member in our team :$ | 21:38 |
| anteaya | yjiang5: might I encourage both of you to read https://wiki.openstack.org/wiki/Gerrit_Workflow | 21:38 |
| anteaya | it is our standard workflow and uses git review | 21:39 |
| clarkb | yjiang5: I have a feeling that if you use git review instead or manually push the missing commit then gerrit will tie them back up again | 21:39 |
| clarkb | yjiang5: for some reason pushing directly allowed gerrit to ignore that one commit | 21:39 |
| yjiang5 | anteaya: I use git review before, changed recently :$ | 21:39 |
| jog0 | the stats on http://status.openstack.org/zuul/ are impressive we hit 500 zuul jobs in an hour | 21:39 |
| clarkb | yjiang5: what unneeded commits are you trying to avoid? | 21:40 |
| jog0 | go team | 21:40 |
| yjiang5 | clarkb: what do you mean of unneeded commits? | 21:40 |
| *** markmcclain has quit IRC | 21:40 | |
| clarkb | yjiang5: earlier you said you use this workflow to avoid pushing unneeded patches | 21:40 |
| clarkb | (I am trying to understand the need so that we can fix git review if there is a bug or missing feature) | 21:41 |
| *** changbl has quit IRC | 21:41 | |
| yjiang5 | clarkb: uneeded means, I get patch set (like >10), and then changes only 1 or 2. IIRC, if using git review, it will update the whole patchset, even if I only change like the last one or two. | 21:41 |
| anteaya | jog0 \o/ | 21:41 |
| *** dkranz has quit IRC | 21:42 | |
| yjiang5 | clarkb: like A, B, C, .... F, G. I get the patches in the morning, and changes F, G, then I do git review, the whole patchset from A~E will be updated. | 21:42 |
| clarkb | yjiang5: if you have an up to date git review that will only happen if it is necessary to resolve merge conflicts with the target branch | 21:43 |
| clarkb | yjiang5: it is a good thing that git review does that otherwise you will fail to merge further down the line | 21:43 |
| openstackgerrit | lifeless proposed a change to openstack-infra/reviewstats: Ng is spelt cmsj. https://review.openstack.org/43954 | 21:43 |
| clarkb | yjiang5: older git review will do it on every push which is a bug and was fixed | 21:43 |
| lifeless | russellb: ^ that just flags Ng's actual lp account properly. | 21:44 |
| Ng | :) | 21:44 |
| yjiang5 | clarkb: what version the 'old' is? | 21:44 |
| clarkb | yjiang5: I will look in the git long, one sec | 21:44 |
| *** markmc has quit IRC | 21:45 | |
| clarkb | yjiang5: >= 1.19 should have the fix | 21:47 |
| yjiang5 | clarkb: strange, mine is 1.22 already. | 21:48 |
| clarkb | yjiang5: 1.22 will rebase but only if it is a good thing | 21:48 |
| yjiang5 | clarkb: thanks for the explain and I will ask my team member to use the right method in future also. | 21:48 |
| clarkb | yjiang5: that case is when you are proposing a change to a branch that you conflict with. The rebase is necessary to resolve the conflicts | 21:48 |
| clarkb | yjiang5: if you really don't want to rebase even when there are conflicts you can use `git review -R` | 21:48 |
| yjiang5 | clarkb: got it. Thanks. | 21:49 |
| clarkb | yjiang5: back to fixing the current problem I would try running git review on your branch | 21:49 |
| *** redbeard2 has quit IRC | 21:49 | |
| clarkb | to push aa1 into gerrit | 21:49 |
| yjiang5 | clarkb: can I just push aa1, i..e git checkout aa1, git review? | 21:50 |
| clarkb | yjiang5: yes | 21:50 |
| Alex_Gaynor | clarkb: Hmm, so https://jenkins02.openstack.org/job/gate-marconi-pypy/1/console does this mean JJB refreshed before the PPA was installed from? | 21:51 |
| clarkb | Alex_Gaynor: possibly, I will take a look at that slave | 21:51 |
| Alex_Gaynor | clarkb: thanks | 21:51 |
| *** ftcjeff has joined #openstack-infra | 21:51 | |
| yjiang5 | clarkb: seems if I git review now, it will update the 39893 , which is in gate already. Will do it after 39893 is done. thanks. | 21:53 |
| clarkb | Alex_Gaynor: http://paste.openstack.org/show/45248/ looking into that now | 21:54 |
| *** mriedem has quit IRC | 21:54 | |
| Alex_Gaynor | clarkb: :/ | 21:55 |
| clarkb | yjiang5: are you using the list that warns you of mutliple changes to make that determination? if your git sha matches gerrit it won't actually update the existing change | 21:55 |
| clarkb | yjiang5: however there is no harm in being careful | 21:55 |
| clarkb | Alex_Gaynor: ppa:pypy/ppa is what the ppa should be according to https://launchpad.net/~pypy/+archive/ppa | 21:56 |
| *** weshay has quit IRC | 21:56 | |
| Alex_Gaynor | clarkb: did I typo a thing? | 21:56 |
| yjiang5 | clarkb: it warns "you have more than one commit that you are about to submit". A bit strange how to make sure the sha will be the same after cherry-pick or after 'git review'? I think after git review, gerrit will get a new sha1 in gerrit, righ? | 21:56 |
| clarkb | Alex_Gaynor: possibly. I am still double checking that that is the correct ppa | 21:57 |
| Alex_Gaynor | clarkb: http://bpaste.net/show/126875/ | 21:57 |
| clarkb | yjiang5: it may or may not get a new sha after git review. depends on whether or not there are conflicts. The warning message is there to help prevent you fro pushing hundreds of commits from a personal branch (it has happened in the past) | 21:58 |
| clarkb | Alex_Gaynor: ya, if what I am looking at pans out that is the fix | 21:58 |
| Alex_Gaynor | clarkb: I wonder if it makes sense for there to be tests for that repo which really spin up an image and verify that stuff installs clean? | 21:59 |
| clarkb | Alex_Gaynor: yes Ithink that is the fix you want | 21:59 |
| Alex_Gaynor | clarkb: sending patch | 21:59 |
| *** markmcclain has joined #openstack-infra | 21:59 | |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Fixed the name of the PPA to install PyPy from https://review.openstack.org/43958 | 21:59 |
| clarkb | Alex_Gaynor: it makes a ton of sense, it is something we have thought about for a while but most of our stuff isn't inteded to cleanly install all in one. The tricky bit is partitioning what you install | 21:59 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/reviewstats: Support for unofficial projects and add savanna https://review.openstack.org/40953 | 22:00 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Fix checkout when preparing a ref https://review.openstack.org/43959 | 22:00 |
| jeblair | clarkb: can you look at that? we should either install that or revert soon; some zuul refs are not correct currently | 22:01 |
| clarkb | jeblair: looking | 22:01 |
| jeblair | clarkb: (the remote update that zuul does at startup was masking then during my testing) | 22:02 |
| *** boris-42 has quit IRC | 22:02 | |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Ng is spelt cmsj. https://review.openstack.org/43954 | 22:02 |
| clarkb | jeblair: it looks good to me. I assume the if block was changed for consistency? | 22:02 |
| clarkb | jeblair: when you are not busy fixing zuul https://review.openstack.org/#/c/43958/ | 22:03 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/reviewstats: Support for unofficial projects and add savanna https://review.openstack.org/40953 | 22:03 |
| *** burt has quit IRC | 22:03 | |
| jeblair | clarkb: yep; two different methods of doing checkouts didn't seem like a good idea | 22:04 |
| *** morganfainberg is now known as morganfainberg|a | 22:05 | |
| jeblair | clarkb: i have aprvd the zuul change; when it merges i will hard restart zuul because the longer it runs the more incorrect it gets. | 22:05 |
| clarkb | jeblair: ok | 22:05 |
| *** SergeyLukjanov has quit IRC | 22:05 | |
| *** morganfainberg|a is now known as morganfainberg | 22:06 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add Jenkins jobs for Tuskar https://review.openstack.org/43461 | 22:07 |
| openstackgerrit | A change was merged to openstack-infra/config: Fixed the name of the PPA to install PyPy from https://review.openstack.org/43958 | 22:07 |
| openstackgerrit | A change was merged to openstack-dev/hacking: Adds ability to ignore hacking validations with noqa https://review.openstack.org/41713 | 22:08 |
| *** senk has quit IRC | 22:09 | |
| *** CaptTofu has joined #openstack-infra | 22:10 | |
| *** shardy is now known as shardy_afk | 22:13 | |
| *** _TheDodd_ has quit IRC | 22:13 | |
| clarkb | Alex_Gaynor: one way we might be able to test that is to run puppet apply for each node in site.pp in its own LXC | 22:13 |
| clarkb | Alex_Gaynor: then we don't have to throw away a ton of slaves to test it, but can get a reasonable check that each node def works | 22:14 |
| *** pentameter has quit IRC | 22:14 | |
| Alex_Gaynor | clarkb: sounds pretty reasonable | 22:14 |
| clarkb | I really like that idea the more I think about it | 22:15 |
| clarkb | the last major hurdle to making it work is supplying enough useful info to the nodes to test something production like. We could possibly do that with an example hiera file | 22:16 |
| jeblair | clarkb, Alex_Gaynor: I mean, we _can_ use a ton of slaves if we want. | 22:17 |
| clarkb | jeblair: I think this is the perfect use case for containers though | 22:18 |
| clarkb | jeblair: we don't need security we just need enough isolation that puppe tdoesn't throw up | 22:18 |
| jeblair | clarkb: i agree. just making conversation. :) | 22:18 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Fix checkout when preparing a ref https://review.openstack.org/43959 | 22:18 |
| jeblair | clarkb: ^ now with more test passing | 22:18 |
| clarkb | jeblair: should the clean happen after the checkout? does it matter? | 22:20 |
| jeblair | clarkb: i updated the commit msg: Also, reset and clean _before_ checking out, since that's supposed | 22:20 |
| jeblair | to clean up from messy merges, etc. | 22:20 |
| jeblair | clarkb: that's what the test caught. | 22:20 |
| openstackgerrit | A change was merged to openstack-infra/config: Added list for Track Chairs Committee for Summit Nov 2013 https://review.openstack.org/43773 | 22:32 |
| *** Bada has quit IRC | 22:35 | |
| openstackgerrit | A change was merged to openstack-infra/config: Fix copy-n-paste example to be clearer. https://review.openstack.org/43870 | 22:36 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Fix checkout when preparing a ref https://review.openstack.org/43959 | 22:37 |
| jeblair | stopping zuul | 22:38 |
| *** jhesketh has joined #openstack-infra | 22:38 | |
| jeblair | starting zuul | 22:39 |
| *** dina_belova has joined #openstack-infra | 22:40 | |
| *** ftcjeff has quit IRC | 22:44 | |
| *** dina_belova has quit IRC | 22:45 | |
| *** ArxCruz has joined #openstack-infra | 22:45 | |
| *** ArxCruz has quit IRC | 22:45 | |
| *** rcleere has joined #openstack-infra | 22:50 | |
| *** senk has joined #openstack-infra | 22:50 | |
| *** dkliban has joined #openstack-infra | 22:50 | |
| Alex_Gaynor | Does nodepool not create more of the centos nodes? | 22:52 |
| jeblair | Alex_Gaynor: currently nodepool only manages devstack-precise nodes; all the others are static | 22:52 |
| jeblair | Alex_Gaynor: we're thinking about having it manage everything | 22:52 |
| Alex_Gaynor | jeblair: seem pretty reasonable | 22:53 |
| *** thomasm has joined #openstack-infra | 22:53 | |
| notmyname | jeblair: are you scripting reverify comments? | 22:54 |
| jeblair | notmyname: yes | 22:54 |
| jeblair | notmyname: i had to restart zuul a couple of times | 22:54 |
| notmyname | jeblair: ok. you may be a little aggressive ;-) re https://review.openstack.org/#/c/42447/ | 22:55 |
| *** portante is now known as portante|afk | 22:55 | |
| *** jhesketh_ has joined #openstack-infra | 22:55 | |
| yjiang5 | clarkb: seems still not working :( | 22:56 |
| jeblair | notmyname: yeah, i had to restart 3 times. i'll get around to making a way to re-enqueue things that don't leave comments someday | 22:57 |
| *** jhesketh has quit IRC | 22:57 | |
| yjiang5 | I cherry-pick everything and then git review -t , but seems 39890 still have parent in my local sha1 | 22:57 |
| clarkb | yjiang5: you only need to push to https://review.openstack.org/#/c/39606/ | 22:58 |
| clarkb | you shouldn't need to cherry pick everything, just push what you have | 22:58 |
| yjiang5 | clarkb: I'm totally lost :$ now I have do the cherry-pick. How can I push that single change? | 22:59 |
| clarkb | yjiang5: just run git review when you have aa1c checked out | 23:00 |
| yjiang5 | clarkb: now in the latest 39890, it's 480a, then I check out that one and git review, right? | 23:00 |
| *** dkliban has quit IRC | 23:01 | |
| clarkb | yjiang5: yes | 23:02 |
| yjiang5 | clarkb: thanks. | 23:02 |
| Alex_Gaynor | clarkb: pypy tests for marconi seem to be going smoothly now! | 23:02 |
| clarkb | Alex_Gaynor: woot | 23:03 |
| clarkb | jhesketh_: jeblair: I have reviewed the Zuul reporters changes | 23:03 |
| jhesketh_ | clarkb: yep, I saw. Thanks for that | 23:04 |
| jhesketh_ | I'll take a look shortly :-) | 23:04 |
| yjiang5 | clarkb: sorry for disturbing again. http://paste.openstack.org/show/45251/ | 23:04 |
| *** jhesketh has joined #openstack-infra | 23:04 | |
| clarkb | yjiang5: aha! I think that is why we have been broken all along | 23:05 |
| yjiang5 | clarkb: ? | 23:05 |
| clarkb | yjiang5: gerrit is being smart and saying that you are not changing anything in the patch you are trying to push so it rejects it | 23:05 |
| clarkb | yjiang5: you can try making a trivial update to the commit message and pushing that | 23:06 |
| yjiang5 | yjiang5: nooo, it's really not easy to get +A :) | 23:06 |
| jeblair | clarkb: should the parent be updated? | 23:06 |
| clarkb | jeblair: the sha has definitely changed so the parent is probably different but gerrit is doing additional checks iirc | 23:06 |
| *** rcleere has quit IRC | 23:07 | |
| clarkb | jeblair: this is related to cherry picking iirc | 23:07 |
| jeblair | clarkb: and the reparenting isn't enough to let it in? perhaps because of the current situation with the parent. | 23:07 |
| *** senk has quit IRC | 23:08 | |
| jeblair | nodepool seems to have done semi-sane things with its image cleanup | 23:09 |
| yjiang5 | jeblair: so anything I can do? Really sorry for bothering. | 23:09 |
| jeblair | yjiang5: i don't have any ideas clarkb doesn't have | 23:09 |
| yjiang5 | jeblair: what's the trival changes in commit message that make sure to contain the +A? | 23:10 |
| clarkb | I think what may have happened is the commit was pushed into gerrit so gerrit knows about it somewhere, but it wasn't pushed into the special refs/for/foo so a change wasn't created for it | 23:10 |
| clarkb | jeblair: yjiang5 so all of these commits are in gerrit's git but not as gerrit patchsets | 23:11 |
| clarkb | jeblair: how are the new zuul refs looking? | 23:12 |
| jeblair | clarkb: the first batch looked good; i'm waiting to check the second batch (after a reset, which is where i saw the drift last time) | 23:12 |
| yjiang5 | clarkb: so possibly I can wait till the parents are merged :) | 23:13 |
| jeblair | clarkb: second batch looks good (it picked up the merged change in the refs created after the reset) | 23:15 |
| clarkb | jeblair: great | 23:15 |
| *** datsun180b has quit IRC | 23:20 | |
| *** vipul is now known as vipul-away | 23:20 | |
| anteaya | I'm tired and going to wrap up, see you tomorrow | 23:21 |
| *** anteaya has quit IRC | 23:21 | |
| *** sarob has joined #openstack-infra | 23:25 | |
| jeblair | clarkb: i think the current times for git processing are about 20-25 seconds to fetch a branch, and another 10-25 seconds to fetch the ref | 23:25 |
| jeblair | clarkb: so it's only marginally better than what we had before :/ | 23:26 |
| jeblair | clarkb: (though, gate queue processing should be much faster because it only incurs the second fetch for changes after the first) | 23:26 |
| jeblair | clarkb: i think that the number of zuul refs in the local repo has an effect. | 23:27 |
| *** michchap has joined #openstack-infra | 23:28 | |
| openstackgerrit | David Kranz proposed a change to openstack-infra/config: Add slow heat job to devstack experimental for debug https://review.openstack.org/43971 | 23:31 |
| *** sarob_ has joined #openstack-infra | 23:36 | |
| *** markmcclain has quit IRC | 23:37 | |
| jeblair | jhesketh_: ping | 23:38 |
| jhesketh | jeblair: pong | 23:38 |
| jeblair | jhesketh: do you really want to keep backwards compat? | 23:38 |
| jhesketh | I'm open to either way | 23:38 |
| jeblair | jhesketh: i don't; i think it's easy enough to change the config file | 23:38 |
| *** sarob has quit IRC | 23:38 | |
| jhesketh | ( lifeless had suggested it) | 23:39 |
| jeblair | jhesketh: and i think if we kept backwards compat, it should be done without the legacy flag; | 23:39 |
| jhesketh | jeblair: so how will openstack's zuul get updated to not break the queues? | 23:39 |
| jeblair | jhesketh: so i was thinking that it would be easier to just drop it | 23:39 |
| jeblair | jhesketh: downtime. but short. | 23:39 |
| jeblair | jhesketh: (we'll merge the config file change, stop zuul, apply the change, start zuul) | 23:40 |
| jhesketh | okay, is there an easy way to snapshot what is in the queue and requeue them after? | 23:40 |
| jeblair | jhesketh: yeah, the graceful shutdown will take care of that | 23:40 |
| *** dina_belova has joined #openstack-infra | 23:40 | |
| jeblair | jhesketh: it's very similar to the multiple trigger change, where we also broke backwards compat | 23:41 |
| jhesketh | okay cool | 23:41 |
| jeblair | jhesketh: soon, we'll release a zuul 2.0, and after that we'll want to try to maintain compatability | 23:41 |
| jhesketh | well if you're happy with no back-compat I'll remove it | 23:41 |
| jeblair | jhesketh: but for now, we're breaking it in several other places, so i'd rather break it in one more and not deal with extra code | 23:42 |
| jhesketh | sure | 23:42 |
| jhesketh | makes sense | 23:42 |
| jeblair | jhesketh: i'll finish reviewing patchset 5, aside from that | 23:43 |
| clarkb | jeblair: maybe packing refs in zuul repos would help? | 23:43 |
| clarkb | jeblair: that makes it hard to delete them though | 23:43 |
| jhesketh_ | jeblair: I have some changes I'm about to push if you want to wait and save double-reviewing | 23:43 |
| jhesketh_ | but I'll remove the backwards compat first | 23:44 |
| jeblair | clarkb: i think they are occasionally getting packed; there were only 400 loose and 36000 packed | 23:44 |
| jeblair | jhesketh_: ok | 23:44 |
| *** dkranz has joined #openstack-infra | 23:44 | |
| jeblair | clarkb: (though it's easy to delete them by just deleting the repo | 23:44 |
| *** dina_belova has quit IRC | 23:45 | |
| *** jhesketh has quit IRC | 23:47 | |
| *** pcrews has quit IRC | 23:47 | |
| clarkb | jeblair: it is amazing how much fun you can have looking at graphite graphs. According to graphite we merged >1k changes the week of the 15th | 23:55 |
| *** jergerber has quit IRC | 23:56 | |
| jeblair | wheee | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!