| *** dboik has quit IRC | 00:00 | |
| *** bknudson has joined #openstack-infra | 00:00 | |
| *** dannywilson has quit IRC | 00:00 | |
| *** dims has joined #openstack-infra | 00:00 | |
| jeblair | (btw, for the record, a while back i checked out javamelody and it looked like the jvm had plenty of memory) | 00:00 |
|---|---|---|
| fungi | there's no publish-and-submit button in 2.10... publish is a separate step now | 00:00 |
| mordred | yah | 00:00 |
| mordred | I would like to register my distate about that | 00:00 |
| fungi | also my submit button press is waiting on the ajaxy callback | 00:01 |
| mordred | haha | 00:01 |
| fungi | that's no good | 00:01 |
| *** rrepaka has quit IRC | 00:01 | |
| fungi | "Working..." | 00:01 |
| *** sdake has quit IRC | 00:01 | |
| mordred | fungi: back in my days, buttons were written by hand. in xlib. | 00:02 |
| fungi | 500 Internal server error | 00:02 |
| openstackgerrit | Merged openstack-infra/system-config: Revert "Upgrade Gerrit from 2.8 to ver 2.10 on review.o.o" https://review.openstack.org/182117 | 00:02 |
| fungi | worked the second time | 00:02 |
| mordred | it took you a whole day to write a button, but when you finished, BY GOD it was rectangular | 00:02 |
| jeblair | i can't find a system-config error in the log | 00:02 |
| *** esker has joined #openstack-infra | 00:03 | |
| fungi | mine were regular quadrilaterals anyway | 00:03 |
| mordred | shall I stop gerrit? | 00:03 |
| fungi | jeblair: i see it from time to time when packets go missing, so nothing new i don't expect | 00:03 |
| jeblair | mordred: ++ | 00:04 |
| *** achanda has quit IRC | 00:04 | |
| mordred | stopp | 00:04 |
| mordred | dumping db | 00:04 |
| fungi | puppetmaster updated | 00:04 |
| fungi | so we're all ready when that step arrives | 00:04 |
| jeblair | cool, added it in where it was critical just to make sure we didn't forget. and we didn't. :) | 00:05 |
| mordred | dump complete - running downgrade script *braces for evil* | 00:05 |
| *** mmedvede has quit IRC | 00:05 | |
| zaro | my kids are kicking at me to goto soccer pracice. need to afk for a few. looking forward to the Aftermath() | 00:05 |
| * greghaynes crosses fingers | 00:05 | |
| *** ivar-laz_ has quit IRC | 00:06 | |
| mordred | finished. no errors | 00:06 |
| hichihara | Is today gerrit maintenance day? | 00:06 |
| *** ivar-lazzaro has joined #openstack-infra | 00:06 | |
| anteaya | hichihara: yes | 00:06 |
| mordred | fungi: all you | 00:06 |
| fungi | hichihara: today is fix the results of saturday's gerrit maintenance day | 00:06 |
| *** enikanorov2 has joined #openstack-infra | 00:06 | |
| fungi | cache and index blown away | 00:07 |
| fungi | looks like puppet is going to reindex for us | 00:07 |
| fungi | mordred: are you doing the apply? | 00:07 |
| hichihara | I got it. Thank you. | 00:07 |
| mordred | fungi: I can | 00:07 |
| mordred | I will also tee into a file like before | 00:08 |
| fungi | go for it | 00:08 |
| fungi | thanks | 00:08 |
| mordred | puppet agent --test --agent_disabled_lockfile=/tmp/alt-lock-file | tee /tmp/upgrade.log | 00:08 |
| mordred | downgrade.log | 00:08 |
| mordred | running | 00:08 |
| fungi | watching | 00:08 |
| mordred | fungi: it seems to be doing something | 00:09 |
| mordred | next time we do one of these, we shoudl do the netcat trick so that other folks can follow along | 00:09 |
| mordred | i mean, I don't really like doing these | 00:10 |
| fungi | and also redirect stderr | 00:10 |
| *** esker has quit IRC | 00:10 | |
| jeblair | mordred: sometimes it emits passwords :( | 00:10 |
| *** esker has joined #openstack-infra | 00:10 | |
| fungi | oh, right, diff of secure.config | 00:10 |
| jeblair | actually, it almost always emits passwords :/ | 00:10 |
| mordred | jeblair: good point | 00:10 |
| mordred | jeblair: we should NOT do that | 00:10 |
| jroll | netcat + sed? :P | 00:11 |
| anteaya | mordred: thanks for thinking of us little people though | 00:11 |
| mordred | jroll: WCPGW? | 00:11 |
| jroll | :) | 00:11 |
| nibalizer | mordred: you can trust us to close our eyes at the right time | 00:11 |
| mordred | what might be neat is a thing to turn on that would emit _Commands_ we're running in to IRC directly, so that we don't have to swap back and forth doing and saying we're doing | 00:11 |
| mordred | I'd want to play with that for a while before using it in anger | 00:12 |
| anteaya | how about not using it in anger at all | 00:12 |
| mordred | anteaya: no chance | 00:12 |
| anteaya | getting tired of the anger personally | 00:12 |
| jroll | 90% of software development is a result of anger IME | 00:13 |
| anteaya | that explains so very much | 00:13 |
| fungi | we should ship mordred some scented candles and chamomile tea | 00:13 |
| mordred | anteaya: sometimes anger happens, usually right when you didn't want any | 00:13 |
| anteaya | fungi: sure to make a change | 00:13 |
| anteaya | I don't recall ever wanting any | 00:13 |
| mordred | anteaya: this is why "used it in anger" is used - because those are the times when things you've put together either work or don't | 00:14 |
| anteaya | shall we get back to our regularly scheduled downgrade | 00:14 |
| mordred | anteaya: things that work nice when things are nice are great and all | 00:14 |
| *** esker has quit IRC | 00:14 | |
| mordred | but things that work during unplanned chaos are the ones you want on your team | 00:14 |
| clarkb | ok I am at computer with food, anything I should do to help? | 00:14 |
| jeblair | did the timeout + false start just happen? | 00:14 |
| mordred | yes | 00:14 |
| fungi | okay, puppet apply timed out on the reindex exec as expected | 00:15 |
| anteaya | clarkb: https://etherpad.openstack.org/p/ZIepsHViGk | 00:15 |
| mordred | the timeout and false start has happened | 00:15 |
| jeblair | clarkb: we're just waiting now | 00:15 |
| mordred | yay. I like it when things happen that we expect to happen | 00:15 |
| * fungi is watching the process table but expects another ~10 minutes | 00:15 | |
| fungi | seems done? | 00:18 |
| *** unicell has quit IRC | 00:18 | |
| fungi | mordred: ready to try starting gerrit now? | 00:18 |
| mordred | fungi: I'd love to try starting gerrit | 00:19 |
| mordred | well that failed | 00:19 |
| *** sputnik13 has quit IRC | 00:19 | |
| jeblair | Caused by: java.lang.SecurityException: class "org.bouncycastle.util.io.TeeOutputStream"'s signer information does not match signer information of other classes in the same package | 00:19 |
| mordred | wat | 00:20 |
| jeblair | so probably some bundled jar cleanup needed? | 00:20 |
| fungi | 1) Error in custom provider, java.lang.SecurityException: class "org.bouncycastle.util.io.TeeOutputStream"'s signer information does not match signer information of other classes in the same package | 00:20 |
| fungi | yeah, looks like we missed the tidy | 00:20 |
| clarkb | jeblair: yes I think so | 00:20 |
| mordred | maybe run puppet again/ | 00:20 |
| mordred | ? | 00:20 |
| fungi | there is indeed a stray bcprov-jdk16-144.jar | 00:20 |
| jeblair | it may not run if it's triggered by init? | 00:20 |
| mordred | these questions are making me thirsty | 00:20 |
| jeblair | maybe try it again, but don't expect it to work and cleanup manually if needed | 00:21 |
| mordred | trying puppet again | 00:21 |
| fungi | also a stray mysql-connector-java-5.1.21.jar though it started fine with that during the upgrade and puppet cleaned it later | 00:21 |
| mordred | sstarting gerrit | 00:21 |
| fungi | looks cleaner this time | 00:21 |
| mordred | it says ok | 00:22 |
| jeblair | it seems to be up | 00:22 |
| fungi | so i think if the exec for the reindex times out, the tidy never gets triggered | 00:22 |
| mordred | I've logged in | 00:22 |
| mordred | and gerrit exists | 00:22 |
| mordred | so that's good | 00:22 |
| *** yamamoto has joined #openstack-infra | 00:22 | |
| *** mmedvede has joined #openstack-infra | 00:22 | |
| jeblair | oh yeah, All-Users | 00:22 |
| *** Swami has quit IRC | 00:22 | |
| fungi | heh, it's creating All-Users.git everywhere. we were going to delete that | 00:22 |
| mordred | gah | 00:23 |
| mordred | should I stop it? | 00:23 |
| mordred | stopping gerrit to remove some all-users carnage maybe | 00:23 |
| *** unicell has joined #openstack-infra | 00:23 | |
| jeblair | it also mentioned All-Projects which i find confusing | 00:24 |
| mordred | I think it always mentions that | 00:24 |
| mordred | and fails | 00:24 |
| mordred | ok - I moved all-users.git out of the way and restarted gerrit | 00:24 |
| jeblair | i'll move it out of the way of the local git replica | 00:25 |
| fungi | yeah, git01 has /var/lib/git/All-Projects.git and /var/lib/git/All-Users.git | 00:25 |
| jeblair | so, er, gerrit can create repos now? | 00:25 |
| jeblair | on replicas? | 00:25 |
| clarkb | neat, but also getting in the way :) | 00:25 |
| mordred | yeah | 00:26 |
| mordred | it can | 00:26 |
| fungi | yeah, i saw that mentioned as new in the changelog (can't remember if it was gerrit or jgit) | 00:26 |
| fungi | in good news, at least cgit isn't publishing them | 00:27 |
| fungi | though i expect they can still be cloned if you know to try | 00:27 |
| jeblair | so i'm guessing 2.10 did the actual creation and 2.8 just said it did? | 00:27 |
| jeblair | com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: No enum constant com.google.gerrit.reviewdb.client.AccountGroup.Type. | 00:27 |
| jeblair | i'm seeing that error ^ | 00:27 |
| mordred | uhm | 00:27 |
| *** aduarte has quit IRC | 00:27 | |
| mordred | ok - diving in to sourcecode | 00:27 |
| jeblair | ssh review gerrit query --all-approvals --patch-sets --format JSON project:'openstack/ironic' branch:master limit:100 --comments | 00:28 |
| jeblair | can repro with that ^ | 00:28 |
| mordred | k. one sec | 00:30 |
| mordred | there are new groups that were made that are missing a value | 00:30 |
| clarkb | thats related to the SYSTEM account type in the db that was removed. | 00:30 |
| clarkb | oh ok mordred is on it | 00:30 |
| * clarkb is just going to stay out of the way and is here if anyone needs extra hands or eyes | 00:30 | |
| mordred | ok. I just did | 00:31 |
| mordred | mysql> update account_groups set group_type='INTERNAL' where group_type=''; | 00:31 |
| mordred | jeblair: we may need to flush caches - but try repro-ing? | 00:31 |
| jeblair | i will flush caches | 00:31 |
| jeblair | mordred: fixed after flushing caches | 00:32 |
| mordred | did I ever mention that my first real job involved executing SQL on live (broken) production systems while angry VPs looked over my shoulder? | 00:32 |
| mordred | that's how I learned SQL, btw | 00:32 |
| mordred | abject. terror. | 00:33 |
| sdake_ | mordred, imo handthem the keyboard | 00:33 |
| mordred | sdake_: they were too busy being angry VPs | 00:33 |
| jeblair | sdake_: heh, "hand them a blunt object?" :) | 00:33 |
| mordred | however, that was the only job I've ever had where in addition to my PC, I had an actual X-Terminal as well | 00:34 |
| clarkb | jeblair: and in the case of our keyboards heavy blunt objects | 00:34 |
| fungi | that's always an effective way to learn anything | 00:34 |
| mordred | jeblair: we should figure out a way to get anyone in infra who wants one an x-terminal that connects to something | 00:35 |
| tchaypo | I wonder if I can hook my Wyse 85 up to my EliteBook | 00:35 |
| sdake_ | trial by fire ftw ;-) | 00:35 |
| fungi | especially when using a model m keyboard as a bludgeon | 00:35 |
| *** marun has joined #openstack-infra | 00:35 | |
| tchaypo | Surely anything called “elite” has to support a serial terminal | 00:35 |
| jeblair | tchaypo: lol! | 00:35 |
| clarkb | tchaypo: it probably has a modem in it | 00:35 |
| tchaypo | *checks* nope, no serial port. which means I’d have to go through a usb->serial dongle | 00:36 |
| tchaypo | clarkb: ooh, it actually does | 00:36 |
| fungi | tchaypo: good luck getting the kernel to add that as a boot-time console ;) | 00:36 |
| jeblair | tchaypo: apparently not an 3l33t800|< | 00:36 |
| mordred | zomg. we should have a special single-host chat network on a host that is only reachable via remote serial | 00:36 |
| openstackgerrit | James Polley proposed openstack-dev/pbr: WIP: Teach pbr to read extras from setup.cfg https://review.openstack.org/181795 | 00:36 |
| clarkb | mordred: cowsay | wall :) | 00:36 |
| fungi | so, post-upgrade carnage seems to have settled | 00:37 |
| greghaynes | tchaypo: dont those have some kind of pci expansion card slot? | 00:37 |
| mordred | that's what I'm talking about | 00:37 |
| fungi | mordred: via ax.25 packet radio | 00:37 |
| fungi | for extra geek cred | 00:37 |
| tchaypo | you people are sick | 00:37 |
| mordred | fungi: if we could get some actually vt220's ... | 00:37 |
| mordred | maybe we could program a full-screen menu-driven interface to OpenStack APIs that you can connect to with an VT220 | 00:38 |
| fungi | mordred: i had a closet full. i eventually junked them move before last because of the room they were taking up. i shed a few tears for them | 00:38 |
| openstackgerrit | greghaynes proposed openstack-infra/shade: Use fakes instead of mocks for dat objects https://review.openstack.org/182126 | 00:38 |
| anteaya | are we still downgrading or are we done? | 00:38 |
| fungi | mordred: ~2 dozen, a mix of ti and wyse terminals. dual-line rs232 support too | 00:38 |
| mordred | anteaya: I believe now we're in the stage called "post terror" | 00:38 |
| jeblair | so, er, gerrit is working and not throwing jgit-zlib errors. so either we're good, or we haven't run long enough to see the problem, or it's US-EOD and we're not generating enough load. | 00:39 |
| anteaya | which I will call done for the moment | 00:39 |
| mordred | anteaya: or, if this were a show, we'd be in the green room eating the catering because the artists would have left the venue | 00:39 |
| anteaya | jhesketh: do you have any questions? | 00:39 |
| openstackgerrit | greghaynes proposed openstack-infra/shade: Use fakes instead of mocks for data objects https://review.openstack.org/182126 | 00:39 |
| clarkb | I keep wanting to stuff my beagleboard into an old terminal and have it be a pianoar jukebox | 00:39 |
| anteaya | jhesketh: as any odd things will be happening on your watch | 00:39 |
| anteaya | mordred: there we go, symbology I understand | 00:39 |
| mordred | anteaya: oh. well, remind me next time and I'll translate to stagehand | 00:40 |
| anteaya | mordred: can do | 00:40 |
| *** baoli has joined #openstack-infra | 00:40 | |
| jhesketh | I don't have any questions but there is also little I can do if things go haywire again | 00:40 |
| anteaya | jhesketh: fair enough | 00:40 |
| mordred | jhesketh: you can drink | 00:40 |
| clarkb | jeblair: should we generate some load? | 00:40 |
| fungi | clarkb: i finally got over my desire to hang onto antiquated computer equipment, and have instead resolved to turn myself into antiquated computer equipment | 00:40 |
| mtreinish | clarkb: I did that with an itanium in college. Pianobar worked surprisingly well on ia64 | 00:41 |
| jhesketh | Heh, little constructive I can do | 00:41 |
| anteaya | ha ha ha | 00:41 |
| fungi | jhesketh: fwiw i will make sure to check in as early as i wake up and see where we are | 00:41 |
| clarkb | we can probably pull a dansmith and push about 50 stacked changes to nova | 00:41 |
| fungi | clarkb: just ask him to rebase one of his | 00:41 |
| jhesketh | I'll do what I can though. But to be clear I wasn't going to be around much today due to personal circumstances | 00:41 |
| clarkb | 181682 should merge shortly (another thing to check the status of) | 00:41 |
| anteaya | jhesketh: yes you did say that earlier | 00:42 |
| dansmith | I don't have any big sets pending I could rebase "for testing" :P | 00:42 |
| anteaya | thanks for the reminder | 00:42 |
| clarkb | dansmith: off your game | 00:42 |
| jhesketh | Also, FYI I'm an apology for tomorrow's meeting | 00:42 |
| dansmith | clarkb: *hangs head* | 00:42 |
| anteaya | jhesketh: thanks for the heads up | 00:42 |
| anteaya | jhesketh: when do you depart for vancouver? | 00:42 |
| clarkb | https://review.openstack.org/#/c/181682/ that merged so we can test, merge, and report | 00:43 |
| jhesketh | Saturday morning | 00:43 |
| *** baoli has quit IRC | 00:43 | |
| jeblair | my plan to prep for tomorrow's meeting today obviously didn't work out | 00:43 |
| anteaya | jhesketh: cool, so in saturday afternoon in Vancouver? | 00:43 |
| clarkb | my plan to check new virtualenv today got postponed | 00:43 |
| clarkb | I should probably spin up a new image with that now? | 00:43 |
| *** baoli has joined #openstack-infra | 00:43 | |
| anteaya | jeblair: but you had a plan | 00:43 |
| jhesketh | Night time I believe | 00:43 |
| anteaya | cool | 00:43 |
| jeblair | i have a draft message to -dev in the bottom of https://etherpad.openstack.org/p/ZIepsHViGk | 00:44 |
| clarkb | I am going to make some bare-trusty images so that I can fix before sdague wakes up if this makes everything worse | 00:44 |
| *** packet has joined #openstack-infra | 00:44 | |
| *** otter768 has joined #openstack-infra | 00:44 | |
| mordred | clarkb: oh - so ... if you want to play with images, btw ... | 00:44 |
| mordred | clarkb: when I was deleting images because of a different break earlier ... | 00:44 |
| harlowja | hmmm, has anyone seen 'https://review.openstack.org/#/q/project:openstack/taskflow' now gets me 'line 0:-1 no viable alternative at input '<EOF>'' :-/ | 00:44 |
| mordred | clarkb: I discovered that we have several images in the 1000 hours age range | 00:44 |
| fungi | jeblair: ml update lgtm | 00:44 |
| clarkb | mordred: fun, I can likely take a look at that too | 00:45 |
| mordred | harlowja: I agree with you | 00:45 |
| harlowja | lol | 00:45 |
| harlowja | k | 00:45 |
| mordred | harlowja: I also get that error | 00:45 |
| anteaya | jeblair: draft email looks good to me | 00:45 |
| jeblair | harlowja: bat url | 00:45 |
| jeblair | bad url | 00:45 |
| clarkb | harlowja: https://review.openstack.org/#/q/project:openstack/taskflow,n,z that works fine | 00:45 |
| jeblair | https://review.openstack.org/#/q/project:openstack/taskflow,n,z | 00:45 |
| clarkb | yes what jeblair said | 00:45 |
| harlowja | hmmm, guess i had something messed up | 00:45 |
| harlowja | k | 00:45 |
| harlowja | thx | 00:45 |
| jeblair | harlowja: yeah, the ,n,z is like the terminating null for the query string :) | 00:45 |
| mordred | https://review.openstack.org/#/q/project:openstack/taskflow,n,z | 00:45 |
| mordred | yah | 00:46 |
| harlowja | :( | 00:46 |
| mordred | gah | 00:46 |
| fungi | it's possible gerrit 2.10 was normalizing urls people were browsing and now those are invalid under 2.8 | 00:46 |
| mordred | yah | 00:46 |
| jeblair | isn't that cool? null-terminated URLs? :) | 00:46 |
| harlowja | jeblair it's something | 00:46 |
| harlowja | from a google product, u'd think they would know urls better | 00:46 |
| harlowja | *from a google opensource project | 00:46 |
| *** ddieterly has joined #openstack-infra | 00:47 | |
| clarkb | the number of insecure platform warnings is amazing | 00:47 |
| mordred | clarkb: yah. uhm . | 00:48 |
| mordred | clarkb: maybe we shoudl upgrade the libraries. also, didnt' we push patches for nodepool to help some of those? | 00:48 |
| clarkb | mordred: you abandoned them | 00:48 |
| mordred | clarkb: I did? | 00:48 |
| mordred | clarkb: that was silly of me | 00:48 |
| mordred | clarkb: because this is annoying | 00:49 |
| jeblair | ml msg sent | 00:49 |
| clarkb | pretty sure, I went hunting to merge them last time I ran into this real hard and seem to have found them abandoned | 00:49 |
| mordred | clarkb: oh - it's because we're using shade to get the client objects RSN isn't it? | 00:49 |
| *** otter768 has quit IRC | 00:49 | |
| clarkb | mordred: except we said that months ago :) | 00:49 |
| *** dboik_ has quit IRC | 00:49 | |
| mordred | yah. :) | 00:49 |
| mordred | to be fair- I do believe that's actually the next patch to land | 00:49 |
| *** ociuhandu has quit IRC | 00:49 | |
| mordred | and greghaynes has it ready to go | 00:49 |
| clarkb | also I am not sure how shade fixes this | 00:50 |
| greghaynes | yep! was going to beg for reviews today but... well, youall know | 00:50 |
| clarkb | since the warnings would be triggered in the clients made by shade | 00:50 |
| mordred | https://review.openstack.org/#/c/168603/ | 00:50 |
| jeblair | status ok Gerrit has been downgraded to version 2.8 due to the issues observed today. Please report further problems in #openstack-infra. | 00:50 |
| clarkb | jeblair: lgtm | 00:50 |
| *** shashankhegde has quit IRC | 00:50 | |
| mordred | clarkb: shade a) depends on the newer versions of things and b) squelches the supider warnings | 00:50 |
| anteaya | in the conversation of what to do next I would like to propose upgrading to gerrit 2.9 | 00:51 |
| anteaya | just to have that on the table | 00:51 |
| clarkb | greghaynes: mordred why does BAD_CLIENT init on import but FAKE_CLIENT does not? | 00:51 |
| mordred | anteaya: well, honestly - I think the next key is to figure out how to reproduce the problem | 00:51 |
| anteaya | yes it is | 00:52 |
| anteaya | but how long to do that is unknown | 00:52 |
| jeblair | yeah, i am not interested in debugging this in production | 00:52 |
| *** dboik_ has joined #openstack-infra | 00:52 | |
| anteaya | no argument | 00:52 |
| clarkb | ++ I can work on getting nova into review-dev | 00:52 |
| clarkb | or neutron or ironic, basically non trivial git repo | 00:52 |
| mordred | clarkb: I think we should turn review-dev into a complete replica | 00:52 |
| jeblair | clarkb: it eventually affected smaller repos too | 00:52 |
| jeblair | clarkb: 11 total | 00:53 |
| mordred | it's been mildly different for a little too long | 00:53 |
| jeblair | and i think the smallest packfile was 1MB | 00:53 |
| mordred | might be time for a tear down and recreate, and a script to sync all the things from review? | 00:53 |
| jeblair | #status ok Gerrit has been downgraded to version 2.8 due to the issues observed today. Please report further problems in #openstack-infra. | 00:53 |
| openstackstatus | jeblair: sending ok | 00:53 |
| greghaynes | clarkb: There was sume rebase funkyness there, I should make that symmetric | 00:53 |
| clarkb | greghaynes: mordred also have either of you tested that against devstack with my intructions to do so? | 00:53 |
| mordred | clarkb: you have instructions to test nodepool against devstack? because we should turn those into a functional test ... | 00:54 |
| greghaynes | clarkb: basically the FAKE_CLIENT stuff was added to master since that patch was made, I had review pushback about initializing client classes at import time | 00:54 |
| mordred | clarkb: I'll attack that tomorrow | 00:54 |
| clarkb | mordred: yes functional tests is next step | 00:54 |
| mordred | sweet | 00:55 |
| clarkb | mordred: problem is its hella slow because nested virt | 00:55 |
| greghaynes | but looks like the FAKE_CLIENT that got merged didnt have the same review pushback | 00:55 |
| mordred | clarkb: ah. yes. I'd imagine so | 00:55 |
| clarkb | mordred: that and pleia2 keeps finding problems iwth them :) | 00:55 |
| mordred | :) | 00:55 |
| mordred | pleia2 gets a gold star | 00:55 |
| mordred | ok. I'm going to dinner. it's food o'clock | 00:55 |
| greghaynes | clarkb: I have not tested against devstack | 00:55 |
| *** ChanServ changes topic to "Discussion of OpenStack Developer and Community Infrastructure | docs http://docs.openstack.org/infra/manual/ http://ci.openstack.org/ | bugs https://storyboard.openstack.org/ | source https://git.openstack.org/cgit/openstack-infra/" | 00:55 | |
| -openstackstatus- NOTICE: Gerrit has been downgraded to version 2.8 due to the issues observed today. Please report further problems in #openstack-infra. | 00:55 | |
| clarkb | ok so I think testing against devstack should be next step there, happy to work with whoever to do that | 00:55 |
| greghaynes | clarkb: is it crazy to test against hpcloud / rax? | 00:56 |
| greghaynes | because, why not | 00:56 |
| greghaynes | my quota should be big enough to make a couple images and nodes... | 00:57 |
| clarkb | greghaynes: I think the idea is to start with a known quantity that we have control over | 00:57 |
| clarkb | greghaynes: becuase as soon as you deal with rax/hpcloud its all bonghits the whole way down | 00:57 |
| greghaynes | heh | 00:57 |
| openstackstatus | jeblair: finished sending ok | 00:58 |
| clarkb | greghaynes: mordred will have a couple comments as soon as I can check one more thing | 00:58 |
| greghaynes | ok | 00:58 |
| clarkb | greghaynes: not really a fan of the use_fakes flag, do you prefer that over just overriding setUpFakes as appropriate? | 01:01 |
| clarkb | greghaynes: method resolution seems like a better solution here to me but I may be missing something | 01:01 |
| greghaynes | clarkb: I have basically 0 opinion either way | 01:02 |
| *** achanda has joined #openstack-infra | 01:03 | |
| clarkb | ok, my only concern is that setUp() is an api we don't define so its weird to change it | 01:03 |
| greghaynes | clarkb: yea, I think youre right, overriding is cleaner | 01:03 |
| greghaynes | clarkb: my biggest question is whether theres other cases I need to test | 01:04 |
| greghaynes | since I only hit those two | 01:04 |
| greghaynes | not sure if theres other config overriding cases we expect to work | 01:04 |
| clarkb | I think every test goes through the methods in the base/db classes to get configs | 01:05 |
| openstackgerrit | Steven Dake proposed openstack-infra/project-config: Add stable/kilo to kolla notifications https://review.openstack.org/182133 | 01:05 |
| *** prad has quit IRC | 01:05 | |
| greghaynes | clarkb: I mean provider configs | 01:06 |
| greghaynes | clarkb: right now im testsing .config/openstack and a nodepool.yaml specified config | 01:06 |
| greghaynes | er, provider config | 01:06 |
| greghaynes | supposedly env vars dont matter? | 01:07 |
| clarkb | ya no env vars | 01:07 |
| clarkb | ok commented | 01:08 |
| *** achanda has quit IRC | 01:08 | |
| clarkb | -1 for the BAD_CLIENT and missing provider.cloud check for reconfiguration, the other things are suggestions I don't feel super strongly about | 01:08 |
| *** sarob has quit IRC | 01:09 | |
| *** mtanino has quit IRC | 01:10 | |
| openstackgerrit | Steven Dake proposed openstack-infra/project-config: Add stable/kilo for all stable branches for kolla https://review.openstack.org/182133 | 01:10 |
| greghaynes | clarkb: ok | 01:10 |
| *** achanda has joined #openstack-infra | 01:11 | |
| *** gyee has quit IRC | 01:12 | |
| clarkb | but otherwise this looks good and even comes with new tests :) | 01:13 |
| greghaynes | clarkb: yea, turns out it was scarily broken | 01:13 |
| greghaynes | so wooo | 01:13 |
| openstackgerrit | James Polley proposed openstack-dev/pbr: WIP: Teach pbr to read extras from setup.cfg https://review.openstack.org/181795 | 01:14 |
| clarkb | bare-centos6 is still puppetting in hpcloud haven't discovered why we can't build them yet | 01:15 |
| clarkb | and new bare-trusty should be online in rax momentarily, will work to confirm latest virtualenv is in use once that is done | 01:15 |
| clarkb | jeblair: mordred fungi zaro et al thanks a bunch for jumping on the gerrit thing this afternoon when I wasn't able to be around | 01:15 |
| fungi | clarkb: you can't _always_ be available. i'm sure you'll get the next one | 01:16 |
| *** banix has quit IRC | 01:17 | |
| clarkb | http://paste.openstack.org/show/220338/ is the centos6 issue, I am guessing that that dir has gone away for some reason :/ | 01:17 |
| clarkb | which is odd beacuse the glance image's name doesn't appear to have changed | 01:17 |
| *** enikanorov2 has quit IRC | 01:17 | |
| *** banix has joined #openstack-infra | 01:17 | |
| clarkb | anyways thats a breadcrumb I can follow up on | 01:17 |
| greghaynes | clarkb: theres a fix up for that | 01:18 |
| clarkb | even better | 01:19 |
| greghaynes | clarkb: https://review.openstack.org/#/c/181750/ | 01:19 |
| *** dboik_ has quit IRC | 01:19 | |
| clarkb | greghaynes: I don't think thats the fix, since the error above happens in not dib builds | 01:20 |
| greghaynes | clarkb: hah | 01:20 |
| clarkb | greghaynes: its happening on the snapshot VM while building | 01:20 |
| greghaynes | clarkb: so, same thing then | 01:20 |
| greghaynes | but yea | 01:20 |
| clarkb | so cloud init is getting disabled but its not installed? thats weird | 01:20 |
| clarkb | since I thought hpcloud base images required cloud init | 01:20 |
| greghaynes | clarkb: yea... maybe its not on centos6? | 01:20 |
| clarkb | possible, I will have to boot an image there and poke around | 01:21 |
| openstackgerrit | Martin André proposed openstack-infra/project-config: Add notifications for all stable branches for kolla https://review.openstack.org/182133 | 01:21 |
| *** tiswanso has joined #openstack-infra | 01:22 | |
| *** patrickeast has quit IRC | 01:22 | |
| *** tiswanso_ has joined #openstack-infra | 01:24 | |
| *** zz_dimtruck is now known as dimtruck | 01:24 | |
| *** shashankhegde has joined #openstack-infra | 01:25 | |
| *** redrobot has quit IRC | 01:25 | |
| *** stevemar has joined #openstack-infra | 01:25 | |
| *** redrobot has joined #openstack-infra | 01:26 | |
| *** redrobot is now known as Guest20306 | 01:26 | |
| *** dstanek has quit IRC | 01:26 | |
| *** dstanek has joined #openstack-infra | 01:26 | |
| *** alexsyip has quit IRC | 01:27 | |
| *** tiswanso has quit IRC | 01:27 | |
| *** otter768 has joined #openstack-infra | 01:28 | |
| jeblair | clarkb: np. and hey, the next gerrit downgrade is all yours if you want it. ;) | 01:28 |
| clarkb | ha | 01:28 |
| jeblair | still not seeing any pack-related exceptions | 01:29 |
| clarkb | I have bare-trusty node in rax-dfw building with new virtualenv, should know shortly how that does | 01:29 |
| fungi | i have evening chores i should switch to. things seem to have calmed down | 01:29 |
| clarkb | I will pop in and out until I get ETIRED so can field questions if they come up | 01:31 |
| *** shashankhegde has quit IRC | 01:31 | |
| *** zhiwei has joined #openstack-infra | 01:32 | |
| *** dimtruck is now known as zz_dimtruck | 01:33 | |
| *** weshay has quit IRC | 01:33 | |
| *** weshay has joined #openstack-infra | 01:34 | |
| openstackgerrit | Merged openstack-infra/project-config: update akanda acls to support branching https://review.openstack.org/182079 | 01:34 |
| *** wenlock has joined #openstack-infra | 01:36 | |
| openstackgerrit | James Polley proposed openstack-dev/pbr: WIP: Teach pbr to read extras from setup.cfg https://review.openstack.org/181795 | 01:38 |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate: [WIP] run centos7 with neutron https://review.openstack.org/179689 | 01:40 |
| *** marun has quit IRC | 01:43 | |
| *** ashleighfarnham has joined #openstack-infra | 01:44 | |
| fungi | oh, btw i will be awol for a few hours tomorrow, but hope to return in time for the infra meeting | 01:44 |
| fungi | that's the earliest we could book the home inspector we wanted. i was originally wanting it to be today, but in retrospect i'm sort of glad that didn't work out | 01:45 |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/devstack-gate: Handle the case of REMAINING_TIME <= 0 https://review.openstack.org/178043 | 01:48 |
| *** baoli has quit IRC | 01:49 | |
| *** weshay has quit IRC | 01:51 | |
| jeblair | yeah, i'm going to bail after the meetings tomorrow to do this evening's postponed errands | 01:52 |
| *** baoli has joined #openstack-infra | 01:52 | |
| clarkb | at least non pypy jobs seem to be running happily with newer virtualenv | 01:54 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 01:54 | |
| clarkb | haven't found a pypy job that ran on a node with the new virtualenv yet | 01:54 |
| clarkb | but I think thats enough to make me happy for tomorrow morning. Now to boot a centos6 image in hpcloud | 01:54 |
| *** weshay has joined #openstack-infra | 01:55 | |
| clarkb | according to glance this image is a couple years old... | 01:56 |
| *** banix has quit IRC | 01:58 | |
| fungi | ha | 01:59 |
| fungi | something tells me that's slightly inaccurate | 01:59 |
| *** packet has quit IRC | 02:00 | |
| *** unicell has quit IRC | 02:02 | |
| *** hdd has quit IRC | 02:03 | |
| *** sarob has joined #openstack-infra | 02:03 | |
| *** dboik has joined #openstack-infra | 02:07 | |
| *** dboik has quit IRC | 02:07 | |
| *** achanda has quit IRC | 02:07 | |
| *** dboik has joined #openstack-infra | 02:08 | |
| *** bknudson has quit IRC | 02:08 | |
| *** achanda has joined #openstack-infra | 02:09 | |
| clarkb | so this centos6 image does have /etc/cloud/cloud.cfg.d | 02:11 |
| clarkb | and if I run the config update dd for it by hand that works, so something earlier in the image build must be breaking us | 02:13 |
| fungi | or trying to refer to it by its chroot path rather than its host system path? | 02:13 |
| fungi | that trips me up regularly with dib since i have trouble remembering what stages run in the chroot and what run from outside it | 02:14 |
| clarkb | no this is a snapshot build | 02:15 |
| clarkb | so no chroots involved | 02:15 |
| *** sarob has quit IRC | 02:15 | |
| clarkb | wild guess is going to be selinux | 02:15 |
| fungi | oh, right | 02:17 |
| fungi | it's clearly getting late over here ;) | 02:17 |
| clarkb | running our yum update now to see if that changes things | 02:19 |
| clarkb | nope that doesn't break it | 02:20 |
| clarkb | so next best guess is puppet and/or selinux, that will take more time to setup so will save for morning | 02:21 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 02:21 | |
| *** craigbr has joined #openstack-infra | 02:22 | |
| *** wenlock has quit IRC | 02:23 | |
| *** craigbr has quit IRC | 02:31 | |
| *** yamahata has quit IRC | 02:31 | |
| *** bhunter71 has joined #openstack-infra | 02:31 | |
| clarkb | wow ok its early on in the process before puppet runs | 02:33 |
| clarkb | we rmove python-setuptools so it removes cloud init | 02:34 |
| clarkb | which we don't actually need we just need to guard against it | 02:34 |
| clarkb | except how do rax images work in that case since they do need cloud init? | 02:35 |
| clarkb | mordred: ^ ? | 02:35 |
| *** banix has joined #openstack-infra | 02:38 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Don't disable cloud-init if not installed part 2 https://review.openstack.org/182142 | 02:41 |
| clarkb | there we go | 02:41 |
| *** ashleighfarnham has quit IRC | 02:44 | |
| *** dims has quit IRC | 02:45 | |
| *** packet has joined #openstack-infra | 02:45 | |
| clarkb | greghaynes: ^ you reviewed the earlier changes in the stack | 02:45 |
| *** rfolco has quit IRC | 02:46 | |
| *** sputnik13 has joined #openstack-infra | 02:46 | |
| *** packet has quit IRC | 02:47 | |
| *** BharatK has quit IRC | 02:48 | |
| greghaynes | lgtm | 02:49 |
| *** ivar-laz_ has joined #openstack-infra | 02:51 | |
| *** ivar-laz_ has quit IRC | 02:52 | |
| *** ivar-lazzaro has quit IRC | 02:55 | |
| *** koolhead17 has joined #openstack-infra | 02:56 | |
| *** julim has joined #openstack-infra | 02:57 | |
| yamamoto | can anyone instruct me a proper way to create a stable branch for a stackforge repo? | 02:58 |
| yamamoto | in my case it's stackforge/networking-ofagent | 02:58 |
| yamamoto | do i need a change like this? https://review.openstack.org/#/c/176724/ | 02:58 |
| clarkb | yamamoto: looking at https://review.openstack.org/#/admin/projects/stackforge/networking-ofagent,access you don't have create permissions so yes you will need a change like that to add the creat epermissions | 02:59 |
| clarkb | I found a pypy job that passed on rax-dfw bare-trusty so I think new virtualenv fixes that | 03:00 |
| clarkb | http://logs.openstack.org/28/181228/2/gate/gate-oslotest-pypy/e974749/console.html | 03:00 |
| yamamoto | clarkb: thank you. i'm wondering why eg. neutron.config doesn't need it. | 03:00 |
| *** sarob has joined #openstack-infra | 03:00 | |
| clarkb | yamamoto: neutron does not need it because the openstack release managers get those permissions for neutron and other openstack projects in our global All-Projects acl config | 03:01 |
| yamamoto | clarkb: makes sense. thank you! | 03:01 |
| *** hdd has joined #openstack-infra | 03:01 | |
| *** baoli has quit IRC | 03:02 | |
| *** oomichi has quit IRC | 03:03 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config: Add "create" permission for networking-ofagent https://review.openstack.org/182144 | 03:05 |
| *** jamesmcarthur has joined #openstack-infra | 03:05 | |
| *** baoli has joined #openstack-infra | 03:12 | |
| *** ddieterly has quit IRC | 03:13 | |
| *** panda|afk has quit IRC | 03:13 | |
| *** panda|afk has joined #openstack-infra | 03:14 | |
| *** sputnik13 has quit IRC | 03:14 | |
| *** sputnik13 has joined #openstack-infra | 03:16 | |
| *** dstanek has quit IRC | 03:16 | |
| *** peterstac has quit IRC | 03:16 | |
| *** niedbalski_ has quit IRC | 03:17 | |
| *** schang has quit IRC | 03:18 | |
| *** schang has joined #openstack-infra | 03:18 | |
| *** niedbalski_ has joined #openstack-infra | 03:19 | |
| *** dstanek has joined #openstack-infra | 03:19 | |
| *** peterstac has joined #openstack-infra | 03:19 | |
| *** sputnik13 has quit IRC | 03:20 | |
| *** bhunter71 has quit IRC | 03:23 | |
| *** tiswanso_ has quit IRC | 03:25 | |
| *** fedexo has joined #openstack-infra | 03:28 | |
| *** baoli has quit IRC | 03:28 | |
| *** amotoki_ has joined #openstack-infra | 03:30 | |
| *** stevemar has quit IRC | 03:31 | |
| *** sarob has quit IRC | 03:36 | |
| *** sarob has joined #openstack-infra | 03:36 | |
| *** marcusvrn has joined #openstack-infra | 03:38 | |
| *** yamahata has joined #openstack-infra | 03:40 | |
| *** otter768 has quit IRC | 03:44 | |
| *** sarob has quit IRC | 03:45 | |
| *** dalgaaf has quit IRC | 03:47 | |
| stevebaker | when I log into gerrit it gives me an eternal "Loading Gerrit Code Review ..." screen. is that expected? | 03:49 |
| clarkb | stevebaker does a hard refresh correct it? | 03:51 |
| clarkb | I wonder if you have newer gerrit client js cached and need to force it back to 2.8 | 03:53 |
| *** marcusvrn has quit IRC | 03:54 | |
| sigmavirus24 | oh man. the review.o.o landing page is so boring again | 03:56 |
| stevebaker | clarkb: nope, empty cache and hard reload -> same result. When it was upgraded I did change some of my settings (switched to new view, tweaked my menues) | 03:57 |
| clarkb | new view should work, and I added a menu but maybe thats related | 03:58 |
| clarkb | is it the change view that fails or the main page? both? | 03:59 |
| *** wenlock has joined #openstack-infra | 03:59 | |
| *** weshay has quit IRC | 03:59 | |
| *** miqui has joined #openstack-infra | 04:00 | |
| stevebaker | clarkb: same issue with firefox, so I don't think it is local caching. It works fine if I clear the cookie and don't log in. But as soon as I log in then main page and change views are bust | 04:01 |
| *** jamespage has quit IRC | 04:05 | |
| *** gnuoy has quit IRC | 04:05 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 04:05 | |
| *** gnuoy has joined #openstack-infra | 04:06 | |
| clarkb | does https://review.openstack.org/#/settings/ load? | 04:06 |
| *** jamespage has joined #openstack-infra | 04:06 | |
| clarkb | but my guess is we may have to surgery your account ^ that page should give us an account number to do it on | 04:07 |
| stevebaker | clarkb: nope, doesn't load | 04:08 |
| *** banix has quit IRC | 04:09 | |
| clarkb | not seeing any obvious errors in the logs related to you | 04:12 |
| clarkb | stevebaker: can you try loading a change and let me know the change number? | 04:12 |
| *** camunoz has quit IRC | 04:12 | |
| *** gsagie has left #openstack-infra | 04:13 | |
| stevebaker | clarkb: lets do https://review.openstack.org/#/c/172386/ | 04:13 |
| clarkb | ok | 04:13 |
| *** ddieterly has joined #openstack-infra | 04:14 | |
| clarkb | haven't seen anything for that change yet and after digging in the db I have found your account, it looks ok to me | 04:16 |
| *** ajmiller has quit IRC | 04:17 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate: [WIP] run centos7 with neutron https://review.openstack.org/179689 | 04:17 |
| stevebaker | clarkb: refreshing a bunch of times | 04:18 |
| *** ddieterly has quit IRC | 04:18 | |
| *** jamesmcarthur has quit IRC | 04:18 | |
| clarkb | still nothing | 04:19 |
| clarkb | stevebaker: can you use chrom* developer tools or firebug to see what your side is trying to do? | 04:19 |
| stevebaker | clarkb: let me just try while logged out, just to see what the browser is loading | 04:20 |
| stevebaker | clarkb: snap | 04:20 |
| openstackgerrit | greghaynes proposed openstack-infra/shade: Use fakes instead of mocks for data objects https://review.openstack.org/182126 | 04:21 |
| *** stevebaker has quit IRC | 04:22 | |
| *** stevebaker has joined #openstack-infra | 04:22 | |
| *** btully has joined #openstack-infra | 04:22 | |
| openstackgerrit | greghaynes proposed openstack-infra/shade: Use fakes instead of mocks for data objects https://review.openstack.org/182126 | 04:23 |
| *** asettle has joined #openstack-infra | 04:24 | |
| *** koolhead17 has quit IRC | 04:24 | |
| stevebaker | clarkb: not logged in http://snag.gy/M7YFW.jpg | 04:25 |
| stevebaker | clarkb: logged in http://snag.gy/2RwJi.jpg | 04:25 |
| clarkb | so that just never grabs the change detail, weird | 04:26 |
| stevebaker | yeah, it just stops | 04:27 |
| clarkb | the change detail request you see on the not logged in example is one of the more important steps, that pulls in much of the info you see on the main change page | 04:27 |
| stevebaker | clarkb: I have a Console error, Uncaught Class$S2141... | 04:29 |
| *** camunoz has joined #openstack-infra | 04:29 | |
| stevebaker | clarkb: only on the logged in one | 04:29 |
| clarkb | and that must be from BF27A....cache.html ? | 04:30 |
| stevebaker | yup. I know enough about gwt to not even bother looking | 04:30 |
| clarkb | but that explains why I don't see errors if they are happening in the client | 04:30 |
| greghaynes | while looking at my console I noticed FF is complaining about our SSL cert | 04:30 |
| greghaynes | due to SHA1 | 04:31 |
| clarkb | greghaynes: it expires next month yes? | 04:31 |
| * clarkb double checks | 04:31 | |
| greghaynes | 7/29/15 | 04:31 |
| clarkb | 2 months, we can fix then | 04:31 |
| *** ildikov has quit IRC | 04:33 | |
| clarkb | stevebaker: can you write a response to the gerrit upgrade/rollback thread with the info above? Its getting late here not going to be able to help debug much more tonight but don't want this to get lost | 04:33 |
| *** dims has joined #openstack-infra | 04:33 | |
| clarkb | stevebaker: as an alternative, you might try gertty? if this is a client issue that should work around it | 04:33 |
| clarkb | stevebaker: or you can use the ssh api too | 04:33 |
| *** reed_ has joined #openstack-infra | 04:36 | |
| *** sdake_ has quit IRC | 04:37 | |
| *** dims has quit IRC | 04:38 | |
| *** camunoz has quit IRC | 04:40 | |
| *** jaypipes has quit IRC | 04:40 | |
| stevebaker | clarkb: are you sure there is no blob of json or somesuch in my user prefs record which could be cleared? I'm guessing I'm one of the few users to futz with my settings while it was on the upgraded version | 04:41 |
| clarkb | stevebaker: its just a few flags in a db table from what I can see | 04:42 |
| *** reed_ has quit IRC | 04:43 | |
| *** ildikov has joined #openstack-infra | 04:46 | |
| *** shashankhegde has joined #openstack-infra | 04:50 | |
| *** mrmartin has joined #openstack-infra | 04:52 | |
| *** camunoz has joined #openstack-infra | 04:52 | |
| *** stevemar has joined #openstack-infra | 04:54 | |
| *** markvoelker has joined #openstack-infra | 04:54 | |
| *** ashleighfarnham has joined #openstack-infra | 04:54 | |
| *** sabeen2 has quit IRC | 04:55 | |
| *** jaypipes has joined #openstack-infra | 04:56 | |
| *** mahito has joined #openstack-infra | 04:57 | |
| *** mahito has quit IRC | 04:58 | |
| *** mahito has joined #openstack-infra | 04:58 | |
| *** ildikov has quit IRC | 04:59 | |
| *** yfried has quit IRC | 05:01 | |
| *** sks has joined #openstack-infra | 05:01 | |
| *** hdd has quit IRC | 05:03 | |
| *** amitgandhinz has quit IRC | 05:06 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate: [WIP] run centos7 with neutron https://review.openstack.org/179689 | 05:06 |
| *** ayoung has quit IRC | 05:06 | |
| *** freerunner has joined #openstack-infra | 05:07 | |
| *** teran has joined #openstack-infra | 05:08 | |
| *** freerunner has quit IRC | 05:11 | |
| *** unicell has joined #openstack-infra | 05:14 | |
| *** ddieterly has joined #openstack-infra | 05:14 | |
| *** Krinkle is now known as Krinkle|detached | 05:18 | |
| *** ddieterly has quit IRC | 05:19 | |
| *** deepakcs has joined #openstack-infra | 05:24 | |
| *** asettle has quit IRC | 05:30 | |
| *** armax has quit IRC | 05:34 | |
| *** ildikov has joined #openstack-infra | 05:38 | |
| *** otter768 has joined #openstack-infra | 05:45 | |
| *** hichihara has quit IRC | 05:47 | |
| *** unicell has quit IRC | 05:50 | |
| *** otter768 has quit IRC | 05:50 | |
| *** unicell has joined #openstack-infra | 05:50 | |
| *** markus_z has joined #openstack-infra | 05:51 | |
| openstackgerrit | Ruslan Kamaldinov proposed openstack-infra/project-config: Add functional test job for python-muranoclient https://review.openstack.org/182082 | 05:56 |
| openstackgerrit | Ruslan Kamaldinov proposed openstack-infra/project-config: Add functional test job for python-muranoclient https://review.openstack.org/182082 | 05:58 |
| *** koolhead17 has joined #openstack-infra | 06:00 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/182158 | 06:01 |
| *** unicell1 has joined #openstack-infra | 06:03 | |
| *** unicell has quit IRC | 06:04 | |
| *** dstanek has quit IRC | 06:08 | |
| *** dstanek has joined #openstack-infra | 06:09 | |
| *** ibiris_away is now known as ibiris | 06:10 | |
| openstackgerrit | Steve Kowalik proposed openstack-infra/infra-specs: Add a specification about building packages https://review.openstack.org/179713 | 06:12 |
| *** scheuran has joined #openstack-infra | 06:14 | |
| *** mmedvede has quit IRC | 06:15 | |
| *** ddieterly has joined #openstack-infra | 06:15 | |
| *** markvoelker has quit IRC | 06:15 | |
| *** afazekas_ has joined #openstack-infra | 06:16 | |
| *** markvoelker has joined #openstack-infra | 06:18 | |
| *** coolsvap|afk is now known as coolsvap | 06:18 | |
| *** ddieterly has quit IRC | 06:20 | |
| *** coolsvap is now known as coolsvap|afk | 06:21 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/182158 | 06:23 |
| *** mmedvede has joined #openstack-infra | 06:24 | |
| *** coolsvap|afk has quit IRC | 06:28 | |
| openstackgerrit | Ruslan Kamaldinov proposed openstack-infra/project-config: Add functional test job for python-muranoclient https://review.openstack.org/182082 | 06:28 |
| *** asettle has joined #openstack-infra | 06:28 | |
| *** coolsvap has joined #openstack-infra | 06:29 | |
| *** sdake_ has joined #openstack-infra | 06:38 | |
| *** soren_ has joined #openstack-infra | 06:38 | |
| *** ashleighfarnham has quit IRC | 06:40 | |
| *** HeOS has quit IRC | 06:40 | |
| openstackgerrit | Merged openstack-infra/project-config: Register openstack-infra-incident with accessbot https://review.openstack.org/182086 | 06:41 |
| *** BharatK has joined #openstack-infra | 06:47 | |
| openstackgerrit | Ruslan Kamaldinov proposed openstack-infra/project-config: Add functional test job for python-muranoclient https://review.openstack.org/182082 | 06:47 |
| *** shashankhegde has quit IRC | 06:49 | |
| *** amitgandhinz has joined #openstack-infra | 06:50 | |
| *** Ala has joined #openstack-infra | 06:53 | |
| *** stevemar has quit IRC | 06:57 | |
| *** jcoufal has joined #openstack-infra | 06:59 | |
| *** mpavone has joined #openstack-infra | 06:59 | |
| *** wenlock has quit IRC | 07:04 | |
| *** asettle has quit IRC | 07:06 | |
| *** panda|afk has quit IRC | 07:13 | |
| *** panda|afk has joined #openstack-infra | 07:14 | |
| openstackgerrit | Merged openstack-infra/system-config: Log openstack-infra-incident https://review.openstack.org/182087 | 07:16 |
| *** ddieterly has joined #openstack-infra | 07:16 | |
| *** dizquierdo has joined #openstack-infra | 07:18 | |
| *** btully has quit IRC | 07:20 | |
| *** ddieterly has quit IRC | 07:21 | |
| *** mpaolino has joined #openstack-infra | 07:22 | |
| *** mpaolino has quit IRC | 07:22 | |
| *** sdake_ has quit IRC | 07:23 | |
| *** katyafervent is now known as katyafervent_awa | 07:24 | |
| *** sshnaidm has joined #openstack-infra | 07:26 | |
| *** katyafervent_awa is now known as katyafervent | 07:26 | |
| *** jistr has joined #openstack-infra | 07:33 | |
| *** mjg is now known as gilliard | 07:33 | |
| *** _nadya_ has joined #openstack-infra | 07:33 | |
| *** sshnaidm has quit IRC | 07:33 | |
| *** koolhead17 has quit IRC | 07:38 | |
| *** koolhead17 has joined #openstack-infra | 07:39 | |
| *** woodster_ has quit IRC | 07:40 | |
| *** _nadya_ has quit IRC | 07:43 | |
| *** sshnaidm has joined #openstack-infra | 07:44 | |
| *** otter768 has joined #openstack-infra | 07:46 | |
| *** jlanoux has joined #openstack-infra | 07:46 | |
| *** mrunge has joined #openstack-infra | 07:47 | |
| *** hashar has joined #openstack-infra | 07:49 | |
| openstackgerrit | Sergey Kraynev proposed openstack-infra/project-config: Add functional test job for python-heatclient https://review.openstack.org/180536 | 07:49 |
| *** hashar has quit IRC | 07:50 | |
| *** hashar_ has joined #openstack-infra | 07:50 | |
| *** shardy_z is now known as shardy | 07:50 | |
| *** otter768 has quit IRC | 07:50 | |
| *** koolhead17 has quit IRC | 07:51 | |
| *** _nadya_ has joined #openstack-infra | 07:52 | |
| *** koolhead17 has joined #openstack-infra | 07:52 | |
| *** openstackstatus has quit IRC | 07:52 | |
| *** openstack has joined #openstack-infra | 07:55 | |
| *** koolhead17 has quit IRC | 07:55 | |
| *** koolhead_ has joined #openstack-infra | 07:55 | |
| *** heyongli has quit IRC | 07:55 | |
| *** _nadya_ has quit IRC | 07:56 | |
| *** HeOS has joined #openstack-infra | 07:56 | |
| davidlenwell | is gerrit broken right now? | 07:56 |
| davidlenwell | getting Exception: Could not connect to gerrit at ssh://dlenwell@review.openstack.org:29418/stackforge/akanda-appliance-builder.git trying to git review | 07:57 |
| anteaya | davidlenwell: what do you see to prompt you to ask | 07:57 |
| *** HeOS has quit IRC | 07:57 | |
| *** HeOS has joined #openstack-infra | 07:57 | |
| *** teran has quit IRC | 07:58 | |
| anteaya | stream events works for me, which is an ssh command | 07:58 |
| *** hashar_ is now known as hashar | 07:58 | |
| davidlenwell | yeah .. I can ping the server too | 07:58 |
| AJaeger | davidlenwell: webui works | 07:59 |
| *** Hal has joined #openstack-infra | 07:59 | |
| anteaya | can you try `ssh -p 29418 <username>@review.openstack.org gerrit stream-events` | 07:59 |
| *** Hal is now known as Guest91208 | 07:59 | |
| davidlenwell | oh I may be having a keychain issue.. | 08:00 |
| davidlenwell | ok .. that was is .. sorry for the false alarm .. | 08:02 |
| anteaya | glad you figured it out | 08:02 |
| davidlenwell | /s/is/it | 08:02 |
| *** arxcruz has joined #openstack-infra | 08:03 | |
| *** yfried has joined #openstack-infra | 08:05 | |
| *** derekh has joined #openstack-infra | 08:09 | |
| *** Longgeek has joined #openstack-infra | 08:09 | |
| *** Guest91208 has quit IRC | 08:10 | |
| openstackgerrit | Jaume Devesa proposed openstack-infra/project-config: Define MidoNet v2 devstack-gate jobs https://review.openstack.org/181932 | 08:10 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Nothing to see here https://review.openstack.org/111011 | 08:16 |
| *** e0ne has joined #openstack-infra | 08:17 | |
| *** e0ne is now known as e0ne_ | 08:17 | |
| *** ddieterly has joined #openstack-infra | 08:17 | |
| *** vponomaryov has joined #openstack-infra | 08:19 | |
| *** isviridov_away is now known as isviridov | 08:21 | |
| *** ddieterly has quit IRC | 08:22 | |
| *** Longgeek has quit IRC | 08:22 | |
| openstackgerrit | Ruslan Kamaldinov proposed openstack-infra/project-config: Add functional test job for python-muranoclient https://review.openstack.org/182082 | 08:22 |
| *** achanda has quit IRC | 08:25 | |
| *** koolhead_ has quit IRC | 08:26 | |
| *** Longgeek has joined #openstack-infra | 08:27 | |
| *** Longgeek has quit IRC | 08:29 | |
| *** koolhead17 has joined #openstack-infra | 08:29 | |
| *** Longgeek has joined #openstack-infra | 08:30 | |
| *** fedexo has quit IRC | 08:30 | |
| openstackgerrit | Jaume Devesa proposed openstack-infra/project-config: Define MidoNet v2 devstack-gate jobs https://review.openstack.org/181932 | 08:35 |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack/requirements: Add InfluxDB https://review.openstack.org/180925 | 08:43 |
| *** e0ne_ is now known as e0ne | 08:46 | |
| *** e0ne is now known as e0ne_ | 08:46 | |
| *** pal has joined #openstack-infra | 08:49 | |
| *** koolhead17 has quit IRC | 08:50 | |
| *** koolhead17 has joined #openstack-infra | 08:51 | |
| *** arxcruz has quit IRC | 08:52 | |
| *** arxcruz has joined #openstack-infra | 08:53 | |
| *** e0ne_ is now known as e0ne | 08:55 | |
| *** koolhead17 has quit IRC | 08:55 | |
| *** yfried is now known as yfried|afk | 08:57 | |
| *** samueldmq has quit IRC | 09:00 | |
| *** ericksonfgds has quit IRC | 09:00 | |
| *** yfried|afk is now known as yfried | 09:01 | |
| *** ssam2 has joined #openstack-infra | 09:03 | |
| *** sshnaidm has quit IRC | 09:03 | |
| *** mahito has quit IRC | 09:04 | |
| *** jtomasek has joined #openstack-infra | 09:06 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate: [WIP] run centos7 with neutron https://review.openstack.org/179689 | 09:12 |
| *** sshnaidm has joined #openstack-infra | 09:15 | |
| *** tnovacik has joined #openstack-infra | 09:16 | |
| *** nfedotov has joined #openstack-infra | 09:17 | |
| *** ddieterly has joined #openstack-infra | 09:18 | |
| *** fhubik has joined #openstack-infra | 09:22 | |
| *** rlandy has joined #openstack-infra | 09:22 | |
| *** ddieterly has quit IRC | 09:22 | |
| *** devvesa has joined #openstack-infra | 09:26 | |
| *** yamamoto has quit IRC | 09:27 | |
| *** yamamoto has joined #openstack-infra | 09:27 | |
| *** yamamoto has quit IRC | 09:32 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate: [WIP] run centos7 with neutron https://review.openstack.org/179689 | 09:35 |
| *** claudiub has joined #openstack-infra | 09:39 | |
| *** otter768 has joined #openstack-infra | 09:39 | |
| *** fhubik is now known as fhubik_afk | 09:40 | |
| *** ZZelle_ has quit IRC | 09:42 | |
| *** ZZelle_ has joined #openstack-infra | 09:42 | |
| jkt | fungi: are you hitting that one as well? | 09:42 |
| *** ihrachyshka has joined #openstack-infra | 09:46 | |
| *** mpaolino has joined #openstack-infra | 09:50 | |
| *** aswadr has joined #openstack-infra | 09:51 | |
| *** Mmike has quit IRC | 09:52 | |
| *** MmikeM has quit IRC | 09:52 | |
| *** cdent has joined #openstack-infra | 09:55 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Update ci for F21 jenkins nodes https://review.openstack.org/182219 | 09:56 |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate: [WIP] run centos7 with neutron https://review.openstack.org/179689 | 09:58 |
| *** yamamoto has joined #openstack-infra | 09:59 | |
| *** Daviey has quit IRC | 10:03 | |
| *** Daviey has joined #openstack-infra | 10:03 | |
| *** Hal has joined #openstack-infra | 10:05 | |
| *** Hal is now known as Guest29018 | 10:06 | |
| *** koolhead17 has joined #openstack-infra | 10:12 | |
| *** jlanoux_ has joined #openstack-infra | 10:13 | |
| *** hashar has quit IRC | 10:14 | |
| *** otter768 has quit IRC | 10:14 | |
| *** jlanoux has quit IRC | 10:16 | |
| *** yfried is now known as yfried|afk | 10:18 | |
| *** ddieterly has joined #openstack-infra | 10:18 | |
| openstackgerrit | Merged openstack-infra/zuul: Merger: ensure_cloned() now looks for '.git' https://review.openstack.org/123437 | 10:19 |
| *** e0ne is now known as e0ne_ | 10:21 | |
| *** Mmike has joined #openstack-infra | 10:21 | |
| *** Mmike has joined #openstack-infra | 10:21 | |
| *** ddieterly has quit IRC | 10:23 | |
| *** yfried|afk is now known as yfried | 10:23 | |
| *** pelix has joined #openstack-infra | 10:26 | |
| *** sshnaidm has quit IRC | 10:27 | |
| *** jlanoux has joined #openstack-infra | 10:29 | |
| *** mpaolino has quit IRC | 10:31 | |
| *** jlanoux_ has quit IRC | 10:32 | |
| *** dims has joined #openstack-infra | 10:33 | |
| openstackgerrit | Merged openstack-infra/project-config: Add "create" permission for networking-ofagent https://review.openstack.org/182144 | 10:33 |
| *** mcdonagm__ has joined #openstack-infra | 10:34 | |
| *** yfried is now known as yfried|afk | 10:34 | |
| *** yamahata has quit IRC | 10:35 | |
| *** bcornec1 has joined #openstack-infra | 10:36 | |
| *** bcornec has quit IRC | 10:36 | |
| *** mcdonagm has quit IRC | 10:36 | |
| *** yfried|afk is now known as yfried | 10:38 | |
| *** e0ne_ is now known as e0ne | 10:39 | |
| *** jlanoux_ has joined #openstack-infra | 10:40 | |
| *** soren_ has quit IRC | 10:43 | |
| *** soren_ has joined #openstack-infra | 10:43 | |
| *** soren_ is now known as soren | 10:43 | |
| *** jlanoux has quit IRC | 10:43 | |
| soren | I'm a little bit confused. I went to jenkins.openstack.org to look at some recent runs of a particular test, but I don't see any record of the builds in Jenkins. Is Jenkins not involved in running tests anymore? | 10:44 |
| soren | Example: https://jenkins.openstack.org/job/gate-glance-python27/ shows no builds. | 10:45 |
| *** zhiwei has quit IRC | 10:46 | |
| *** fhubik_afk is now known as fhubik | 10:47 | |
| *** pc_m has joined #openstack-infra | 10:47 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/jenkins-job-builder: Query all jobs once https://review.openstack.org/181364 | 10:50 |
| AJaeger | soren: we have z jenkins - and zuul distributes them. | 10:50 |
| soren | AJaeger: "z jenkins"? | 10:50 |
| AJaeger | Sorry, 7 | 10:50 |
| *** ociuhandu has joined #openstack-infra | 10:50 | |
| soren | AJaeger: Oh, I see. Ok. | 10:51 |
| *** coolsvap is now known as coolsvap|afk | 10:52 | |
| AJaeger | there's jenkins1, jenkins2 - if you want to have an overview about running jobs, better use other means | 10:52 |
| AJaeger | can't help further with that right now... | 10:52 |
| *** yfried is now known as yfried|afk | 10:54 | |
| soren | Sure, thanks. I was just curious what a typical runtime for glance unit tests were. This was my first instinct. | 10:54 |
| *** Krinkle|detached is now known as Krinkle | 10:56 | |
| AJaeger | soren: just check via review.openstack.org | 10:56 |
| StevenK | AJaeger: http://paste.openstack.org/show/220524/ sorry for the wrapping badness | 10:56 |
| AJaeger | And there's http://graphite.openstack.org/ | 10:56 |
| *** yfried|afk is now known as yfried | 10:57 | |
| AJaeger | StevenK: Glad you figured it out! | 10:58 |
| yamamoto | after https://review.openstack.org/#/c/182144/ being merged, is there anything for me to do to enable the permission? | 11:01 |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add port resource methods https://review.openstack.org/179968 | 11:02 |
| openstackgerrit | Sergey Reshetnyak proposed openstack-infra/project-config: Make gate-sahara-tox-py27-scenario-unit job voting https://review.openstack.org/182238 | 11:03 |
| *** links has joined #openstack-infra | 11:04 | |
| AJaeger | yamamoto: just use it - see the Infra manual | 11:04 |
| AJaeger | yamamoto: http://docs.openstack.org/infra/manual/drivers.html#create-proposed-branch applies | 11:05 |
| yamamoto | AJaeger: https://review.openstack.org/#/admin/projects/stackforge/networking-ofagent,branches doesn't have a form for me | 11:08 |
| yamamoto | AJaeger: and https://review.openstack.org/#/admin/projects/stackforge/networking-ofagent,access doesn't show the permission | 11:09 |
| yamamoto | AJaeger: so i thought some human intervention was necessary. | 11:09 |
| openstackgerrit | Louis Taylor proposed openstack-infra/project-config: Add experimental glance_store functional test job https://review.openstack.org/182000 | 11:10 |
| kragniz | AJaeger: gah, thanks for reviewing my mistakes | 11:10 |
| *** pal has quit IRC | 11:12 | |
| AJaeger | you're welcome, kragniz | 11:12 |
| *** Mmike has quit IRC | 11:12 | |
| AJaeger | kragniz: I wonder whether you can't combine the various drivers instead of starting a new job for each of them. | 11:13 |
| *** panda|afk has quit IRC | 11:13 | |
| AJaeger | I'll add that as comment as well | 11:14 |
| *** Mmike has joined #openstack-infra | 11:14 | |
| *** Mmike has joined #openstack-infra | 11:14 | |
| AJaeger | but if it runs for 60 mins... | 11:14 |
| *** panda|afk has joined #openstack-infra | 11:14 | |
| *** sergsh has joined #openstack-infra | 11:15 | |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add keystone service resource methods https://review.openstack.org/177621 | 11:15 |
| *** dguitarbite has joined #openstack-infra | 11:15 | |
| *** ddieterly has joined #openstack-infra | 11:19 | |
| *** ddieterly has quit IRC | 11:23 | |
| *** woodster_ has joined #openstack-infra | 11:29 | |
| *** marcusvrn has joined #openstack-infra | 11:30 | |
| *** marcusvrn has quit IRC | 11:31 | |
| *** marcusvrn has joined #openstack-infra | 11:31 | |
| *** samueldmq has joined #openstack-infra | 11:34 | |
| samueldmq | morning | 11:34 |
| *** panda|afk is now known as panda | 11:35 | |
| *** panda is now known as Guest79688 | 11:35 | |
| *** Guest79688 is now known as panda | 11:36 | |
| *** ldnunes has joined #openstack-infra | 11:38 | |
| *** stevebaker has quit IRC | 11:39 | |
| *** asahlin has quit IRC | 11:43 | |
| *** weshay has joined #openstack-infra | 11:43 | |
| *** ddieterly has joined #openstack-infra | 11:43 | |
| *** asahlin has joined #openstack-infra | 11:43 | |
| *** deepakcs has quit IRC | 11:44 | |
| *** smarcet has joined #openstack-infra | 11:44 | |
| *** viktors has quit IRC | 11:44 | |
| *** maishsk has joined #openstack-infra | 11:45 | |
| kragniz | AJaeger: yeah, that's a good point | 11:45 |
| *** jistr is now known as jistr|class | 11:46 | |
| *** jistr|class is now known as jistr | 11:47 | |
| kragniz | AJaeger: I'm just using the filesystem driver as a guinea pig for the time being, but I'll refactor them into a single job if it becomes clearer it would be better later on | 11:47 |
| *** Krinkle is now known as Krinkle|detached | 11:48 | |
| *** jistr is now known as jistr|class | 11:50 | |
| *** maishsk has quit IRC | 11:52 | |
| *** maishsk has joined #openstack-infra | 11:52 | |
| *** hashar has joined #openstack-infra | 11:53 | |
| *** mpavone has quit IRC | 11:55 | |
| *** rfolco has joined #openstack-infra | 11:56 | |
| *** jistr|class is now known as jistr | 11:57 | |
| samueldmq | dhellmann, AJaeger do you know the plans to graduate oslo-incubator quota.py to its own library (possibly oslo.quota ? ) | 11:58 |
| *** kgiusti has joined #openstack-infra | 11:58 | |
| *** e0ne is now known as e0ne_ | 12:00 | |
| yolanda | fungi, clarkb, i'm testing apache on centos 7, to use it on cgit, and httpd.services timeout , it never ends booting. Have you ever experimented with that? | 12:00 |
| fungi | yolanda: i have not, i don't think clarkb has either | 12:00 |
| fungi | but i'll let him answer when he's awake | 12:01 |
| yolanda | having some error like systemctl status httpd.service timeout, seems to be related with systemd | 12:01 |
| mordred | soren: we also track stats counters in graphite | 12:01 |
| yolanda | i wanted to start moving git_backend to centos 7, but i'm quite blocked on that | 12:01 |
| *** yamamoto has quit IRC | 12:01 | |
| soren | mordred: What are "stats counters"? | 12:02 |
| *** dprince has joined #openstack-infra | 12:02 | |
| *** ildikov has quit IRC | 12:02 | |
| mordred | oh - hrm, we track success and failure - don't seem to have job runtimeme | 12:02 |
| mordred | in graphite | 12:02 |
| *** dims_ has joined #openstack-infra | 12:03 | |
| fungi | mordred: i thought we had the runtimes in a separate stat tree from the counts? | 12:03 |
| mordred | ah - yeah - there they are | 12:03 |
| fungi | soren: statsd collection | 12:03 |
| *** dkranz has joined #openstack-infra | 12:04 | |
| fungi | soren: zuul emits events about job run times and results and they're collected for future analysis | 12:04 |
| fungi | soren: and http://graphite.openstack.org/ provides a frontend to query that | 12:05 |
| soren | fungi: Let's just for the sake of the argument assume I'm a complete idiot. Hypothetically, of course. | 12:05 |
| soren | fungi: Where would I need to click to see how long a typical unit test run takes for, say, glance. | 12:05 |
| *** dims has quit IRC | 12:06 | |
| *** zul has quit IRC | 12:06 | |
| *** dkranz has quit IRC | 12:09 | |
| samueldmq | fungi, oh graphite is nice, so we can compose our chart with whatever metric we want :) | 12:09 |
| samueldmq | we just click on metrics and they're added to the chart | 12:09 |
| *** dkranz has joined #openstack-infra | 12:09 | |
| *** mpavone has joined #openstack-infra | 12:09 | |
| mordred | samueldmq: yup | 12:10 |
| mordred | soren: I'm poking in the data to find such a thing | 12:11 |
| mordred | fungi: stats.timers.pipeline.gate.job.gate-glance-python27.*.median doesn't so much seem to be showing anything | 12:12 |
| *** bswartz has quit IRC | 12:12 | |
| *** mjturek1 has quit IRC | 12:13 | |
| *** mjturek1 has joined #openstack-infra | 12:13 | |
| samueldmq | mordred, cool .. and all those info is caught by events which are sent when running the jobs ? | 12:13 |
| fungi | mordred: over what timespan? glance hasn't merged a new change in 4 days, and only 4 in the past week and a half | 12:13 |
| mordred | fungi: good point | 12:14 |
| *** otter768 has joined #openstack-infra | 12:15 | |
| soren | mordred: Cool. | 12:15 |
| samueldmq | fungi, there's an option there : 'Select a Data Range' which can be useful | 12:16 |
| fungi | samueldmq: yep, i'm just not at a computer i can do that from easily. still have morning waking up things going on | 12:16 |
| samueldmq | fungi, I meant *Date* Range, | 12:16 |
| *** ildikov has joined #openstack-infra | 12:17 | |
| *** freerunner has joined #openstack-infra | 12:17 | |
| samueldmq | fungi, yeah I understand your point .. to show a data range which can show significant data by default | 12:17 |
| *** gordc has joined #openstack-infra | 12:18 | |
| *** zul has joined #openstack-infra | 12:19 | |
| *** otter768 has quit IRC | 12:20 | |
| *** mpaolino has joined #openstack-infra | 12:23 | |
| *** mpaolino has quit IRC | 12:23 | |
| *** rbradfor has joined #openstack-infra | 12:25 | |
| *** Longgeek has quit IRC | 12:26 | |
| *** Longgeek has joined #openstack-infra | 12:27 | |
| *** rbradfor has quit IRC | 12:28 | |
| samueldmq | I wonder if there is an API to make queries on those data regarding jobs stats .. | 12:28 |
| samueldmq | failing / success rate per project, etc | 12:29 |
| samueldmq | but not sure it would be useful :p | 12:29 |
| samueldmq | those could possibly be queried on graphite | 12:29 |
| mordred | there is, I believe | 12:30 |
| *** aysyd has joined #openstack-infra | 12:30 | |
| mordred | samueldmq: pabelanger and timrc have been working on spinning up a grafana which is an external thing that talks to the graphite api to make different graphs | 12:31 |
| samueldmq | mordred, nice, I would like to see that :) | 12:33 |
| samueldmq | pabelanger, timrc let me know if you have something published | 12:33 |
| *** e0ne_ is now known as e0ne | 12:34 | |
| *** ddieterly has quit IRC | 12:36 | |
| vponomaryov | Hello, does anybody know why CI does not save logs if job timeout was reached? Here is example - http://logs.openstack.org/80/181380/4/check/gate-manila-tempest-dsvm-neutron-no-share-servers/4eafef4/logs/ | 12:36 |
| *** zz_dimtruck is now known as dimtruck | 12:36 | |
| *** dims_ has quit IRC | 12:40 | |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add keystone service resource methods https://review.openstack.org/177621 | 12:40 |
| *** dims has joined #openstack-infra | 12:42 | |
| *** mestery has joined #openstack-infra | 12:42 | |
| *** banix has joined #openstack-infra | 12:43 | |
| dhellmann | jeblair, fungi, clarkb, anteaya, pleia2, and anyone else who might have contributed: the "depends-on" gerrit tag is my favorite OpenStack feature for Kilo: https://review.openstack.org/#/q/topic:bp/remove-namespace-packages,n,z | 12:45 |
| *** baoli has joined #openstack-infra | 12:45 | |
| *** baoli has quit IRC | 12:45 | |
| * ttx would like Depends:Same-ChangeID-merges-in-master-first | 12:46 | |
| *** baoli has joined #openstack-infra | 12:48 | |
| *** bhunter71 has joined #openstack-infra | 12:48 | |
| *** Adri2000 has quit IRC | 12:49 | |
| *** radez_g0n3 is now known as radez | 12:49 | |
| *** baoli has quit IRC | 12:50 | |
| *** Adri2000 has joined #openstack-infra | 12:50 | |
| *** baoli has joined #openstack-infra | 12:50 | |
| *** bswartz has joined #openstack-infra | 12:51 | |
| *** tiswanso has joined #openstack-infra | 12:52 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Stop leaking server objects https://review.openstack.org/181513 | 12:56 |
| *** fhubik is now known as fhubik_afk | 12:56 | |
| *** maishsk has quit IRC | 12:57 | |
| AJaeger | dhellmann: yeah, it's really handy! | 12:57 |
| sdague | ttx: well, just don't use Idempotent id copy for the backport, then you can get it :) | 12:57 |
| ttx | we kinda like it though :) | 12:57 |
| ttx | (the idempotent id copy) | 12:58 |
| AJaeger | dhellmann: wow, nice patch series! | 12:58 |
| *** btully has joined #openstack-infra | 12:58 | |
| dhellmann | AJaeger: sed is my friend :-) | 12:59 |
| sdague | ttx: well that's why you can't have nice things :) | 12:59 |
| *** zul has quit IRC | 12:59 | |
| *** fhubik_afk is now known as fhubik | 13:00 | |
| ttx | sdague: we could say across the board that if two things have the same changeID and one of them is in master and the others in other branches, it shall merge first | 13:00 |
| *** amitgandhinz has quit IRC | 13:00 | |
| *** hdd has joined #openstack-infra | 13:01 | |
| *** dkranz has quit IRC | 13:01 | |
| ttx | that would provide a nice safety net to make sure backports always land in master and avoid regressions there | 13:01 |
| *** sdake has joined #openstack-infra | 13:01 | |
| kragniz | dhellmann: wow, that's impressive! | 13:01 |
| SpamapS | ttx: +1 | 13:01 |
| *** kgiusti has quit IRC | 13:02 | |
| *** mpavone has quit IRC | 13:02 | |
| *** sdake_ has joined #openstack-infra | 13:03 | |
| *** zul has joined #openstack-infra | 13:03 | |
| *** maishsk has joined #openstack-infra | 13:03 | |
| dhellmann | AJaeger, kragniz : I'm going to pump up my stats early this cycle so I can take it easy at the end. ;-) | 13:03 |
| *** jistr is now known as jistr|mtg | 13:04 | |
| kragniz | dhellmann: hehe | 13:04 |
| AJaeger | dhellmann: go for it! | 13:04 |
| *** tiswanso has quit IRC | 13:05 | |
| *** jordanP has joined #openstack-infra | 13:06 | |
| *** sdake has quit IRC | 13:06 | |
| *** ildikov has quit IRC | 13:07 | |
| *** Ala has quit IRC | 13:07 | |
| jordanP | hi guys. I am running a 3rd party ci without Zuul, and without nodepool. Just the plain devstack-gate script. It worked well until "Add errexit to setup & cleanup functions" landed. This patch is a good thing though. My CI fails because when it runs the "copy_mirror_config" which tries to copy a "~/.pydistutils.cfg". Apparently I don't have this file (install exists with 'no source file'). Where is this file supposed to come from ? | 13:09 |
| *** Ala has joined #openstack-infra | 13:12 | |
| mordred | jordanP: our puppet adds that file to /home/jenkins | 13:13 |
| fungi | vponomaryov: the log copying happens as part of the job. if the job reaches a jenkins timeout, jenkins just kills the worker, so no logs copied | 13:13 |
| mordred | jordanP: it's where we put mirror config setting (or one of the places) | 13:13 |
| jordanP | mordred, okay, thanks. I don't use puppet, my base image is a base ubuntu 14 image | 13:13 |
| fungi | vponomaryov: we build in-job timeout mechanisms to work around that, for example devstack-gate scripts have a timeout option which we set a little lower than the jenkins timeout for the job so that the process group for the tests gets killed and allows the job to finish copying logs before the jenkins timeout is reached | 13:14 |
| mordred | it probably wouldnt' be terrible to put an if [-f ~/.pydistutils.cfg] around that copy | 13:14 |
| jordanP | mordred, +1 :) Can I submit a patch for this ? | 13:15 |
| fungi | dhellmann: you have to know deep down that you're never going to let yourself "take it easy" | 13:15 |
| dhellmann | fungi: a boy can dream | 13:15 |
| *** mpavone has joined #openstack-infra | 13:15 | |
| mordred | jordanP: hrm. looking at it - that's part of the setup ... let me look 3 more seconds before I say yes submit a patch .. | 13:16 |
| jordanP | mordred, sure :) | 13:16 |
| *** freerunner has quit IRC | 13:16 | |
| samueldmq | dhellmann, hi ... do you know the plans to graduate oslo-incubator quota.py to its own library (possibly oslo.quota ? ) | 13:17 |
| mordred | jordanP: yeah - I'd say wrapping that block in an if [ -f ~/.pydistutils.cfg ] shoudn't be terrible | 13:17 |
| samueldmq | dhellmann, I am asking you because I saw you contributed to incubator :) | 13:17 |
| *** sdake_ is now known as sdake | 13:18 | |
| jordanP | mordred, so far my CI was running okay, so I assume that pydistutils.cfg is not super critical (I lived without before). Do you confirm that not having this file is not too bad ? | 13:18 |
| mordred | jordanP: it's fine - the only thing we use it for is to point things to our pypi mirror | 13:19 |
| jordanP | okay great thanks. Going to submit a change now. | 13:19 |
| *** freerunner has joined #openstack-infra | 13:19 | |
| *** banix has quit IRC | 13:19 | |
| *** peristeri has joined #openstack-infra | 13:21 | |
| *** yamamoto has joined #openstack-infra | 13:22 | |
| *** ildikov has joined #openstack-infra | 13:23 | |
| *** rfolco_ has joined #openstack-infra | 13:23 | |
| *** freerunner has quit IRC | 13:23 | |
| *** sarob has joined #openstack-infra | 13:24 | |
| *** sarob has quit IRC | 13:24 | |
| *** rfolco has quit IRC | 13:24 | |
| *** sarob has joined #openstack-infra | 13:25 | |
| *** banix has joined #openstack-infra | 13:25 | |
| dhellmann | samueldmq: let's move to #openstack-oslo to chat about that | 13:27 |
| *** ddieterly has joined #openstack-infra | 13:27 | |
| *** tiswanso has joined #openstack-infra | 13:29 | |
| *** prad has joined #openstack-infra | 13:29 | |
| samueldmq | dhellmann, sure | 13:30 |
| pabelanger | samueldmq, this is what I have so far: https://github.com/pabelanger/grafyaml | 13:31 |
| pabelanger | inspiration from JJB, you'd run grafana-dashboards update path/to/dashboard.yaml | 13:32 |
| pabelanger | right now, just a single yaml file is published into grafana, plan to update it to add recursive directories this / next week | 13:32 |
| pabelanger | also basic support for grafana items right now, rows, panels (text / dashlist) | 13:33 |
| pabelanger | still need to add validation for others and unit tests | 13:33 |
| pabelanger | but, because we are using python voluptuous it should be simple to add more coverage into grafana | 13:33 |
| * mordred looks forward to such things | 13:33 | |
| *** annegentle has joined #openstack-infra | 13:35 | |
| *** sarob_ has joined #openstack-infra | 13:35 | |
| *** dboik has quit IRC | 13:35 | |
| openstackgerrit | Jordan Pittier proposed openstack-infra/devstack-gate: Test for `~/.pydistutils.cfg` existence before copying it https://review.openstack.org/182294 | 13:36 |
| mordred | flaper87: are you ok with 172095 now? | 13:36 |
| *** dustins has joined #openstack-infra | 13:37 | |
| fungi | no responses to jeblair's repo-discuss post so far. and #gerrit is still basically silent aside from semi-idle chatter | 13:37 |
| *** mattfarina has joined #openstack-infra | 13:38 | |
| flaper87 | mordred: yeah, go ahead. I've the patch ready! | 13:38 |
| *** amitgandhinz has joined #openstack-infra | 13:38 | |
| flaper87 | mordred: thanks for checking | 13:38 |
| *** sarob has quit IRC | 13:38 | |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add keystone endpoint resource methods https://review.openstack.org/180328 | 13:39 |
| mordred | fungi: wow. that's SO EXCITING | 13:40 |
| fungi | makes me wonder if anyone's actually using 2.10 under any appreciable load | 13:40 |
| sdague | fungi: so, we are the biggest gerrit install right? And I also assume with the 200+ CI systems attached to it the inbound voting load is far more than anyone else has. | 13:42 |
| fungi | probably. i don't actually know how heavy the traffic is on android's gerrit, or wikimedia's | 13:43 |
| fungi | eclipse foundation uses it too, but i do suspect theirs is lower volume than ours | 13:43 |
| *** links has quit IRC | 13:44 | |
| *** smarcet has quit IRC | 13:44 | |
| *** Ala has quit IRC | 13:45 | |
| *** sarob_ has quit IRC | 13:45 | |
| *** sarob has joined #openstack-infra | 13:45 | |
| mordred | gerrit's gerrit hasn't crossed the 100k changes mark | 13:45 |
| *** mriedem_away is now known as mriedem | 13:46 | |
| fungi | i can't remember whether gerrit's gerrit is the one hosting android development, or whether they're separate instances | 13:46 |
| mordred | neither has cyanogenmod's although they're close | 13:46 |
| mordred | https://android-review.googlesource.com | 13:46 |
| *** bauzas has quit IRC | 13:46 | |
| mordred | has at least 150k changes - so they're at least similar in data size | 13:47 |
| *** asrangne has joined #openstack-infra | 13:47 | |
| *** bauzas has joined #openstack-infra | 13:47 | |
| fungi | ahh, yep | 13:47 |
| fungi | and running 2.11 | 13:47 |
| mordred | I'm fairly certain that our use of the gerrit api for automated systems dwarfs everyone else by large orders of magnitude though | 13:48 |
| fungi | ~50 commits behind the build running on gerrit's gerrit | 13:48 |
| *** arielb has joined #openstack-infra | 13:49 | |
| *** arielb has joined #openstack-infra | 13:49 | |
| *** aswadr has quit IRC | 13:50 | |
| sdague | android seems to be doing about 10k changes in 60 days, and we do so in ~40. So our change inbound rate is double | 13:51 |
| sdague | oh... android dev uses drafts? | 13:51 |
| mordred | they like their secrets | 13:52 |
| *** sarob has quit IRC | 13:52 | |
| sdague | I've also yet to find a change with > 3 revisions | 13:52 |
| *** mestery has quit IRC | 13:53 | |
| fungi | that matches my quality expectations for their platform | 13:53 |
| mordred | we should count both changes and change-revions in our talking about scale stuff | 13:53 |
| mordred | fungi: :) | 13:53 |
| *** e0ne is now known as e0ne_ | 13:53 | |
| *** markvoelker has quit IRC | 13:53 | |
| sdague | yes, we should | 13:53 |
| BharatK | sdague: fungi: I sent patch to "Archive GlusterFS logs" https://review.openstack.org/#/c/171500/ | 13:54 |
| *** Ala has joined #openstack-infra | 13:54 | |
| BharatK | sdague: fungi and this got merged. But after job run I am getting all glusterfs log files with zero size | 13:54 |
| BharatK | sdague: fungi http://logs.openstack.org/54/182254/3/check/check-tempest-dsvm-full-glusterfs-nv/2096420/logs/glusterfs/ | 13:54 |
| BharatK | sdague: fungi May I know the reason for this? | 13:54 |
| sdague | so... I'm actually extra confused now, do they have a different change id allocator? Because when I randomly pick change numbers, unless they end in a 0 I seem to get an error | 13:54 |
| *** e0ne_ is now known as e0ne | 13:55 | |
| mordred | weird | 13:55 |
| *** maishsk has quit IRC | 13:55 | |
| openstackgerrit | Russell Bryant proposed openstack-infra/reviewstats: Revert "Fix compat with gerrit 2.9" https://review.openstack.org/182300 | 13:56 |
| *** bknudson has joined #openstack-infra | 13:57 | |
| *** dboik has joined #openstack-infra | 13:58 | |
| fungi | BharatK: i don't see any errors around where those logs are copied to /opt/stack/logs or subsequently scp'd to the logserver. is it possible that whatever writes to those file descriptors hasn't flushed its buffers? | 13:58 |
| *** maishsk has joined #openstack-infra | 13:59 | |
| *** maishsk has quit IRC | 13:59 | |
| *** fhubik is now known as fhubik_afk | 13:59 | |
| *** banix has quit IRC | 14:00 | |
| *** stevebaker has joined #openstack-infra | 14:00 | |
| sdague | russellb: it looks like the base patch never merged anyway | 14:00 |
| BharatK | fungi: No. actually in local environment I am able to see the logs there. Through job it is coping files correctly but not the content. | 14:00 |
| russellb | sdague: oh, well that's convenient | 14:00 |
| sdague | yeh | 14:00 |
| russellb | sure enough, thanks :) | 14:00 |
| sdague | it hit a merge conflict | 14:00 |
| *** sarob has joined #openstack-infra | 14:01 | |
| BharatK | fungi: Is there any way to check whether glusterfs files have some content or not while running the job. | 14:01 |
| fungi | BharatK: i suppose you could make an empty glusterfs patch which depends-on a throwaway devstack-gate patch containing some additional debug lines around the log copying to wc -l them or tail them or something | 14:02 |
| fungi | er, glusterfs plugin patch or whatever your repo is | 14:03 |
| fungi | or some project which has the glusterfs variant job in its experimental pipeline | 14:03 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: WIP: Update ci for F21 jenkins nodes https://review.openstack.org/182219 | 14:04 |
| dims | sdague: lack of python3 a show stopper for global requirements repo? (python-kubernetes) | 14:05 |
| sdague | dims: yes, pretty much | 14:05 |
| openstackgerrit | Jaume Devesa proposed openstack-infra/project-config: Define MidoNet v2 devstack-gate jobs https://review.openstack.org/181932 | 14:05 |
| *** armax has joined #openstack-infra | 14:05 | |
| BharatK | fungi: Yeah got it. I will send some test patch to stackforge/devstack-plugin-glusterfs project. Thanks for the idea. | 14:05 |
| sdague | unless there is a *really* good reason | 14:05 |
| *** ildikov has quit IRC | 14:05 | |
| openstackgerrit | Merged openstack-infra/shade: Update server API for get/list/search interface https://review.openstack.org/180293 | 14:05 |
| openstackgerrit | Merged openstack-infra/shade: Update images API for get/list/search interface https://review.openstack.org/180600 | 14:05 |
| openstackgerrit | Merged openstack-infra/shade: Rewrite extension checking methods https://review.openstack.org/181935 | 14:07 |
| *** Ala has quit IRC | 14:08 | |
| *** rossella_s has quit IRC | 14:08 | |
| *** Ala has joined #openstack-infra | 14:08 | |
| zxiiro | mordred: drafts don't have to be "secret" one of the projects I work on we setup drafts to be public. | 14:08 |
| fungi | mordred: looks like we have a gerrit acl update which didn't get applied during yesterday's carnage. what's the current recommended way to rerun manage-projects? via ansible or safe to just run locally on gerrit? need to do it as gerrit2 or root? | 14:08 |
| *** rossella_s has joined #openstack-infra | 14:09 | |
| pabelanger | So, I'm just starting to look at shade. I wanted to see if anybody had some code up to use shade to launch a single_use_slave? I know I could use launch_node in system-config, but the honest answer is, I just want to launch a single jenkins node, without connecting it into puppet, so I can then install devstack a top of it | 14:09 |
| mordred | fungi: guh - I think it's safe to run locally as gerrit2 | 14:10 |
| fungi | mordred: thanks | 14:10 |
| mordred | pabelanger: yes, this is very possible | 14:10 |
| *** freerunner has joined #openstack-infra | 14:10 | |
| zxiiro | mordred: what's the progress on that patch to allow setting nodepool's public/private network to a custom network? I noticed the Gerrit patch I was following was closed is there a new one? | 14:10 |
| mordred | pabelanger: you may want to learn all about the new os_server module I wrote using shade for ansible | 14:10 |
| mordred | zxiiro: we already support configuring custom networks ... | 14:11 |
| vponomaryov | fungi: according to timeouts of jobs, in mentioned case by me - "devstack-gate"'s timeout is not reached (60 min) and reached only CI's which is 65 minutes, | 14:11 |
| sdague | sadly, the android gerrit doesn't enable ssh at all | 14:11 |
| *** ajmiller has joined #openstack-infra | 14:11 | |
| sdague | so adhocing to figure out their activity level is not nearly so simple | 14:11 |
| zxiiro | mordred: oh so it was fixed recently? I'll have to checkout the latest nodepool then... | 14:11 |
| mordred | pabelanger: https://github.com/ansible/ansible-modules-core/blob/devel/cloud/openstack/os_server.py | 14:11 |
| mordred | zxiiro: no, it's been in nodepool for quite some time - perhaps I'm not understanding which thing you need | 14:12 |
| *** StevenK has quit IRC | 14:12 | |
| zxiiro | mordred: awhile back you suggested I hack this patch together for nodepool until you fixed nodepool to support this: https://github.com/zxiiro/nodepool/commit/7bc6b47df5464cf7b97e4b75a8eb6591ced71ce4 | 14:13 |
| mordred | pabelanger: https://github.com/ansible/ansible-modules-core/blob/devel/cloud/openstack/os_server.py#L302 can show you the code you need if you just want to write a shade script directly though | 14:13 |
| *** BharatK has quit IRC | 14:13 | |
| fungi | ghub = github.Github(secure_config.get("github", "username"), | 14:13 |
| fungi | NoSectionError: No section: 'github' | 14:13 |
| fungi | i wonder if we recently broke jeepyb | 14:13 |
| mordred | hrm | 14:15 |
| mordred | that would be annoying | 14:15 |
| mordred | I do remember approving a jeepyb patch recently | 14:16 |
| *** mpavone has quit IRC | 14:16 | |
| *** smccully has joined #openstack-infra | 14:16 | |
| *** otter768 has joined #openstack-infra | 14:16 | |
| *** _smccully has joined #openstack-infra | 14:16 | |
| *** _smccully has quit IRC | 14:16 | |
| *** koolhead17 has quit IRC | 14:16 | |
| *** smccully has quit IRC | 14:16 | |
| fungi | oh, gerrit2 doesn't have read access to /etc/github/github-projects.secure.config | 14:17 |
| *** smccully has joined #openstack-infra | 14:17 | |
| mordred | fungi: last jeepyb patch was May 1 - I'm pretty sure we would have heard screaming if project creation had been broken that long | 14:17 |
| *** zul has quit IRC | 14:17 | |
| fungi | -r--r----- 1 root github | 14:17 |
| *** sarob has quit IRC | 14:17 | |
| mordred | oh! well them - maybe we run that script at root then | 14:17 |
| *** koolhead17 has joined #openstack-infra | 14:17 | |
| fungi | yje "github" group has no members | 14:17 |
| fungi | er, the | 14:17 |
| *** stevemar has joined #openstack-infra | 14:17 | |
| *** claudiub has quit IRC | 14:18 | |
| *** fhubik_afk is now known as fhubik | 14:18 | |
| *** wenlock has joined #openstack-infra | 14:18 | |
| *** zul has joined #openstack-infra | 14:19 | |
| *** fhubik is now known as fhubik_afk | 14:20 | |
| *** wenlock has quit IRC | 14:20 | |
| *** ildikov has joined #openstack-infra | 14:21 | |
| *** otter768 has quit IRC | 14:21 | |
| *** wenlock has joined #openstack-infra | 14:21 | |
| *** annegentle has quit IRC | 14:21 | |
| *** deepakcs has joined #openstack-infra | 14:21 | |
| *** annegentle has joined #openstack-infra | 14:22 | |
| *** koolhead17 has quit IRC | 14:22 | |
| *** dimtruck is now known as zz_dimtruck | 14:22 | |
| *** fhubik_afk is now known as fhubik | 14:22 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/requirements: Allow Magnum to conform to global requirements process https://review.openstack.org/170830 | 14:24 |
| *** nelsnelson has joined #openstack-infra | 14:24 | |
| dims | mordred: sdague: dropped the python-kubernetes, it's just docker-py that's needed. reworked the commit with info requested in README. please take a look when you get a chance. ^^^ | 14:24 |
| *** signed8bit has joined #openstack-infra | 14:25 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:25 | |
| openstackgerrit | Merged openstack/requirements: Update xstatic-font-awesome to 4.3.0 https://review.openstack.org/162142 | 14:27 |
| *** rfolco_ has quit IRC | 14:27 | |
| *** michchap has quit IRC | 14:27 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Add kilo compat jobs to tempest https://review.openstack.org/181907 | 14:28 |
| *** rfolco has joined #openstack-infra | 14:28 | |
| clarkb | yolanda: fungi I have not tested on centos7 yet. If I had to guess what the issue is centos7 uses apache 2.4 like trusty and the vhost needs to be updated? | 14:28 |
| *** devvesa has quit IRC | 14:29 | |
| mtreinish | anteaya, AJaeger, sdague: ^^^ if you get a sec, assuming I didn't screw that up, it should be pretty simple | 14:29 |
| mordred | dims: done | 14:30 |
| *** zz_dimtruck is now known as dimtruck | 14:30 | |
| openstackgerrit | Merged openstack/requirements: Bump cryptography to 0.8.2 for thread safety fix https://review.openstack.org/172672 | 14:30 |
| dims | thank you sir! | 14:30 |
| *** maishsk has joined #openstack-infra | 14:31 | |
| *** sarob has joined #openstack-infra | 14:33 | |
| anteaya | mtreinish: you put in a stanza for -kilo-nv but I don't see any -kilo-nv jobs added in zuul/layout.yaml in your patch | 14:33 |
| openstackgerrit | Valeriy Ponomaryov proposed openstack/requirements: Bump ddt to min version 0.7.0 https://review.openstack.org/166162 | 14:34 |
| *** thedodd has joined #openstack-infra | 14:34 | |
| mtreinish | anteaya: I figured it'd be better to have it there for future use, it was there for the other branches | 14:34 |
| fungi | hrm, so the ultimate issue with the stale acls i'm seeing is that /etc/project-config hasn't been updated | 14:34 |
| *** dangers_away is now known as dangers | 14:34 | |
| fungi | on review.o.o | 14:34 |
| *** Ala has quit IRC | 14:35 | |
| *** sarob has quit IRC | 14:35 | |
| fungi | Notice: Skipping run of Puppet configuration client; administratively disabled (Reason: 'reason not specified'); | 14:35 |
| openstackgerrit | Clint Adams proposed openstack-infra/system-config: Add #kolla to the list of statusbot channels https://review.openstack.org/181673 | 14:35 |
| fungi | there we go | 14:35 |
| fungi | we never reenabled puppet after yesterday's maintenance | 14:35 |
| mordred | fungi: hahahahaha | 14:36 |
| fungi | forgot we were applying with an alternative lockfile | 14:36 |
| zxiiro | mordred: does my github patch ring any bells? | 14:37 |
| openstackgerrit | Merged openstack/requirements: global-requirements: Add ovs https://review.openstack.org/177891 | 14:37 |
| mordred | zxiiro: it does not. but we currently use nodepool with specific neutron networks in our hp cloud region, so I know that works | 14:37 |
| *** Ala has joined #openstack-infra | 14:38 | |
| mordred | zxiiro: oh! one sec ... Ijust saw your link above | 14:38 |
| mordred | zxiiro: AHHHHHHHHHHHH | 14:38 |
| zxiiro | mordred: yeah about a month ago we were discussing how nodepool currently doesn't allow us to set a network with a name other than "public" or "private" | 14:38 |
| mordred | zxiiro: ok. I grok all the things now | 14:38 |
| mordred | sorry - I have a broken brain often | 14:38 |
| zxiiro | no worries, fast paced world we all live in ;) | 14:39 |
| *** jistr|mtg is now known as jistr | 14:39 | |
| mordred | zxiiro: so - no, we don't have that - but we do have some work coming that will fix that | 14:39 |
| *** Guest20306 is now known as redrobot | 14:39 | |
| openstackgerrit | Merged openstack/requirements: Uncap pymongo https://review.openstack.org/172095 | 14:39 |
| zxiiro | mordred: yeah I was following a gerrit review but it was closed. Was wondering if there was a newer Gerrit or bug i should be following to get updates? | 14:39 |
| mordred | zxiiro: because honestly what should be being checked there is the network property about whether the network is externally routable, I believe | 14:39 |
| pabelanger | mordred, Thanks checking it out now | 14:40 |
| mordred | zxiiro: we're working on porting nodepool to use the shade library - and we're teaching shade hwo to be smarter about all of these things | 14:40 |
| clarkb | mordred: pretty sure that property does not exist | 14:40 |
| anteaya | mtreinish: the only place both juno-nv and icehouse-nv are used are in the devstack-gate jobs in zuul/layout.yaml, how would you feel about saving some time and just offering kilo-nv jobs to devstack-gate in the same patch, or a dependent one? | 14:40 |
| mordred | clarkb: it does, in fact | 14:40 |
| clarkb | mordred: pretty sure my devstack test cloud has not had that | 14:40 |
| mordred | clarkb: on the neutron network/router? | 14:40 |
| clarkb | mordred: but its gone now, so not easy to check | 14:41 |
| clarkb | mordred: no nova net | 14:41 |
| mordred | right. this is a neutron setting | 14:41 |
| clarkb | mordred: then we can't rely on it | 14:41 |
| fungi | oh, fun. github api still fails most of the time on a full manage-projects run | 14:41 |
| mordred | clarkb: hang on | 14:41 |
| fungi | SSLError: _ssl.c:495: The handshake operation timed out | 14:41 |
| mtreinish | anteaya: sure I'll respin it right now to add the d-g jobs | 14:41 |
| mordred | clarkb: we already have to do checks to see what sort of networking you have - currently we do it by shape of the network dict | 14:41 |
| anteaya | mtreinish: thank you | 14:41 |
| mordred | clarkb: that tells us whether your networking is nova-net or neutron | 14:41 |
| *** StevenK has joined #openstack-infra | 14:42 | |
| *** emagana has joined #openstack-infra | 14:42 | |
| mordred | clarkb: if we detect that your networking is neutron rather than nova net, then in that fork it's actually both more efficient and less racy to use neutron floating ip | 14:42 |
| mordred | clarkb: so - we already have forking logic | 14:42 |
| mordred | clarkb: and we can, in that forking logic, be smarter | 14:42 |
| mordred | the forking logic we have now is very brittle | 14:42 |
| clarkb | its based on existing public clouds because you can do whatever you want as opendaylight has done | 14:43 |
| mordred | clarkb: yes. I know this | 14:43 |
| mordred | clarkb: what I'm saying is that opendaylight's thing is totally detectable | 14:43 |
| clarkb | mordred: right and I am saying I don't understand how since its arbitrary | 14:43 |
| clarkb | mordred: we could say anything that isn't "private" is public | 14:43 |
| clarkb | but that remains brittle | 14:43 |
| mordred | clarkb: how about we look for the externally-routable property? | 14:44 |
| mordred | clarkb: if it's a neutron network | 14:44 |
| openstackgerrit | Clint Adams proposed openstack-infra/system-config: Add #kolla to the list of statusbot channels https://review.openstack.org/181673 | 14:44 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Add kilo compat jobs to tempest and devstack-gate https://review.openstack.org/181907 | 14:44 |
| clarkb | that assumes your "public" net is externally routable | 14:44 |
| mtreinish | anteaya: ^^^ | 14:44 |
| mordred | clarkb: I believe that property is set when it's the network you use to get outside of your current cloud | 14:45 |
| *** mtanino has joined #openstack-infra | 14:45 | |
| mordred | clarkb: there are 4 possibilities - nova-net direct route, nova-net floating ip, neutron direct route, neutron floating-ip. currently we have no known clouds that are nova-net floating ip - so let's ignore that for a sec | 14:45 |
| clarkb | mordred: I am reading api docs now and unfortunately that response parameter is not documented other than being in the example json | 14:45 |
| mordred | for nova-net direct route, the best we can do is look at network name | 14:45 |
| mordred | clarkb: ok. how about we look for the flag, and if it's there, we use it, and if it's not, we fall back to heuristics basd on network name? | 14:46 |
| *** ashleighfarnham has joined #openstack-infra | 14:46 | |
| anteaya | mtreinish: thank you | 14:46 |
| clarkb | mordred: also the example network is called "private-network" but it is router:external = true | 14:46 |
| fungi | i agree that public vs private is a poor distinction for generalized use cases. if "externally-routable" means "the interface through which i reach my default gateway" then that's getting closer. but ultimately i think people want "the address for the interface through which nodepoold will know/reach me" | 14:47 |
| *** freerunner has quit IRC | 14:47 | |
| mordred | clarkb: laptop rebooted. one sec | 14:47 |
| dguerri | http://imgur.com/TVffcx4 | 14:49 |
| dguerri | I am pretty sure that there is a pattern here | 14:49 |
| clarkb | looking at hpcloud this should work there, not sure if it would work in general neutron case | 14:49 |
| *** sdake_ has joined #openstack-infra | 14:49 | |
| dguerri | could be morse, could be binary | 14:49 |
| mordred | clarkb: agree | 14:50 |
| mordred | clarkb: so - I think we have a better case to tell people "we're going to try to loko for a network which is marked as router:external=true | 14:50 |
| fungi | jeblair: on the "tracking age of zuul-merger git refs" front, at one point we determined that we couldn't find arbitrary refs even if we encoded something like creation epoch time into them... but i was just thinking, what if we used lightweight tags? e.g. have zuul create a corresponding unique tag for each and include a timestamp in the tag name? then we could enumerate, sort and cull tags beyond a | 14:50 |
| fungi | specific marker | 14:50 |
| mordred | clarkb: and if your neutron is weird, we'll handle that as a best-effort next step | 14:50 |
| clarkb | mordred: would probably be good to have zxiiro confirm that for the opendaylight network since we are trying to address zxiiro's issue | 14:51 |
| *** fhubik is now known as fhubik_afk | 14:51 | |
| mordred | indeed | 14:51 |
| *** carl_baldwin has joined #openstack-infra | 14:51 | |
| mordred | zxiiro: can you check your neutron "opendaylight" network and see if it has router:external=true set? | 14:51 |
| *** fhubik_afk is now known as fhubik | 14:51 | |
| *** e0ne is now known as e0ne_ | 14:51 | |
| clarkb | zxiiro: can you do a `neutron net-show opendaylight` and `neutron net-show private` and tell us if the router:external flag values are correct | 14:51 |
| *** fhubik is now known as fhubik_afk | 14:51 | |
| clarkb | mordred: we need it on the private network too because if that is also external then we will get confused and have a bad time | 14:52 |
| zxiiro | where's this neutron command come from? | 14:52 |
| *** achanda has joined #openstack-infra | 14:52 | |
| clarkb | zxiiro: python-neutronclient installation | 14:52 |
| *** sdake has quit IRC | 14:52 | |
| clarkb | nodepool should install it if you run from that env | 14:52 |
| zxiiro | ah one sec let me see if it has it | 14:52 |
| mordred | clarkb: yah - we would hope that opendaylight has router:external and private does not | 14:53 |
| clarkb | oh wait nodepool currently only uses novaclient? I may be wrong about it installing neutronclient for you. Sorry | 14:53 |
| *** smarcet has joined #openstack-infra | 14:54 | |
| *** jaypipes has quit IRC | 14:54 | |
| mordred | yah. that's one of the reasons we haven't tried to do smarter things in this area with nodepool yet, because then we'd have to plumb through a neutronclient | 14:54 |
| *** reed_ has joined #openstack-infra | 14:54 | |
| *** deepakcs has quit IRC | 14:54 | |
| mordred | which is why I was saying it'll be easier when we have shade available because multi-client is better supported there | 14:54 |
| mordred | we'll also be able to cut down on our api calls for floating ips, which will be nice | 14:55 |
| mordred | but - those are not happening this week or probably this month | 14:55 |
| zxiiro | yeah my nodepool install doesn't have it. i'm guessing i can pip install it? | 14:55 |
| mordred | yah | 14:55 |
| jeblair | fungi: wait, why wouldn't "refs/zuul/<year>/<month>/<random>" work? | 14:56 |
| clarkb | mordred: found why it wasn't documented. Its a "Network provider extended attributes" extension to the nova api. The docs for that indicate the value should be true if network is externally accessible | 14:56 |
| annegentle | whew! clarkb :) | 14:56 |
| mordred | clarkb: ah - so you can also get to it via nova api? | 14:56 |
| *** maishsk has quit IRC | 14:56 | |
| *** freerunner has joined #openstack-infra | 14:56 | |
| clarkb | mordred: so not all clouds will have it (why are extensions a good thing?) but thats fine. Bigger issue is if the flag isn't set consistently | 14:56 |
| mordred | yah | 14:56 |
| *** reed_ has quit IRC | 14:57 | |
| mordred | but it's certainly a thing we can add into the front-line of checking - and one that we can be more rigteously indignant about if it's set to true and the network is not, in fact, externally routable :) | 14:57 |
| clarkb | mordred: hpcloud doesn't like novaclient doing net-list or network-list | 14:57 |
| zxiiro | i tried pip install neutronclient and it's nto a package | 14:57 |
| zxiiro | we're using rackspace if that helps? | 14:57 |
| mordred | zxiiro: pip install python-neutronclient | 14:57 |
| fungi | jeblair: oh, good point. for some reason i thought we'd ruled out specific refnames in the past but can't remember why now | 14:57 |
| zxiiro | ah there we go, strange that a pypi package has python- prefix | 14:58 |
| *** e0ne_ is now known as e0ne | 14:58 | |
| fungi | zxiiro: historical reasons... stemming from when it was released on launchpad instead | 14:59 |
| *** rwsu has joined #openstack-infra | 14:59 | |
| mordred | clarkb: yeah - so - honestly I'm not going to spend a ton of time solving for the nova-network case, beacuse 100% of the nova-network providers we have give us ips and set them in accessIPv4 properly | 14:59 |
| fungi | zxiiro: repository names were matched to distro package names, then got repeated on pypi | 14:59 |
| mordred | clarkb: neutron gets hairier - but also has api bits we can introspect | 14:59 |
| clarkb | mordred: ya I think my only concern now is that opendaylight will be external false or privte will be external true and we won't actually fix zxiiro's issue | 14:59 |
| clarkb | but thats when we/you/zxiiro yells at the cloud in question to fix it | 15:00 |
| mordred | yah | 15:00 |
| *** achanda has quit IRC | 15:01 | |
| clarkb | and if you don't have the extension we continue to do the brittle thing we do today | 15:01 |
| mordred | clarkb: OH - there may be another use case here that we may still want to solve with configuration ... | 15:01 |
| mordred | clarkb: which is "what if you have a network that is self-contained (all private) and you run both nodepool and your tests in it- and none of your nodes need external access" | 15:01 |
| *** alexpilotti has joined #openstack-infra | 15:01 | |
| mordred | I tink in that case, we'll require that you configure somethign explicitly | 15:01 |
| mordred | becasue there's no way we're going to figure that out | 15:01 |
| mordred | but I think you shoudl not have to explicitly configure for non-pathological cases | 15:02 |
| *** dustins_ has joined #openstack-infra | 15:02 | |
| clarkb | annegentle: is it worth filing a bug that a network called "private-network" is externally routable? | 15:02 |
| clarkb | annegentle: sorry in the network v2.0 api doc response example | 15:02 |
| mordred | clarkb: I think so. I think that's confusing | 15:03 |
| *** sputnik13 has joined #openstack-infra | 15:03 | |
| annegentle | clarkb: hm. the sample name is confusing? Yeah, sure. low-hanging-fruit too :) | 15:03 |
| annegentle | we can use it for the upstream training :) | 15:03 |
| mordred | might be less work to submit a patch :) | 15:03 |
| fungi | though _technically_ you can have a functionally private network assigned from a globally-routable allocation but completely firewalled or even air-gapped | 15:03 |
| fungi | but yes, i agree it's confusing to use such a configuration in a documentation example | 15:04 |
| clarkb | fungi: ya, I think for the illustration of what it means in the context of 90% of neutron deployments this is confusing | 15:04 |
| mordred | yup. I think we accomodate the pathological case in openstack too much when we talk about what you can vs. what you should od | 15:04 |
| mordred | do | 15:04 |
| mordred | you SHOULD have a network that gets off the cloud - that network SHOULD be marked router:external | 15:04 |
| mordred | you _Can_ do anything | 15:05 |
| fungi | be careful to separate "accommodate pathological configurations" from "be liberal in what you accept" | 15:05 |
| zxiiro | odd i get unauthorized | 15:05 |
| zxiiro | but my nova command works | 15:05 |
| *** dustins has quit IRC | 15:05 | |
| mordred | fungi: I'd say "have a network labeled private that is publically routable ips but is air-gapped" is pathological | 15:05 |
| fungi | the former is a design-by-committee complication issue, the latter is a robust design principle (and sometimes they can look similar) | 15:05 |
| clarkb | zxiiro: maybe try a `nova network-show private` instead? | 15:05 |
| *** ams__ has joined #openstack-infra | 15:06 | |
| *** zz_jgrimm is now known as jgrimm | 15:06 | |
| mordred | fungi: while neutron will let you do that - it shoudn't be the focus of the docs :) | 15:06 |
| fungi | completely agree | 15:06 |
| zxiiro | ERROR (CommandError): No network with a name or ID of 'private' exists. | 15:06 |
| zxiiro | same when i change it to "opendaylight" too | 15:06 |
| *** sks has quit IRC | 15:06 | |
| clarkb | so I think this is where we get into rax neutron deployment is special | 15:06 |
| *** btully has quit IRC | 15:07 | |
| clarkb | and I have no idea how to deal with that as we don't use it. We just rely on them to give us an IP address that works | 15:07 |
| mordred | awesome | 15:07 |
| mordred | zxiiro: ok - so we may still have work to do for you | 15:07 |
| clarkb | (its weird to me that more and more people are actively bypassing that in rax since it is rax's best feature) | 15:07 |
| *** sarob has joined #openstack-infra | 15:07 | |
| mordred | zxiiro: but it'll probably be at least a month until we're able to think about it for real | 15:07 |
| openstackgerrit | Sirushti Murugesan proposed openstack-infra/project-config: Enable non-voting py34 check job for Heat https://review.openstack.org/182339 | 15:07 |
| mordred | clarkb: youknow - _all_ of the other clouds I've tested give you an ip for your vm | 15:08 |
| zxiiro | mordred: sure no problem we have a workaround that's working for us right now so we're fine for the time being | 15:08 |
| mordred | zxiiro: awesome | 15:08 |
| *** banix has joined #openstack-infra | 15:08 | |
| mordred | clarkb: well, I need to go re-test dreamhost - I didn't check that last time I poked them - they may also require floaters | 15:08 |
| zxiiro | I will be at the OpenDaylight Summit too | 15:08 |
| *** hemnafk is now known as hemna | 15:09 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack-infra/project-config: Make gate-sahara-tox-py27-scenario-unit job voting https://review.openstack.org/182238 | 15:09 |
| openstackgerrit | Sirushti Murugesan proposed openstack-infra/project-config: Enable non-voting py34 check job for Heat https://review.openstack.org/182339 | 15:09 |
| *** fhubik_afk is now known as fhubik | 15:11 | |
| *** cdent has quit IRC | 15:12 | |
| *** panda has quit IRC | 15:13 | |
| *** banix has quit IRC | 15:13 | |
| *** panda has joined #openstack-infra | 15:13 | |
| *** aswadr has joined #openstack-infra | 15:14 | |
| *** patrickeast has joined #openstack-infra | 15:14 | |
| *** hdd has quit IRC | 15:14 | |
| *** fhubik has quit IRC | 15:14 | |
| *** pcrews has quit IRC | 15:14 | |
| openstackgerrit | Ruby Loo proposed openstack/requirements: Avoid httpretty 0.8.9 as it breaks unittests https://review.openstack.org/182340 | 15:14 |
| *** afazekas_ has quit IRC | 15:15 | |
| *** shashankhegde has joined #openstack-infra | 15:15 | |
| *** asrangne has quit IRC | 15:17 | |
| *** arxcruz has quit IRC | 15:17 | |
| *** ayoung has joined #openstack-infra | 15:17 | |
| *** sabeen has joined #openstack-infra | 15:18 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Deal with yum --downloadonly CentOS 6.x quirks https://review.openstack.org/181574 | 15:19 |
| *** dkranz has joined #openstack-infra | 15:19 | |
| *** redixin has joined #openstack-infra | 15:19 | |
| *** aswadr has quit IRC | 15:20 | |
| redixin | hi. could anybody tell why gerrit user ci50 can't post comments? | 15:21 |
| clarkb | redixin: it can't post comments at all or it can't vote? | 15:21 |
| redixin | clarkb, it cant post at all | 15:21 |
| redixin | it has form for comment in web ui, form for Vote: and Code-Review: | 15:22 |
| redixin | i can submit this form, but got invalid json in response | 15:22 |
| clarkb | annegentle: https://bugs.launchpad.net/openstack-api-site/+bug/1454302 | 15:23 |
| openstack | Launchpad bug 1454302 in openstack-api-site "Network v2.0 API network show response confusing." [Undecided,New] | 15:23 |
| redixin | here what i got in reply to POST: )]}' {"labels":{"Verified":0,"Code-Review":0}} | 15:23 |
| clarkb | redixin: the invalid json is expected and documented by the gerrit rest api docs | 15:24 |
| clarkb | redixin: if you want to vote you need to be in that projects ci group | 15:24 |
| redixin | clarkb, ok. but i neet to post. and it is in projects ci group | 15:25 |
| *** dannywilson has joined #openstack-infra | 15:25 | |
| *** derekh_ has joined #openstack-infra | 15:25 | |
| clarkb | redixin: can you show me what your POST looks like? (remove the password) | 15:25 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Deal with yum --downloadonly CentOS 6.x quirks https://review.openstack.org/181574 | 15:25 |
| *** markus_z has quit IRC | 15:25 | |
| redixin | clarkb, {"labels":{"Verified":0, "Code-Review":0}, "strict_labels":true, "drafts":"PUBLISH", "message":"test comment"} | 15:26 |
| *** krtaylor has quit IRC | 15:26 | |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add keystone service resource methods https://review.openstack.org/177621 | 15:26 |
| redixin | I see this in firefox inspector | 15:26 |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add keystone endpoint resource methods https://review.openstack.org/180328 | 15:26 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add grafyaml into openstack-infra https://review.openstack.org/182045 | 15:26 |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add floating IP pool resource methods https://review.openstack.org/180255 | 15:26 |
| clarkb | fungi: https://review.openstack.org/#/c/181742/1 is the base of a stack that you have otherwise approved. Were you going to review that one too? | 15:26 |
| redixin | clarkb, also I trying to post via ssh, but got "unathorized" | 15:26 |
| openstackgerrit | Davide Guerri proposed openstack-infra/shade: Add port resource methods https://review.openstack.org/179968 | 15:27 |
| redixin | clarkb, but i still can run "gerrit stream-events" | 15:27 |
| clarkb | redixin: can you show me your ssh gerrit review command? | 15:27 |
| redixin | clarkb, I use paramiko | 15:27 |
| clarkb | redixin: ok, can you show me the command that you are running against gerrit? | 15:27 |
| *** Ala has quit IRC | 15:27 | |
| redixin | clarkb, give me a minute. ill try via ssh command line | 15:28 |
| *** dimtruck is now known as zz_dimtruck | 15:28 | |
| *** scheuran has quit IRC | 15:28 | |
| *** esker has joined #openstack-infra | 15:28 | |
| clarkb | redixin: if I had to guess your account doesn't have voting permissions and I should be able to check all that if given a concrete review command | 15:28 |
| *** derekh has quit IRC | 15:29 | |
| *** sergsh has quit IRC | 15:29 | |
| *** dguitarbite has quit IRC | 15:31 | |
| *** zz_dimtruck is now known as dimtruck | 15:32 | |
| *** annegentle has quit IRC | 15:32 | |
| *** sputnik13 has quit IRC | 15:32 | |
| *** sputnik1_ has joined #openstack-infra | 15:32 | |
| anteaya | I can't find a reviewer ci50 using gerrit's autocomplete | 15:32 |
| redixin | clarkb, ssh -p 29418 ci50@review.openstack.org gerrit review -m '"Test comment"' eaf8718c296b3 | 15:32 |
| redixin | clarkb, it return no error, but comment does not appear | 15:33 |
| redixin | clarkb, the same ssh -p 29418 ci50@review.openstack.org gerrit review -m '"Test comment"' --verified=-1 eaf8718c296b3 | 15:33 |
| *** dkranz has quit IRC | 15:33 | |
| *** jtomasek has quit IRC | 15:33 | |
| *** julim has quit IRC | 15:33 | |
| anteaya | redixin: you are testing on the ci-sandbox repo, correct? http://git.openstack.org/cgit/openstack-dev/ci-sandbox/ | 15:34 |
| redixin | anteaya, no | 15:34 |
| *** yamahata has joined #openstack-infra | 15:34 | |
| anteaya | please do so | 15:34 |
| redixin | anteaya, it was working last week | 15:34 |
| openstackgerrit | Vincent Fournier proposed openstack-infra/project-config: Surveil:py34 checks https://review.openstack.org/182353 | 15:34 |
| anteaya | no repo wants ci's testing on their patches | 15:34 |
| anteaya | redixin: what is the name of the account, besides ci50? | 15:35 |
| redixin | anteaya, Mirantis Rally CI | 15:35 |
| clarkb | redixin: I see multiple test comments on that change | 15:35 |
| clarkb | https://review.openstack.org/#/c/180144/ | 15:35 |
| clarkb | so I think it is working | 15:35 |
| anteaya | this one: https://review.openstack.org/#/q/reviewer:%22Mirantis+Rally+CI+%253Credixin%2540yandex.ru%253E%22,n,z | 15:36 |
| anteaya | there are two Mirantis Rally CI accounts | 15:36 |
| *** hdd has joined #openstack-infra | 15:36 | |
| clarkb | pleia2: I have to go pick up my mom from the airport shortly but want to finish working through the nodepool + devstack testing doc today if you are around | 15:36 |
| *** yamahata has quit IRC | 15:36 | |
| redixin | clarkb, oh. sorry. Toggle CI was disabled somehow :-\ | 15:37 |
| *** dizquierdo has quit IRC | 15:37 | |
| *** yamahata has joined #openstack-infra | 15:37 | |
| fungi | redixin: are you perhaps browsing with... | 15:37 |
| fungi | yep, was about to suggest that ;) | 15:37 |
| *** krtaylor has joined #openstack-infra | 15:38 | |
| *** koolhead17 has joined #openstack-infra | 15:38 | |
| *** koolhead17 has quit IRC | 15:38 | |
| *** koolhead17 has joined #openstack-infra | 15:38 | |
| fungi | redixin: it defaults to hiding comments from all ci systems to keep the reviews more readable | 15:38 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Switch on requirements check for Magnum https://review.openstack.org/182337 | 15:39 |
| *** sabeen2 has joined #openstack-infra | 15:39 | |
| fungi | okay, disappearing for a while to run some errands. i'll try to get back in via wireless modem as time allows, and hope to be back by time for the infra meeting | 15:39 |
| *** gyee has joined #openstack-infra | 15:41 | |
| *** freerunner has quit IRC | 15:41 | |
| *** Swami has joined #openstack-infra | 15:42 | |
| *** erikmwil_ has joined #openstack-infra | 15:43 | |
| *** sabeen has quit IRC | 15:43 | |
| *** banix has joined #openstack-infra | 15:43 | |
| *** erikmwil_ is now known as erikmwilson_ | 15:43 | |
| *** erikmwilson is now known as Guest28213 | 15:43 | |
| *** erikmwilson_ is now known as erikmwilson | 15:43 | |
| *** erikmwilson_ has joined #openstack-infra | 15:44 | |
| openstackgerrit | greghaynes proposed openstack-infra/shade: Use fakes instead of mocks for data objects https://review.openstack.org/182126 | 15:44 |
| clarkb | I should note that I expect to be at the meeting so pleia2 maybe we can plan for post lunch nodepool fixing? | 15:45 |
| openstackgerrit | Merged openstack-infra/project-config: Don't require NODEPOOL_SCRIPTDIR to be set https://review.openstack.org/181742 | 15:45 |
| *** jaypipes has joined #openstack-infra | 15:45 | |
| openstackgerrit | Andy McGuigan proposed openstack-infra/project-config: Add the coverage jobs for VPNaaS and FWaaS https://review.openstack.org/182359 | 15:46 |
| clarkb | greghaynes: ^ my plan is to work through the rest of the nodepool + devstack testing stuff, then run you and mordred's use shade change against it | 15:48 |
| greghaynes | clarkb: ok, I should do the same | 15:49 |
| *** dkranz has joined #openstack-infra | 15:49 | |
| *** Krinkle|detached is now known as Krinkle | 15:49 | |
| clarkb | greghaynes: https://review.openstack.org/#/c/179840/3/doc/source/testing.rst is the doc if you don't have it handy anymore | 15:49 |
| greghaynes | seems like a nice thing to have setup for nodepool dev | 15:49 |
| *** links has joined #openstack-infra | 15:50 | |
| *** jistr has quit IRC | 15:50 | |
| mordred | clarkb: cool | 15:50 |
| mordred | clarkb: I expect the use shade change to be a complete no-op, btw | 15:50 |
| mordred | clarkb: but I agree, _testing_ that it's a no-op would be fanastic | 15:50 |
| clarkb | yup | 15:50 |
| clarkb | I am going to try it in both a "new" install basically nothing in the DB and as an existing install with an image/node already there | 15:51 |
| clarkb | I expect it to just work but would rather find out before we go to production | 15:51 |
| *** jamesmcarthur has joined #openstack-infra | 15:52 | |
| *** freerunner has joined #openstack-infra | 15:52 | |
| greghaynes | I thought we all decided yesterday that downgrades are fun though? | 15:52 |
| *** Guest29018 has quit IRC | 15:53 | |
| *** pcrews has joined #openstack-infra | 15:53 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Test for `~/.pydistutils.cfg` existence before copying it https://review.openstack.org/182294 | 15:56 |
| mordred | clarkb: +100 | 15:56 |
| jeblair | greghaynes: they seem to work better than upgrades | 15:57 |
| *** dkranz has quit IRC | 15:57 | |
| *** mtreinish has quit IRC | 15:58 | |
| *** rwsu has quit IRC | 15:58 | |
| greghaynes | hah | 15:59 |
| *** rwsu has joined #openstack-infra | 15:59 | |
| *** gema_ has quit IRC | 15:59 | |
| openstackgerrit | Merged openstack-infra/project-config: Don't disable cloud-init if it is not installed https://review.openstack.org/181750 | 15:59 |
| openstackgerrit | Merged openstack-infra/project-config: Don't disable cloud-init if not installed part 2 https://review.openstack.org/182142 | 15:59 |
| *** spredzy is now known as spredzy|afk | 15:59 | |
| *** mtreinish has joined #openstack-infra | 16:00 | |
| *** smarcet has quit IRC | 16:01 | |
| mordred | hahahaha | 16:01 |
| *** dustins_ has quit IRC | 16:01 | |
| greghaynes | clarkb: re ^ , centos6 in rax is not an option as a result? | 16:01 |
| *** gema_ has joined #openstack-infra | 16:01 | |
| *** banix has quit IRC | 16:01 | |
| yolanda | hi clarkb, i was testing some changes in httpd.conf to make apache boot, it was giving some errors but now the config was fine | 16:02 |
| *** unicell has joined #openstack-infra | 16:02 | |
| yolanda | seems i need to load some other onfigs | 16:02 |
| yolanda | configs | 16:02 |
| *** rbradfor has joined #openstack-infra | 16:03 | |
| *** whoops has joined #openstack-infra | 16:03 | |
| *** dustins has joined #openstack-infra | 16:03 | |
| *** dustins has quit IRC | 16:04 | |
| *** thedodd has quit IRC | 16:04 | |
| *** unicell1 has quit IRC | 16:04 | |
| *** thedodd has joined #openstack-infra | 16:04 | |
| *** pcrews has quit IRC | 16:04 | |
| *** kgiusti has joined #openstack-infra | 16:05 | |
| *** dustins has joined #openstack-infra | 16:05 | |
| *** jlanoux_ has quit IRC | 16:05 | |
| *** marun has joined #openstack-infra | 16:06 | |
| zxiiro | any core jenkins-job-builder reviewers mind reviewing this small patch? https://review.openstack.org/#/c/182116/ | 16:06 |
| openstackgerrit | Adam Gandelman proposed openstack-infra/system-config: Add #akanda to openstack bot's channel list https://review.openstack.org/182368 | 16:07 |
| *** gema_ has quit IRC | 16:07 | |
| *** ivar-lazzaro has joined #openstack-infra | 16:07 | |
| clarkb | greghaynes: it is using snapshots | 16:08 |
| *** dkranz has joined #openstack-infra | 16:08 | |
| clarkb | yolanda: so it was missing apache config? | 16:08 |
| *** gema_ has joined #openstack-infra | 16:08 | |
| greghaynes | clarkb: yea, I dont mean our nodepool-dib stuff | 16:09 |
| *** thedodd has quit IRC | 16:09 | |
| clarkb | greghaynes: oh, I think rax does an out of band cloud init install that is unaffected | 16:09 |
| yolanda | clarkb, so notnownikki solved that, it needed a Include conf.modules.d/*.conf , to load systemd specific config | 16:09 |
| greghaynes | huh | 16:09 |
| clarkb | greghaynes: because rax hand makes all of their own images rather than pulling from the distros like hpcloud | 16:09 |
| greghaynes | well, thats good for us in a bad way | 16:10 |
| clarkb | yup :) | 16:10 |
| pleia2 | clarkb: after lunch nodepool docs fiddling sounds good :) | 16:10 |
| *** ivar-lazzaro has quit IRC | 16:10 | |
| clarkb | pleia2: cool, good catch on the needing to create db and using disable password with adduser. | 16:11 |
| *** ivar-lazzaro has joined #openstack-infra | 16:11 | |
| *** openstackgerrit_ has quit IRC | 16:11 | |
| clarkb | pleia2: as far as what connection string to use for the DB I think devstack-gate (which I was using because my ultimate goal is to run tests like this upstream) sets up different passwords than devstack default :/ | 16:11 |
| *** unicell has quit IRC | 16:12 | |
| clarkb | pleia2: we can probably just ask the localrc config for what that value is instead | 16:12 |
| *** ivar-lazzaro has quit IRC | 16:12 | |
| pleia2 | clarkb: hopefully just a few more little things like that until we're there, tricky when I can't log into the nodepool vm to see what's going on because it seems to not complete the ssh bit | 16:13 |
| *** ivar-lazzaro has joined #openstack-infra | 16:13 | |
| pleia2 | clarkb: and yeah, I tried a few things with the database, devstack is weird o_o | 16:13 |
| *** btully has joined #openstack-infra | 16:13 | |
| *** dustins has quit IRC | 16:14 | |
| clarkb | pleia2: ya ssh is tricky, I had a really hard time getting ubuntu user to work before I realized cloud-init was overwriting my ssh authorized keys for me :/ | 16:15 |
| clarkb | thats why I went the route of hard coding the jenkins user in, but that obviously has some fragility of its own | 16:15 |
| pleia2 | clarkb: yeah, I'll try a few more things this morning, sleeping on it was good | 16:15 |
| *** thedodd has joined #openstack-infra | 16:16 | |
| *** craigbr has joined #openstack-infra | 16:16 | |
| pabelanger | how long as the 'Cherry Pick To' button been there on gerrit? | 16:16 |
| pabelanger | I just noticed it now | 16:16 |
| mordred | pabelanger: a while now - 2.8 I think added it | 16:16 |
| *** nfedotov has quit IRC | 16:17 | |
| anteaya | we are back to 2.8 | 16:17 |
| *** alexsyip has joined #openstack-infra | 16:17 | |
| pabelanger | Ya, odd how I just realized it was there | 16:17 |
| anteaya | I never looked on 2.10 to see if it was there | 16:17 |
| *** otter768 has joined #openstack-infra | 16:17 | |
| anteaya | pabelanger: maybe you never needed it before? | 16:17 |
| *** davideagnello has quit IRC | 16:17 | |
| pabelanger | anteaya, Right, never needed, and even now. Tempted to click it to see what happens... hehe | 16:17 |
| pabelanger | at la http://www.reddit.com/r/thebutton | 16:18 |
| zaro | morning | 16:18 |
| anteaya | pabelanger: go to the sandbox repo and try it there | 16:18 |
| *** btully has quit IRC | 16:18 | |
| *** claudiub has joined #openstack-infra | 16:18 | |
| anteaya | morning zaro | 16:18 |
| mordred | it works well - it pops up a thing and asks you want you want to cherry pick it to | 16:18 |
| mordred | makes things like "cherry pick to stable" nice | 16:19 |
| clarkb | I think it doesn't do the cherry-pick -x flag through right? | 16:19 |
| clarkb | which stable maintainers like | 16:19 |
| *** davideagnello has joined #openstack-infra | 16:19 | |
| *** ildikov has quit IRC | 16:19 | |
| zaro | so anybody know a good way to simulate load on review-dev to see if we can repro that gerrit zlib issue? | 16:20 |
| pabelanger | anteaya, Cool, that is pretty handy actually | 16:20 |
| greghaynes | Are there some kind of access logs? | 16:20 |
| anteaya | pabelanger: glad you like it | 16:20 |
| greghaynes | age old method of simulating load | 16:20 |
| clarkb | zaro: we probably want to put a larger repo in review-dev (say nova) then write a script that can push changes/patchsets/comments in a loop | 16:20 |
| clarkb | greghaynes: sort of, we have the apache logs which can probably be distilled into gerrit operations, but gerrit itself doesn't log that iirc | 16:21 |
| zaro | would there any problems with just putting entire review db onto review-dev? | 16:21 |
| clarkb | zaro: we would have to sanitize it first to clean out passwords | 16:21 |
| anteaya | zaro: that was my interpretation of what jeblair wants | 16:21 |
| *** otter768 has quit IRC | 16:21 | |
| clarkb | but other than that we should be able to push that over into the dev server | 16:22 |
| sdague | clarkb: there are passwords in gerrit? | 16:22 |
| clarkb | sdague: your http api password yet | 16:22 |
| clarkb | *yes | 16:22 |
| sdague | oh, gotcha. | 16:22 |
| zaro | will also need the review_site/git directory as well | 16:23 |
| clarkb | sdague: also the reason that Gerrit gerrit and Android gerrit don't ssh is because they are all googley | 16:23 |
| clarkb | sdague: so they are barely even Gerrit as we know it | 16:23 |
| jeblair | we should also make sure it doesn't send email | 16:23 |
| jeblair | either by manipulating the db, gerrit configuration, or mta config on the host | 16:24 |
| anteaya | clarkb: I don't see the "cherry picked from ..." in the commit message when I use the cherry pick button, so I'm going with no on the -x flag question | 16:24 |
| clarkb | I would not be surprised if they run a completely different jgit too. We do know they don't use the ssh stuff because they have that large http infrastructure and they don't use a SQL database. Instead its like bigtable and something else | 16:24 |
| clarkb | oh and iirc their git stuff isn't on a filesystem its in a distributed DB like bigtable | 16:24 |
| *** davideagnello has quit IRC | 16:25 | |
| *** davideagnello has joined #openstack-infra | 16:26 | |
| sdague | clarkb: gotcha, I'm tempted to write some code that actually lists changes against there API to actually judge the size of it compared to ours | 16:26 |
| clarkb | sdague: this is why gerrit uses guice extensively despite not actually needing it in the open source version | 16:26 |
| clarkb | sdague: there is very little dependency injection that is configurable for us in the real world | 16:27 |
| clarkb | I think we can choose jetty or some other http server and thats about it | 16:28 |
| *** freerunner has quit IRC | 16:28 | |
| sdague | ok | 16:28 |
| zaro | ohh they also do some funky thing with their http passwords. i think they set it up to be stored somewhere else, not in gerrit. | 16:29 |
| zaro | so if some infra core wants to prep the db data and git file contents i can work on putting that on review-dev. | 16:31 |
| *** julim has joined #openstack-infra | 16:31 | |
| *** dboik_ has joined #openstack-infra | 16:31 | |
| *** tiswanso has quit IRC | 16:31 | |
| *** aduarte has joined #openstack-infra | 16:31 | |
| zaro | with email turned off of course, probably just disable the config or just turn off smtp. | 16:31 |
| *** tiswanso_ has joined #openstack-infra | 16:32 | |
| clarkb | ya probably easiest to iptables or exim rule email away | 16:32 |
| clarkb | since gerrit doesn't make email super configurable | 16:32 |
| *** sputnik13 has joined #openstack-infra | 16:33 | |
| *** dboik has quit IRC | 16:33 | |
| *** nijaba has quit IRC | 16:34 | |
| clarkb | I have to head out now or would start looking at a db dump | 16:34 |
| clarkb | zaro: ping me after the meeting if no one else has volunteered | 16:34 |
| *** ivar-lazzaro has quit IRC | 16:34 | |
| *** afazekas_ has joined #openstack-infra | 16:35 | |
| *** yamahata has quit IRC | 16:35 | |
| *** ssam2 has quit IRC | 16:36 | |
| *** annegentle has joined #openstack-infra | 16:36 | |
| *** sputnik1_ has quit IRC | 16:36 | |
| zaro | jeblair: i don't think we need this anymore, https://review.openstack.org/#/c/182060/ would you mind removing your workflow+1 vote? i don't want zuul to run and merge it. | 16:37 |
| *** mestery has joined #openstack-infra | 16:37 | |
| *** mestery has quit IRC | 16:38 | |
| *** claudiub has quit IRC | 16:38 | |
| zaro | jeblair: ohh, actually i just abandoned. | 16:39 |
| *** gema_ has quit IRC | 16:39 | |
| jeblair | that works too :) | 16:39 |
| *** claudiub has joined #openstack-infra | 16:41 | |
| *** gema_ has joined #openstack-infra | 16:41 | |
| *** annegentle has quit IRC | 16:41 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Update ci for F21 jenkins nodes https://review.openstack.org/182219 | 16:42 |
| *** afazekas_ has quit IRC | 16:42 | |
| *** pcrews has joined #openstack-infra | 16:43 | |
| *** dustins has joined #openstack-infra | 16:43 | |
| *** dustins has quit IRC | 16:43 | |
| *** patrickeast has quit IRC | 16:44 | |
| *** derekh_ has quit IRC | 16:45 | |
| *** unicell has joined #openstack-infra | 16:47 | |
| *** emagana has quit IRC | 16:48 | |
| *** freerunner has joined #openstack-infra | 16:48 | |
| *** jamesmcarthur has quit IRC | 16:50 | |
| *** emagana has joined #openstack-infra | 16:50 | |
| *** jcoufal has quit IRC | 16:52 | |
| openstackgerrit | Vincent Fournier proposed openstack-infra/project-config: Surveil:py34 checks https://review.openstack.org/182353 | 16:54 |
| *** rfolco has quit IRC | 16:55 | |
| *** yfried has quit IRC | 16:55 | |
| *** banix has joined #openstack-infra | 16:58 | |
| *** hdd_ has joined #openstack-infra | 17:02 | |
| *** emagana has quit IRC | 17:02 | |
| *** dboik_ has quit IRC | 17:02 | |
| *** mtreinish has quit IRC | 17:02 | |
| *** hdd has quit IRC | 17:02 | |
| *** esker has quit IRC | 17:02 | |
| *** ashleighfarnham has quit IRC | 17:02 | |
| *** dprince has quit IRC | 17:02 | |
| *** vponomaryov has quit IRC | 17:02 | |
| *** rkukura has quit IRC | 17:02 | |
| *** gilliard has quit IRC | 17:02 | |
| *** jogo has quit IRC | 17:02 | |
| *** pfallenop has quit IRC | 17:02 | |
| *** bcourt has quit IRC | 17:02 | |
| *** tteggel has quit IRC | 17:02 | |
| *** jpeeler has quit IRC | 17:02 | |
| *** sc` has quit IRC | 17:02 | |
| *** gibi has quit IRC | 17:02 | |
| *** asilenkov has quit IRC | 17:02 | |
| *** jesusaurus has quit IRC | 17:02 | |
| *** plol has quit IRC | 17:02 | |
| *** ptoohill has quit IRC | 17:02 | |
| *** devananda has quit IRC | 17:02 | |
| *** asilenkov_ has joined #openstack-infra | 17:02 | |
| *** mtreinish_ has joined #openstack-infra | 17:02 | |
| *** Guest25580 is now known as dan | 17:02 | |
| *** sc`_ has joined #openstack-infra | 17:02 | |
| *** plol has joined #openstack-infra | 17:02 | |
| *** sc`_ is now known as sc` | 17:02 | |
| *** rkukura has joined #openstack-infra | 17:02 | |
| *** gibi has joined #openstack-infra | 17:02 | |
| *** dprince has joined #openstack-infra | 17:02 | |
| *** ptoohill has joined #openstack-infra | 17:02 | |
| *** devananda has joined #openstack-infra | 17:02 | |
| *** pfallenop has joined #openstack-infra | 17:02 | |
| *** jogo has joined #openstack-infra | 17:02 | |
| *** ashleighfarnham has joined #openstack-infra | 17:02 | |
| *** mtreinish_ is now known as mtreinish | 17:02 | |
| *** emagana has joined #openstack-infra | 17:02 | |
| *** gilliard has joined #openstack-infra | 17:02 | |
| *** jpeeler has joined #openstack-infra | 17:02 | |
| *** dboik has joined #openstack-infra | 17:02 | |
| *** yamahata has joined #openstack-infra | 17:03 | |
| *** bcourt has joined #openstack-infra | 17:03 | |
| *** esker has joined #openstack-infra | 17:03 | |
| *** marcusvrn has quit IRC | 17:03 | |
| *** vponomaryov has joined #openstack-infra | 17:03 | |
| *** jesusaurus has joined #openstack-infra | 17:03 | |
| *** tteggel has joined #openstack-infra | 17:03 | |
| *** flaper87 has quit IRC | 17:03 | |
| *** cpallares has quit IRC | 17:04 | |
| *** cpallares has joined #openstack-infra | 17:04 | |
| zaro | ohh nice work on the new groovy formatted irc archives! | 17:05 |
| *** BharatK has joined #openstack-infra | 17:05 | |
| *** flaper87 has joined #openstack-infra | 17:05 | |
| *** flaper87 has quit IRC | 17:06 | |
| *** flaper87 has joined #openstack-infra | 17:06 | |
| rcarrillocruz | zaro: halo halo | 17:07 |
| *** rfolco has joined #openstack-infra | 17:07 | |
| rcarrillocruz | do you know if it's possible create a user on Gerrit via the SSH interface? From what I see, I can only do with the UI, unless I mess with SQL and such... | 17:08 |
| rcarrillocruz | we run on 2.4, i'm hoping on 2.10 there's something for that? | 17:08 |
| zaro | rcarrillocruz: isn't there a ssh command for that? | 17:08 |
| zaro | rcarrillocruz: ohh 2.4 is pretty limited. | 17:08 |
| rcarrillocruz | well, yeah, but i need to have an admin user already created via UI for that , right? | 17:08 |
| rcarrillocruz | i should've specified that user creation on boostrapping phase... | 17:09 |
| zaro | rcarrillocruz: if there's a command then yeah only admins can create new users | 17:09 |
| rcarrillocruz | and i can't create an admin user via SSH interface, can I? | 17:09 |
| rcarrillocruz | for a Gerrit with SSO auth that is | 17:09 |
| *** sdake has joined #openstack-infra | 17:11 | |
| zaro | yeah, i don't know if you can do that easliy at bootstrap. i think some people have tried and are succesful but process is convoluted. | 17:11 |
| zaro | i have never tried. | 17:11 |
| rcarrillocruz | i'll mail gerrit mailing list or something | 17:12 |
| rcarrillocruz | zaro: btw, y ou going to Vancouver, aren't you? | 17:12 |
| zaro | rcarrillocruz: yes | 17:12 |
| zaro | rcarrillocruz: you? | 17:12 |
| *** ivar-lazzaro has joined #openstack-infra | 17:13 | |
| *** jordanP has quit IRC | 17:13 | |
| *** sergsh has joined #openstack-infra | 17:13 | |
| rcarrillocruz | yup, we'll meet there then. I'll pick your brain during a dinner, we are planning to do the Gerrit migration, so I'd like to hear from you caveats, past experiences, etc | 17:13 |
| fungi | i believe the ssh api had a user create method on 2.4 (and even earlier) | 17:13 |
| zaro | rcarrillocruz: why do you want to do it at bootstrap? seems like it's only a one time thing since you gonna reuse the db on subsequent gerrit updates. | 17:13 |
| *** links has quit IRC | 17:14 | |
| *** shashankhegde has quit IRC | 17:14 | |
| openstackgerrit | Adam Gandelman proposed openstack-infra/project-config: Adds initial akanda dsvm functional tests https://review.openstack.org/182391 | 17:14 |
| rcarrillocruz | fungi: yes, but that requires you connect with a user that has already an SSH key registered in Gerrit... i.e. you must login at least once with the browser and set your user as admin | 17:14 |
| fungi | rcarrillocruz: oh, you want to create the project bootstrappers account so that you have something to use to push in the configuration? | 17:14 |
| *** sdake_ has quit IRC | 17:14 | |
| zaro | rcarrillocruz: which version are you planning to upgrade to? | 17:15 |
| rcarrillocruz | zaro , fungi: i'm writing a tool with ansible that brings up an openstack infra automated, bare services: jenkins, gerrit, zuul, nodepool, etc... As I have it now, the ansible module pauses when Gerrit is installed and asks you 'please create your user on Gerrit on URL foo with your user key and type your chosen username' | 17:15 |
| rcarrillocruz | and i'd like to get rid of that | 17:15 |
| fungi | rcarrillocruz: yolanda scripted that in a former position i think, so she can probably explain what it entails. i think the crux of it was that the bootstrap account was created by directly manipulating the database and possibly groups file in all-projects repo | 17:15 |
| rcarrillocruz | but i can only see doing it with messing sql | 17:16 |
| rcarrillocruz | fungi: yah, she lost access to thse scripts :D :D | 17:16 |
| rcarrillocruz | i already asked her | 17:16 |
| rcarrillocruz | it's a shame | 17:16 |
| rcarrillocruz | zaro: we are going to 2.8 | 17:16 |
| rcarrillocruz | and i'm hoping we will upgrade to 2.10 shortly after that | 17:16 |
| rcarrillocruz | to keep as close to upstream as possible | 17:16 |
| yolanda | ah fungi, i should have kept a copy somewhere but i didn't | 17:17 |
| yolanda | it was mostly sql | 17:17 |
| fungi | rcarrillocruz: so anyway, i believe that's still going to be necessary. chicken and egg problem... you need an account to create an account via any existing api (unless _maybe_ 2.8 added that option for the mostly undocumented "gerrit code review" account which authenticates with the gerrit host key) | 17:17 |
| rcarrillocruz | yeah, i was hoping gerrit would have some sort of initial admin user... but seems not | 17:18 |
| zaro | yeah, unfortunately gerrit has not done anything to help with this siutation | 17:18 |
| rcarrillocruz | the admin is created for the first logged in user | 17:18 |
| fungi | the only things i'm aware you can use that pseudoaccount for are gsql and impersonate functions | 17:18 |
| rcarrillocruz | it's an odd pattern | 17:18 |
| zaro | rcarrillocruz: do you know about this? https://blueprints.launchpad.net/openstack-ci/+spec/gerrit-2.8-upgrade | 17:18 |
| fungi | rcarrillocruz: it's not that uncommon a pattern in the world of classic webapps | 17:18 |
| *** sergsh has quit IRC | 17:18 | |
| fungi | rcarrillocruz: often the assumption was that the admin performed the installation and so they would be the first to log into the service before they made it accessible to the outside | 17:19 |
| *** tiswanso_ has quit IRC | 17:19 | |
| *** hashar has quit IRC | 17:19 | |
| zaro | rcarrillocruz: that's our history of upgrading from 2.2 (i think) to 2.8 | 17:19 |
| fungi | (often by setting it up in the build room via a cross-over cable or with a web browser installed and pointed at the loopback interface, then wheeling it into the datacenter and hooking it up to the big pipe) | 17:20 |
| rcarrillocruz | heh | 17:20 |
| fungi | really wasn't _that_ long ago that we still deployed servers that way. i would imagine some organizations _still_ do that | 17:21 |
| rcarrillocruz | zaro: yeah, glanced at it this week, but need to dive... when back from Vancouver I guess | 17:21 |
| *** tiswanso_ has joined #openstack-infra | 17:21 | |
| *** harlowja has quit IRC | 17:22 | |
| *** harlowja has joined #openstack-infra | 17:22 | |
| *** mpaolino has joined #openstack-infra | 17:22 | |
| *** marcusvrn has joined #openstack-infra | 17:22 | |
| *** mpaolino has quit IRC | 17:22 | |
| *** patrickeast has joined #openstack-infra | 17:25 | |
| zaro | rcarrillocruz: i think wenlock might have worked on this same issue for forj | 17:26 |
| rcarrillocruz | oh yeah | 17:26 |
| rcarrillocruz | you're right | 17:26 |
| rcarrillocruz | hadn't thought about forj folks.... | 17:26 |
| rcarrillocruz | wenlock: ^ you around? | 17:26 |
| *** tiswanso_ has quit IRC | 17:26 | |
| wenlock | 1 sec, leme read | 17:26 |
| * rcarrillocruz goes check forj github site | 17:26 | |
| wenlock | rcarrillorcruz, what time is it good to start reading for the thread? | 17:27 |
| *** Vinod___ has joined #openstack-infra | 17:27 | |
| rcarrillocruz | 20min ago | 17:28 |
| *** tiswanso has joined #openstack-infra | 17:28 | |
| rcarrillocruz | was asking basicly if it's possible to create an admin user on a just installed Gerrit | 17:28 |
| wenlock | rcarrillocruz, yeah, so for first admin user, we have a puppet manifest doing that for us | 17:28 |
| clarkb | yes git review tests do it | 17:28 |
| rcarrillocruz | sweet | 17:29 |
| wenlock | leme point you to the code we use for that, 1 sec | 17:29 |
| *** signed8bit is now known as signed8bit_ZZZzz | 17:29 | |
| clarkb | wenlock ever co sider pushing that up? | 17:29 |
| *** HeOS has quit IRC | 17:29 | |
| clarkb | should be much easier now with the split out module but maybe not | 17:29 |
| wenlock | we did, we lost traction on it when we switched to getting DevOps inner source work going | 17:29 |
| wenlock | i still need to circle around to that clarkb, i have lots of abandoned changs that havn't gone through the system-config switch | 17:30 |
| *** amitgandhinz has quit IRC | 17:30 | |
| wenlock | rcarrillocruz https://github.com/forj-oss/redstone/blob/master/puppet/modules/gerrit_config/manifests/createfirstaccount.pp | 17:30 |
| rcarrillocruz | aww | 17:30 |
| rcarrillocruz | awesome | 17:30 |
| rcarrillocruz | thx a bunch wenlock | 17:30 |
| *** achanda has joined #openstack-infra | 17:30 | |
| wenlock | rcarrillocruz np, leme know if you run into problems using those guys | 17:31 |
| wenlock | rcarrillocruz, this one too https://github.com/forj-oss/redstone/blob/master/puppet/modules/gerrit_config/manifests/firstopenidadmin.pp | 17:31 |
| *** pc_m has quit IRC | 17:31 | |
| wenlock | 1 is for the service account the other is for a person | 17:32 |
| fungi | rcarrillocruz: on 2.8 at least, it looks like you can probably `ssh -i review_site/etc/ssh_host_rsa_key -p 29418 'Gerrit Code Review'@127.0.0.1 gerrit create-account ...` | 17:32 |
| wenlock | indeed | 17:32 |
| rcarrillocruz | oh... | 17:32 |
| * rcarrillocruz goes try | 17:32 | |
| clarkb | fungi we do it in git review tests | 17:33 |
| clarkb | you have to manipulate db directly | 17:33 |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-cgit: Updated config for CentOS 7 and Apache 2.4 https://review.openstack.org/182394 | 17:33 |
| wenlock | but yeah, it's a chicken/egg problem | 17:33 |
| wenlock | you have to have an account to do the create-account thing to begin with | 17:33 |
| rcarrillocruz | damn, i teared down my gerrit 1h ago, i'll create a new one | 17:33 |
| fungi | clarkb: oh, really? using the "gerrit code review" pseudoaccount doesn't work? | 17:33 |
| fungi | the help output indicates that create-account is available but i haven't tried it | 17:34 |
| clarkb | maybe it does now, definitely didnt before | 17:34 |
| fungi | i know in the 2.4 days it only allowed suexec and gsql | 17:34 |
| fungi | but it's possible they've broadened that restriction | 17:34 |
| asselin | my 3rd party ci zuul stopped working. Recent change? apache error log has "File does not exist: /var/lib/zuul/www/lib" | 17:35 |
| wenlock | fungi that would be nice | 17:35 |
| asselin | zuul ui | 17:35 |
| wenlock | means all you would need is something like java -jar ... gerrit.war create-account blah | 17:35 |
| wenlock | right? | 17:35 |
| wenlock | no need for gerrit to be even running yet | 17:35 |
| fungi | wenlock: no idea about that. i'm talking about with the command i pasted above | 17:36 |
| fungi | using the host key to authenticate as the built-in "Gerrit Code Review" account name | 17:37 |
| *** annegentle has joined #openstack-infra | 17:37 | |
| *** dprince has quit IRC | 17:37 | |
| wenlock | fungi, ic... but that command would require an account with the key setup.. | 17:37 |
| fungi | wenlock: nope. there's no actual account called "Gerrit Code Review" | 17:37 |
| yolanda | clarkb, fungi : https://review.openstack.org/#/c/182394/ - to run cgit on centos 7 | 17:38 |
| fungi | wenlock: it's a built-in pseudoaccount identified by the gerrit ssh service's host key | 17:38 |
| *** tmcpeak has joined #openstack-infra | 17:38 | |
| asselin | I see the change Ib0cfdda5ded53af8e89aa239bd2f6e442f2537e3.. but not sure how it gets adjusted.... | 17:38 |
| wenlock | fungi nice | 17:38 |
| *** sc68cal has quit IRC | 17:38 | |
| fungi | it's very, very underdocumented but does still seem to exist on 2.8 | 17:38 |
| jlvillal | clarkb: You had +2 this before, but I had to rebase and then made a change for AJaeger I was hoping you could look at it again as the Ironic PTL would like to enable the Python 3.4 checks to gate Ironic. https://review.openstack.org/#/c/181034/ | 17:39 |
| *** sc68cal has joined #openstack-infra | 17:39 | |
| *** sc68cal has quit IRC | 17:39 | |
| *** sc68cal has joined #openstack-infra | 17:39 | |
| fungi | wenlock: rcarrillocruz: i just confirmed it on review-dev.openstack.org running gerrit 2.10 as well | 17:40 |
| rcarrillocruz | fungi: you are legend :-) | 17:40 |
| fungi | wenlock: rcarrillocruz: ssh -i review_site/etc/ssh_host_rsa_key -p 29418 'Gerrit Code Review'@127.0.0.1 gerrit create-account testuser | 17:40 |
| rcarrillocruz | thx | 17:40 |
| rcarrillocruz | that's way better than messing with sql | 17:40 |
| wenlock | thats awesome | 17:40 |
| rcarrillocruz | will test on my side, spinning gerrit right now... | 17:40 |
| wenlock | fungi, i'll add a user story on our side to update that script | 17:41 |
| wenlock | no need to do it in sql if that works | 17:41 |
| rcarrillocruz | indeed | 17:41 |
| *** annegentle has quit IRC | 17:42 | |
| fungi | just double-checked the account_external_ids table and there is definitely now an account with an external_id of "username:testuser" so it definitely worked | 17:44 |
| *** shashankhegde has joined #openstack-infra | 17:44 | |
| fungi | gerrit documentation really doesn't seem to talk much about that special built-in ssh account | 17:44 |
| *** dboik has quit IRC | 17:45 | |
| *** dboik has joined #openstack-infra | 17:46 | |
| *** claudiub has quit IRC | 17:46 | |
| *** craigbr has quit IRC | 17:46 | |
| *** chlong has quit IRC | 17:46 | |
| *** signed8bit_ZZZzz is now known as signed8bit | 17:47 | |
| *** pc_m has joined #openstack-infra | 17:47 | |
| *** e0ne is now known as e0ne_ | 17:47 | |
| *** craigbr has joined #openstack-infra | 17:49 | |
| pabelanger | Figured I'd ask, however are 3rd party infra vendors launching test nodes of their system-config? For example you want to launch a new nodepool. Aside from manually creating the VM, ssh into back, clone git, etc, anybody have some scripts / apps for it yet? I know I can do manual, but figure somebody must have a script someplace automating most of this, specifically for a test environment | 17:50 |
| *** ihrachyshka has quit IRC | 17:51 | |
| pleia2 | pabelanger: there's a spec coming together to make that all easier, let me dig it up | 17:51 |
| pleia2 | pabelanger: http://specs.openstack.org/openstack-infra/infra-specs/specs/openstackci.html | 17:53 |
| pleia2 | pabelanger: clarkb is also working on some manual nodepool testing docs, that should turn into a scriptable thing in the near future (just ironing out some final wrinkles) https://review.openstack.org/#/c/179840/ | 17:53 |
| asselin | pabelanger, we're using this repo: https://github.com/rasselin/os-ext-testing and trying to migrate to a common solution mentioned in that spec ^^ | 17:54 |
| *** nfedotov has joined #openstack-infra | 17:55 | |
| pabelanger | pleia2, asselin thanks, reading now | 17:55 |
| *** e0ne_ is now known as e0ne | 17:56 | |
| *** tiswanso has quit IRC | 17:59 | |
| pabelanger | Does nodepool support manually launching jobs? EG: from the CLI, vs gerrit | 17:59 |
| pabelanger | I don't think so | 17:59 |
| *** e0ne has quit IRC | 17:59 | |
| *** chlong has joined #openstack-infra | 17:59 | |
| *** tiswanso has joined #openstack-infra | 18:00 | |
| asselin | pabelanger, nodepool doens't launch jobs...it just keeps a poool of jenkins slaves ready to use | 18:00 |
| fungi | pabelanger: nodepool does not, but there are tools in zuul's repo to inject work requests into its gearman queue | 18:00 |
| fungi | pabelanger: or you could trigger them directly in jenkins | 18:00 |
| fungi | either via its webui or its api | 18:00 |
| fungi | you'd just need to supply the same parameters they expect to get from zuul if you go that route | 18:00 |
| *** ajmiller_ has joined #openstack-infra | 18:01 | |
| *** rbradfor has quit IRC | 18:01 | |
| *** chris6131 has joined #openstack-infra | 18:01 | |
| pabelanger | asahlin, right, just thinking a loud. | 18:01 |
| *** dboik_ has joined #openstack-infra | 18:01 | |
| *** btully has joined #openstack-infra | 18:02 | |
| *** ajmiller has quit IRC | 18:03 | |
| pabelanger | fungi, right, I was trying to reduce the dependency on zuul / jenkins / gerrit / etc for this case | 18:04 |
| *** dboik has quit IRC | 18:04 | |
| pabelanger | basically, first step bootstrap single_use_slave. Looks like shade will be my starting point | 18:05 |
| fungi | pabelanger: so to emulate jenkins, you would ssh into the worker nodepool created, set some envvars in your environment and run the same scripts the jobs are configured to run | 18:05 |
| fungi | pabelanger: i have some notes in the devstack-gate repo's readme with a great example of that | 18:05 |
| zaro | fungi: did you fix the replication key to github on review-dev only or on heira? | 18:05 |
| pabelanger | fungi, ya, that's basically what I'm trying to do. For the purpose of testing | 18:05 |
| pabelanger | fungi, Ya, that is what I am readying now too | 18:06 |
| fungi | zaro: locally on review-dev. puppet has likely reverted it | 18:06 |
| pabelanger | was trying to automate some of it | 18:06 |
| *** weshay has quit IRC | 18:06 | |
| *** btully has quit IRC | 18:06 | |
| *** jog0 has joined #openstack-infra | 18:06 | |
| fungi | zaro: after the meeting i'll try to remember to check/fix it in hiera too | 18:06 |
| zaro | fungi: i disabled the agent on review-dev so should still be there. | 18:07 |
| openstackgerrit | Merged openstack-infra/project-config: Make gate-sahara-tox-py27-scenario-unit job voting https://review.openstack.org/182238 | 18:07 |
| fungi | pabelanger: turbo-hipster may be worth examining. it's basically meant to operate as a gearman worker which runs shell commands, as a possible future alternative to our current use of jenkins | 18:07 |
| *** BharatK has quit IRC | 18:09 | |
| *** maishsk has joined #openstack-infra | 18:09 | |
| pabelanger | fungi, okay, will check it out | 18:10 |
| *** btully has joined #openstack-infra | 18:11 | |
| *** jogo has quit IRC | 18:11 | |
| *** jog0 has quit IRC | 18:11 | |
| *** jogo has joined #openstack-infra | 18:12 | |
| *** amotoki_ has quit IRC | 18:15 | |
| *** btully has quit IRC | 18:15 | |
| fungi | just a heads up, i'm about to drop off the wireless modem to finish some quick errands. i'm still hoping to get back home for the start of the meeting but there's a chance i'll miss the first few minutes | 18:16 |
| *** weshay has joined #openstack-infra | 18:18 | |
| nibalizer | pabelanger: there is also a spec we wrote to do acceptance testing of the nodepool images before setting them to the active test nodes, which might be in the vein of what you are doing | 18:20 |
| *** nfedotov has quit IRC | 18:20 | |
| nibalizer | I'm not entirely understanding your goal but that might be related | 18:20 |
| russellb | Hrm .. I have a requirements.txt entry for neutron that doesn't work anymore. networking-ovn and networking-odl use this (at least) ... The entry is: -e git://git.openstack.org/openstack/neutron.git#egg=neutron ... and failure is http://logs.openstack.org/93/177893/3/check/gate-networking-ovn-pep8/d2d179c/console.html ... any ideas? | 18:21 |
| pabelanger | nibalizer, well, the honest answer is. I'm tried of login into horizon, to launch base image. Then, ssh into it, apt-get update / install, apt-get install git, git clone system-config, puppet apply, etc. Just to get a single_use_slave. | 18:22 |
| pabelanger | much rather type a command and come back to node ready | 18:22 |
| pabelanger | I'm lazy | 18:22 |
| pabelanger | :) | 18:22 |
| *** claudiub has joined #openstack-infra | 18:22 | |
| anteaya | something to keep in mind | 18:26 |
| anteaya | the monday of summit is a holiday in Canada | 18:26 |
| anteaya | so banks and so on will likely be closed | 18:26 |
| pabelanger | anteaya, and beer stores! | 18:27 |
| anteaya | that is correct | 18:27 |
| anteaya | and good thinking | 18:27 |
| anteaya | I don't drink so that wouldn't occur to me but thanks for mentioning that | 18:27 |
| pleia2 | anteaya: ah, good to know | 18:27 |
| pleia2 | monday following summit is a holiday in the US | 18:28 |
| cody-somerville | Might be a good thing. I wonder if they'll let people buy a one day pass. | 18:28 |
| cody-somerville | So local can come and check things out that wouldn't be able to otherwise. | 18:28 |
| *** amitgandhinz has joined #openstack-infra | 18:28 | |
| anteaya | pleia2: good to know | 18:28 |
| pabelanger | anteaya, local transit might be on reduced hours too | 18:28 |
| pabelanger | at least they do that in Ottawa when holidays happen | 18:29 |
| anteaya | cody-somerville: by a one day pass to the conference? | 18:29 |
| *** cdent has joined #openstack-infra | 18:29 | |
| anteaya | pabelanger: I was thinking that as well | 18:29 |
| cody-somerville | anteaya: Some folks might not be able to get time off work normally but if its a holiday there might be folks who want to come in on just the Monday for expo hall and keynotes and what not. | 18:29 |
| cody-somerville | If the foundation isn't taking advantage of the holiday to get more folks exposure, it'll be a shame. Hopefully they've already thought of this | 18:30 |
| anteaya | interesting | 18:31 |
| anteaya | I had never thought of it myself | 18:31 |
| anteaya | I was just making plans and was told the store hours for next week | 18:31 |
| anteaya | closed monday | 18:31 |
| *** freerunner has quit IRC | 18:33 | |
| *** nfedotov has joined #openstack-infra | 18:33 | |
| *** ivar-laz_ has joined #openstack-infra | 18:35 | |
| *** ddieterly has quit IRC | 18:36 | |
| *** ivar-lazzaro has quit IRC | 18:37 | |
| *** arielb has quit IRC | 18:38 | |
| *** rbradfor has joined #openstack-infra | 18:38 | |
| pabelanger | anteaya, what time do you get into vancouver? | 18:40 |
| mrmartin | fungi, re | 18:41 |
| anteaya | thursday afternoon about 5pm I think | 18:41 |
| pabelanger | Also, any info about -infra meet ups happening? Like diners? | 18:41 |
| anteaya | I have plans on friday | 18:41 |
| anteaya | we just text each other | 18:41 |
| pabelanger | anteaya, this thursday? | 18:41 |
| anteaya | yes | 18:41 |
| pabelanger | wow | 18:41 |
| pabelanger | early | 18:41 |
| anteaya | when do you arrive | 18:41 |
| pabelanger | sunday morning, 9ma | 18:41 |
| pabelanger | 9am* | 18:42 |
| anteaya | I got invited to go to an event so I said yes | 18:42 |
| *** craigbr has quit IRC | 18:42 | |
| anteaya | pabelanger: in time for the board/tc meeting then, you will find more than a few of us there | 18:42 |
| *** craigbr has joined #openstack-infra | 18:42 | |
| pabelanger | ya, plan to saunter to downtown | 18:42 |
| pabelanger | see what is cooking at conferences | 18:43 |
| anteaya | cool, you should trip over a few of us then | 18:43 |
| boris-42 | AJaeger: hi there | 18:44 |
| boris-42 | AJaeger: could you put +1 workflow to this patch https://review.openstack.org/#/c/181343/ | 18:44 |
| boris-42 | AJaeger: recheck didn't help | 18:44 |
| AJaeger | boris-42: will do now since the other patch merged | 18:44 |
| boris-42 | AJaeger: thank you | 18:46 |
| *** shashankhegde has quit IRC | 18:47 | |
| *** arielb has joined #openstack-infra | 18:48 | |
| *** ddieterly has joined #openstack-infra | 18:48 | |
| *** aduarte has quit IRC | 18:48 | |
| *** puranamr has joined #openstack-infra | 18:50 | |
| *** shashankhegde has joined #openstack-infra | 18:50 | |
| *** e0ne has joined #openstack-infra | 18:51 | |
| *** afazekas_ has joined #openstack-infra | 18:51 | |
| nibalizer | jeblair: idunno if this goes in meeting agenda or at begining of meeting or where exactly, 'downstream-puppet' from priority specs has an update | 18:51 |
| nibalizer | i remember we were trying to identify which specs have updates versus need discussion | 18:52 |
| anteaya | nibalizer: add something to the agenda that is a bullet point for the update | 18:52 |
| anteaya | nibalizer: it will serve as a reminder that you need some air time | 18:52 |
| *** Rockyg has joined #openstack-infra | 18:53 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul: Add option to ignore ref-updated events emitted by branch deletions https://review.openstack.org/178833 | 18:54 |
| clarkb | ok back | 18:57 |
| openstackgerrit | Merged openstack-infra/project-config: Add Rally job to Murano https://review.openstack.org/181343 | 18:57 |
| *** mjturek2 has joined #openstack-infra | 18:57 | |
| *** yfried has joined #openstack-infra | 18:57 | |
| openstackgerrit | Swaminathan Vasudevan proposed openstack-infra/project-config: Make neutron-dvr job voting at the gate https://review.openstack.org/180230 | 18:57 |
| openstackgerrit | Joshua Harlow proposed openstack/requirements: Kazoo had a new release with fixes projects would like https://review.openstack.org/182412 | 18:58 |
| *** Krinkle is now known as Krinkle|detached | 18:58 | |
| anteaya | clarkb: glad you made it | 18:59 |
| openstackgerrit | George Peristerakis proposed openstack-infra/nodepool: Add REST API to Nodepool https://review.openstack.org/136884 | 18:59 |
| *** mjturek1 has quit IRC | 18:59 | |
| clarkb | russellb: it looks like neutron's setup hook is broken with an import error | 19:00 |
| jeblair | meeting time! | 19:00 |
| *** Krinkle|detached is now known as Krinkle | 19:00 | |
| clarkb | russellb: mordred and or lifeless may be able to decipher more from that message | 19:00 |
| russellb | clarkb: yeah, found the neutron commit that broke it. :) | 19:00 |
| russellb | clarkb: thanks for looking! | 19:00 |
| SpamapS | we doing a meeting? | 19:00 |
| *** Sukhdev has joined #openstack-infra | 19:00 | |
| clarkb | russellb: any chance you can point it out to me? (for my own curiousity) | 19:00 |
| mordred | russellb: yah - lifeless will definitely want to see that | 19:01 |
| *** tiswanso has quit IRC | 19:02 | |
| *** banix has quit IRC | 19:02 | |
| *** afazekas_ has quit IRC | 19:03 | |
| *** dboik_ has quit IRC | 19:03 | |
| russellb | clarkb: mordred https://review.openstack.org/#/c/182413/ | 19:03 |
| lifeless | someone invoketh me? | 19:03 |
| *** ayoung has quit IRC | 19:03 | |
| russellb | don't know exactly what busted it ... but i think the neutron/__init__.py change to add the use of six | 19:04 |
| *** dboik has joined #openstack-infra | 19:04 | |
| *** HeOS has joined #openstack-infra | 19:04 | |
| mordred | lifeless: russellb has a pbr issue | 19:04 |
| russellb | lifeless: I was trying to figure out this error: http://logs.openstack.org/93/177893/3/check/gate-networking-ovn-pep8/d2d179c/console.html | 19:04 |
| russellb | lifeless: and it turned out the problem was introduced by the neutron commit i reverted here: https://review.openstack.org/#/c/182413/ | 19:04 |
| russellb | because reasons | 19:05 |
| *** ayoung has joined #openstack-infra | 19:05 | |
| *** banix has joined #openstack-infra | 19:05 | |
| *** mjturek2 has left #openstack-infra | 19:05 | |
| *** ildikov has joined #openstack-infra | 19:05 | |
| *** dguerri is now known as _dguerri | 19:06 | |
| *** jamesmcarthur has joined #openstack-infra | 19:06 | |
| lifeless | pep8 is running under py2 | 19:06 |
| lifeless | does the py2 unittest fail too? | 19:06 |
| russellb | lifeless: yes | 19:06 |
| lifeless | ok so consistent | 19:06 |
| lifeless | now that looks like a hook referenced in setup.cfg | 19:07 |
| lifeless | which has become unimportable | 19:07 |
| lifeless | russellb: how did your patch merge in the first place? | 19:07 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Stop leaking server objects https://review.openstack.org/181513 | 19:07 |
| rcarrillocruz | hmm | 19:08 |
| *** tiswanso has joined #openstack-infra | 19:08 | |
| anteaya | mordred: where on the schedule is the ansible session? | 19:08 |
| rcarrillocruz | peristeri: hi | 19:08 |
| russellb | lifeless: no idea, wasn't my patch ... noticed when it broke jobs running on stackforge repos that had the neutron git repo in their requirements.txt | 19:08 |
| mordred | anteaya: it's not | 19:08 |
| lifeless | ok | 19:08 |
| mordred | anteaya: it's not part of the summit schedule at all - it's just in the same building | 19:08 |
| anteaya | mordred: ah okay I will put a tracker on you and follow you | 19:08 |
| lifeless | oh man | 19:08 |
| lifeless | mordred: you will be sad and angry | 19:08 |
| lifeless | mordred: look at neutron.hooks | 19:09 |
| peristeri | rcarrillocruz, hi | 19:09 |
| SpamapS | There's an Ansible working "session" | 19:09 |
| lifeless | mordred: wearing your CI reliability and consistency hat. | 19:09 |
| lifeless | mordred: I give you full permission to rant. | 19:09 |
| rcarrillocruz | peristeri: hey, are you george peristerakis re: https://review.openstack.org/#/c/136884/5 ? | 19:09 |
| SpamapS | mordred: http://sched.co/3HEp ? | 19:09 |
| dims | sdague: anyone: seen this yet? "ImportError: neutron.hooks.setup_hook" | 19:09 |
| lifeless | we're just looking now at that | 19:10 |
| mordred | lifeless: no ranting needed - that's been there for ages | 19:10 |
| dims | ah thanks. | 19:10 |
| lifeless | mordred: hidden from update.py.... | 19:10 |
| mordred | lifeless: I've always ignored it because it's for windows support and we don't have any of that | 19:10 |
| russellb | dims: This should fix it https://review.openstack.org/182413 | 19:10 |
| peristeri | rcarrillocruz, yes, I rebased the patch. | 19:10 |
| rcarrillocruz | sweet, good to see people interested on it | 19:10 |
| rcarrillocruz | cos it's been a bit stale :-) | 19:11 |
| rcarrillocruz | you going to the summit by any chance? | 19:11 |
| mordred | SpamapS: yes, that's it. pabelanger, tchaypo, anteaya: ^^ see SpamapS link above | 19:11 |
| rcarrillocruz | we could talk about it, i was planning to chat with some infra folks on that topic | 19:11 |
| dims | russellb: sweet thanks. will recheck mine after that merges | 19:11 |
| peristeri | rcarrillocruz, not this one, maybe in Tokyo. | 19:11 |
| lifeless | anyhow | 19:12 |
| lifeless | that failing indicates that neutron.__init__ failed to import | 19:12 |
| rcarrillocruz | k, will look into that patch not too far, there's been quite a bit of nodepool changes so i assume it needs some further rebase | 19:12 |
| rcarrillocruz | thx for the patchset anyway | 19:12 |
| lifeless | which means this https://review.openstack.org/#/c/181277/4/neutron/__init__.py | 19:12 |
| lifeless | is the prolem | 19:12 |
| peristeri | rcarrillocruz, np | 19:13 |
| anteaya | mordred SpamapS thank you | 19:13 |
| lifeless | russellb: I think gettext.install() is failing | 19:13 |
| *** panda has quit IRC | 19:13 | |
| lifeless | russellb: since any errors thrown during import result in an ImportError | 19:13 |
| lifeless | russellb: and the hook file didn't change | 19:13 |
| *** afazekas_ has joined #openstack-infra | 19:14 | |
| *** panda has joined #openstack-infra | 19:14 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Stop leaking server objects https://review.openstack.org/181513 | 19:14 |
| lifeless | russellb: no indication of a pbr bug per se, but if we can confirm this perhaps pbr can do something to help | 19:14 |
| mordred | yup. that would be it | 19:14 |
| mordred | those hooks would be running at import time, which means that things that require requirements being imported in __init__ is going to break things | 19:14 |
| russellb | ok, and the patch that broke it added use of six | 19:15 |
| russellb | so that's the root cause | 19:15 |
| *** afazekas_ has quit IRC | 19:15 | |
| *** dimtruck is now known as zz_dimtruck | 19:15 | |
| lifeless | so gettext and all the things needed to make gettext.install('neutron', unicode=1) are now setup_requires for neutron | 19:15 |
| lifeless | mordred: and ^ is why you should have cared, even though its for windows | 19:15 |
| lifeless | and six | 19:15 |
| lifeless | russellb: or six, yes | 19:16 |
| mordred | well, I'm not sure how it passed in the first place | 19:16 |
| *** openstackgerrit_ has joined #openstack-infra | 19:16 | |
| *** afazekas_ has joined #openstack-infra | 19:16 | |
| fungi | voodoo | 19:16 |
| mordred | the gate shuold have failed the patch that added the import of six into __init__ | 19:16 |
| lifeless | mordred: me neither, Was six installed globally? | 19:16 |
| mordred | lifeless: venvs | 19:16 |
| lifeless | mordred: and now its not? | 19:16 |
| mordred | shouldn't have been | 19:16 |
| mordred | lifeless: OH | 19:16 |
| mordred | lifeless: I betcha I know | 19:16 |
| mordred | lifeless: pbr used to depend on pip | 19:16 |
| mordred | lifeless: thus - six. thus - voila | 19:16 |
| *** tim_o has joined #openstack-infra | 19:16 | |
| lifeless | mordred: still does | 19:17 |
| mordred | lifeless: we didn't land that? | 19:17 |
| lifeless | mordred: we only just +A'd it | 19:17 |
| mordred | hrm | 19:17 |
| lifeless | right? 17m ago or something | 19:17 |
| tchaypo | mordred: thanks | 19:17 |
| tchaypo | SpamapS: thanks too :) | 19:18 |
| * russellb stares at openstackgerrit and openstackgerrit_ | 19:18 | |
| lifeless | mordred: oh | 19:18 |
| lifeless | mordred: nope, thats running 0.11.0 release of pbr | 19:18 |
| morganfainberg | russellb: i think wee need more openstackgerrit-like names in the channels :) | 19:18 |
| lifeless | mordred: all changes to pip use are future | 19:18 |
| fungi | ooh! openstackgerrit_ where'd you come from? | 19:19 |
| lifeless | and pip hasn't relesed 7.0.0 yet so it not the topo sort stuff | 19:20 |
| fungi | openstackgerrit_ looks like a test bot someone's got running on a not-our-gerrit somewhere in hpcloud | 19:20 |
| fungi | rcarrillocruz: ^ ? | 19:20 |
| openstackgerrit | Merged openstack-infra/project-config: Have gate-ironic-python34 be a voting job https://review.openstack.org/181034 | 19:20 |
| *** zz_dimtruck is now known as dimtruck | 19:20 | |
| mordred | fungi: HAHAHAHAHAHA | 19:20 |
| russellb | i can't wait to see what it reports! | 19:21 |
| russellb | openstackgerrit_: tell me all your secrets | 19:21 |
| pleia2 | bots on the loose | 19:21 |
| *** afazekas_ has quit IRC | 19:22 | |
| *** Krinkle is now known as Krinkle|detached | 19:25 | |
| *** openstackgerrit_ has quit IRC | 19:25 | |
| *** hashar has joined #openstack-infra | 19:26 | |
| *** mrunge has quit IRC | 19:28 | |
| *** annegentle has joined #openstack-infra | 19:31 | |
| *** spredzy|afk is now known as spredzy | 19:31 | |
| *** freerunner has joined #openstack-infra | 19:31 | |
| *** hashar has quit IRC | 19:32 | |
| *** hashar has joined #openstack-infra | 19:32 | |
| *** achanda has quit IRC | 19:33 | |
| *** ayoung has quit IRC | 19:35 | |
| *** ayoung has joined #openstack-infra | 19:36 | |
| *** marcusvrn has quit IRC | 19:37 | |
| *** _dguerri is now known as dguerri | 19:38 | |
| *** thedodd has quit IRC | 19:38 | |
| tchaypo | My bank has me use a special form to send them messages. That form does not allow me to use apostrophes. This feels me with *great* confidence. | 19:38 |
| openstackgerrit | Dan Smith proposed openstack-infra/reviewstats: Add melwitt to nova-core and drop cyeoh-0 https://review.openstack.org/182429 | 19:38 |
| anteaya | tchaypo: I can feel your confidence from here | 19:39 |
| lifeless | tchaypo: security! | 19:40 |
| *** thedodd has joined #openstack-infra | 19:41 | |
| lifeless | mordred: russellb: ok, I'm paging this out. All I can think of is that we were insufficiently isolated, and we are now. | 19:41 |
| lifeless | technically you could put six as a setup_requires to let this work, but it would be better to move the windows deps to an environment marker. | 19:41 |
| lifeless | I can put up a patch for that if you like | 19:42 |
| lifeless | in fact, let me do that right now | 19:42 |
| openstackgerrit | Doug Hellmann proposed openstack/requirements: Avoid httpretty 0.8.9 as it breaks unittests https://review.openstack.org/182340 | 19:43 |
| *** markvoelker has joined #openstack-infra | 19:45 | |
| *** Krinkle|detached is now known as Krinkle | 19:45 | |
| *** turul_ has joined #openstack-infra | 19:46 | |
| *** marcusvrn has joined #openstack-infra | 19:46 | |
| lifeless | mordred: pbr.hooks.setup_hook isn't needed anymore right? we Just Work through entry points? | 19:48 |
| mordred | lifeless: that is correct | 19:48 |
| *** luqas__ has joined #openstack-infra | 19:48 | |
| mordred | anyone that sets that in their setup.cfg is just old cruft | 19:48 |
| *** afazekas has quit IRC | 19:50 | |
| *** luqas has quit IRC | 19:51 | |
| *** chris6131 has left #openstack-infra | 19:51 | |
| *** gyee has quit IRC | 19:52 | |
| jogo | zaro: ping random gerrit question | 19:53 |
| jogo | zaro: would it be feasible to make tags reviewable items in gerrit? | 19:54 |
| jogo | in a future version of gerrit that is | 19:54 |
| fungi | jogo: that's been near or at the top of our wishlist since somewhere around the beginning | 19:54 |
| lifeless | mordred: russellb: https://review.openstack.org/182433 should fix the problem more permanently. | 19:55 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for osc stable/kilo novaclient caps bug 1454397 https://review.openstack.org/182434 | 19:55 |
| openstack | bug 1454397 in python-openstackclient "python-novaclient 2.25.0 release breaks stable/kilo because of uncapped osc on stable/kilo" [Undecided,In progress] https://launchpad.net/bugs/1454397 | 19:55 |
| jogo | fungi: is that on the radar of upstream gerrit? | 19:55 |
| fungi | jogo: anything's possible, it's software after all. the challenge will be coming up with a generalized solution that upstream will actually approve | 19:55 |
| russellb | lifeless: nice! thanks! | 19:55 |
| fungi | jogo: afaik there is no current work on that | 19:55 |
| *** aduarte has joined #openstack-infra | 19:56 | |
| lifeless | russellb: de nada | 19:56 |
| *** mjturek1 has joined #openstack-infra | 19:56 | |
| mordred | jogo: I've been asking for that feature since we sstarted using gerrit | 19:56 |
| zaro | jogo: people have asked but there have been no activity, that i am aware of, for that. | 19:56 |
| *** claudiub|2 has joined #openstack-infra | 19:57 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Enable pacemaker via the resource registry https://review.openstack.org/181522 | 19:57 |
| jogo | zaro: is there an open issue for it? | 19:58 |
| clarkb | zaro: we can chat about the logs here, can you expand a bit more on which logs are in question and what does not accessible from gerrit mean? | 19:58 |
| jogo | I didn't see anything in https://code.google.com/p/gerrit/issues yet | 19:58 |
| *** dboik_ has joined #openstack-infra | 19:58 | |
| russellb | lifeless: so this is fun ... i think the revert is blocked by the bug it introduced. maybe we should just get your patch in instead | 19:58 |
| jogo | zaro: ahh https://code.google.com/p/gerrit/issues/detail?id=1433 | 19:59 |
| *** claudiub has quit IRC | 19:59 | |
| jogo | wow nothing in over a year, almost as bad as nova bugs | 20:00 |
| lifeless | russellb: hah, eek. | 20:01 |
| *** dboik has quit IRC | 20:01 | |
| lifeless | russellb: can you track my patch? I've just thrown it up there - if it doesn't work grab me and we can look at whats happening in more detail | 20:01 |
| russellb | lifeless: yep, on it | 20:01 |
| clarkb | zaro: looking at a random noop job log on review-dev the issue is likely in the zuul-dev config | 20:02 |
| clarkb | zaro: zuul is telling gerrit the logs are on logs.openstack.org but we don't copy files there for review-dev | 20:02 |
| *** rlandy has quit IRC | 20:02 | |
| zaro | clarkb: ok. thanks. i see if i can fix that up. | 20:02 |
| tmcpeak | anybody here particularly involved with this: https://github.com/openstack-infra/pypi-mirror | 20:03 |
| tmcpeak | ? | 20:03 |
| jeblair | Rockyg: once you write that spec and we approve it and start spinning up servers, the running of the refstack service will be an official project activity under the infrastructure project | 20:03 |
| *** Krinkle is now known as Krinkle|detached | 20:03 | |
| clarkb | zaro: there should be something like a ZUUL_LOG_PATH that is used by zuul to generate the report to gerrit and is also passed to the jobs themselves so they can copy to the correct path | 20:03 |
| jeblair | Rockyg: the _code_ for refstack itself also probably wants to be a big-tent openstack project too | 20:03 |
| jeblair | under its own right | 20:03 |
| clarkb | tmcpeak: its mostly dead at this point but I think there are a few individuals still using and caring for it | 20:03 |
| *** banix has quit IRC | 20:03 | |
| clarkb | tmcpeak: whats up? | 20:03 |
| *** armax has quit IRC | 20:03 | |
| greghaynes | clarkb: what, no | 20:04 |
| *** achanda has joined #openstack-infra | 20:04 | |
| dims | russellb: can't put genie back in the bottle :) | 20:04 |
| greghaynes | clarkb: theres patches that one day will merge that use it | 20:04 |
| clarkb | greghaynes: no you don't care for it anymore? or no it isn't dead? | 20:04 |
| greghaynes | hsh | 20:04 |
| fungi | tmcpeak: we tried to abandon/retire it and some people who still rely on it stepped up to maintain sort of it | 20:04 |
| tmcpeak | clarkb: I'm interested in adding a feature into it to download the accompanying signature for PyPI packages which have signatures | 20:04 |
| greghaynes | I mean, I kinda care for it | 20:04 |
| Rockyg | jeblair: Thanks for the info, and yeah, refstack wants to be big tent, but we need a few more people for critical mass | 20:04 |
| Rockyg | We'll be recruiting at the summit | 20:05 |
| tmcpeak | fungi: ^ | 20:05 |
| jeblair | Rockyg: cool, though we have pretty low standards when it comes to that now. :) | 20:05 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Add new project: puppet-refstack https://review.openstack.org/182437 | 20:05 |
| greghaynes | tmcpeak: go for it :) | 20:05 |
| krotscheck | Rockyg: ^^ | 20:05 |
| Rockyg | Kewl! thanks krotscheck! | 20:05 |
| tmcpeak | greghaynes: cool, just normal gerrit review process? | 20:05 |
| greghaynes | tmcpeak: yep | 20:05 |
| tmcpeak | greghaynes: cool, sounds good, thanks | 20:05 |
| russellb | dims: that's what it feels like | 20:05 |
| greghaynes | tmcpeak: Im not the best at watching that review queue, so feel free to ping when you post a patch | 20:05 |
| clarkb | I was just able to build bare-centos6 in hpcloud-b1, going to kick off builds for the remaining 4 "providers" | 20:06 |
| krotscheck | Rockyg: np | 20:06 |
| tmcpeak | greghaynes: cool, will do | 20:06 |
| *** ayoung has quit IRC | 20:07 | |
| jeblair | krotscheck, Rockyg: i'd like to get the spec written+approved before we pull the module in, is that ok? | 20:07 |
| openstackgerrit | Cyril Roelandt proposed openstack-infra/project-config: Neutron: enable a voting py34 gate on master https://review.openstack.org/182439 | 20:07 |
| krotscheck | jeblair: Sure! Ball's in your court, Rockyg :) | 20:07 |
| Rockyg | jeblair: yes | 20:07 |
| Rockyg | can you point me to an example spec? I'm reading the add server doc now | 20:08 |
| fungi | Rockyg: http://git.openstack.org/cgit/openstack-infra/infra-specs/tree/template.rst | 20:08 |
| fungi | Rockyg: also the README.rst in that same directory has some helpful instructions | 20:09 |
| *** dboik_ has quit IRC | 20:09 | |
| Rockyg | kk Thought maybe someone else had already done one like this...no such luck :-) | 20:09 |
| fungi | Rockyg: and the specs subdirectory has lots of approved specs | 20:09 |
| *** dboik has joined #openstack-infra | 20:10 | |
| Rockyg | fungi: I'll see if there's one that resembles what we are doing | 20:10 |
| clarkb | hrm new hpcloud buildsare going straight to error | 20:10 |
| *** ashleighfarnham has quit IRC | 20:11 | |
| fungi | Rockyg: migrate_askbot is probably the closest approved spec at the moment, though it has some migration-specific steps since we needed to move data from an existing production server to a new one under the infrastructure team's control | 20:11 |
| fungi | Rockyg: also don't worry about getting the first revision perfect. that's why it goes through review. if we want more or different information then we'll ask once we see it | 20:12 |
| clarkb | pleia2: I just held 23.253.234.137 and will be using it for nodepool + devstack things once the job running on it completes | 20:14 |
| pleia2 | clarkb: ok | 20:14 |
| *** craigbr has quit IRC | 20:14 | |
| *** craigbr has joined #openstack-infra | 20:15 | |
| clarkb | I am slowly able to get bare-centos6 builds going so working on that in the mean time | 20:15 |
| *** craigbr has quit IRC | 20:15 | |
| openstackgerrit | Merged openstack-infra/project-config: Add bifrost to OpenStack https://review.openstack.org/182022 | 20:16 |
| *** marun has quit IRC | 20:17 | |
| *** otter768 has joined #openstack-infra | 20:19 | |
| *** ayoung has joined #openstack-infra | 20:19 | |
| *** flaper87 has quit IRC | 20:20 | |
| *** flaper87 has joined #openstack-infra | 20:20 | |
| openstackgerrit | Merged openstack-infra/reviewstats: Add melwitt to nova-core and drop cyeoh-0 https://review.openstack.org/182429 | 20:21 |
| *** ayoung has quit IRC | 20:21 | |
| *** ayoung has joined #openstack-infra | 20:21 | |
| *** achanda has quit IRC | 20:21 | |
| *** Krinkle|detached is now known as Krinkle | 20:23 | |
| clarkb | woot finally have all four builds going | 20:23 |
| *** otter768 has quit IRC | 20:24 | |
| clarkb | pleia2: will ping once I hop on that node to do nodepool things | 20:24 |
| pleia2 | clarkb: sounds good | 20:24 |
| *** ashleighfarnham has joined #openstack-infra | 20:26 | |
| tchaypo | so this is entirely anecdotal, but… every git review I’ve done since the downgrade has taken a long time to respond. | 20:26 |
| openstackgerrit | James Polley proposed openstack-dev/pbr: WIP: Teach pbr to read extras from setup.cfg https://review.openstack.org/181795 | 20:26 |
| tchaypo | naturally with the exception of that one | 20:26 |
| tchaypo | which took 6s, because i was running it in time. | 20:27 |
| greghaynes | clarkb: im going through the nodepool+devstack on a hpcloud node | 20:28 |
| greghaynes | clarkb: just fyi | 20:28 |
| clarkb | greghaynes: cool, I think pleia2's comments are basically all on point so if you run into any trouble you should check them | 20:28 |
| greghaynes | clarkb: I have one note about missing mysql flush privs | 20:29 |
| *** dizquierdo has joined #openstack-infra | 20:29 | |
| *** markvoelker has quit IRC | 20:30 | |
| clarkb | mordred: when you are not TCing, whats the easiest way to grep the mysql db dump for review.o.o? is it going to be loading it into a mysql? | 20:30 |
| clarkb | I just need the one row but its 2.2GB and grep fails spectacularly | 20:30 |
| mordred | clarkb: you need the new or the old db? | 20:31 |
| mordred | clarkb: I have the pre-upgrade db loaded on my laptop | 20:31 |
| mordred | if you want a line out of a different dump, you'll have to load it, sadly | 20:31 |
| clarkb | ya I need old | 20:31 |
| *** Krinkle is now known as Krinkle|detached | 20:31 | |
| clarkb | mordred: select * from accounts where full_name = "Steve Baker"; | 20:31 |
| *** nfedotov1 has joined #openstack-infra | 20:31 | |
| clarkb | mordred: want to compare that to what we have in the post downgraded version | 20:32 |
| mordred | | 2012-06-16 18:00:13 | Steve Baker | sbaker@redhat.com | 2013-03-03 17:12:12 | 100 | N | N | NULL | CHECKOUT | N | ISO | HHMM_24 | N | 4571 | NULL | EXPAND_RECENT | SIDE_BY_SIDE | N | N | N | | 20:32 |
| clarkb | thanks | 20:34 |
| *** nfedotov has quit IRC | 20:35 | |
| *** mjturek1 has left #openstack-infra | 20:35 | |
| *** achanda has joined #openstack-infra | 20:36 | |
| *** openstackgerrit has quit IRC | 20:37 | |
| clarkb | stevebaker: I only see two differences. The first is your change screen selection and the second is your date format | 20:37 |
| *** openstackgerrit has joined #openstack-infra | 20:37 | |
| clarkb | stevebaker: I definitely don't expect the change screen 2 selection to be a problem, but the date format may not be valid in the old client? looking into that now | 20:38 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Initial manual for infra-cloud https://review.openstack.org/180796 | 20:39 |
| *** maishsk has quit IRC | 20:39 | |
| mordred | SpamapS: ^^ updated your doc there - added more words | 20:40 |
| mordred | SpamapS: also added some numbers | 20:40 |
| *** jamesmcarthur has left #openstack-infra | 20:40 | |
| stevebaker | clarkb: morning, date format sounds plausable | 20:41 |
| anteaya | Rockyg: there are female operators as well | 20:41 |
| *** ashleighfarnham has quit IRC | 20:41 | |
| TheJulia | Anyone with admin access to gerrit up for adding me to bifrost-core ? | 20:42 |
| pleia2 | sure, sec | 20:42 |
| TheJulia | Merci | 20:43 |
| clarkb | stevebaker: ya I think I may have confirmed it, gerrit 2.8 seems to honor, STD, US, ISO, and EURO formats but you have UK | 20:43 |
| *** dimtruck is now known as zz_dimtruck | 20:43 | |
| pleia2 | TheJulia: done | 20:44 |
| *** maishsk has joined #openstack-infra | 20:44 | |
| clarkb | stevebaker: I can switch you back to STD and see if that fixes (then you can select from preferences whatever you like and is valid) | 20:44 |
| *** radez is now known as radez_g0n3 | 20:44 | |
| clarkb | stevebaker: or I guess your old value was ISO so can use that | 20:44 |
| *** mriedem has quit IRC | 20:44 | |
| *** pabelanger has quit IRC | 20:44 | |
| fungi | clarkb: i suppose we need to query the whole table to see if there are others outside that set | 20:44 |
| clarkb | fungi: probably | 20:44 |
| fungi | fix them before they discover they're broken | 20:44 |
| fungi | i guessing this is the only one, but you never know | 20:45 |
| clarkb | UPDATE accounts SET date_format = 'ISO' WHERE account_id = 4571 ; that look correct? | 20:45 |
| anteaya | never having looked at our gerrit db that looks like SQL to me, sorry best feedback I have | 20:47 |
| mordred | clarkb: yes | 20:48 |
| mordred | clarkb: wait | 20:48 |
| mordred | clarkb: oh - yes | 20:48 |
| mordred | clarkb: that looks goo | 20:48 |
| mordred | clarkb: you're re-setting the entry to what it was in the old db, yeah? | 20:48 |
| clarkb | I am going to run a quick select to see if any other accounts need cleanup first | 20:48 |
| clarkb | mordred: yup | 20:48 |
| stevebaker | no Queen's data format for me | 20:49 |
| *** tnovacik has quit IRC | 20:50 | |
| clarkb | stevebaker: you are the only winner in this competition :) | 20:50 |
| *** koolhead17 has quit IRC | 20:50 | |
| openstackgerrit | Antoine Musso proposed openstack-infra/zuul: Zuul references cleaner https://review.openstack.org/109276 | 20:50 |
| clarkb | mordred: I don't have your practice editing production DBs with VPs looking over your shoulder :) | 20:50 |
| clarkb | mordred: but that update looks good ya? | 20:51 |
| fungi | clarkb: that query looks correct, yes | 20:51 |
| *** marun has joined #openstack-infra | 20:52 | |
| clarkb | I should probably not use gsql because its a terrible client, but meh | 20:52 |
| fungi | though i'm lazy and tend to just type the commands lower-case. trying not to put too many miles on my shift key | 20:52 |
| *** e0ne has quit IRC | 20:52 | |
| *** nfedotov1 has quit IRC | 20:53 | |
| mordred | clarkb: yah. it's great | 20:53 |
| mordred | fungi: ++ | 20:53 |
| mordred | clarkb: and wow, you use gsql? | 20:53 |
| *** e0ne has joined #openstack-infra | 20:53 | |
| fungi | i've just been using mysqlclient directly for that stuff and then flushing caches as needed | 20:53 |
| mordred | yup | 20:54 |
| clarkb | stevebaker: can you try logging in now? | 20:54 |
| *** kgiusti has left #openstack-infra | 20:54 | |
| stevebaker | clarkb: IT WORKS \o/ | 20:54 |
| clarkb | woot that was it then | 20:54 |
| stevebaker | clarkb: thanks, I owe you a $beverage | 20:55 |
| clarkb | stevebaker: it was tricky because looking at the row without the old side looked fine | 20:55 |
| *** Longgeek has quit IRC | 20:56 | |
| clarkb | anyways glad we could get that sorted out | 20:56 |
| clarkb | bare-centos6 builds failed cloning bifrost :/ trying again | 20:57 |
| clarkb | pleia2: that node is done running stuff so about to try nodepool things as soon as I get bare-centos6 builds going again | 20:57 |
| mordred | yay for backups | 20:57 |
| *** dangers is now known as dangers_away | 20:57 | |
| *** zz_dimtruck is now known as dimtruck | 20:57 | |
| pleia2 | clarkb: sure | 20:58 |
| *** mrmartin has quit IRC | 21:00 | |
| *** Sukhdev has quit IRC | 21:00 | |
| *** gyee has joined #openstack-infra | 21:01 | |
| *** armax has joined #openstack-infra | 21:01 | |
| tchaypo | We seem to have an ops work session on “packaging” (no convenor, nothing in the etharpad) scheduled at the same time as the oslo session on reviewing the release process | 21:01 |
| fungi | is that reviewing the _oslo_ release process? | 21:02 |
| fungi | if so, i'm not terribly concerned about the overlap | 21:02 |
| pleia2 | I believe it's related to the thread StevenK started on the -infra list | 21:02 |
| *** shardy is now known as shardy_z | 21:02 | |
| mordred | there is an ops work session related to an infra topic? | 21:02 |
| *** maishsk has left #openstack-infra | 21:02 | |
| pleia2 | oh, ops | 21:02 |
| fungi | so long as it's not reviewing the infrastructure tooling around how projects release, i'm fine with that oslo overlap | 21:03 |
| mordred | phew. because I haven't been adding those to my schedule at all | 21:03 |
| *** e0ne has quit IRC | 21:03 | |
| sdague | hey, could anyone give the second +2 on this - https://review.openstack.org/#/c/175531/ - which lets us test keystone in wsgi mode in grenade, which is what we do in the primary testing | 21:03 |
| mordred | sdague: oh - hah. my brain thought the opposite thing from your sentence | 21:03 |
| fungi | sdague: it passes? | 21:04 |
| sdague | fungi: yes, it does, see test results | 21:04 |
| fungi | d'oh! right, self-testing ;) | 21:04 |
| fungi | for some reason i was thinking job definition, not d-g selection logic | 21:05 |
| *** aysyd has quit IRC | 21:05 | |
| clarkb | pleia2: reading up on how devstack does mysql passwords it appears to primpt you for it if it isn't set, so I will just update the comment on the doc | 21:05 |
| pleia2 | clarkb: yeah, reran devstack and it was the first question it asks during stack.sh | 21:06 |
| pleia2 | setting the database password | 21:06 |
| clarkb | yup, so I iwll just leave a comment in there | 21:06 |
| sdague | pleia2: yes, we prompt for a couple of passwords if not set | 21:07 |
| *** mriedem has joined #openstack-infra | 21:07 | |
| sdague | I think 5 last time I started completely from scratch | 21:07 |
| pleia2 | yeah, it was something like that | 21:07 |
| openstackgerrit | Antoine Musso proposed openstack-infra/zuul: Zuul references cleaner https://review.openstack.org/109276 | 21:07 |
| *** erikmwilson has quit IRC | 21:07 | |
| sdague | pleia2: was there some concern about it? | 21:08 |
| *** stevemar has quit IRC | 21:08 | |
| pleia2 | sdague: no, just working on clarkb's nodepool against devstack instructions | 21:08 |
| sdague | ok | 21:08 |
| tchaypo | https://etherpad.openstack.org/p/YVR-ops-packaging has no details (stevenk, if this is your session, could you add something?) | 21:09 |
| fungi | *grumble* something else is still pulling in the python-nose distro package as a transitive dependency from devstack's rpm set on at least one branch. no idea what. https://review.openstack.org/181574 will allow us to stop caring though | 21:09 |
| pleia2 | tchaypo: I was mistaken, I didn't see "ops" in your comment | 21:09 |
| pleia2 | tchaypo: StevenK is working on *infra* packaging things | 21:09 |
| tchaypo | ah | 21:09 |
| tchaypo | la la la | 21:10 |
| russellb | soooo ... lifeless mordred fungi neutron is wedged. If you look at what failed here: https://review.openstack.org/#/c/182413/1 | 21:10 |
| *** annegentle has quit IRC | 21:10 | |
| tchaypo | https://etherpad.openstack.org/p/YVR-oslo-release-process-review - "We had enough issues with semver in some of the other projects that the release team is considering centralizing client library release management." | 21:10 |
| lifeless | russellb: actually | 21:10 |
| russellb | the job that's broken is pulling neutron master, and not the patch from gerrit | 21:10 |
| tchaypo | yeah, they sound like there’s not a lot of overlap | 21:10 |
| lifeless | russellb: I think we should make that job nonvoting | 21:10 |
| lifeless | russellb: since its not testing the commit in question anyway | 21:10 |
| russellb | lifeless: OK, i guess that would fix it | 21:10 |
| *** tiswanso has quit IRC | 21:11 | |
| russellb | yeah, that job is just wrong | 21:11 |
| russellb | and that also explains how this merged in the first place | 21:11 |
| russellb | because it wasn't actually testing the patch | 21:11 |
| lifeless | oh man | 21:11 |
| mordred | fungi: +A | 21:11 |
| fungi | yeah, all the plugins are putting git urls to neutron branches in their requirements | 21:11 |
| lifeless | # This project does depend on neutron as a library, but the | 21:11 |
| lifeless | # openstack tooling does not play nicely with projects that | 21:11 |
| lifeless | # are not publicly available in pypi. | 21:11 |
| lifeless | # -e git+https://git.openstack.org/openstack/neutron#egg=neutron | 21:11 |
| lifeless | What the everlasting F | 21:11 |
| russellb | OK, I'll put up a patch to make it non-voting | 21:11 |
| *** ashleighfarnham has joined #openstack-infra | 21:12 | |
| russellb | so we can un wedge this effing thing | 21:12 |
| lifeless | 'openstack tooling' | 21:12 |
| lifeless | Python ecosystem more like | 21:12 |
| fungi | 'hand waving' | 21:12 |
| *** ayoung has quit IRC | 21:12 | |
| mordred | yeah. honestly - we should set up a job that clones neutron via zuul-cloner and also networking plugin in question | 21:12 |
| mordred | it turns out we know how to do multi-repo thigns :) | 21:13 |
| russellb | mordred: yar. | 21:13 |
| *** gyee has quit IRC | 21:13 | |
| lifeless | we should also put the servers on pypi | 21:13 |
| fungi | _we_ do, the people responsible for writing the neutron-.*aas-dsvm-api jobs don't even know what that means, i think | 21:13 |
| lifeless | and have the plugin depend on 'neutron' (which we can satisfy locally via a location requirement) | 21:13 |
| mordred | lifeless: that too | 21:13 |
| anteaya | fungi: correct | 21:13 |
| lifeless | ttx: hi | 21:14 |
| mordred | lifeless: except for the one where we do not have the pypi name | 21:14 |
| lifeless | mordred: which one? | 21:14 |
| *** Krinkle|detached is now known as Krinkle | 21:14 | |
| fungi | lifeless: just keystone, nothing important | 21:14 |
| fungi | well, also tempest, but that's not really a server | 21:14 |
| lifeless | so that we can call python-keystone on pypi, or openstack-keystone or something | 21:14 |
| lifeless | distribution name doesn't need to equal package name | 21:15 |
| *** gyee has joined #openstack-infra | 21:15 | |
| lifeless | I know its PITA for our tooling | 21:15 |
| lifeless | but pip etc can cope | 21:15 |
| *** bswartz has quit IRC | 21:15 | |
| fungi | for 'openstack tooling' ;) | 21:15 |
| lifeless | CI specifically | 21:15 |
| mordred | lifeless: biggest problem is actually that changing the name that we publish to pypi would mean a change in the name of the tarball | 21:15 |
| openstackgerrit | Russell Bryant proposed openstack-infra/project-config: Make neutron-lbaas-dsvm-api non-voting. https://review.openstack.org/182455 | 21:16 |
| lifeless | mordred: we can come up with a solution | 21:16 |
| mordred | lifeless: and I could imagine more than one person is likely consuming http://tarballs.openstack.org/$project-master.tar.gz | 21:16 |
| lifeless | mordred: is this the only reason? | 21:16 |
| mordred | lifeless: sure we can | 21:16 |
| mordred | lifeless: just listing things to be cognizant of | 21:16 |
| *** pc_m has quit IRC | 21:16 | |
| lifeless | I mean, there's no point me haring off to organise a group to tackle this | 21:16 |
| russellb | patch above will unwedge neutron ^^^ | 21:16 |
| lifeless | if there's areason we don't /want/ to do this | 21:16 |
| clarkb | pleia2: basically i have updated things and am going from start to finish, will push my changes up as soon as things are working | 21:16 |
| fungi | we could in theory transform the tarball/wheel filenames when uploading to pypi too | 21:17 |
| pleia2 | clarkb: sounds good | 21:17 |
| *** ajmiller__ has joined #openstack-infra | 21:17 | |
| lifeless | fungi: so pbr needs to generate the name | 21:18 |
| lifeless | fungi: because it matches the req to the tarball name | 21:18 |
| lifeless | fungi: I'd do the translation when putting them on tarballs. | 21:18 |
| lifeless | fungi: s/translation/copy-to/ | 21:18 |
| mordred | or a symlink | 21:19 |
| lifeless | fungi: s/translation/copy-to/ln-s/ | 21:19 |
| lifeless | :P | 21:19 |
| mordred | lifeless just invented new perl syntax | 21:19 |
| fungi | lifeless: i think we already have at least one library where we're extracting the tarball we retrieve and then finding the correct name from setup.cfg before uploading. i need to hunt down which one again | 21:19 |
| *** dboik_ has joined #openstack-infra | 21:19 | |
| mordred | fungi: all of them | 21:19 |
| mordred | fungi: we always extract the tarball, look at setup.cfg, and then upload, iirc | 21:20 |
| fungi | mordred: well, i mean we set it up that way to accommodate one in particular | 21:20 |
| fungi | because it didn't actually match | 21:20 |
| lifeless | anyhow, point is - setup.cfg has to change | 21:20 |
| lifeless | because thats the distribution name | 21:20 |
| fungi | but now i don't recall which one it was | 21:20 |
| lifeless | the package name doesn't have to change. | 21:20 |
| *** ajmiller_ has quit IRC | 21:21 | |
| *** dizquierdo has quit IRC | 21:21 | |
| *** ldnunes has quit IRC | 21:21 | |
| fungi | aha, i think it was https://pypi.python.org/pypi/WSME so more of a normalization issue | 21:22 |
| *** dboik has quit IRC | 21:22 | |
| fungi | on tarballs.o.o they're "wsme" but on pypi they're "WSME" | 21:22 |
| fungi | hrm, no we upload the tagged tarballs as "WSME" on tarballs.o.o too | 21:23 |
| *** dboik_ has quit IRC | 21:23 | |
| *** ddieterly has quit IRC | 21:23 | |
| *** craigbr has joined #openstack-infra | 21:24 | |
| *** chlong has quit IRC | 21:24 | |
| greghaynes | clarkb: question about the nodepool test thing, why not have nodepool build and upload the image? | 21:25 |
| ttx | lifeless: on the cross-project meeting. What's up ? | 21:25 |
| clarkb | greghaynes: it does | 21:25 |
| *** ayoung has joined #openstack-infra | 21:25 | |
| greghaynes | clarkb: why am I having to glance image create then? | 21:25 |
| clarkb | greghaynes: because we are using nodepool's snapshot functionality | 21:25 |
| mordred | greghaynes: base image | 21:25 |
| clarkb | greghaynes: using that because every cloud so far has supported it but every cloud doesn't support dib | 21:26 |
| greghaynes | hrm | 21:26 |
| clarkb | however you may need to change your base image if not using devstack | 21:26 |
| *** ashleighfarnham has quit IRC | 21:26 | |
| greghaynes | I devstack'd, so will see how it goes... | 21:27 |
| mordred | there's a tshirt slogan | 21:27 |
| greghaynes | haha | 21:27 |
| *** ibiris is now known as ibiris_away | 21:27 | |
| *** enikanorov2 has joined #openstack-infra | 21:28 | |
| greghaynes | clarkb: the prepare_node.sh in nodepool.yaml - where is that path relative to? | 21:29 |
| fungi | "well, it worked in greghaynes's devstack" | 21:29 |
| clarkb | greghaynes: relative to the scripts dir at the top of the file | 21:30 |
| clarkb | greghaynes: I used the same fully rooted path for both | 21:30 |
| clarkb | greghaynes: should I make that more clear? | 21:30 |
| *** emagana has quit IRC | 21:30 | |
| greghaynes | ah, I see, where you write it out | 21:30 |
| *** emagana has joined #openstack-infra | 21:30 | |
| *** erikmwilson_ is now known as erikmwilson | 21:30 | |
| clarkb | oh ya its not there in the image portion of the config, I can add a comment there | 21:30 |
| *** amitgandhinz has quit IRC | 21:31 | |
| *** juggler has quit IRC | 21:33 | |
| pleia2 | that one almost got me too, but then I just assumed the scripts dir | 21:34 |
| *** SpamapS has quit IRC | 21:34 | |
| openstackgerrit | Russell Bryant proposed openstack-infra/project-config: Neutron: enable a voting py34 gate on master https://review.openstack.org/182439 | 21:34 |
| *** jodah has left #openstack-infra | 21:34 | |
| *** SpamapS has joined #openstack-infra | 21:34 | |
| *** mattfarina has quit IRC | 21:35 | |
| clarkb | I am currently waiting for nodepool to ssh into the image node it built to make the image | 21:35 |
| clarkb | this is really slow with qemu | 21:35 |
| clarkb | oh hrm image build failed for some reason | 21:36 |
| openstackgerrit | Merged openstack-infra/system-config: Move hardcoded values into jenkins class params https://review.openstack.org/167288 | 21:36 |
| pleia2 | clarkb: how much ram are you requiring? might be the flavor issue I ran into | 21:36 |
| *** hashar has quit IRC | 21:37 | |
| pleia2 | but waiting for it to ssh in is where I've ended up too, eventually it times out | 21:37 |
| lifeless | ttx: server releases on pypi | 21:37 |
| pleia2 | (not ssh, nodepool's timeout) | 21:37 |
| lifeless | ttx: any thoughts? | 21:37 |
| lifeless | ttx: neutron-lbaas requirements.txt is invalid as it doesn't specify the dependency on neutron | 21:37 |
| greghaynes | clarkb: youre making an actual test with this? | 21:37 |
| openstackgerrit | Merged openstack-infra/project-config: Remove custom taskflow sqlalchemy environments https://review.openstack.org/181687 | 21:37 |
| lifeless | ttx: (because neutron isn't on pypi) | 21:38 |
| openstackgerrit | Merged openstack-infra/project-config: Remove extra unused environments from tooz jobs https://review.openstack.org/182021 | 21:38 |
| openstackgerrit | Merged openstack-infra/project-config: Deal with yum --downloadonly CentOS 6.x quirks https://review.openstack.org/181574 | 21:38 |
| openstackgerrit | Merged openstack-infra/project-config: NPM-Run jobs are now voting. https://review.openstack.org/182049 | 21:38 |
| fungi | lifeless: i think they'd still run into it because they want to co-develop with neutron, not with what's been released as neutron | 21:38 |
| openstackgerrit | Merged openstack-infra/project-config: Added NPM-based unit tests for merlin https://review.openstack.org/182084 | 21:38 |
| openstackgerrit | Merged openstack-infra/project-config: Surveil:py34 checks https://review.openstack.org/182353 | 21:38 |
| openstackgerrit | Merged openstack-infra/project-config: Add functional test job for python-muranoclient https://review.openstack.org/182082 | 21:38 |
| clarkb | pleia2: I bumped to 2048 as suggested | 21:38 |
| openstackgerrit | Merged openstack-infra/project-config: Add devstack gate job for Cue project https://review.openstack.org/180774 | 21:38 |
| clarkb | greghaynes: depends on how bad qemu ends up being | 21:38 |
| lifeless | fungi: so no, its more subtle than that | 21:39 |
| lifeless | fungi: we need to separate out 'CI deploy' and 'install_requires' | 21:39 |
| clarkb | greghaynes: its possible we can have a dib based image build test instead that makes a distro that doesn't take a million years to boot | 21:39 |
| fungi | lifeless: well, and that they feel they need to declare a package dependency on neutron in some way | 21:39 |
| lifeless | fungi: do you agree that install_requires for something that imports neutron should include neutron. | 21:39 |
| clarkb | greghaynes: basically cirros + sftp | 21:39 |
| lifeless | (in its install_requires) | 21:39 |
| greghaynes | clarkb: what do you want to test? that the image boots? | 21:39 |
| greghaynes | oh, that you can ssh in too | 21:39 |
| greghaynes | clarkb: the problem with cirros in dib is grub | 21:40 |
| fungi | lifeless: yes, do those plugins import neutron or does it import them? | 21:40 |
| lifeless | fungi: from neutron.db import agents_db | 21:40 |
| lifeless | fungi: it depends on neutron | 21:40 |
| lifeless | (that import is from neutron_lbaas/agent_scheduler.py) | 21:40 |
| fungi | got it. so anyway yes it solves the deployment issue. not the testing issue | 21:40 |
| lifeless | right | 21:40 |
| lifeless | the testing issue should either preinstall neutron @ zuul_ref | 21:40 |
| clarkb | greghaynes: I want to test image build + nodes booting. For snapshots that requires working sftp | 21:41 |
| lifeless | or edit the requirements.txt during testing to specify a git url with the zuul ref. | 21:41 |
| clarkb | greghaynes: ya it would likely have to be ubuntu minimal | 21:41 |
| *** dkranz has quit IRC | 21:41 | |
| greghaynes | clarkb: if you just want to test booting then id set init to be something super simple like touch file; shutdown | 21:41 |
| greghaynes | and then inspect the image after | 21:41 |
| *** ayoung has quit IRC | 21:41 | |
| clarkb | greghaynes: no I need to be able to ssh in, thats how the image build works | 21:42 |
| greghaynes | ok, so yea if you need ssh then ubuntu-minimal ftw | 21:42 |
| fungi | lifeless: yep. separately providing (and using a local filesystem url in the requirements list of the project, maybe, if absolutely needed) is the way to go from a testing perspective | 21:42 |
| *** salv-orlando has quit IRC | 21:42 | |
| lifeless | fungi: it would be a git url pointing at zuul, no? whereverthe zuul ref is findable | 21:42 |
| fungi | because pip doesn't grok our git repo caching setup | 21:42 |
| clarkb | OH! I know the problem I think | 21:42 |
| clarkb | gah | 21:42 |
| lifeless | fungi: the pip git location is configurable, --source=/opt/stack/new probably is all it needs | 21:43 |
| fungi | lifeless: so we still want to not have pip go cloning neutron in its entirety | 21:43 |
| fungi | yep, that's probably fine then | 21:43 |
| lifeless | fungi: if you look at the failure in that job yo ucan see it cloned it in its entirety :) | 21:43 |
| clarkb | greghaynes: pleia2 security groups | 21:43 |
| *** ddieterly has joined #openstack-infra | 21:43 | |
| lifeless | fungi: so, we can do better, but one is correctness and one is optimisation :) | 21:43 |
| fungi | yes, and that leads to broken because cloning neutron over and over from the network is pain | 21:44 |
| pleia2 | clarkb: oh bother | 21:44 |
| lifeless | fungi: Oh, I agree. | 21:44 |
| greghaynes | clarkb: hhahah | 21:44 |
| fungi | lifeless: mere details. always broken vs randomly broken. still broken | 21:44 |
| lifeless | problem is this job seems very naive, on lots of counts. So, how to help them do better. | 21:44 |
| *** nelsnelson has quit IRC | 21:44 | |
| lifeless | who knows the most about this split out layout? russellb ? anteaya ? markmcclain ? | 21:44 |
| lifeless | e.g. who is the one person we should educate | 21:44 |
| anteaya | lifeless: dougwig | 21:45 |
| lifeless | dougwig: hi, whats your email? I'm going to do a brain dump of the issues here to -dev and I'd like to cc you :) | 21:45 |
| *** Vinod___ has quit IRC | 21:45 | |
| *** ildikov has quit IRC | 21:45 | |
| anteaya | pcm or pc_m is a good candidate as well but dougwig is the best point person | 21:45 |
| lifeless | whats a pcm? | 21:46 |
| *** banix has joined #openstack-infra | 21:46 | |
| anteaya | if you want to trouble yourself (I'm slow) in educating me, I'm grateful and will pass on what I learn to the best of my ability but I'm afraid you may decide your time is not worth it | 21:46 |
| dougwig | lifeless: hiya, dougwig@parkside.io, but i'll see it on -dev as well | 21:46 |
| lifeless | dougwig: cool | 21:46 |
| lifeless | anteaya: I'll cc you as well | 21:46 |
| ttx | lifeless: I think last time that was considered we said that was a stupid idea. But I guess if things start to depend on something, there is a use case | 21:46 |
| anteaya | pcm is a nick for paul michail I think | 21:46 |
| anteaya | lifeless: feel free | 21:47 |
| *** cdent has quit IRC | 21:47 | |
| clarkb | pleia2: greghaynes yup opening that up allowed image build to move onto snapshotting | 21:47 |
| bookwar | hi, can someone direct me, how Gerrit UI Parser differentiates between Jenkins check jobs and Jenkins gate jobs so those are shown as different sections in results table? | 21:47 |
| bookwar | is it based on some keyword in result message? | 21:47 |
| ttx | lifeless: food for the "release model" session probably | 21:47 |
| lifeless | ttx: ok | 21:47 |
| pleia2 | clarkb: great! | 21:48 |
| lifeless | fungi: are you sheparding the neutron-lbaas nonvoting patch through? | 21:48 |
| clarkb | bookwar: yes, iirc check jobs say "we ran in check" and gate jobs say "we ran in gate" and hideci.js just matches on that | 21:48 |
| fungi | ttx: last time we said "maybe part of horizon, because something else imports it" but now with the driver/backend/services splits going on in projects, that situation is coming back magnified manyfold | 21:48 |
| clarkb | bookwar: https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/gerrit/hideci.js is the source | 21:48 |
| clarkb | bookwar: you should be able to see it in your browsers debugger too | 21:49 |
| fungi | lifeless: it's in the gate with some unrelated changes ahead of it | 21:49 |
| bookwar | clarkb: thanks! | 21:49 |
| clarkb | ok bare-centos6 finally updated in hpcloud | 21:49 |
| ttx | fungi: right, I don't think we ever said "never", there was probably no use for it. So that can easily be reconsidered | 21:49 |
| fungi | lifeless: er, i'm thinking about a different patch, but yes | 21:49 |
| *** SergeyL_irccloud has joined #openstack-infra | 21:50 | |
| *** annegentle has joined #openstack-infra | 21:50 | |
| *** dkehn has quit IRC | 21:50 | |
| fungi | lifeless: eta is 6 minutes to merge and then a few more before zuul knows to stop voting on that | 21:50 |
| fungi | hopefully ~21:05 utc depending on where it falls in the puppetage | 21:51 |
| *** ociuhandu has quit IRC | 21:51 | |
| openstackgerrit | Merged openstack-infra/project-config: Add experimental glance_store functional test job https://review.openstack.org/182000 | 21:51 |
| fungi | um, 22:05 utc ;) | 21:51 |
| * fungi is clearly living in the past | 21:51 | |
| *** dkehn has joined #openstack-infra | 21:52 | |
| dtroyer | fungi mentions Criswell, then lives in the past… | 21:54 |
| *** peristeri has quit IRC | 21:54 | |
| fungi | you gotta admit though, he's _amazing_ | 21:54 |
| *** ivar-laz_ has quit IRC | 21:54 | |
| openstackgerrit | Dolph Mathews proposed openstack-infra/elastic-recheck: add CLI command for executing elastic-recheck queries https://review.openstack.org/137234 | 21:54 |
| *** ivar-lazzaro has joined #openstack-infra | 21:54 | |
| fungi | or was, anyway, until he failed to predict his death in the 1980s | 21:54 |
| clarkb | greghaynes: pleia2 my script to install the ssh authorized_keys file doesn't seem to work yet but everything else does so pushing new patchset | 21:55 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Document running Nodepool for testing https://review.openstack.org/179840 | 21:55 |
| dougwig | lifeless, fungi - so, scrolling back, i see that you folks are frustrated. the current state of affairs, unless i'm batshit insane, was worked out with infra way back when it occurred. we discussed exactly what you're proposing now, and it was rejected at the time. the job is clearly borked, but can be fixed by having tox pull in the local repo for the | 21:55 |
| dougwig | devstack job. | 21:55 |
| *** armax has quit IRC | 21:56 | |
| pleia2 | clarkb: yeah, need to sudo bash -c or something, because sudo and piping is no good | 21:56 |
| pleia2 | clarkb: I gave an example inline in patchset 3 | 21:56 |
| clarkb | pleia2: ya I thought I addressed that but maye not well enough | 21:56 |
| fungi | dougwig: not tox, no. we need to preinstall the neutron ref being tested in devstack-gate (not tox) | 21:56 |
| clarkb | pleia2: I did write to file in homdir then sudo mv | 21:56 |
| pleia2 | clarkb: oh I see now, yeah | 21:56 |
| *** juggler has joined #openstack-infra | 21:57 | |
| dougwig | fungi: that should already be happening. but the tox invocation doesn't use it. | 21:57 |
| openstackgerrit | Merged openstack-infra/project-config: Add functional test job for python-heatclient https://review.openstack.org/180536 | 21:57 |
| openstackgerrit | Merged openstack-infra/project-config: Make neutron-lbaas-dsvm-api non-voting. https://review.openstack.org/182455 | 21:59 |
| fungi | dougwig: i must be confused. i don't see tox used in the {pipeline}-neutron-lbaas-dsvm-api{branch-designator} definition unless it's called by your gate script | 21:59 |
| fungi | or one of the hook scripts at any rate. digging in the neutron-lbaas repo now to confirm | 21:59 |
| dougwig | fungi: yes, i copied the gate hook from another api job, which calls tox, and missed the two repo connection. | 22:00 |
| dougwig | fungi: right here: https://github.com/openstack/neutron-lbaas/blob/master/neutron_lbaas/tests/contrib/post_test_hook.sh | 22:00 |
| fungi | dougwig: also we said "don't do this" and people said "you're not doing the work, you can't tell us how to design this" so we stopped caring | 22:00 |
| clarkb | pleia2: the trick to debugging this is you boot that image but provide your key to nova so cloud init sets it up on the ubuntu user, then you can dig around and see why that script broke | 22:01 |
| fungi | i'm happy to leave neutron unable to merge code if that's their preference, but i'm working on the assumption it's not | 22:01 |
| zaro | fungi: have you look into putting correct github key in heira for review-dev? | 22:01 |
| clarkb | pleia2: its still super slow but seems to eventually get there | 22:01 |
| dougwig | fungi: whoa, i'm sorry, i *NEVER* meant to convey that. | 22:01 |
| *** ildikov has joined #openstack-infra | 22:01 | |
| fungi | zaro: no, i should probably do that. thanks--switching gears | 22:01 |
| *** Swami has quit IRC | 22:02 | |
| *** arielb has quit IRC | 22:02 | |
| *** mjturek1 has joined #openstack-infra | 22:03 | |
| lifeless | dougwig: nope, I'm not frustrated, nor playing the blame game. | 22:05 |
| pleia2 | clarkb: jenkins@ubuntu-1431468036:~$ | 22:05 |
| pleia2 | huzzah! | 22:06 |
| clarkb | pleia2: nice, so my script worked for you? | 22:06 |
| *** hdd_ has quit IRC | 22:06 | |
| lifeless | dougwig: we've identified bugs in the current implementation, and I want to make sure knowledge about the heuristics around this area, and why, is spread | 22:06 |
| clarkb | pleia2: I am waiting for my test node to finish booting so I can look around on it | 22:06 |
| lifeless | dougwig: rather than just fixing it once and waiting for it to happen again. | 22:06 |
| pleia2 | clarkb: actually, I was using my previously prepared one where I was using sudo bash | 22:06 |
| clarkb | pleia2: even better, maybe I should just use that | 22:06 |
| *** Guest38967 has quit IRC | 22:09 | |
| *** dimtruck has quit IRC | 22:09 | |
| *** mjturek1 has quit IRC | 22:10 | |
| fungi | dougwig: and as lifeless points out, part of this is due to prior conscious choices by release management not to release api server packages to pypi, but that can certainly be revisited | 22:10 |
| dougwig | lifeless, fungi - i certainly appreciate the help getting things unwedged, and getting things sorted out properly. | 22:10 |
| fungi | i strongly suspect the current job was hurriedly put into place to avoid regressing immediately following the services split work, and was "good enough" to test the neutron-lbass repo changes on the assumption that neutron master was known working, but where the problem came in was making that job run on neutron changes. it doesn't even know about the neutron changes it would supposedly test | 22:10 |
| *** dimtruck_ has joined #openstack-infra | 22:10 | |
| fungi | the job simply wasn't designed properly to test changes to anything other than the neutron-lbass repo itself | 22:10 |
| *** dimtruck_ is now known as dimtruck | 22:11 | |
| fungi | it would be akin to running keystone unit tests for nova changes | 22:11 |
| *** jcooley has joined #openstack-infra | 22:11 | |
| greghaynes | clarkb: that was my experience with qemu booting for dib testing too | 22:12 |
| greghaynes | clarkb: too slow :( | 22:12 |
| clarkb | pleia2: aha! I need a sync I think | 22:12 |
| clarkb | pleia2: the file is there with no content | 22:12 |
| pleia2 | clarkb: which file? | 22:12 |
| lifeless | dougwig: anteaya: mail sent. | 22:12 |
| clarkb | pleia2: authorized_keys file for jenkins user | 22:12 |
| pleia2 | clarkb: ah :) I just started a nodepool run again trying your way | 22:13 |
| * pleia2 wait&see | 22:13 | |
| *** jaypipes has quit IRC | 22:13 | |
| anteaya | lifeless: thank you | 22:14 |
| annegentle | reed: around? | 22:14 |
| reed | annegentle, yep | 22:14 |
| annegentle | reed: great! Have a moment to talk about the TC comm plan? | 22:14 |
| reed | sure | 22:14 |
| reed | switch to #openstack-community? | 22:15 |
| annegentle | reed: sounds good | 22:15 |
| reed | unless it requires infra | 22:15 |
| greghaynes | clarkb: the heredoc within a heredoc is hilarious | 22:16 |
| dougwig | lifeless: thank you | 22:16 |
| clarkb | greghaynes: sshhhh I don't want to scare anyone off with my bad bash | 22:16 |
| *** doude has quit IRC | 22:17 | |
| mordred | clarkb: we've been scared for a while | 22:17 |
| *** soren has quit IRC | 22:17 | |
| clarkb | greghaynes: I am not actually sure there is a better way to do that for the "just copy paste this thing here and it works"ness of that script and the inner script | 22:17 |
| greghaynes | clarkb: I think what you have there is correct | 22:18 |
| *** bknudson has quit IRC | 22:18 | |
| lifeless | dougwig: I'm happy to discuss more, obviously | 22:18 |
| greghaynes | clarkb: its just hilarious | 22:18 |
| lifeless | dougwig: and the great folk here know way more than I about the CI plumbing and where to tweak things | 22:18 |
| dougwig | lifeless: heh, i was just typing more | 22:18 |
| pleia2 | clarkb: your instructions work fine :) have jenkins@ubuntu-1431468814:~$ | 22:18 |
| *** enikanorov2 has quit IRC | 22:19 | |
| openstackgerrit | Merged openstack-infra/subunit2sql: Add command to calculate failure count and graph it https://review.openstack.org/176043 | 22:19 |
| clarkb | pleia2: woot, I am adding a sleep then sync should make it work on this test node | 22:19 |
| pleia2 | clarkb: yeah, I'm testing on virtualbox, so it may be slightly faster than yours (though only by a little) | 22:19 |
| *** otter768 has joined #openstack-infra | 22:20 | |
| clarkb | pleia2: now you should try stopping nodepool (^C is fine) then grabbing greghaynes and mordreds use shade change, run the pip install -U ./nodepool again then start nodepool | 22:20 |
| zaro | clarkb: took at the logs from review-dev. i think i've miscommunicated myself | 22:20 |
| mordred | fungi, clarkb: so - it's not super important - but we approved a change to os-client-config an hour ago and it never seems to have hit zuul - so I just re-approved and I still see nothing in status ... | 22:21 |
| mordred | mentioning in case it's an indication that something is stuck | 22:21 |
| dougwig | lifeless, fungi - i don't disagree with anything in your email, and feel goofy for missing the two repo link there. ultimately getting the servers in pypi is the answer, and solves a lot of other issues as well. shorter-term fix, getting that job to use the local neutron instead of cloning master should get us limping along? or do you have other | 22:21 |
| dougwig | suggestions? | 22:21 |
| zaro | clarkb: so i'm wondering if it would make sense to make zuul-dev run jobs then post logs to swift store. | 22:21 |
| openstackgerrit | Dolph Mathews proposed openstack-infra/release-tools: Add a script to safely update feature branches https://review.openstack.org/126719 | 22:21 |
| clarkb | zaro: it can do that we would just need to configure a swift user for it | 22:21 |
| clarkb | mordred: what change? | 22:21 |
| fungi | mordred: i see other recent events in zuul... what change? | 22:22 |
| pleia2 | clarkb: happen to have the shade change # handy? | 22:22 |
| fungi | mordred: any chance it had no jenkins +1 on it? | 22:22 |
| mordred | fungi, clarkb: 182072 | 22:22 |
| zaro | clarkb: does having that setup on zuul-dev help us debug issues around that stuff? | 22:22 |
| mordred | nope. has a +1 - I also both recheck and second-+A'd it | 22:22 |
| zaro | or do you not really care? | 22:22 |
| clarkb | pleia2: https://review.openstack.org/#/c/168603/14 | 22:23 |
| clarkb | greghaynes: ^ looks like you haven't addressed my comments? | 22:23 |
| *** signed8b_ has joined #openstack-infra | 22:23 | |
| fungi | mordred: next guess is depends-on loop | 22:23 |
| greghaynes | clarkb: no :( | 22:23 |
| clarkb | zaro: I don't think it helsp a whole lot, we are moving to having os-loganalyze test against swift via devstack | 22:23 |
| *** tnovacik has joined #openstack-infra | 22:23 | |
| fungi | mordred: nope, not a loop. just depending on a not-approved change | 22:23 |
| greghaynes | clarkb: ETHINGS | 22:23 |
| *** redixin has quit IRC | 22:23 | |
| pleia2 | clarkb: well, I'll keep this VM around so I can test when it's ready | 22:23 |
| clarkb | mordred: its because you depend on a thing that isn't merged | 22:24 |
| lifeless | dougwig: so servers in pypi is orthogonal -because you still need the git version of neutron that you're testing. | 22:24 |
| fungi | mordred: https://review.openstack.org/182061 needs to be reviewed/approved first | 22:24 |
| *** markvoelker has joined #openstack-infra | 22:24 | |
| *** thedodd has quit IRC | 22:24 | |
| clarkb | pleia2: it should be mostl ready, the only thing we have to be aware of is if we change the cloud config item in nodepool.yaml we will have to restart the server | 22:24 |
| *** Rockyg has quit IRC | 22:24 | |
| clarkb | pleia2: so I would go ahead and give it a go | 22:24 |
| lifeless | dougwig: changes to neutron-lbass *could* use neutron-last-release, thats a separate discussion | 22:24 |
| pleia2 | clarkb: ok, having a look | 22:24 |
| lifeless | dougwig: changes to neutron *must* use that changed neutron. | 22:24 |
| *** otter768 has quit IRC | 22:24 | |
| lifeless | dougwig: so the basic approach is fine, but you need to encode the ZUUL_REF into the git url you give pip | 22:25 |
| fungi | dougwig: since you're running within devstack-gate, it should already have checked out the neutron change to be tested under /opt/stack | 22:25 |
| lifeless | dougwig: the pip docs describe how to do this | 22:25 |
| lifeless | or | 22:25 |
| clarkb | fungi: yup thats how I would fix this | 22:25 |
| *** signed8bit has quit IRC | 22:25 | |
| fungi | in this case don't even need to worry about zuul_ref i don't think, because devstack-gate already took care of it | 22:25 |
| clarkb | fungi: in fact thats why d-g is used iirc | 22:25 |
| lifeless | assuming fungi is right, just use /opt/stack/neutron#egg_info=neutron | 22:25 |
| clarkb | so that neutron et al are installed at the right versions | 22:25 |
| openstackgerrit | Merged openstack-infra/subunit2sql: Improve run_time graph formatting https://review.openstack.org/178276 | 22:25 |
| mordred | clarkb, fungi; thank you HEADDESK | 22:25 |
| clarkb | lifeless: no everything is already installed | 22:25 |
| clarkb | lifeless: we don't need to install it again | 22:26 |
| greghaynes | clarkb: oh hey, my node is building | 22:26 |
| lifeless | clarkb: pip disagrees with you | 22:26 |
| clarkb | lifeless: ok then maybe they have removed more of devstack than I thought | 22:26 |
| fungi | clarkb: well, in that job there's a gate hook running stuff with tox | 22:26 |
| lifeless | clarkb: let me check the log | 22:26 |
| clarkb | it used to be that all those jobs ran devstack so that it would setup neutron and the root wrap rules | 22:26 |
| fungi | clarkb: so neutron is not preinstalled into the tox venv by devstack-gate | 22:26 |
| *** weshay has quit IRC | 22:26 | |
| lifeless | clarkb: it may be pip was just checking the project name, but since tha twas supplied it shouldn't have needed to. | 22:26 |
| zaro | clarkb: ok, maybe asked differently.. do we care whether we store job logs from review-dev at all? | 22:26 |
| clarkb | zaro: I personally don't, I can always just look at jenkins-dev but others may | 22:27 |
| clarkb | zaro: I thought you did :) | 22:27 |
| lifeless | dougwig: anyhow - either a file path or perhaps a file:// url with the egg name supplied should work fine | 22:27 |
| fungi | clarkb: unless maybe tox is set to do site-packages to pick up the devstack-installed neutron instead | 22:27 |
| clarkb | fungi: no I think this tox stuff is all new | 22:27 |
| fungi | oh fun | 22:27 |
| clarkb | fungi: the old functional jobs for neutron* ran devstack for the test env | 22:27 |
| clarkb | fungi: sounds like the use tox instead now | 22:27 |
| lifeless | but if neutron is already installed, there'd be no need to monkey with requirements at all | 22:27 |
| lifeless | it should just work | 22:28 |
| *** dims_ has joined #openstack-infra | 22:28 | |
| clarkb | fungi: which is fine except we should stop using devstack-gate entirely and just make a tox job | 22:28 |
| clarkb | unless, do these need root? our tox jobs will explode on that :/ | 22:28 |
| fungi | clarkb: yeah, i'm dissecting it now | 22:28 |
| *** esker has quit IRC | 22:29 | |
| mordred | we also may need to figure out the tox/zuul-cloner intersection we were discussing the other day | 22:29 |
| mordred | because so far none of our tox things install from on-host git repos | 22:29 |
| clarkb | mordred: if the idea is that running this locally will work then yes | 22:29 |
| mordred | clarkb: yah | 22:29 |
| *** sdake_ has joined #openstack-infra | 22:29 | |
| clarkb | but currently the assumption seems to be d-g will set things up for us | 22:29 |
| clarkb | at least partially, which is fine | 22:30 |
| lifeless | hah, pbr is broken by neutron too | 22:30 |
| lifeless | we need to get that hooks patch in soon :) | 22:30 |
| lifeless | http://logs.openstack.org/85/181785/4/check/check-pbr-installation-dsvm/f78dad8/console.html#_2015-05-12_21_31_43_747 | 22:30 |
| *** dims has quit IRC | 22:30 | |
| zaro | clarkb: i only care because when i test changes on review-dev I want to make sure that the round trip for a change works as expected, gerrit->zuul->jenkins->gerrit. I guess that doesn't have to include the logging | 22:31 |
| zaro | i mean job logs | 22:32 |
| mordred | lifeless: I support all of your patches | 22:32 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Document running Nodepool for testing https://review.openstack.org/179840 | 22:32 |
| zaro | clarkb: so i guess no big to me either. | 22:32 |
| lifeless | turns out I proposed it to kilo. OOPS. | 22:32 |
| clarkb | pleia2: greghaynes ^ that works for me with current nodepool master, trying the shade change now | 22:32 |
| greghaynes | clarkb: missed my comment | 22:33 |
| *** tim_o has quit IRC | 22:33 | |
| greghaynes | it worked for me, but we should flush privs still | 22:33 |
| *** sdake has quit IRC | 22:33 | |
| pleia2 | clarkb: applied the change to my local nodepool, do I still run with my old nodepool.yaml for greghaynes' change? | 22:33 |
| fungi | lifeless: i do see "check-neutron-lbaas-dsvm-api: in progress (non-voting)" on neutron changes in zuul now, so looks like it took effect | 22:33 |
| greghaynes | pleia2: yes | 22:33 |
| pleia2 | greghaynes: thanks | 22:33 |
| clarkb | greghaynes: doesn't the update command avoid needing that? | 22:34 |
| clarkb | greghaynes: you need to flush if editing the table directly | 22:34 |
| clarkb | ok first shade comment, I am compiling lxml right now | 22:34 |
| lifeless | how do I change the branch on https://review.openstack.org/#/c/182433/ ? | 22:34 |
| clarkb | lifeless: you push a new change | 22:34 |
| lifeless | clarkb: I just did | 22:34 |
| lifeless | clarkb: I rebased it on origin/master and did git review | 22:35 |
| greghaynes | clarkb: oh? everywhere ive read says to flush privs after that | 22:35 |
| greghaynes | clarkb: I dont actually know for sure thoughn | 22:35 |
| mordred | clarkb: yup. keystoneclient | 22:35 |
| clarkb | lifeless: and that didn't make a new change in gerrit? | 22:35 |
| mordred | clarkb: it's in the todolist to make go away | 22:35 |
| clarkb | greghaynes: mordred will know. Tell us when I must flush privs | 22:35 |
| mordred | what update command? | 22:35 |
| clarkb | OpenStackCloudException: CA Cert /opt/stack/data/CA/int-ca/ca-chain.pem does not exist | 22:36 |
| greghaynes | mordred: https://review.openstack.org/#/c/179840/5/doc/source/testing.rst | 22:36 |
| mordred | are you editing a table in the mysql database? | 22:36 |
| greghaynes | mordred: is it needed after the grant / create? | 22:36 |
| lifeless | clarkb: looks like not. Perhaps russellb pushed a copy aimed at master already or something... | 22:36 |
| clarkb | mordred: no just doing GRANTs | 22:36 |
| lifeless | ahh he did | 22:36 |
| mordred | clarkb: you do not need flush after grants | 22:36 |
| clarkb | mordred: thanks | 22:36 |
| lifeless | https://review.openstack.org/#/c/182438/ is the one we need to land | 22:36 |
| *** ashleighfarnham has joined #openstack-infra | 22:36 | |
| clarkb | ok, can someone explain to me why novaclient is perfectly happy with devstack's SSL setup but shade isn't? | 22:36 |
| clarkb | especially since I am not using ssl | 22:37 |
| lifeless | and nuts, I've reset the gate timer I suspect | 22:37 |
| mordred | clarkb: might be a bug | 22:37 |
| mordred | clarkb: but we use it against devstack | 22:37 |
| mordred | clarkb: so I'm confused as to what's different in your setup | 22:37 |
| clarkb | mordred: yes iirc you guys worked around it rather than fixing it | 22:37 |
| mordred | that it's not working for you | 22:37 |
| mordred | clarkb: ok. well, we should fix that then | 22:37 |
| clarkb | mordred: you unset some openstack stackrc thing let me go look at the test | 22:37 |
| mordred | clarkb: I have a hunch I'm about to say that devstack has a bug if it's putting a reference to a non-existent CA into localrc | 22:38 |
| clarkb | I don't think that was it | 22:38 |
| mordred | kk | 22:38 |
| mordred | clarkb: in any case - I agree with you, we should not be giving cert errors/warnings if you're not using https | 22:39 |
| clarkb | unset OS_CACERT is what the functional shade test hook does | 22:39 |
| mordred | I suggest that in this case too - because setting OS_CACERT is telling keystoneclient to use a cert | 22:40 |
| mordred | and if that cert does not exist, then that is an error | 22:40 |
| clarkb | but ya OS_CACERT points at a thing that does not exist so likely also a devstack bug | 22:40 |
| clarkb | mordred: keystone is using a cert for what? | 22:40 |
| *** tnovacik has quit IRC | 22:40 | |
| clarkb | mordred: I specifically told it to use http | 22:40 |
| greghaynes | tokens? | 22:40 |
| mordred | you did - but you also told it you had a cert | 22:40 |
| mordred | nono | 22:40 |
| mordred | this is for self-signed CA support | 22:40 |
| clarkb | mordred: and novaclient (which I thought uses keystoneclient) doesn't explode | 22:41 |
| mordred | or, local signed CA | 22:41 |
| greghaynes | ah | 22:41 |
| mordred | clarkb: I'd argue that they're doing the wrong thing | 22:41 |
| clarkb | mordred: ok so this is for trustying the endpoint | 22:41 |
| mordred | clarkb: beause if you tell them to use a cert | 22:41 |
| mordred | and the cert doesn't exist | 22:41 |
| mordred | I'd want it to explode | 22:41 |
| clarkb | and even though its http they want to trust ith | 22:41 |
| *** isviridov is now known as isviridov_away | 22:41 | |
| clarkb | mordred: I want it to explode if I https | 22:41 |
| mordred | if you don't want to use the cert, don't tell it to use the cert | 22:41 |
| mordred | sure | 22:41 |
| clarkb | mordred: it should never explode if using a different protocol | 22:41 |
| mordred | so - let's make two patches there | 22:41 |
| mordred | one to shade to set insecure=true if the auth_url == http:// | 22:41 |
| mordred | and one to devstack to not put OS_CACERT into the file if it's not doing SSL | 22:42 |
| mordred | solve it in both directions? | 22:42 |
| clarkb | pleia2: looks like we have to unset OS_CACERT then rebase greghaynes/mordred's change on top of master to get my fixes for not having jenkins/zmq/etal | 22:42 |
| clarkb | mordred: insecure == true if auth_url does not start iwth https | 22:42 |
| greghaynes | clarkb: ok, ill rebase on master in a moment when im done with your comment fixes | 22:42 |
| mordred | clarkb: yes | 22:42 |
| mordred | clarkb: so - unset OS_CACERT for now, then I'll work up the change for the other thing | 22:42 |
| *** Krinkle is now known as Krinkle|detached | 22:42 | |
| clarkb | mordred: ya sounds good | 22:42 |
| pleia2 | clarkb: yeah, I was getting some ssl errors | 22:43 |
| *** salv-orlando has joined #openstack-infra | 22:43 | |
| mordred | clarkb: now - not to make you sad ... | 22:43 |
| mordred | clarkb: but we don't actually handle auth_url AT ALL | 22:43 |
| mordred | clarkb: because it's not actually a keystone param for real | 22:43 |
| *** sdake has joined #openstack-infra | 22:43 | |
| mordred | it's a pluggable param that is not always present | 22:43 |
| mordred | for instance, apparently x509 auth plugin does not come along with an auth_url param | 22:43 |
| clarkb | mordred: then maybe keystoneclient needs fixing? | 22:43 |
| mordred | clarkb: I think so | 22:43 |
| mordred | clarkb: that said- I'm thining "if 'auth_url' in auth and auth['auth_url'].startswith('http'): insecure=True" | 22:44 |
| clarkb | ok I think I have nodepool running with shade now | 22:44 |
| mordred | clarkb: will get it 99% of the time | 22:44 |
| mordred | neat! | 22:44 |
| *** ihrachyshka has joined #openstack-infra | 22:44 | |
| clarkb | mordred: ya that seems reasonable for shade to protect against if the underlying thing is broken | 22:44 |
| clarkb | since that is what shade is there for | 22:44 |
| mordred | yah | 22:45 |
| clarkb | mordred: greghaynes shade needs to squelch iso8601 debug logging | 22:45 |
| mordred | oh! that's new - I haven't seen that yet | 22:45 |
| greghaynes | hah | 22:45 |
| greghaynes | the things you discover when you actually run the code | 22:45 |
| *** salv-orlando has quit IRC | 22:46 | |
| fungi | clarkb: lifeless: dougwig: so i _think_ where this is breaking is when tox is creating the virtualenv for tempest to use, it's installing neutron-lbaas (to get some tempest tests from that repo? or just because it doesn't know any better?) and that's dragging in the neutron master git requirement, which then fails to install | 22:46 |
| clarkb | pleia2: with that unset and rebased I am building a node | 22:46 |
| greghaynes | \O/ | 22:46 |
| mordred | clarkb: AH. actually | 22:46 |
| clarkb | pleia2: if that succeeds, I am going to delete my image and have it make a new one of those too | 22:46 |
| lifeless | fungi: yes, as I said- that url in the requirements.txt needs the ZUUL_REF | 22:46 |
| mordred | clarkb: foudn it - shade is being extra helpful and erroring if you pass a cacert path to it that doesnt' exist | 22:46 |
| *** sdake_ has quit IRC | 22:46 | |
| clarkb | pleia2: maybe you want to test it with a fresh db? | 22:46 |
| clarkb | pleia2: just drop database nodepool; create database nodepool when nodepool is stopped | 22:47 |
| fungi | lifeless: or none at all. i'm not sure why neutron is technically needing to be installed into tempest's virtualenv | 22:47 |
| dougwig | fungi: it's definitely the tox venv setup, and it's because it's hardcoding neutron master. | 22:47 |
| mordred | clarkb: so - it actually is coming from us | 22:47 |
| pleia2 | clarkb: yeah, I think I do | 22:47 |
| dougwig | fungi: in effect, it is not testing what zuul is telling it to. | 22:47 |
| clarkb | mordred: gotcha, I think it should not do that if https is not used | 22:47 |
| mordred | yup | 22:47 |
| lifeless | fungi: if its not needed, removing the addition of neutron to the requirements.txt should address that | 22:47 |
| mordred | doing now | 22:47 |
| lifeless | dougwig: ^ | 22:47 |
| fungi | lifeless: dougwig: but tempest is going to test the neutron which is installed and running on the system, not the one in the tox virtualenv | 22:47 |
| clarkb | mordred: but we should also fix devstack, I can track that down | 22:47 |
| dougwig | fungi: no, tox will run it all under the venv it creates. | 22:47 |
| lifeless | dougwig: know any neutron cores? we need to get https://review.openstack.org/#/c/182438/ +A'd again | 22:48 |
| dougwig | fungi: the env that dsvm created is *CORRECT*. it's tox plus the hardcoded git url that is the bug. | 22:48 |
| pleia2 | clarkb: so just envvar unset OS_CACERT? | 22:48 |
| lifeless | dougwig: cause I messed it up by pushing unnecessarily | 22:48 |
| clarkb | pleia2: yup and do the rebase onto master | 22:48 |
| fungi | lifeless: dougwig: so the one that's been installed in the global system context by devstack-gate has the proposed neutron patch, but the thing getting (presumably unnecessarily) dragged into the virtualenv for tempest is uninstallable | 22:48 |
| dougwig | lifeless: yes, standby | 22:48 |
| lifeless | dougwig: thanks | 22:48 |
| *** whoops has quit IRC | 22:49 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Don't error on missing certs when not https https://review.openstack.org/182488 | 22:49 |
| mordred | clarkb, greghaynes: ^^ | 22:49 |
| clarkb | mordred: from devstack: # Set the pointer to our CA certificate chain. Harmless if TLS is not used. | 22:51 |
| *** tmcpeak has quit IRC | 22:52 | |
| clarkb | mordred: so I guess the expectation is that devstack is not at fault here and clients should be smart | 22:52 |
| *** ddieterly has quit IRC | 22:53 | |
| mordred | kk | 22:53 |
| greghaynes | mordred: heh, I dont think that is valid in python | 22:53 |
| mordred | clarkb: so - potentially we shoudl just remove the validation from shade altogether? | 22:53 |
| greghaynes | its somewhere between a statement and expression | 22:53 |
| clarkb | mordred: ya, I think let that go through to the existing clients/sdk | 22:53 |
| mordred | clarkb: kk | 22:53 |
| pleia2 | clarkb: cool, got past the bit where it errored earlier | 22:53 |
| pleia2 | clarkb: state is ready \o/ | 22:54 |
| *** banix has quit IRC | 22:54 | |
| clarkb | pleia2: and this was starting with a fresh db? | 22:54 |
| pleia2 | so wfm with OS_CACERT unset | 22:54 |
| pleia2 | clarkb: yeah | 22:54 |
| clarkb | nice | 22:54 |
| clarkb | so if my starting from dirty db works we can wait for greghanyes to address my comments then +2 | 22:55 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Don't error on missing certs https://review.openstack.org/182488 | 22:55 |
| mordred | k. just remove the validation | 22:55 |
| *** davideagnello has quit IRC | 22:56 | |
| openstackgerrit | greghaynes proposed openstack-infra/nodepool: Use shade to get the client objects https://review.openstack.org/168603 | 22:56 |
| *** davideagnello has joined #openstack-infra | 22:56 | |
| greghaynes | clarkb: ^ | 22:56 |
| mordred | clarkb: so - are you close to a devstack job for this then? because co-gating shade and nodepool would be killercool | 22:57 |
| *** Swami has joined #openstack-infra | 22:57 | |
| * mordred is excited by clarkb, pleia2 and greghaynes this afternoon | 22:57 | |
| clarkb | mordred: sort of, I am not convinced that hpcloud qemu will keep up | 22:57 |
| clarkb | greghaynes: did you get it working on hpcloud? | 22:57 |
| greghaynes | "it"} | 22:58 |
| greghaynes | ? | 22:58 |
| greghaynes | oh, booting the node? | 22:58 |
| clarkb | greghaynes: image built and node booted with qemu on hpcloud | 22:58 |
| greghaynes | no | 22:58 |
| greghaynes | it halted there | 22:58 |
| clarkb | mordred: maybe you can remove some of the speed up loops in hpcloud for us | 22:58 |
| clarkb | otherwise this will have to be a single cloud job, or we have to build our own base image that is light enough to work on hpcloud | 22:58 |
| clarkb | which we could do but may take some time to get right | 22:59 |
| greghaynes | I would really like the make a qemu bootable image for hpcloud | 22:59 |
| greghaynes | I want that for dib testing | 22:59 |
| greghaynes | *really* want that | 22:59 |
| mordred | ok. I'm going to dinners | 23:00 |
| pleia2 | enjoy | 23:00 |
| clarkb | greghaynes: basically we just need bash, ssh, and sftp for nodepool | 23:01 |
| clarkb | greghaynes: cirros lacks sftp | 23:01 |
| greghaynes | clarkb: cirros also cant do the grub install we need | 23:01 |
| clarkb | but whatever is light weight enough to have those three things should be useable | 23:01 |
| clarkb | greghaynes: we don't need that | 23:01 |
| greghaynes | how will it boot? | 23:01 |
| clarkb | greghaynes: cirros has a bootloader right? | 23:01 |
| greghaynes | oh, not using dib | 23:01 |
| greghaynes | psh | 23:01 |
| clarkb | but ya I don't care what we use as long as its useable, could try ubuntu-minimal but that may not be minimal enough | 23:02 |
| *** freerunner has quit IRC | 23:02 | |
| greghaynes | clarkb: I tried, didnt work | 23:02 |
| clarkb | maybe we can just get smoser to add sftp to cirros | 23:03 |
| *** tmcpeak has joined #openstack-infra | 23:03 | |
| clarkb | dropbear does support it, likely a different flag needed at build time | 23:03 |
| greghaynes | do you actually need to do something useful after you ssh in? | 23:03 |
| *** freerunner has joined #openstack-infra | 23:04 | |
| *** freerunner has quit IRC | 23:04 | |
| clarkb | greghaynes: create a user, edit its authorized_keys file | 23:04 |
| greghaynes | hrm | 23:04 |
| greghaynes | thats a bit much for my idea of dropbear + dracut | 23:04 |
| *** derekh has joined #openstack-infra | 23:04 | |
| mtreinish | clarkb: according to the googles you need to build sftp-server from openssh to use sftp with dropbear | 23:05 |
| clarkb | ah ok | 23:05 |
| jesusaurus | clarkb: how did you get those test runtimes in your comment on 76057? | 23:06 |
| *** emagana has quit IRC | 23:06 | |
| clarkb | jesusaurus: I increased the test timeout in tox.ini | 23:06 |
| jesusaurus | no, i mean, i dont really understand how testr works, how did you get the actual output? | 23:07 |
| clarkb | jesusaurus: just run tox -epy27 thats part of the output | 23:07 |
| jesusaurus | oh, i guess i've been glancing over that | 23:07 |
| *** markvoelker has quit IRC | 23:08 | |
| fungi | zaro: i've finally found a moment to correct the replication key for review-dev in hiera | 23:09 |
| fungi | all done now | 23:09 |
| *** mestery has joined #openstack-infra | 23:09 | |
| clarkb | jesusaurus: you can also do `testr slowest` to get those numbers from the last testr run | 23:09 |
| clarkb | ok my node off of newly built image just went ready so the nodepool use shade clients change lgtm | 23:11 |
| clarkb | now to go review the latest patchset | 23:11 |
| jesusaurus | clarkb: ah, thanks | 23:11 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for osc stable/kilo novaclient caps bug 1454397 https://review.openstack.org/182434 | 23:12 |
| openstack | bug 1454397 in python-openstackclient "python-novaclient 2.25.0 release breaks stable/kilo because of uncapped osc on stable/kilo" [Undecided,In progress] https://launchpad.net/bugs/1454397 | 23:12 |
| *** smccully has quit IRC | 23:14 | |
| jesusaurus | clarkb: it might also be worth noting that i tried testr with --analyze-isolation and everything passed on its own, but it might just be that we're so close to the timeout and we loose enough with context switching to put us over | 23:14 |
| *** chlong has joined #openstack-infra | 23:15 | |
| *** carl_baldwin has quit IRC | 23:16 | |
| *** esker has joined #openstack-infra | 23:16 | |
| *** sabeen2 has quit IRC | 23:16 | |
| *** annegentle has quit IRC | 23:17 | |
| jogo | fungi: http://logs.openstack.org/08/152108/4/check/gate-hacking-integration-nova/176583f/console.html#_2015-05-12_23_15_31_371 I have a hunch I should just drop --depth=1 | 23:17 |
| openstackgerrit | Maru Newby proposed openstack-infra/project-config: Clean up run-tox.sh to make it easier to reuse https://review.openstack.org/161414 | 23:17 |
| *** craigbr has quit IRC | 23:20 | |
| *** signed8b_ is now known as signed8bit_ZZZzz | 23:21 | |
| *** esker has quit IRC | 23:22 | |
| clarkb | mordred: greghaynes +2 | 23:22 |
| greghaynes | clarkb: w00t | 23:23 |
| *** aduarte has quit IRC | 23:23 | |
| *** smccully has joined #openstack-infra | 23:24 | |
| openstackgerrit | Joe Gordon proposed openstack-dev/hacking: Don't shallow clone from file:// in integration test https://review.openstack.org/182494 | 23:24 |
| clarkb | greghaynes: so on hpcloud booting ubuntu trusty, how long until you get login prompt on console log? | 23:24 |
| clarkb | greghaynes: and any idea if vivid + systemd is any quicker? | 23:25 |
| *** yamamoto has quit IRC | 23:25 | |
| * clarkb tests vivid on rax with nested qemu | 23:26 | |
| greghaynes | haha | 23:26 |
| greghaynes | clarkb: I killed it | 23:26 |
| *** dimtruck is now known as zz_dimtruck | 23:26 | |
| greghaynes | didnt check | 23:26 |
| * greghaynes can rerun | 23:26 | |
| *** yamamoto has joined #openstack-infra | 23:27 | |
| clarkb | because if that works we can fedora/centos/vivid/anythingelse | 23:27 |
| openstackgerrit | Joe Gordon proposed openstack-dev/hacking: Drop py33 as a target https://review.openstack.org/182496 | 23:29 |
| openstackgerrit | Joe Gordon proposed openstack-dev/hacking: whitelist bash in integration test https://review.openstack.org/182497 | 23:29 |
| *** wenlock has quit IRC | 23:30 | |
| clarkb | it would be nice if ubutnu cloud images didn't try to put colors into the console log | 23:31 |
| clarkb | also grub boot timeouts should go away | 23:31 |
| *** yamamoto has quit IRC | 23:31 | |
| morganfainberg | ugh. the new gerrit UI doesn't have a hot-key to show all files | 23:31 |
| morganfainberg | only next/previous file | 23:32 |
| anteaya | morganfainberg: old gerrit ut | 23:32 |
| anteaya | ui | 23:32 |
| *** weshay has joined #openstack-infra | 23:32 | |
| anteaya | we are back on 2.8 | 23:32 |
| anteaya | the version we had for a year | 23:32 |
| clarkb | morganfainberg: 'f' | 23:32 |
| morganfainberg | anteaya: huh. | 23:32 |
| morganfainberg | clarkb: not in the "new" view | 23:32 |
| morganfainberg | clarkb: doesn't exist. | 23:32 |
| anteaya | ah sorry, the new view, my mistake | 23:32 |
| clarkb | oh I don't use the new view | 23:32 |
| clarkb | its bad so I don't use it | 23:33 |
| morganfainberg | clarkb: i got flipped to the "new" view when the 2.10 upgrade happened | 23:33 |
| *** bswartz has joined #openstack-infra | 23:33 | |
| morganfainberg | there was a nice added feature: "conflicts with" | 23:33 |
| morganfainberg | but with 2.8 no reason to use the new ui | 23:33 |
| anteaya | clarkb: thanks I never knew about f | 23:33 |
| clarkb | greghaynes: so vivid took about 220 seconds and trusty was closer to 400 | 23:33 |
| morganfainberg | anteaya: yeah 'f' is awesome | 23:34 |
| greghaynes | clarkb: oh! nice | 23:34 |
| greghaynes | clarkb: but... can it ssh | 23:34 |
| openstackgerrit | Merged openstack-dev/hacking: tests: allow to run the same test for different code https://review.openstack.org/152108 | 23:34 |
| clarkb | greghaynes: the initial connection is slow | 23:35 |
| clarkb | greghaynes: but was on trusty too | 23:35 |
| clarkb | likely that overhead is constant, only boottime is going to change with vivid | 23:35 |
| openstackgerrit | John Griffith proposed openstack-infra/project-config: Add config for Cinder LVM Thin configuration https://review.openstack.org/182499 | 23:35 |
| greghaynes | clarkb: mine was so slow that ssh was timing out | 23:36 |
| clarkb | jgriffith: with 182499 why not just update the default in devstack? | 23:36 |
| greghaynes | but if it works then sounds good | 23:36 |
| clarkb | jgriffith: devstack is branched so you don't have to worr about kilo and juno and icehouse | 23:36 |
| clarkb | jgriffith: and that change to devstack will be self testing | 23:37 |
| jgriffith | clarkb: I was going to do that actually, but there seems to be "concern" among some folks that thin isn't ready | 23:37 |
| clarkb | :( | 23:38 |
| jgriffith | clarkb: not following the statement WRT devstack and branching, I realize it's not branched, did I miss something? | 23:38 |
| clarkb | jgriffith: no, just pointing out that liberty vs kilo is solved for you in devstack | 23:38 |
| jgriffith | clarkb: Oh... "THAT" yeah.. my comment there | 23:38 |
| clarkb | ya | 23:38 |
| *** bhuvan has joined #openstack-infra | 23:39 | |
| jgriffith | clarkb: is it a PITA to run non-gating for a week then flip the default? | 23:39 |
| *** yamamoto has joined #openstack-infra | 23:39 | |
| jgriffith | clarkb: I do know some would like both paths tested... I'm not one of those people :) | 23:40 |
| clarkb | jgriffith: well we would delete the job then make it default in devstack to do that | 23:41 |
| clarkb | jgriffith: so basically we are just making throwaway work | 23:41 |
| clarkb | jgriffith: its not a huge pita but its also not going to stick around verylong | 23:41 |
| *** ivar-lazzaro has quit IRC | 23:42 | |
| jgriffith | clarkb: I guess I could just run it in my own CI setup | 23:42 |
| *** ivar-lazzaro has joined #openstack-infra | 23:42 | |
| clarkb | jgriffith: left some comments on the change for getting it working as proposed | 23:43 |
| jgriffith | clarkb: the point being it may not make it to being default... and even if it does there are still two predominent configs | 23:43 |
| clarkb | jgriffith: if ou push a patch to devstack that enables it by default openstack ci will test it for you | 23:43 |
| jgriffith | clarkb: https://review.openstack.org/#/c/182498/ | 23:43 |
| jgriffith | clarkb: I realize what you're saying | 23:44 |
| jgriffith | clarkb: what I'm saying is that ultimately I wanted to test both configs | 23:44 |
| jgriffith | clarkb: existing users "switching to thin" isn't going to happen | 23:44 |
| jgriffith | clarkb: but it would be nice to finally get new deployments using thin | 23:44 |
| clarkb | jgriffith: if you want to test both then we should decide which one is the default, change devstack to that by default, then have a second job that only runs against cinder to test the non default config | 23:44 |
| fungi | jogo: yes, just ditch the shallow clone there, for the same reasons that you can't do it from gerrit (the git version on trusty wants write access to the originating repo, this is sort of the same with local clone). also there is no reason this should ever run against nova changed, yes? (want to make sure we avoid a repeat of the neutron-lbaas integration test situation) | 23:45 |
| jgriffith | clarkb: so for now that's what I had intended this projects patch to be for (only cinder) | 23:45 |
| *** hichihara has joined #openstack-infra | 23:45 | |
| clarkb | jgriffith: ya I think the thing that is throwing me off is that you are adding the new default as the special thing rather than just making it default | 23:45 |
| jgriffith | clarkb: so thin would only run against cinder jobs. Maybe the statement about makign it default should just be removed as it seems to muddy the waters | 23:46 |
| clarkb | jgriffith: possibly | 23:46 |
| *** banix has joined #openstack-infra | 23:46 | |
| jgriffith | clarkb: oop.. plugin-repo line... thnks | 23:47 |
| jogo | fungi: https://review.openstack.org/182494 | 23:47 |
| clarkb | jgriffith: also check the cover comment, couple other things there | 23:47 |
| jgriffith | clarkb: will do | 23:47 |
| jogo | fungi: no reason to run this against pending nova changes correct | 23:48 |
| *** armax has joined #openstack-infra | 23:48 | |
| jogo | fungi: left the --shallow when pulling across the network since that can take some time on everything but the fastest network | 23:48 |
| jgriffith | clarkb: perfect, thanks for the tips on the remaining steps | 23:48 |
| clarkb | jgriffith: no problem, happy to help if I wasn't verbose enough in the comment too | 23:51 |
| jgriffith | clarkb: I think I got it... like adding the job here: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/projects.yaml#L2034 | 23:55 |
| clarkb | jgriffith: yup | 23:56 |
| *** yamamoto has quit IRC | 23:56 | |
| clarkb | jogo: where do we set REPO_ROOT outisde of https://review.openstack.org/#/c/182494/1/integration-test/test.sh ? | 23:57 |
| clarkb | jogo: eg is it ever going to match that first condition? | 23:57 |
| *** mahito has joined #openstack-infra | 23:57 | |
| jogo | clarkb: project-config/jenkins/jobs/hacking.yaml | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
