19:02:44 #startmeeting infra 19:02:45 Meeting started Tue Aug 12 19:02:44 2014 UTC and is due to finish in 60 minutes. The chair is jeblair. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:02:46 o/ 19:02:46 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:02:49 The meeting name has been set to 'infra' 19:02:50 thanks! 19:02:51 welcome grantbow 19:02:55 #link agenda: https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting 19:02:55 o/ 19:02:58 #link previous meeting: http://eavesdrop.openstack.org/meetings/infra/2014/infra.2014-08-05-19.02.html 19:03:14 #topic Puppet module Split out (jesusaurus/nibalizer) 19:03:24 okay so i guess we decided some stuff last time that i forgot 19:03:37 but sounds like step 1 is to split out a guniea pig 19:03:42 so i can do that with storyboard 19:03:54 i probably should use the agree command more 19:04:11 * krotscheck perks up. 19:04:14 jeblair: you had no chance, i had to do wedding stuff(not mine) last week so my brain was fried 19:04:27 krotscheck: so yea your patches that haven't landed, we can pivot those to be applied to the new storyboard module 19:04:29 nibalizer: btw, once the guinea pig is done, you're welcome to add me to people who can help with splitting out modules 19:04:37 nibalizer: Works for me. 19:04:43 the puppet-storyboard projcet has already been created so thats easy 19:04:44 we didn't land the big storyboard refactor? 19:04:49 pleia2: awesome :) 19:04:52 jeblair: We did. 19:04:56 jeblair: We’re now building on that. 19:04:58 i think we should have a little discussion about acls 19:04:59 ah cool 19:05:02 jeblair: ya krotscheck is a machine 19:05:03 hi 19:05:14 i.e. does infra-core retain the +2+a on the modules 19:05:24 please add me to the list of people who can help with the module split as well. 19:05:24 or do we give it krotchchek on the storyboard module? 19:05:24 yes 19:05:45 krotscheck, nibalizer: does the new refactor work? ie, is it stable enough for us to pivot to now? 19:05:50 * SergeyLukjanov2 lurking 19:05:56 jeblair: It’s currently in use. 19:06:05 jeblair: But there’s one thing left that.... 19:06:08 jeblair: that ^^ 19:06:15 once we approve https://review.openstack.org/#/c/99990/ we will make a story for each module 19:06:25 and then all the folks that want to volunteer can assign that story to themselves 19:06:29 jeblair: Well, there’s one remaining patch that I want to land to make it “stable" 19:06:36 jeblair: great 19:06:51 s/that story/those stories/ 19:07:13 nibalizer: can we land that first, then focus on catching the external module up? 19:07:27 nibalizer jeblair: This one - https://review.openstack.org/113616 19:07:28 jeblair: lets land krotchekcs patch for stability 19:07:35 then catch the external module up 19:07:55 #agreed land https://review.openstack.org/113616 to internal storyboard module, then begin moving to external module 19:08:16 woot 19:08:34 nice use of agreed 19:09:17 anything else on this? 19:09:22 o/ 19:09:24 * krtaylor notices "agreed", hm I'll have to use that.. 19:09:34 jeblair: acls? 19:09:52 o/ 19:09:57 heh, i guess we should expand on mordred's answer to that :) 19:10:05 :) 19:10:18 oh did mordred say something? 19:10:20 nibalizer: infra-core will have +2 on all openstack-infra/ projects (including these) 19:10:30 oh he said yes 19:10:45 nibalizer: if you need help with the acl in the patch, let me know, I can help you 19:10:53 anteaya: okay 19:11:06 aight sweet then i think im good on this topic 19:11:08 nibalizer: once they are split out, we can consider giving folks +2 on individual projects 19:11:27 jeblair: cool 19:11:46 but probably not right away, so for sanity sake, i'd just put infra-core in the acl files for now 19:11:46 we generally accomplish that, as needed, by modifying the acl to use a new core reviewer group and have it include the infra-core group 19:11:52 fungi: ++ 19:12:22 #topic Project renames (flaper87,jraim) 19:12:38 I'd like to do this renaming 19:12:51 SergeyLukjanov2: i'd like that too! :) 19:12:53 https://review.openstack.org/#/c/113614/ and https://review.openstack.org/#/c/111244/ are also renames 19:12:53 flaper87 wasn't going to be around for the meeting sounded like, but said something to the effect of "the sooner the better" 19:12:59 Rename Marconi to Zaqar 19:13:06 Ajaeger: yep 19:13:13 and just pushed: Move openstack-security-notes to attic 19:13:33 ack 19:13:40 Ajaeger: cool, can you add that to https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Upcoming_Project_Renames 19:13:41 i tacked on jraim so we could get some clarification on whether the kite rename should also include python-kiteclient 19:13:49 jraim: ping 19:13:51 jeblair: will add directly 19:14:00 fungi: i think i was catching us up to the governance repo 19:14:10 but wasn't able to get jraim's attention earlier, so he's probably also not around at the moment 19:14:21 fungi: and kiteclient isn't in there 19:14:23 jeblair: yep, i figured that out from the wiki history 19:14:36 bug that sort of suggests the question of whether that was an omission from the gov repo 19:14:54 mainly wanted to sync up on whether it should be, and whether we should delay the kite rename until it happens 19:15:15 so that they get renamed at the same time 19:15:54 so anyway, it sounds like maybe just the marconi and security notes renames are ready to go in near term 19:16:02 yup 19:16:14 yeah, let's keep trying to get clarification on that, and defer kite until we do 19:16:34 ++ 19:16:40 who has dates/times they really want to avoid? should we do friday pst afternoon? earlier? 19:16:47 so, here is the guide for renaming - http://ci.openstack.org/gerrit.html?highlight=rename#renaming-a-project am I right? 19:17:05 SergeyLukjanov2: yep. it may or may not be 100% current, but it's close 19:17:15 SergeyLukjanov2: yep; we'll make sure at least one other person is there with you to double check 19:17:18 I'd like to avoid my sunday (your sat/sun night) 19:17:22 we keep finding that little things have changed 19:17:54 I think there were no changes in gerrit since last renaming, so, it should be going good 19:17:55 friday is ard for me but early saturday morning pst should be ok 19:18:02 SergeyLukjanov2: generally we try to put together an etherpad with the timeline and cut+paste of the grittier commands so they can be peer reviewed 19:18:21 also we want to make sure the pending rename commits are fully reviewed and current (not in need of rebases) 19:18:37 SergeyLukjanov2: what's the latest time friday that would be okay for you? 19:18:41 early sat morning is ok for me 19:18:55 let me calc re friday 19:19:14 so, I'm UTC-4 19:19:22 clarkb: 1600 utc saturday okay? 19:19:29 * clarkb does math 19:19:34 that is 9am? that should work 19:19:42 and 3 am is time when I'm falling asleep 19:19:59 fungi: will you be around? 19:20:01 1600 utc saturday is ok for me too 19:20:06 looks like that's 8pm SergeyLukjanov2's time? 19:20:08 * clarkb is wondering how many schedules we are trying to juggle? 19:20:15 clarkb: i'm fine with it 19:20:22 correction: UTC+4 :) 19:20:26 cool why don't we aim for then then? 19:20:48 #agreed project renames 1600 utc aug 16 19:20:51 ok, sounds like we agreed in 1600 utc saturday 19:20:56 :) 19:21:10 did i get that right? :) 19:21:14 I'll prepare etherpad tomorrow and share with you folks 19:21:31 jeblair, I think so ;) 19:21:36 SergeyLukjanov2: i can get you links to earlier ones if you need examples 19:21:44 fungi, it'll be great, thanks! 19:21:58 #topic Publish python-jenkins 0.3.3 (last release without pbr) tarball to pypi. (zaro) 19:22:32 zaro: so 0.3.3 fixes the issues with 0.3.1? 19:22:39 yes, it does. 19:22:53 #action jeblair Publish python-jenkins 0.3.3 (last release without pbr) tarball to pypi 19:22:55 last one without pbr 19:23:11 that's a manual upload too, right? 19:23:16 fungi: yup 19:23:38 previous one had a bug, so sort of defeated the purpose :( 19:24:06 (of having a published version before we went and changed _lots_ of stuff) 19:24:24 cool, end of topic i guess. 19:24:34 #topic Administering third party ci gerrit accounts: setting http passwords (anteaya) 19:24:54 hi 19:25:03 so there was an email thread 19:25:14 #link 19:25:14 http://lists.openstack.org/pipermail/openstack-infra/2014-August/001685.html 19:25:16 #link http://lists.openstack.org/pipermail/openstack-infra/2014-August/001685.html 19:25:19 thanks 19:25:27 nope, you got it right :) 19:25:42 and there is the ability for gerrit admins to set an http password for third party cis 19:25:50 vmware would like a password 19:25:53 why? 19:26:21 from the email: We are looking to use some of the Gerrit REST API operations that require authentication 19:26:28 which ones? 19:26:28 that is all I have for the why 19:26:43 ryan was going to try to be here at the meeting 19:26:50 I have forgotten his irc nick 19:26:54 I don't know which ones 19:26:57 :( 19:27:10 you don't need admin to create http password. 19:27:19 for third party ci? 19:27:26 which don't have gui access? 19:27:27 i believe account holder can do themself 19:27:34 via ssh? 19:27:58 ohh, sorry. didn't realize no UI access. then no they cannot. 19:28:06 i don't think there's a good way for us to set those without actually knowing the passwords. and i don't want us to become the password reset help desk any more than we already are. 19:28:11 okay yes, so they can't by themselves 19:28:18 kk 19:28:22 so i think there's a pretty high bar for deciding this is a good idea 19:28:31 * anteaya nods 19:28:42 especially when they can just use a normal account to make rest api queries 19:28:50 ++ 19:28:54 also, third-party ci only needs to do two things authenticated, and ssh access permits both 19:28:56 and the ssh api exposes all the things too 19:29:01 jeblair: ++ 19:29:35 okay so I will share the meeting logs in response to the email and if he needs more air time hopefully he will attend in future 19:29:37 thanks 19:29:39 I'm done 19:29:47 anteaya: thank you! 19:29:54 #topic Bug Day proposal: August 19th, alt suggestions: August 26th, September 9th (pleia2) 19:30:10 so I noticed our bug numbers creeping up and tossed some dates out 19:30:22 FYI, September 9th the Documentation team will do a Bug day - but that's not a conflict IMHO 19:30:25 (skipped labor day week) 19:30:25 next week is hard for me particularly with the trusty and tox switches coming up 19:30:49 I would +1 the 26th 19:30:59 26th sounds great to me 19:31:03 I'm better on the 26th 19:31:05 ++26 19:31:07 great 19:31:09 ++26 19:31:16 should we incorporate storyboard projects into this bug day? 19:31:27 mordred: do you think we'll be able to import by then? 19:31:32 I don't actually know anything about storyboard's API for writing a script (it has one, right? :)) 19:31:32 yes 19:31:39 jeblair: I'm finishing off teh import script right now 19:31:45 do we normally incorporate anything besides the openstack-ci project bugs? 19:31:47 https://review.openstack.org/113624 fwiw 19:32:03 fungi: well, storyboard has some of our projects that were origially openstack-ci on launchpad 19:32:22 pleia2: oh, yeah makes sense then 19:32:24 it might be a good way to kick the tires on storyboard too 19:32:34 clarkb: that's what I'm thinking 19:32:45 I haven't actually done anything with storyboard aside from reading it :) 19:32:49 storyboard tire-kicking day ;) 19:33:07 pleia2: i created a test story 19:33:10 well, krotscheck and I will both be not here 19:33:12 then found out you cant delete them :/ 19:33:20 nibalizer: haha, oops 19:33:30 nice 19:33:31 oh yeah krotscheck will be honeymooning 19:33:34 thats good to know 19:33:48 ok, so maybe bump storyboard tire kicking until next time 19:33:50 which is slightly different from mooninhoneys 19:33:57 :D 19:33:59 lulz 19:34:06 krotscheck: also, congrats :) 19:34:09 i think we can kick storyboard's tires 19:34:11 yeah, congrats 19:34:22 Well, y’all can hit as many bugs as you want to, and file them as stories, just don’t expect much movement on them. 19:34:25 i'm fine with it, as long as the tires don't kick back 19:34:29 alright 19:34:35 i'm not interested in using launchpad anymore for infra bugs; it's time to start really using storyboard 19:34:38 all the bugs for when krotscheck returns \o/ 19:34:44 jeblair: script almost done! 19:34:52 krotscheck: btw - I found a bug in StoryTags 19:35:15 so is there a way to query storyboard from a script? 19:35:24 jeblair: ++ we haev to switch at some point and every time I have used storyboard I have been happy 19:35:31 curl? 19:35:59 #agreed bug day august 26 (and bugs will be imported into storyboard by then) 19:36:03 pleia2: curl would work, although i would love if someone would work on python bindings 19:36:14 pleia2: python-storyboardclient is a TDL item 19:36:19 ok 19:36:26 well, I'll come up with something 19:36:41 can't be worse than launchpadlib anyway :) 19:36:45 haha 19:36:47 indeed 19:37:01 pleia2: thanks! 19:37:04 Oh man, I’m glad I created a sorting patch :) 19:37:07 ZOMG launchpadlib 19:37:07 #topic Refactor of artifact upload scripts (rcarrillocruz) 19:37:11 that's me 19:37:16 well, to be fair launchpadlib is decent. It's launchpad APi that is horrible :P 19:37:16 #link https://review.openstack.org/#/c/109816/ 19:37:29 i've been chatting with fungi and clarkb about this 19:37:41 and also chatted with jeblair at mid-cycle sprint 19:37:55 i'd like to get consensus on the change 19:38:02 should it be a refactor for just java things? 19:38:22 or are we open to have a generic framework/script to uplod all sorts of artifacts? 19:38:25 pypi 19:38:26 wheels 19:38:29 jenkins 19:38:31 maven 19:38:32 ... 19:38:43 rcarrill`: as i recall, it was mostly the java stuff that you needed to consolidate for your use-case, right? so either will work for you? 19:39:04 consolidate and genericize 19:39:12 yes, but clarkb says he's not very keen on having a monolithic script for all kind of artifacts 19:39:22 i.e. not -t ARTIFACT_TYPE 19:39:22 my opposition to doing everything in a giant complicated script is that we have been very unixy so far and makinga one size fits all script is not how anything else works there 19:39:41 rcarrill`: i think most of those other artifact types use widely varying tools/commands to prepare and upload, and the script wouldn't really do much in common ultimately 19:39:53 i agree with clarkb that we should have the One Way to do maven, then a different One Way to do pypi 19:40:03 clarkb: but there's some commonality in the download part yeah? 19:40:15 for most of them, the bulk of the duplication we'd avoid would be in the license boilerplate comment block 19:40:18 clarkb: i'm guessing you'll say "have a download script, and N upload scripts" :) 19:40:19 yes 19:40:20 jeblair: yes, the download part has some commonality. the difference are in the upload to $repo 19:40:26 jeblair: that would be fine 19:40:34 to be fair, i think we could have a common artifact upload thingy , but bash is not the way to go 19:40:41 Is there a way to create a library that can be included for some common functions? 19:40:45 so, imho we could just have my change for java stuff 19:40:56 and then we can work on something on later that is more flexible than bash 19:41:15 yeah, it seems like maven nexus and jenkins-ci are probably similar enough that they benefit 19:41:44 fungi: they are the same thing. the only differences are nexus location and filenames 19:41:45 agreed on having the change on just nexus, and upload either java package OR jenkins package? that way no point of differentiating if the stuff is jenkins of java 19:42:00 clarkb: ^, yup, i agree with you 19:42:20 Ajaeger: yes, that would be possible 19:42:24 after our chat last week, i give you that 19:42:29 (so the download part could be 'sourced') 19:42:37 rcarrill`: yeah, could probably branch on the filename extension and not need explicit artifact type parameters even 19:42:42 not exactly the same, jenkins-ci is artifactory while maven central is nexus. 19:43:01 but curl one-liner would work the same, not zaro? 19:43:13 for download only i think 19:43:46 for maven central (nexus) they employ a two step upload process. 19:43:50 zaro: could I work with you on that? 19:43:55 yes. 19:43:57 it's a PITA to open an account myself for testing 19:44:05 we could just use openstack-infra account for that 19:44:21 you'll need the password. 19:44:36 iirc, oss sonatype site requires an approval process to get an account 19:44:42 it's not something immediate to get 19:44:48 rcarrill`: right, you have to open a jira ticket 19:44:59 just do it! 19:45:04 and then wait for them to process it 19:45:09 anyways, overall the change looks good? 19:45:21 pulling the -t switches 19:45:29 (not entirely unlike our third-party ci service account request workflow) 19:45:33 and just copying .hpi / .jar artifacts 19:45:42 ? 19:45:46 now that i think about it. i don't think it will work for maven central. 19:46:25 you need to upload then do another request to "approve" to actually publish it. 19:46:43 anyways. i'll review again. 19:46:49 rcarrill`: ya overall it is fine. I think we should just rip out the branch logic 19:46:52 cool 19:47:03 #topic Process to deprecate drivers/plugins and their corresponding CIs (emagana) 19:47:07 ohh, i agree with clark there as well. 19:47:15 hi 19:47:24 we have a deal then 19:47:49 Just wondering if there is a process in place to deal with the plugins/drivers that are being deprecated 19:48:00 zaro: will make another patchset and ping you then 19:48:08 as well as the process to deprecate them if they do not have a CI 19:48:09 emagana: what plugins/drivers are being deprecated? 19:48:23 in Neutron: OVS, LB, NXOS 19:48:27 emagana: that's more a decision on the part of the projects in which those drivers exist or integrate 19:48:29 jeblair: maybe more 19:48:31 so in that case, neutron 19:48:57 fungi: understood! 19:49:15 if there are third-party ci systems which correspond with them and are being taken offline as well, we appreciate a heads up so we can disable their service accounts 19:49:28 yep, and the wiki pages for them should be updated too 19:49:36 fungi: That is the kind of guidence I am looking for 19:49:51 I am planning to run an audit for the plugins/drivers CI 19:50:02 I know, I will not have many friends 19:50:31 The idea is to be sure that all drivers/plugins are being tested properly 19:50:36 emagana: ++ 19:51:23 emagana: that would be awesome; thanks 19:51:25 emagana, what is the output of the audit? 19:51:35 I will update NeutronPolicies for Plugin and Drivers with the results: https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers 19:51:53 emagana, understood 19:52:01 krtaylor: ensure quality! and deprecated anything that is not being properly tested 19:52:12 emagana, feel free to bring this up in the next third-party meeting as well, we can spread the news 19:52:19 Thanks! 19:52:29 ok, will provide results next week 19:52:39 #topic Open discussion 19:52:39 that is all I wanted to share about this 19:52:57 o/ 19:53:00 anteaya: what's the status of account renames? 19:53:01 jeblair: I have a question about Multi-node testing 19:53:24 right now I am hoping either you or someone else can help me append CI to the end of them all 19:53:30 I have a small nodepool feature I'd like to run by you all 19:53:30 then we can merge teh js patch 19:53:43 anteaya: can you prepare an etherpad with a list of the names? 19:53:45 after I get gerrit permissions then go through them and chance the name 19:53:56 jeblair: oh goodness I have been working on it for 3 months 19:54:13 anteaya: yeah, that's too long. i'm ready to just name them myself. 19:54:16 anteaya: I think we need a clear old name to new name map 19:54:22 #link https://etherpad.openstack.org/p/automated-gerrit-account-naming-format 19:54:29 jeblair: go for it 19:54:30 then whoever can do it. 19:54:45 if you want it I will support you and clean up afterward 19:55:36 We've been having some problems with nodepool and the ready-script recently. Our ready-scripts do some nice things like update git repos, set hostnames, but because we generate our images every night, they're not absolutely essential. 19:55:36 i will tend to give them _very bad names_ 19:55:42 So we'd like to have an option that lets us tolerate errors in the ready-script, so if it fails to run, the node is still used. 19:55:52 like "Unknown Function Third Party CI" 19:55:53 jeblair: there are no names they will like 19:56:13 jedimike: the whole point of a ready script imo is to make sure the node is ready. if it fails the node is not ready. 19:56:20 jedimike: tolerating failures should happen in the script 19:56:34 jeblair: During the Neutron mid-cycle sprint in Minnesota, we discussed the need for having a Multi-node testing environment, specially for DVR (Distributed Virtual Router). So, I created a spec in Infra: https://review.openstack.org/#/c/106495/ 19:56:43 jeblair: do it, they will yell at me and I will take it 19:56:59 and I don't have to keep thinking about how to make everyone happy, because I can't 19:56:59 jedimike: yeah, i think updating git repos is probably not something that should be in a ready script 19:57:04 clarkb, yes, but sometimes nodepool just isn't running the script, and because it doesn't log anything useful when it fails to run the ready script, we've had days where no jobs are running and we don't know why, and our scripts run fine as far as we can see 19:57:07 jeblair: I just wanted to know if you have this cover by a different spec.. anteaya recommended to bring this question here 19:57:15 I have a patch currently in review to fix that though, https://review.openstack.org/#/c/113287/ 19:57:24 jedimike: well, "just not running the script" is a pretty clear bug :) 19:57:27 jedimike: we do it in our prep scripts instead, and then pull whatever has changed that day at the start of jobs 19:57:47 rather than when the node goes ready 19:57:49 jedimike: part of the reasoning for that is that nodes can sit in ready state for a long time 19:57:57 i see 19:57:58 emagana: there isn't another spec. afazekas is just doing the work 19:58:05 so if you want something to be current, doing it at the start of the job is the best place 19:58:21 emagana: it is something that was made possible months ago by a new nodepool featuer and no one did anything with it until afazekas stepped up 19:58:35 jedimike: ++ on better logging 19:58:56 clarkb: should we abandon the spec that I created and just let the work continue? 19:59:13 jeblair, yeah, it's caused headaches that the scripts look fine, run fine when we run them, but nodepool just threw away stdout and stderr so we can't see what happened 19:59:36 clarkb: I think afazekas could point his work to this spec 19:59:41 emagana: I don't necessarily thing it needs abandoning as it does spell out the neutron use case. But I don't think the spec was required to start the work 19:59:45 emagana: yup 19:59:49 so, I'll make a note to look at our ready scripts and see if they can be more minimal too, thanks 20:00:09 time 20:00:30 thanks all! 20:00:32 #endmeeting