19:02:44 #startmeeting infra 19:02:45 Meeting started Tue May 28 19:02:44 2013 UTC. The chair is jeblair. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:02:46 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:02:48 The meeting name has been set to 'infra' 19:03:06 #topic agenda 19:03:17 #link https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting 19:03:27 #topic testr in project progress 19:03:39 clarkb: how's it goin? 19:04:17 not much new to report since the last meeting 19:04:39 poo 19:04:44 I think most of the remaining projects are targetting H-2 for testr'ing and H-1 things are taking a front seat right now 19:04:58 I migrated a few more folkses I thought 19:05:09 or at least submitted patches 19:05:31 I think patches are floating around but no merges yet 19:05:37 you and jgriffith started on cinder 19:06:00 i believe we finished cinder, no? also, I have testtools up and initial testr up for swift 19:06:15 (thank you long boring plane flight) 19:06:53 if you did I missed it. This is a good thing :) 19:07:33 #action clarkb get up to speed on recent testr'ing 19:09:04 #topic What can infra do to enable remote participation at the next Design Summit? 19:09:20 reed: what's going on in this area? 19:09:56 jeblair, the discussion on the mailing list didn't really give me the feeling that this is something uber-important 19:11:32 reed: are you leaning toward getting some phone lines, or doing ad-hoc irc/etherpad chats? 19:11:43 I have little interest in setting up something that is voice only, usable basically only for the summits and only for a handful of people 19:12:15 jeblair, not sure about phone lines, but if that's all we need I can ask to add them to the budget 19:14:58 reed: okay 19:15:03 #topic Mailing lists 19:15:08 reed: this one is yours too 19:15:16 yeah ... 19:15:37 rockstar has been working with Launchpad adminstrators to get a pickle from the General list 19:15:55 he got one but it's 'funny looking' so he needs another one, hopefully it will be delivered today 19:16:22 occasionally I love the sentences that we speak around here 19:16:33 once we get the pickle we will be able to test that indeed it contains the full list of subscribers and configurations from LP into lists.openstack.org 19:16:39 it's hard to pickle correctly, and possibly dangerous if you do it wrong 19:17:19 once that is tested, we will have to decide how to migrate from one list to another 19:17:32 and it won't be as simple as switching a DNS 19:17:50 I think we need a clear cutoff date 19:17:54 since we don't control launchpad, i think we'll have to just set a flag day and make the switch 19:17:58 and communication 19:18:03 yep 19:18:14 and the day of the switch we'll need to get the pickle again 19:18:28 so we'll have to coordinate with LP admins 19:18:51 we're keeping notes on https://etherpad.openstack.org/mailmain-migration-notes 19:19:17 #link https://etherpad.openstack.org/mailmain-migration-notes 19:19:45 #info we'll need to set a date for a cutoff and do the switch between Launchpad and lists.o.o 19:20:10 reed: so we'll wait on rockstar to propose a date based on when he's happy with the pickle 19:20:18 ? 19:20:21 yes 19:20:47 #info we'll wait on rockstar to propose a date based on when he's done testing the migration between LP and lists.o.o 19:20:54 okay, that all sounds good 19:22:11 what should we do to the LP list on cutover? 19:22:28 the archives will be on lists.o.o at that point? can we just kill it on LP? 19:22:49 yeah, just wondering what "kill" means in this context... 19:23:36 we could subscribe the new list to the old one right before the cutover to get anything in flight, and then make membership to the old list require approval for a bit 19:23:37 we should move the archives too 19:24:13 fungi, why? 19:24:14 reed: is rockstar working on that? 19:24:32 " 19:24:35 "Deactivating this list will stop it from accepting posted messages. You can reactivate the list later, without having to get administrator approval." 19:24:42 so that's an option in launchpad ^ 19:24:48 jeblair, I'll make sure he does 19:24:52 reed: only if we care about messages which are pending delivery to the old list at the time of cutover, i guess 19:24:55 i think we could do that at cutover time 19:25:55 fungi, I get it now, thanks 19:26:03 reed: if messages in people's outbound queues at that moment (keeping in mind that temporary connectivity issues can make that days) are an acceptable loss, then no need 19:26:53 fungi: i think we'd have to create a launchpad user in order to do that 19:27:00 * fungi nods 19:27:09 (whose email addr is the new list addr) 19:27:25 it's nontrivial, i agree. the ossg list did that for a while 19:27:38 i don't know if it's worth the effort, just putting it out there 19:27:38 i lean toward "acceptable loss"... 19:27:50 me too, acceptable loss 19:27:52 especially since everyone will need to know about the new list and cutover time anyway 19:28:01 acceptable 19:28:05 yeah, not worth worrying over in that case 19:28:15 ++ 19:28:54 reed: anything else on this topic? 19:28:59 nope 19:29:15 cool, thanks (and i'm looking forward to the move!) 19:29:21 me too :) 19:29:24 #topic OpenStack Operations Guide - tooling, licensing, publishing discussion 19:29:42 a few of us chatted with annegentle and folks from oreilly about their atlas system 19:30:28 it sounds like it will probably be used mostly for annegentle, the book authors, and oreilly to collaborate for a limited period during which oreilly will be working on editing the book for publication 19:30:48 and that we're not looking at making it part of a long-term workflow for docs/books at this point 19:31:01 Sounds right jeblair -- if we reach a contract agreement. 19:31:15 Limited use, limited time periods. 19:31:44 so that seems like a great position to be in for an experiment 19:31:51 yay for labs 19:32:04 ++ 19:32:29 also, as a point of interest, their web editor is built around http://ace.ajax.org/ 19:32:54 #link http://ace.ajax.org/ 19:33:07 #topic TripleO Testing (TOCI) 19:33:17 so, dprince wrote this to do tripleo testing: https://github.com/toci-dev/toci 19:33:30 which is essentially tripleo's incubator notes.md put into scripts and tweaked a bit to run on its own for testing purposes (cool!) 19:33:51 he currently has it running on a physical box, I've done testing on my own physical boxes last week 19:33:55 Not just dprince though. Derek and Lucus chipped in quite a bit too! 19:34:00 hooray :) 19:34:08 so on our side, dprince put together this to get the syntax right on actually adding TOCI to devstack-gate https://review.openstack.org/#/c/29978/ 19:34:11 yay for collaboration! 19:34:20 however, this ends up being testing "openstack on openstack" on hpcloud, an openstack VM, so I guess that's like... quintuple o 19:34:35 I tested this today, hpcloud VM doesn't have the kvm module, so we can't boot the bootstrap node as-is :( 19:34:47 I even tried anyway to see if it would do something clever, alas, no: http://paste.openstack.org/show/37829/ 19:34:50 fwiw that change is ready for merging imo. Just need a good time to batch up the zuul layout changes 19:35:09 wow. no kvm module makes me a sad panda 19:35:14 mordred: me too 19:35:19 well... if it is only going to fail ATM there is no sense sending it in yet. 19:35:24 Would it run on RAX? 19:35:24 dprince: right 19:35:41 I don't know if they're any different in this regrad 19:35:43 regard 19:35:45 I mean... we *can* add the KVM module there right? 19:35:54 it's not loadable by default 19:36:05 but I just discovered this about 30 minutes ago, so more investigation is required 19:36:24 * jeblair considers kexecing into a kvm-enabled kernel 19:36:44 jeblair is more leet than morganfainberg 19:36:44 would that help? KVM requires cpu things that aren't availabe to VMs iirc 19:36:46 gah 19:36:50 jeblair is more leet than mordred 19:36:53 jeblair: that's not going to work 19:36:54 Right. But on next gen RAX I don't think they prevent you from using your own kernel, etc. 19:37:01 Just throwing it out there if it helps. 19:37:05 sdague: i'm sure; but it's an amusing thought. ;) 19:37:08 kvm needs access to the processor hardware extensions :) 19:37:12 sdague: yeah 19:37:21 this problem exists with RAX and HPCloud 19:37:22 I mean eventually we want bare metal anyway... so got plans for that? 19:37:31 which if hp cloud isn't running with nested in the host, then you're done 19:37:36 dprince: well we wanted to do both, starting with virtual 19:37:47 do we need kvm? I thought the tripleo stuff would also do qemu? 19:37:47 (we're working out details about who would give us a rack that we can do scary things to) 19:37:51 but do you actually need kvm 19:37:54 I got a rack offer 19:38:00 mordred: that may be possible, if slow 19:38:06 mordred: do you have a people offer? 19:38:11 but we're also working out details of people to deal with it 19:38:13 :) 19:38:26 pleia2: it would be worth trying it without real kvm to figure out how slow it really is 19:38:36 sdague: yeah, maybe that's my next step 19:38:53 so I'll give it a spin with qemu and keep everyone updated, other suggestions will be welcome :) 19:38:54 and not assume it's unworkable until proven so, because my experience is sometimes this stuff surprises you 19:38:57 since we've already lost access to one set of hardware, i'm going to be really cautious about having any openstack gating tests depend on that. 19:39:09 Okay. There are a couple ways to go here from my prospective. It would be useful to have this as a feedback loop for upstream tripleO stuff (soonish) 19:39:32 So if we think cloud isn't going to work out, with qemu or fancy kvm module loading, or whatever... 19:39:42 Then I could 3rd party it up on bare metal. 19:39:52 I mean... I *know* how to do that. :) 19:40:04 i agree with sdague we ought to try qemu first 19:40:17 so there was that session we had on baremetal where we talked about what requirements we'd have for baremetal, was that ever written down somewhere? (aside from the etherpad) 19:40:48 like, isolated on network, provide on-site hands, don't ever trust these machines again 19:41:29 i don't recall seeing it summarized beyond that etherpad 19:42:50 neither do I 19:43:04 ok, so I'll test w/ qemu and see what we can do on the virtualization side, and also put together some baremetal requirements from that etherpad so we have something to give to people who offer us hardware 19:43:16 ++ 19:43:20 pleia2: sounds good 19:43:42 pleia2: this may be as simple as passing an alternate --engine to configure-bootstrap-vm. 19:44:01 dprince: yeah, going to test locally and compare speeds, then test on hpcloud 19:44:02 if nothing else, seeing how slow it is under nested qemu provides some numbers which allow sizing the bare metal environment 19:44:17 * pleia2 nods 19:44:32 cool, thanks everyone 19:44:57 #topic infra bootcamp 19:45:33 mordred and I have been discussing ways to help new people contribute to openstack-infra 19:46:11 i've started a rework of our documentation to orient it to answer questions like "how do i contribute" and "what are all these tools and what the heck do they do?" 19:46:22 awesome 19:46:58 that's going to be super helpful 19:47:09 i'm extremely excited 19:47:13 I just tried to give an overview of things to a person on friday - it got ... complex ... quickly 19:47:19 mordred has been trying to get companies to dedicate some fte to us... 19:47:30 which has actually gotten really good response 19:47:48 so we figured that if we can round up a few potential new contribs 19:48:01 we should try to get them together and have a sort of bootcamp 19:48:09 ++ 19:48:20 to help deal with the information overload of becoming a new contrib 19:48:52 yes. it will be helpful 19:49:01 are you thinking an inperson hackathon type thing? 19:49:18 or just a this is what we are all doing this week sort of thing? 19:49:27 I think both 19:49:30 I've been involved in several onboarding initiatives over in Ubuntu land, so I'm happy to help as I can 19:49:33 we're looking at getting together in NYC june 27-28 19:49:58 * mordred ninja-voted for nyc because he's tired of travel, and also has a show the 26th 19:50:16 heh 19:50:20 i'm okay with nyc because it's been a while and i'd like to visit. :) 19:50:20 * mordred predicts that we'll eat at meatball shop 19:50:38 mordred: keep me in the loop on that, because I could probably just hop on a train down for a day regardless 19:50:39 i'm okay with nyc because it's not far from me for a change 19:50:48 sdague: awesome. we'd love to have you 19:51:19 mordred: is this something for a couple of infra guys plus new people... or is this a entire infra team meetup? 19:51:20 yes, and pleia2, we'd love help figuring out how to do this. :) 19:52:11 (and I can support online or in person - mordred just let me know) 19:52:30 I've done plenty of both 19:54:17 dprince: that's a good question. one the one hand, it would be great to have everyone. on the other hand, if the ENTIRE infra team is in a meeting for 2 days, god only knows what will happen to the project :) 19:54:50 jeblair: ^^ thoughts? 19:54:59 dprince: i think it's not just for new people -- i mean the focus will be on getting new people up to speed, but i'm anticipating there may be a lot of context and orienting that existing contributors could benefit from 19:55:27 jeblair: ++ especially since as we have grown there has definitely been a lot more partitioning of knowledge 19:55:49 * mordred has yet to grok logstash, for instance 19:55:50 * dprince sounds interesting 19:55:54 so if the topic of 'helping to understand what's going on in openstack-infra and how to contribute' sounds interesting, you should come regardless of how long you've been involved 19:55:55 (which isn't a bad thing, but I think there is a lot that other peopel know and understand that I have very little understanding of) 19:56:22 * olaph can't wait 19:56:23 but i don't think it will be a design-summit/hackathon thing; i think the focus on new contributions is the main thing 19:57:41 we should probably make a signup thing or something 19:57:45 if some of us show up and get busy fixing things, just plug us into extra projectors and let new/potential contributors see what goes into the sausage 19:57:48 sounds really great \o/ 19:57:48 so that I can size location and stuff 19:57:56 mordred: yeah that's probably a good idea. 19:57:57 fungi: ooh. nice 19:58:05 fungi: yeah, afterall, that's a great way to learn 19:58:18 or scare people away, either way... 19:58:27 nah, we do fun things :) 19:58:38 #action mordred make a signup thingy for bootcamp 19:59:02 i think basic topics are necessary as well. if you've never used gerrit & git you'll be behind the 8 ball. 19:59:28 zaro: yeah totally, live Gerrit_Workflow 19:59:36 * fungi agrees... i wonder if upstream university would be a good prereq for people unfamiliar with that stuff? 19:59:57 fungi: oh good idea, maybe we should also suggest some homework before attendance 19:59:59 i know they've done some 100% online sessions 20:00:08 or try what the GNOME womens outreach program does and force people to submit a patch beforehand 20:00:18 though we're certainly not above helping people grok gerrit and git. :) 20:00:32 okay, i think that's time for us... thanks all! 20:00:37 #endmeeting