20:01:28 #startmeeting Glance 20:01:28 Meeting started Thu Apr 25 20:01:28 2013 UTC. The chair is markwash. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:01:29 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:01:32 The meeting name has been set to 'glance' 20:01:49 We've got a lot of blueprints to talk about today! 20:02:14 I posted a new standing agenda here https://wiki.openstack.org/wiki/Meetings/Glance 20:02:27 but we probably will only get into the new blueprints part 20:02:46 I want to leave time for open discussion at the end though, so please yell at me if we go over 50 minutes 20:02:48 sigh 20:03:00 #topic new blueprints 20:03:32 The summit was great, and we had a lot of fruitful discussions 20:03:42 thanks, session leaders! 20:04:16 I've co-opted the the glance meeting agenda etherpad to try to streamline things today 20:04:29 #link https://etherpad.openstack.org/glance-meeting-agenda 20:05:14 just curious, do we have ameade nikhil jbresnah present? 20:05:20 nod 20:05:21 wave 20:05:38 here 20:05:43 phew 20:05:51 iccha is traveling, correct? 20:05:55 ack 20:06:00 yeah, i believe she is 20:06:00 yes markwash 20:06:08 actually on vacay 20:06:14 :-) 20:06:34 no one is on vacation from openstack! 20:06:45 it's like 2 am where she is though 20:06:50 :-) 20:06:59 openstack will find you wherever you are 20:07:15 So looking at the etherpad, anyone have thoughts on moving forward on quotas? 20:07:15 true 20:07:39 (not me) 20:07:52 ummm 20:08:02 is the idea to manage it via keystone? 20:08:12 not as far as I am concerned 20:08:26 markwash: is that your top priority? 20:08:54 is that effort mostly about enforcement then? 20:08:56 not really, the list is ordered more by summit session than anything else 20:08:56 i'm a little curious about folks assigned to the existing blueprints as such 20:09:04 what was happening with the whole centralized quota thing? how would that even work? 20:09:17 did anybody other than me go to the centralize quota session? 20:09:23 i was there 20:09:24 i was there 20:09:45 having keystone manage the information seemed broken to me 20:09:55 I'm not sure I got a clear impression of it, but it sounds like 1) there are issues with the proposed implementation 20:09:56 like... working around the procedure to create a new service 20:09:56 and then we sync-ed up later in the common lobby/hall 20:10:08 and 2) even if there weren't, it would take a while to get it rolling 20:10:11 I don't think that's a good idea, FWIW 20:10:13 not everyone uses keystone 20:10:17 lets just do it in glance? 20:10:29 ameade: +! 20:10:31 ameade: +1 20:10:39 I'm at +1 for that 20:10:51 how badly do we need quotas? 20:10:51 markwash: ameade do we need to create an action item to sync with folks who are working on it in keystone? 20:11:07 i dunno, i like the idea of having it shared across all projects 20:11:11 i really think a multi-component solution would be better 20:11:14 (agree to non-centralized) 20:11:25 (what jbresnah said) 20:11:40 so many things will be needed every where 20:11:46 no i mean multi-component in the sense of something that multiple components can use, not just glance 20:11:47 I think the quotas thing should start moving but perhaps we should try to sync with other projects as well. I mean, there's some code duplicated between nova and cinder 20:11:59 we should contribute back to oslo and use that 20:12:15 flaper87: +1 20:12:28 i would like a general solution, but not a centralized one 20:12:44 I was going to take care of that in the next few weeks since I'm trying to reduce duplicated code throughout openstack 20:12:48 rosmaita: jbresnah: I"m all for centralizing the UX of managing quotas, but I never want to have to make a call across a region to determine a users quota limit 20:13:04 i feel there is a discussion going on somewhere else where folks are pushing towards the other idea 20:13:08 well, a centralized solution is needed if you are worried about totla bandwith usage in the cloud for example 20:13:28 is anyone worried about that? 20:13:29 nikhil: I feel that you are right 20:13:38 ameade: I see 20:13:47 you are looking for a shared code base but not a shared service? 20:13:59 rosmaita: good point, drats, this is a can of worms! 20:14:20 so I agree with the action item to find out more info before working on this in glance 20:14:22 that's a really good idea (not sure how fast it might move in oslo though) 20:14:38 well the other thing here, is who should do the enforcement? 20:14:45 guessing the best route would be to develop a good solution and then make it generic 20:14:48 i am sure there are good OS level things to do this 20:15:01 one person in one of the sessions was strongly against trying to do it in python 20:15:13 interesting 20:15:16 think there are a lot of them 20:15:21 if it is at the host level having limits across sevices is easier (i think) 20:15:28 jbresnah: josh harlow was against ratelimiting in python 20:15:32 my guess is it's kind of a tie 20:15:55 #action research approaches to centralize quota management, limit storage, enforcement, and usage monitoring 20:15:56 ameade: was it for streaming limits only? 20:16:06 or limiting rate of request/connection also? 20:16:15 okay, let's table this for a bit, looks like we don't have a stong consensus 20:16:20 nod 20:16:21 markwash: nice! 20:16:22 +1 20:16:24 +1 20:16:24 +1 20:16:28 i can look into that action item also 20:16:32 +! 20:16:33 +1 20:16:52 Okay, protected properties (again from the etherpad) 20:17:07 I think this is pretty uncontroversial? 20:17:17 +1 20:17:24 iccha and smclaren and mikal were possible assignees 20:17:26 +1 20:17:39 yes 20:17:58 #action markwash talk to iccha and smclaren and mikal about who will lead api-v2-property-protection 20:18:03 moving on 20:18:10 markwash: one sec 20:18:18 can we get the etherpad in the bp 20:18:38 looks like folks are out of sync (at least from what i sense here) 20:18:41 who in the what now? 20:18:53 yeah lets update teh blueprints to have all the info 20:19:05 +1 20:19:07 the link to the etherpad which has details about this protected properties implementation proposal 20:19:14 ah 20:19:20 do you have that link? I don't recall it? 20:19:22 be in the bp * 20:19:27 one min 20:19:48 #action nikhil to update api-v2-property-protection blueprint with link to etherpad discussion/proposal 20:19:55 anything else before we move on? 20:20:09 https://etherpad.openstack.org/public-glance-protected-props 20:20:11 ok 20:20:18 thanks! 20:20:19 Upload/Download Workflow: https://blueprints.launchpad.net/glance/+spec/upload-download-workflow 20:20:41 this was discussed during the summit, I'm proposing we split it up into two blueprints 20:20:56 but I'm not sure that I captured all the details we need to cover 20:21:41 rosmaita: any thoughts here? 20:21:45 mhh, my understanding is that users will be able to download the image directly from swift 20:21:49 am i right ? 20:22:16 flaper87: in some deployments yes, in others perhaps not 20:22:25 (what mark said) 20:22:40 I think the idea is that direct download and copy-to would have potentially different policies 20:22:42 and I guess the direct download won't happen using glance's client, right? 20:22:44 so the idea is that you put an image some place and then register it with glance? 20:22:55 jbresnah: that's a little different 20:22:58 'some place' being something outside of the scope of glance? 20:23:04 the idea is that you put an image someplace, and then glance downloads it from there 20:23:11 async 20:23:16 oh this is copy from? 20:23:30 oh i see now 20:23:37 ameade: that's what I'm proposing, that "import" is really just "copy-from" 20:23:42 and "export" is just "copy-to" 20:23:55 though I'm not married to any particular verbal scheme here 20:24:01 ok 20:24:13 so are copy-from and copy-to already implemented? 20:24:17 so 'conversion/verification' do not apply here? 20:24:22 or are you talking conceptually? 20:24:26 copy-from exists in v1 20:24:32 what is this bp proposing? 20:24:40 it seems to be a lot 20:24:46 generic workflow 20:24:59 markwash: what is your proposal for splitting? 20:24:59 jbresnah: I think copy-from may have some room for conversion / verification support 20:25:05 it a conceptual ameade 20:25:18 got it, so it would have a bunch of dep bps 20:25:37 rosmaita: two blueprints, one for adding copy-from like behavior to v2, another for adding copy-to like behavior to v2 20:25:48 that makes sense to me! 20:26:04 prolly could be an action item here ameade (get all the stuff implemented noted down and add bp for things to be created) good point 20:26:12 can we spell out use cases for this? 20:26:24 i am having some trouble getting my head around what it is trying to do 20:26:26 jbresnah: you beat me 20:26:29 sure 20:26:32 yeah, i dunno why we need copy to if we have copy from 20:26:33 and i have some interest in surrounding areas 20:26:45 upload/download 20:27:01 well, in the usual case, you can just directly upload the image bits on creation 20:27:21 but in the "import" case, the implication is that the bits you upload are not necessarily the bits that get stored 20:27:27 b/c of possible conversion, etc. 20:27:31 (almost halfway through meeting time btw) 20:28:11 wanna add the usecases to the bp? 20:28:17 that could be an action item 20:28:18 perhaps a few of us should take this offline and repackage it to better explain to others? 20:28:25 +1 20:28:28 +1 20:28:31 +1 20:28:33 +1 20:28:44 +1 20:28:44 rosmaita, how about you and me? 20:28:46 let's have a team speak channel for glance? 20:28:53 may be i went a bit too optimistic on that 20:29:25 #action rosmaita & markwash to clarify upload/download workflow bp 20:29:26 sure 20:29:48 How about rate limits? Summit discussion was mostly "nope" as I understood it 20:30:08 agreed, nope 20:30:10 i think that one will fall out of the quota reseach 20:30:21 jbresnah: I agree they're related 20:30:27 +! 20:30:29 +1 20:30:31 :-) 20:31:08 so, can I mark it superseded by quotas, but say consensus is leading towards not implementing it in glance? 20:31:41 yeah the lazy consensus at the summit was we dont have to worry about it in glance 20:31:43 I'd say superseded by quotas, period :P 20:31:46 it == https://blueprints.launchpad.net/glance/+spec/transfer-rate-limiting 20:31:51 right, it seems better done by something else so glance never sees the requests in the first place 20:32:25 kk, now on to image workers https://blueprints.launchpad.net/glance/+spec/async-glance-workers 20:32:32 which is sort of a new one, sort of an old one 20:33:01 like! 20:33:17 +1 20:33:21 gotta be done somehow 20:33:22 lol 20:33:47 this would dovetail into the upload/download workflow to give us a chance at image conversion 20:33:59 +1 20:34:01 I like the idea 20:34:01 markwash: is there a strong focus on queue based async? 20:34:08 it seems to be the same line of thinking as upload/download... 20:34:14 and also possibly https://blueprints.launchpad.net/glance/+spec/iso-image-metadata 20:34:14 i have not processed this one yet 20:34:45 nikhil: I'm not sure. . I think its a good idea in some possible implementations 20:34:57 nikhil: but I could also see a devstack-focused one that doesn't bother with a queue 20:35:05 i know i may sound like a broken record on this but i feel glance needs to get its story strat on replica management before it should do conversion 20:35:33 +1 20:35:37 for example: is a qcow2 image that was converted from a raw image a replica? 20:35:47 markwash: cool, was just gunna ask about metadata sync and exceptions on the queue based proposal 20:35:56 if the answer is "no" does that make things simpler? 20:36:08 does a user look to glance to: find me all ubuntu11.11 base images? 20:36:09 ummm 20:36:25 it might, but then it makes me ask 'why is glance doing conversion' 20:36:38 I think the async-workers bp can be related to conversion but it isn't. It's useful for other operations as well. 20:36:45 at some level glance is a discovery service for images 20:36:45 jbresnah: sorry, didn't mean for that to come off as jerk-ish :-) 20:36:52 it didnt 20:37:00 (didn't even entry my mind that way) 20:37:04 phew 20:37:22 if glance is just tracking image blobs 20:37:34 then the formats would not be replicas at all 20:37:49 but if it is just doing blobs, why is it in the business of conversion 20:37:56 TBH, I'd also like glance to be more "image aware" and be capable of introspecting images (extracting metadata and useful info from them) 20:38:00 i have no objection either way, i just think it should be well defined 20:38:08 but that maybe could be discussed in another meeting 20:38:19 I was definitely in the "just blobs" mindset 20:38:21 we need to have a strong typing on the service or feature creap happens 20:38:29 12 mins to go 20:38:51 just blobs works, but then why does glance convert? 20:38:52 but then there would still be a justification for conversion, in that a given deployer knows that he can't actually run qcow2 internally 20:39:02 for example 20:39:32 so, is the idea is to have a image conversion service under the glance umbrella? 20:39:38 maybe there is more discussion here that should follow on or be a part of the upload/download workflow? 20:39:41 I'm not sure about having conversions in Glance, TBH 20:39:51 jbresnah: not quite 20:40:09 does glance deal with bits or not? 20:40:13 we just need to define that 20:40:20 then we can go headstrong into whatever 20:40:34 yeah, i think glance's role could be more clearly defined 20:40:47 so, action item is to refine the pitch for conversion? 20:40:48 as for now, I like to think it doesn't 20:40:50 and if it was, i think i would be siginificantly less of a PITA :-( 20:40:58 is glance == metadata + image data? 20:41:24 perhaps a specific meeting to discuss glance's hats? 20:41:25 or glance == metadata + image directory 20:41:34 i think that glance should allow users to upload images in wome enumeration of formats and should make sure that when the image is used, it's in a format that the cloud the glance is in can use 20:41:39 jbresnah: +1 20:41:48 there is yes and no there either way esheffield 20:41:51 ? 20:42:18 quick, somebody #action something! :-) 20:42:51 you might have done that markwash 20:42:53 ! 20:43:02 on "something" :) 20:43:09 heh 20:43:23 #action markwash schedule meeting to further discuss image conversion and glance's role with image data 20:43:31 awesome 20:43:32 +1 20:43:33 +1 20:43:36 +1 20:43:41 +1 20:43:41 okay, we aren't going to get through all of the ones on the list today, but that's def okay 20:44:04 I'd like to skip out of the summit session bps down to the "Other new Blueprints" section of https://etherpad.openstack.org/glance-meeting-agenda 20:44:36 and, going out of order, flaper87 had some discussion points around functional testing 20:44:54 wow, that domain model will need a lot of work 20:45:04 guessing 20:45:24 flaper87: functional test bp notes? 20:46:34 well, not many. THere are a couple of things I think we should improve there: 1) Reduce the number of tests that need an up&running glance service (perhaps converting some to unittests) 20:46:53 +1 20:46:54 jbresnah: has been doing an amazing job on improving fuinctional tests performance 20:46:58 +1000 20:47:12 really i jsut did the impl, markwash did the hard part 20:47:17 yeah shoutout to jbresnah 20:47:30 redirects to markwash 20:47:44 well, and also the fix to port selection has landed 20:47:48 o/ 20:47:56 oh another one 20:47:58 I'm curious if folks are still seing a lot of ECONNREFUSED jenkins blowouts 20:48:08 2) We should stop using glance-control for functional tests 20:48:10 T_T 20:48:11 yeah i would like to know that too 20:48:17 ECONN 20:48:30 one thing i wanted to mention with regard to reviews, which has been better lately but still, is requiring unit tests and not just functional 20:48:34 ah yes, I forgot alla bout glance-control over night 20:49:09 ameade: what do you mean? 20:49:11 markwash: I haven't sent the email because it's holiday here and my gf took advantage of that :P 20:49:30 I'll send it later 20:49:30 :-) 20:49:33 cool 20:49:33 :) 20:49:43 #action flaper87 send that glance-control email 20:49:54 jbresnah: i just mean that in the past i've seen functional test coverage be good enough for a patch and that shouldnt be 20:50:05 not a big deal since i havent noticed lately 20:50:06 more than 50 mins now 20:50:08 markwash: ^^ 20:50:12 about ECONN, I haven't seen it lately 20:50:16 oh i see 20:50:17 yeah 20:50:18 ameade: that is a really good point 20:50:26 +1 20:50:36 there are a few items on open discussion 20:50:41 #topic open discussion 20:50:52 i have one question 20:50:56 go for it 20:51:37 scott was assign to public glance and wondering if he's working on it? 20:51:46 assigned* 20:51:55 nikhil: good point, we need to follow up with him 20:52:08 thanks, action item? 20:52:23 #action markwash follow up with smoser on public glance bp 20:52:26 I'd like to know if we plan to support legacy commands. They've been deprecated since folsom, AFAIK and some of them have given users some issues 20:52:31 perfect! 20:52:49 I think we should drop them based on the fact that new commands have been around for 2 releases 20:52:54 and we should support those 20:53:00 +1 20:53:00 flaper87: +1 20:53:07 flaper87: ^^ 20:53:14 * jbresnah agrees 20:53:21 its not an issue to drop support for them? 20:53:40 markwash: well we version glance-client dont we? 20:53:46 they're all covered by the new commands 20:53:52 so, a big NOTE in the change log 20:54:00 and release notes should be enough 20:54:02 IMHO 20:54:07 this would be a major version change I think 20:54:10 plus, they won't see them anymore :P 20:54:14 user legacy scripts could be scripting around the old calls 20:54:20 markwash: yes definitely 20:54:23 but they have been deprecated for some time 20:54:37 we dont do "releases" for clients do we? 20:54:37 #action bcwaldon consider dropping legacy shell commands in glance client and let us know why or why not 20:54:41 and setting a cultural understanding that deprecation means it is probably a good thing 20:55:01 bcwaldon really still owns this for now, I haven't taken the time to get up to speed 20:55:20 releases for glance client too? (good point ameade) 20:55:38 jbresnah: did you want to try to schedule a meeting for talking about the transfer service? 20:55:59 i do 20:56:06 but i thnk i will send email about it 20:56:08 I also wanted to let folks now that I submitted 2 patches for review related to the new registry's api. I'd love to have some feedback from you and make sure we agree with what have been written there 20:56:19 cool 20:56:24 i want to have a few more thoughts clear and available for pre-meeting consumption 20:56:31 and in general I'd like to remind folks that 20:56:34 Registry's V2 https://review.openstack.org/#/c/27359/ 20:56:41 #info glance review and bug squashing day tomorrow! 20:56:49 ww000000000000000000000000000000000ttttttttttttttttt 20:56:54 * jbresnah is looking forward to it! 20:57:03 heck yeah 20:57:10 In an hour it will be tomorrow for me 20:57:11 some hard core coding and reviewing action 20:57:21 flaper87: i was looking over that review and i think i need some interactive help understanding some things 20:57:22 let the squashing begin! 20:57:29 also, its probably obvious that this meeting format is still a little under-construction 20:57:33 flaper87: perhaps later today if not too late for you? 20:57:33 flaper87: where are you located physically? 20:57:41 rosmaita: Italy 20:57:49 jbresnah: it's never late :P 20:57:51 anybody got quick feedback? also you can email me longer feedback as well 20:57:52 jbresnah: sounds great! 20:58:06 markwash: i am really happy about these meetings FYI 20:58:16 markwash: like on the meetings! 20:58:17 markwash: i think they are going well 20:58:18 * flaper87 is happy too 20:58:20 markwash: it is making it so much easier for me to get involved in things 20:58:23 we just have so much to discuss atm 20:58:23 +1 20:58:24 nod 20:58:42 okay cool. . feel free to share ideas for improvment though. I'm just improvising at this point :-) 20:58:55 well good job so far markwash 20:59:00 free food 20:59:00 +1 20:59:04 nod 20:59:10 lets's close it there, we can discuss the bps we missed next week or offline during the week 20:59:14 flaper87: ciao! 20:59:15 nikhil: lol! 20:59:19 kk 20:59:24 rosmaita: ciao :D 20:59:28 #endmeeting