19:01:00 <markwash> #startmeeting glance
19:01:01 <openstack> Meeting started Wed Apr 10 19:01:00 2013 UTC.  The chair is markwash. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:02 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:01:04 <openstack> The meeting name has been set to 'glance'
19:01:30 <markwash> So, I didn't really lay out the agenda in the wiki
19:02:00 <markwash> but the rough idea from my perspective is: follow up on caching discussion, more summit topic discussion, and more blueprint triage
19:02:13 <flaper87> +1
19:02:19 <brianr-g1ne> +1
19:02:21 <markwash> but anybody else who has important topics feel free to throw them out now, we can work them in
19:02:46 <ameade> i want more code reviews
19:03:19 <markwash> ah, I've got a quick note on that
19:03:31 <markwash> #topic review and bug squashing days
19:03:40 <flaper87> w00000t
19:03:45 <markwash> last meeting flaper87 suggested we have bug squashing days
19:04:01 <markwash> then privately made the suggestion that we ought to have review squashing as well
19:04:17 <markwash> which struck me as even more critical, as it seems
19:04:24 <iccha_> is there a glance openstack room? where we can all discuss and coordinate during these days?
19:04:25 <markwash> I know I haven't been doing enough review lately :-( sorry folks
19:04:38 <markwash> I don't believe so, but we could make one
19:04:39 <flaper87> openstack-glance
19:04:52 <ameade> +1 on both types of squash
19:05:07 <markwash> any thoughts on when we should have the first one? week after the summit?
19:05:17 <ameade> i saw that last meeting the concern was that we dont have enough bugs
19:05:24 <jbresnah> I agree with the review days for sure
19:05:37 <flaper87> that sounds ok, the week after the summit
19:05:39 <ameade> so maybe we just have a "squash" day and do both? i dont want reviews to wait til squash days though
19:05:42 <jbresnah> bug squash sounds good too, tho that is something that could end up being more async for people
19:06:00 <flaper87> the suggestion was to have them together
19:06:00 <markwash> ameade: agreed, we shouldn't put them off until squash day
19:06:07 <iccha_> +1 that
19:06:23 <markwash> yeah, one day for both as needed
19:06:28 <flaper87> starting with the one in worse shape
19:06:54 <nikhil> +1
19:06:59 <markwash> #agreed have a shared review/bug squashing day during the week after the summit
19:07:00 <ameade> would it be more effective to have constant reviews if we do the traditional core review days where a core is assigned to a day?
19:07:21 <markwash> ameade: I think we should have constant reviews regardless of squashing days
19:07:35 <markwash> ameade: and I hope we aren't at a point where we need review days like nova
19:07:49 <ameade> markwash: yeah i dont think we are that bad
19:08:12 <ameade> i agree, lets just try out the squash days and if we need more than that we will revaluate
19:08:20 <markwash> #action markwash schedule a squash day during the week after the summit
19:08:24 <flaper87> I think we should review daily and that review days should be used as a speed up on reviews and "shared reviews" day
19:08:33 <markwash> ++
19:08:36 <ameade> +1
19:08:37 <nikhil> agree
19:08:40 <iccha_> +1
19:08:43 <nikhil> +1
19:08:51 <iccha_> approved :p
19:08:52 <markwash> any other thoughts on reviews?
19:09:22 <markwash> #topic glance caching
19:09:38 <markwash> last week we talked about new approaches to managing cache
19:10:21 <markwash> jbresnah has posted his ideas about future approaches to caching here https://tropicaldevel.wordpress.com/2013/04/09/free-cache/
19:10:39 <ameade> did we decide to not have a summit session on this?
19:11:12 <markwash> I'm not sure that we decided, but one was not proposed
19:11:46 <markwash> I do feel like we could work out a reasonable solution outside of the summit, however
19:12:08 * flaper87 wont be at the summit :(
19:12:20 <nikhil> so, this seems to be something that could be benefitial to discuss before exposing glance
19:12:37 <nikhil> would help on some design decisions
19:12:54 <nikhil> thoughts?
19:13:08 <iccha_> should caching not work almost the same way either glance is exposed or not?
19:13:45 <jbresnah> I am not sure what exposed means here exactly...
19:14:01 <nikhil> from the previous meeting logs, looks like locations is something discussed
19:14:13 <iccha_> jbresnah: it refers to being able to use glance directly
19:14:23 <iccha_> as an independent service
19:14:25 <nikhil> my bad, it mean glance as a public deployment
19:14:34 <jbresnah> oh right, sorry i forgot that in some places it is not public, thanks.
19:14:55 <jbresnah> i think that multiple locations is a big part of the cache
19:15:23 <jbresnah> nikhil: so i agree locations is something to discuss
19:15:37 <nikhil> jbresnah: on the similar note
19:15:54 <nikhil> if we decide on multiple locations and related glance cache
19:16:01 <markwash> one thing I would note, is that with jbresnah's proposal, can we still keep the local cached copies urls hidden from regular users? if not I worry about exposing individual nodes to targeted dos attacks
19:16:35 <jbresnah> i think you can
19:16:48 <iccha_> we could have additional way to indicate provate urls maybe?
19:16:54 <jbresnah> in one sense glance api becomes another user of the relica catalog
19:16:55 <flaper87> I think it is important to be able to scale out cache instances
19:16:55 <markwash> cool, wanted to make sure that wasn't antithetical
19:17:12 <jbresnah> so, if a user goes to download an image from glance, glance looks at the catalog and chooses the best location
19:17:15 <jbresnah> then streams it
19:17:26 <markwash> flaper87: yes, in general you had other ideas for how we might approach caching
19:17:31 <jbresnah> if the locations are exposed to an end user, then can make that decisions
19:17:34 <markwash> flaper87: any code or docs for us to look at?
19:17:42 <jbresnah> if not glance can make it internally and then stream it
19:18:31 <jbresnah> i agree that horizontal scalability is crucial
19:18:42 <jbresnah> i hope i have not spoken against that
19:18:57 <flaper87> markwash: I was writing it and then I thought that it should exist along the lines of what jbresnah proposed. The idea would be to use the multi-location implementation for caches so that cached images would be just a new location for an existing image but under a dedicated API.
19:19:15 <nikhil> jbresnah: guess not, rather proved a thought on how to achieve this
19:19:25 <flaper87> that will allow us to just disable glance-api paths and have instances dedicated just for caches
19:19:26 <nikhil> s/proved/provoked/
19:19:49 <jbresnah> the original problem was here was to deal with a cache management service
19:19:58 <jbresnah> clearing the cache, listing images in it etc
19:20:24 <jbresnah> i think that all of the problems there will also be in maintaining the consistency of multiple locations
19:20:44 <jbresnah> so we should solve the multiple locations issue and accept the cache issue as a lesser included
19:20:56 <nikhil> i guess that is a challenge by itself
19:21:07 <markwash> that is interesting, and we do need some motivation for the multiple locations api
19:21:21 <nikhil> that's the reason why i wished to include public glance in this discussion
19:21:48 <nikhil> basically, maintaining consistency would be an issue
19:22:37 <markwash> I don't know about everybody else, but I'd like a little more time to think on this, and we can certainly discuss it more over libations at the summit (those who can attend)
19:22:44 <nikhil> markwash: we'r hoping to have the same uuid for copied/cache image to other location
19:22:47 <iccha_> jbresnah: do you envision cache management being a sepaarte servoce?
19:22:52 <ameade> +1 libations
19:22:54 <nikhil> markwash: sure
19:22:58 <jbresnah> +1 libations
19:23:10 <jbresnah> iccha_: perhaps
19:23:13 <jbresnah> or a tool
19:23:40 <jbresnah> it gets complicated because issues come up like: Does the user who registered the replicated image deligate delete rights to the service
19:23:40 * markwash searches for an action item. . .
19:23:41 <jbresnah> etc
19:23:56 <markwash> hmm, interesting
19:24:04 <jbresnah> there are some other interesting bits that come out of it too that we can discuss when appropriate
19:24:27 <jbresnah> for example: nova-compute could register a local replica instead of maintaining its own cache
19:24:33 <jbresnah> tho, that could be out of scope also
19:24:40 <jbresnah> just a use case example really
19:24:50 <nikhil> jbresnah: very interesting
19:24:59 <jbresnah> my main point is this:
19:25:06 <nikhil> may be a pluggable, though
19:25:09 <jbresnah> multiple-locations makes glance a replica service
19:25:20 <jbresnah> a copy on a local disk is just another replica
19:25:34 <jbresnah> special case code will cause complications
19:25:46 <jbresnah> nod pluggable
19:25:56 <markwash> very thought provoking
19:26:04 <markwash> hmm
19:26:10 <ameade> +1 to an image living anywhere just being another location
19:26:38 <iccha_> do we see this tying into image cloning?
19:26:57 <nikhil> jbresnah: don't wanna go off topic, though one thing that stikes me is local copy is not at a known state and a snapshot on that is a different image (looks like deeper issue, may be?)
19:26:57 <flaper87> iccha_: I kind of think it like that
19:27:38 <jbresnah> nikhil: i may not follow, but if the checksum changes it is no longer a replica
19:27:48 <nikhil> kk
19:27:59 <markwash> I'm a little worried about scope creep here
19:28:06 <nikhil> may be i on a different thought train :)
19:28:11 <flaper87> again, I think it is important to be able to scale that out and being able to identify which images are indeed a cached image and which not.
19:28:13 <ameade> images shouldnt go stale anyways, they dont change?
19:29:05 <jbresnah> the issue about datasets changing is exactly why we will need some sort of consistency management with multiple locations
19:29:45 <jbresnah> which feeds into my point about that feature subsuming cache management
19:30:11 <jbresnah> i think for the purpose of this conversation the replicas are blogs of data, not images
19:30:20 <jbresnah> tho i could definitely be missing critical info there
19:30:21 <flaper87> another thing that we might also want to consider is being able to have some auto-caching algorithms
19:31:06 <markwash> so for next steps here
19:31:08 <jbresnah> flaper87: perhaps that is true, but i think that is a future feature wrt the conversation here
19:31:22 <jbresnah> flaper87: but a good feature nonetheless
19:31:27 <nikhil> +1
19:31:32 <markwash> I here a lot of interesting ideas, but maybe we need someone to synthesize these into some smaller concrete proposals for next steps?
19:31:41 <markwash> *ehar
19:31:43 <markwash> *hear
19:31:44 <markwash> there we go
19:32:15 <jbresnah> i think the first step is to see the current multiple-locations effort through
19:32:27 <ameade> +1
19:32:31 <jbresnah> and i would suggest that we stall the cache effort until then
19:32:45 <nikhil> +1
19:32:48 <jbresnah> see exactly what is needed at the end of that
19:32:55 <flaper87> +1, once that's done we could talk about how to replicate it
19:32:59 <markwash> sounds good
19:33:14 <nikhil> markwash: besides the proposed topic for glance at the summit, whay else would you like to discuss?
19:33:24 <nikhil> *topics
19:33:34 <markwash> lets' leave caching off here, and move on to glance design summit topics
19:33:46 <jbresnah> sounds good
19:33:49 <iccha_> sounds good
19:33:58 <markwash> all I have left is summit topics in general (which is probably a big talk) and then just blueprint triage
19:34:13 <iccha_> https://wiki.openstack.org/wiki/Summit/Havana/Etherpads is the wiki page for etherpads for summit sessions
19:34:17 <markwash> but I'd be happy to leave off blueprint triage for other topics of interest
19:34:22 <markwash> #topic glance summit sessions
19:34:42 <markwash> 3 sessions have been selected and scheduled so far
19:34:52 <markwash> #link http://openstacksummitapril2013.sched.org/overview/type/design+summit/Glance#.UWWzb6tg_58
19:35:08 <markwash> that leaves 2 sessions left to schedule
19:35:36 <markwash> and one of the topics we have to hit relates to http://summit.openstack.org/cfp/details/47
19:35:43 <iccha_> i see only 2 sessions unreviewed
19:35:59 <markwash> but possibly more generally just image upload/download performance
19:36:02 <markwash> and nova boot performance
19:36:38 <markwash> improving image transfer performance is a big topic, so I'm wondering 1) should it be split into two? 2) does anyone have any interest in rolling db migrations?
19:37:17 <rosmaita> markwash: we definitely are interested in rolling db migrations
19:37:34 <iccha_> +1
19:37:48 <nikhil> markwash: we'r also interested in interoperability/inter hypervisor compatibility as well
19:37:52 <ameade> lets just unconference everything :)
19:37:53 <nikhil> +1
19:37:59 <markwash> so splitting the performance topic into two sessions would come at some cost
19:38:52 <markwash> I have a summit pitch about performance that I'd love to make now
19:39:15 <jbresnah> cool
19:39:25 <jbresnah> i would like to hear that pitch
19:39:28 <rosmaita> we are all ears
19:39:33 <flaper87> go go go
19:39:40 <markwash> take this as a straw man if you like
19:39:51 <markwash> but I propose that Glance should not be worried about performance
19:39:54 <markwash> at least not directly
19:40:18 <markwash> Glance should concern itself with exposing the information that other performance-oriented clients need in order to work efficiently
19:40:36 <ameade> +1
19:40:37 <jbresnah> that sounds right to me
19:40:40 <flaper87> +1
19:40:45 <markwash> most of the proposals that I've seen say:
19:40:55 <markwash> If we make images X, then we can have nova do Y, which is faster
19:41:34 <markwash> well, never mind that last train of thought. . not sure where it was going to derail
19:41:43 <ameade> but dont forget about glance being a 1st class API
19:41:55 <nikhil> +1
19:41:59 <ameade> we still need glance to do transfers at some level right?
19:42:21 <rosmaita> and i believe caching is done for performance improvements?
19:42:26 <nikhil> transfers and sync too
19:43:06 <markwash> This behavior feels kind of like legacy compatibility to me
19:43:09 <jbresnah> I would like to couple resource management with transfer performance
19:43:36 <jbresnah> when consoidering going super fast you have to consider resource usage
19:43:41 <jbresnah> and how that effects others
19:44:09 <jbresnah> also i would like to add that as glance stands right now it cannot make proper decisions about either
19:44:15 <jbresnah> because it is only on 1 side of the transfer
19:44:36 <markwash> jbresnah: but if glance isn't concerning itself with those things, the responsibility for managing resource usage would be elsewhere. . glance isn't any side of the transfer
19:44:58 <jbresnah> markwash: oh yeah +1 that
19:45:24 <flaper87> I think glance should be more worried about knowing more from images than knowing more about improving performances
19:45:36 <ameade> I'd really like to see glance taken out of actual transfers and maybe just negotiate transfers?
19:45:52 <iccha_> it boils down to what is the purpose of glance and what we want it to be once is it is public
19:45:56 <markwash> yeah
19:46:00 <markwash> iccha_: +1
19:46:14 <markwash> so glance didn't originally exist alongside nova
19:46:19 <nikhil> looks like everyone wants to decouple the glance and image data management service
19:46:21 <jbresnah> I love this convoersation so far
19:46:43 <markwash> it was created because public clouds didn't want people to be able to boot just anything, since all the broken nova boots would cause a support nightmare
19:47:13 <markwash> it was also created as a central place to manage image data, however, since swift isn't really up to the challenge
19:47:55 <markwash> I think we should figure out a way to formally adopt the former purpose as something like a mission statement
19:48:09 * markwash is really out on a limb here
19:48:16 <flaper87> I agree
19:48:20 <jbresnah> i am with you
19:48:40 <jbresnah> in a past life i worked on grid computing.... i will save the details for when beer is near by...
19:48:47 <markwash> and help out other projects so that we can move away from the latter purpose
19:48:56 <jbresnah> but to me it makes sense to have replica management and data transfer as separate things
19:49:21 <jbresnah> tho i do understand the convenience of coupling them as well
19:49:29 <markwash> jbresnah: I think it makes sense to keep track of different replicas, yes, assuming it is useful to keep multiple replicas of the same thing
19:49:41 <markwash> which seems likely
19:50:04 <markwash> wow, I've gotten kind of far from what I meant to do
19:50:07 <jbresnah> markwash: replica might be too strong for me to be using, perhaps 'registry' is better
19:50:08 <iccha_> Yeah its a different thing maintaining info about replicas vs transfering them or making the replicas
19:50:47 <ameade> lets not get into redefining Glance maybe?
19:50:50 <markwash> going back to the summit topic, I guess the default option is to make a "booting and snapshotting, download and upload performance" session
19:51:06 <markwash> and approve the db rolling upgrades session
19:51:06 <iccha_> sounds good
19:51:08 <ameade> +1
19:51:11 <iccha_> +1
19:51:12 <flaper87> +1
19:51:33 <markwash> jbresnah: one session is not a lot of time
19:51:59 <jbresnah> nod, i will try not to talk too much
19:52:10 <markwash> jbresnah: lol, not what I was trying to suggest :-)
19:52:12 <iccha_> we can try doing a glance grab a beer/dinner/coffee thing during the summit too
19:52:14 <jbresnah> heh
19:52:22 <jbresnah> most of my thinking is on my blog (i think)
19:52:30 <ameade> s/beer/beers/
19:52:33 <markwash> there are also some folks who will want to talk about volumes as images
19:52:43 <iccha_> ah yeah thats always been there
19:52:45 <ameade> markwash: can we ignore them?
19:52:48 <ameade> lol
19:52:54 <jbresnah> heh
19:53:01 <flaper87> :P
19:53:06 <markwash> well, we can, but perhaps at our peril
19:53:15 <jbresnah> i was also thinking of proposing a 'unconference' topic on an image transfer sercice
19:53:21 <jbresnah> could get time that way i suppose
19:53:40 <iccha_> yeah unconferences are a good way to do that
19:53:41 <markwash> zhiyan and IBM have an amazingly fast provisioning system they have built on top of volume-like abstractions, and want to expose that in openstack
19:53:43 <jbresnah> this is my first summit tho, so i may not have the culture quite right
19:54:06 <iccha_> yeah i think i see a glance-cinder-driver blueprint too
19:54:56 <jbresnah> markwash: can you point me at more details on that?
19:55:06 <markwash> jbresnah: I will have to find a link
19:55:11 <rosmaita> well, having glance not transfer any image data would definitely improve its performance
19:55:15 * nikhil is having a hard time finding out action item for this except for a philosophical discussion with glance folks
19:55:15 <nikhil> deadlock?
19:55:18 <markwash> rosmaita: :-)
19:55:24 <ameade> yeah i'm actually pretty excited about boot from volumes and stuff
19:55:43 <markwash> as far as action items, I'm pretty sure the boot-from-volume stuff needs session time
19:56:19 <flaper87> will all this be recorded, written, G+ ?
19:56:27 <markwash> so I'll check the other topics to see if its sufficiently covered
19:56:38 <flaper87> I'd love to participate somehow and catch up with everything that happens at the summit
19:56:41 <markwash> and if not I'd like to boot the rolling db upgrades to an unconference
19:56:42 <iccha_> flaper87: we will take notes
19:56:52 <flaper87> iccha_: :D thank you so much!
19:57:15 <nikhil> flaper87: they would be live streaming these sessions as well, i hope
19:57:16 <ameade> flaper87: I know they said they weren't gonna stream sessions but maybe enough people were unhappy about that and they changed their minds
19:57:24 <markwash> #action markwash finish scheduling the summit sessions asap
19:57:26 <iccha_> flaper87: https://wiki.openstack.org/wiki/Summit/Havana/Etherpads has etherpad links for summit sessions
19:57:42 <markwash> #topic open discussion
19:57:52 <markwash> sorry I wandered so far afield today folks
19:58:16 <jbresnah> i enjoyed it :-)
19:58:19 <ameade> markwash: just pretend that's what you meant to do :P
19:58:26 <flaper87> ahhahaa
19:58:30 <markwash> lol
19:58:30 <ameade> i'm really hyped for the summit
19:58:34 <ameade> gonna be a doosey
19:58:51 <iccha_> i am glad we re having glance meetings :) and we all should try hanging out in the openstack-glance channel flaper87 mentioned
19:58:55 <markwash> flaper87: I'll make sure we keep use the etherpads as much as possible during the summit
19:59:29 <nikhil> +1
19:59:36 <flaper87> markwash: thank you, I'm really sad I wont be there! :(
19:59:55 <flaper87> Notes will be really useful for catching up
20:00:00 <flaper87> and commenting
20:00:09 <markwash> there was some mention of irc being used during the meetings
20:00:26 <markwash> not sure how well that would work, but if anyone has any ideas we could give it a try
20:00:36 <ameade> we could always google hangout
20:00:42 <markwash> the problem with AV solutions is that the A always sucks
20:01:03 <ameade> we appear to be out of time...
20:01:06 <markwash> indeed
20:01:18 <markwash> #endmeeting