14:02:13 <nikhil_k_> #startmeeting glance drivers
14:02:13 <openstack> Meeting started Tue Aug 18 14:02:13 2015 UTC and is due to finish in 60 minutes.  The chair is nikhil_k_. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:02:14 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:02:16 <openstack> The meeting name has been set to 'glance_drivers'
14:02:18 <mfedosin> o/
14:02:21 <rosmaita> flaper87: i am running behind, have not got comments for you yet
14:02:25 <nikhil_k_> #topic agenda
14:02:28 <nikhil_k_> #link https://etherpad.openstack.org/p/glance-drivers-meeting-agenda
14:02:39 <nikhil_k_> mfedosin: I guess the topis are from you
14:02:43 <sigmavirus24_awa> o/
14:02:45 <mfedosin> yep
14:02:46 <nikhil_k_> if you wanna go ahead?
14:02:47 <flaper87> rosmaita:np :)
14:02:56 <flaper87> I do have 2 specs requests but mfedosin please, go ahead
14:03:00 <nikhil_k_> #topic  multithreading support
14:03:06 <mfedosin> so, my first question is about multithreading in swift
14:03:07 <nikhil_k_> https://review.openstack.org/#/c/207075/
14:03:30 <mfedosin> I wrote a spec for this and had several comments from swift team
14:03:53 <mfedosin> about using SLO by default, mainly
14:04:18 <sigmavirus24_awa> mfedosin: you mean in the swift store right? ;)
14:04:23 <nikhil_k_> we still have the concurrency issue with dlos ?
14:04:38 <mfedosin> afair, we discuss this feature on the last summit and it was stated as 'ok' :)
14:04:49 <mfedosin> sigmavirus24_awa, yes, sure
14:05:02 <mfedosin> I told you about that, remember?
14:05:10 <flaper87> I need to read the spec in detail but I'd definitely like to see mclaren's comments on it since he's got experience with that store
14:05:39 <mfedosin> I made several tests and the performance was increased several times on upload
14:06:01 <flaper87> mfedosin: using SLO ?
14:06:03 <mfedosin> yesterday they gave me a small hardware cluster to test it more detailed
14:06:36 <mfedosin> flaper87, you can read Samuel comment there about SLO
14:06:49 <mfedosin> (he called us squirrels, btw)
14:07:01 <sigmavirus24_awa> mfedosin: your performance note in the spec seems odd
14:07:07 <flaper87> mfedosin: I will read it
14:07:13 <sigmavirus24_awa> a multithreaded 10 GB upload can take as long as 40 minutes?
14:07:27 <sigmavirus24_awa> or are 6-30/2-40 not ranges?
14:07:50 <sigmavirus24_awa> is that 6 minutes, 30 seconds?
14:07:54 <sigmavirus24_awa> 2 minutes 40 seconds?
14:08:07 <sigmavirus24_awa> if so, please be more explicit about that, it doesn't read that well to me
14:08:07 <mfedosin> sigmavirus24_awa, when I tested it on Juno it was 6 minutes and 30 seconds
14:08:24 <flaper87> aaaaaaaaaaahhh
14:08:27 <nikhil_k_> guess, that's something on action item to update the spec
14:08:39 <mfedosin> but when I used multithreading it was like 2 minutes and 40 seconds
14:09:05 <nikhil_k_> mfedosin: the config will have to be in the store repo to reduce the redundancy and outofsycn issues with other possible clients consuming it
14:09:27 <nikhil_k_> mfedosin: the performance stats may need the following
14:09:46 <nikhil_k_> 1. total concurrent image uploads at a given time
14:10:13 <nikhil_k_> 2. total number of objs in the container already present at the time of tests
14:10:23 <nikhil_k_> 3. n/w b/w provisioning for the tests
14:10:57 <nikhil_k_> 4. tradeoffs seen while uploading with different thread numbers
14:11:05 <nikhil_k_> 5. image sizes that this was tested on
14:11:09 <mfedosin> as I mentioned they gave me a cluster only yeasterday, so was not able to perform all the tests, but for sure I will do that in the near future
14:11:31 <flaper87> mfedosin: np, I think it's a good start and what nikhil_k_ proposed makes sense to have on the spec
14:11:32 <nikhil_k_> yeah, np. these are suggestions as the performance is a very custom parameter
14:11:38 <sigmavirus24_awa> mfedosin: yeah please make that more explicit that would be super helpful because I think most people will read those as ranges
14:11:59 <flaper87> sigmavirus24: YOU ARE HERE NOW!!!!
14:12:07 * flaper87 stfu
14:12:14 <nikhil_k_> mfedosin: guess, we have something to start on
14:12:22 <sigmavirus24> lol
14:12:24 <nikhil_k_> should we move on to next one?
14:12:32 <sigmavirus24> flaper87: I had an IRC client connected with this nick on a different laptop
14:12:36 <sigmavirus24> 1st world problems right there
14:12:38 <mfedosin> I don't mind
14:12:43 <nikhil_k_> thanks
14:12:45 <nikhil_k_> #topic list of available storage shemes
14:12:49 <nikhil_k_> https://review.openstack.org/#/c/207063/
14:13:01 <nikhil_k_> mfedosin: that's yours again, i sup
14:13:02 <mfedosin> if you can please add all of these suggestions to my spec
14:13:33 <mfedosin> I hoped that Timur from Horizon joined too
14:13:43 <mfedosin> but he's not available
14:14:35 <mfedosin> the main thing that Horizon needs a list of available store shemes to download images from
14:14:42 <flaper87> to be completely honest, I'm not very keen on having this kind of information disclosed by the API :/
14:14:47 <flaper87> but I understand the use-case
14:14:58 <flaper87> and it's not really harmful
14:14:59 <nikhil_k_> mfedosin: umm, I am not sure how I totally feel about it. it looks good at the first impression but there may be gotchas that I don't recall
14:15:12 <flaper87> nikhil_k_: yeah, my exact feelings
14:15:22 <nikhil_k_> racy?
14:15:39 <mfedosin> racy?
14:15:48 <nikhil_k_> may be client caches what's in glance api call and then assumes the store availability, possibly a deploy can change it
14:16:01 * flaper87 races
14:16:21 <nikhil_k_> also, the store capabilities would need to be exposed
14:16:26 <nikhil_k_> likely acls :P
14:16:27 <flaper87> mmh, yeah. One other thing is that we'd be disclosing deployment-specific information
14:16:37 <flaper87> yup
14:16:51 <nikhil_k_> some may be read only, some r/w
14:17:01 * flaper87 needs to think this through
14:17:13 <flaper87> I'm not fully opposed but I gotta put some thoughts on it
14:17:24 <nikhil_k_> agreed
14:17:28 <flaper87> That based on the assumption I know how to think
14:17:33 <nikhil_k_> lol
14:17:36 <flaper87> :P
14:17:44 <mfedosin> nikhil_k_, it doesn't matter about read-only, because store will be used to download image only
14:18:04 <mfedosin> my another suggestion to write this information in error message
14:18:26 <nikhil_k_> I see. the motivation para would benefit from incl it :)
14:18:37 <mfedosin> Like 'Upload from this external source is not supported. List of available sources are: '
14:18:54 <flaper87> mfedosin: but then we'll have people doing random posts to get the list of stores
14:18:57 <flaper87> which is not idea
14:19:05 <sigmavirus24> yeah
14:19:13 <flaper87> We already return images' schema
14:19:15 <sigmavirus24> I don't like this at all. I feel like it's exposing more detail than most people need
14:19:19 <mfedosin> flaper87, but there won't be new api call
14:19:29 <flaper87> lets see how we can play with that, if we really need this
14:19:31 <sigmavirus24> I'd need to see people actually *need*ing this in Horizon rather than thinking they need it
14:19:35 <rosmaita> mfedosin: did you withdraw the spec to make this an api call?
14:19:40 <rosmaita> sigmavirus24: +1
14:19:52 <sigmavirus24> Like are there actual user studies from the Horizon team showing that people absolutely need this and have really good reasoning for it
14:20:01 <sigmavirus24> If not, are these just "power" users asking for it
14:20:17 <sigmavirus24> I put "power" in quotes because there are never any real "power" users as far as I'm concerned. just people who break things and then whine about it
14:20:34 <flaper87> The cases where I think this could be useful are those that need to interact with locations
14:20:45 <mfedosin> there are several customers requests for this
14:20:56 <flaper87> which is something that users from outside the cloud shouldn't do, IMHO.
14:21:01 <sigmavirus24> mfedosin: the customer is not always right
14:21:06 <nikhil_k_> so, why do we need to list available schemas for download if the location is exposed?
14:21:13 <mfedosin> because they wanted to create images from ftp, but it's not supported by glance_store
14:21:20 <nikhil_k_> it might be useful for uploads with lack of awareness
14:21:37 <nikhil_k_> that's not download
14:21:42 <flaper87> mfedosin: I think that's lack of knowledge of what the cloud you're sitting on top of provides
14:21:50 <flaper87> which is not Glance's fault
14:21:57 <sigmavirus24> that's the cloud provider's fault
14:22:01 <sigmavirus24> but also, what nikhil_k_ said
14:22:02 <flaper87> :)
14:22:03 <sigmavirus24> that's not download
14:22:12 <nikhil_k_> yeah, I think let's have horizon people request this clearly here or on ml
14:22:50 <rosmaita> i am really confused, the spec says the call will be allowed for people who can create images
14:22:54 <rosmaita> so that's not read only
14:22:54 <sigmavirus24> And let's have horizon people back this up with a study that shows the real use-cases this provides rather than "I tried to upload using a rando url and it failed"
14:22:56 <flaper87> An update on the spec wouldn't be bad
14:23:01 <flaper87> that'd be a good start
14:23:02 <mfedosin> the actual issue to prevent users from creating images from wrong sources
14:23:14 <nikhil_k_> would be good to have some weight on criticality of this change, use cases supporting it and scope of change (like we should state if this is going to be download only)
14:23:15 <flaper87> To have more details and to understand the problem it tries to solve better
14:23:26 <mfedosin> I'll give a link to the bug
14:23:31 <sigmavirus24> == flaper87
14:23:32 <mfedosin> wait a sec please
14:23:54 <nikhil_k_> flaper87: you had 2 specs?
14:24:01 <flaper87> yup, pasted on the etherpad
14:24:04 <flaper87> but I can paste them here
14:24:06 <nikhil_k_> I think we can cover one today and may be another on -glance
14:24:13 <flaper87> the first one should be straightforward
14:24:19 <flaper87> it's small and it's got 2 +2s
14:24:24 <flaper87> well, 1 if we don't count mine
14:24:29 <nikhil_k_> oh yeah, I was about to approve it yday :)
14:24:35 <flaper87> The code has been reviewed by sigmavirus24 and myself too
14:24:44 <flaper87> and the second one is tasks but we can move to -glance for that
14:24:46 <mfedosin> https://bugs.launchpad.net/horizon/+bug/1476657
14:24:47 <openstack> Launchpad bug 1476657 in OpenStack Dashboard (Horizon) "Attempt to upload image via ftp makes queued image" [Undecided,In progress] - Assigned to Vlad Okhrimenko (vokhrimenko)
14:24:50 <flaper87> if you folks have time now
14:24:57 <flaper87> otherwise, I'll just blame rosmaita
14:25:02 <nikhil_k_> :)
14:25:07 <nikhil_k_> I can be available if others are
14:25:16 <rosmaita> i can be available
14:25:17 <nikhil_k_> good to have it in the open while people are around
14:25:22 <flaper87> awesome, lets do it there
14:25:27 <sigmavirus24> I can be available
14:25:31 <flaper87> nice nice
14:25:47 <flaper87> nikhil_k_: it'd be cool to approve the S3 one since it's tiny
14:25:53 <sigmavirus24> mfedosin: so I disagree that the spec you've written is the right fix for that bug
14:25:53 <flaper87> I'm done
14:25:57 <nikhil_k_> surely
14:26:23 <nikhil_k_> flaper87: if there are amends, we can follow up in a commit by whoever wants to
14:26:27 <sigmavirus24> I can see having Glance return a 4xx error if the scheme is unsupported, but I see no reason to return a list of available schemes as a new endpoint
14:26:54 <sigmavirus24> That said, the best solution to that bug is to make an FTPStore =P
14:26:55 <nikhil_k_> sigmavirus24: new endpoint??
14:27:02 <mfedosin> sigmavirus24, of course not - fix for this bug is https://review.openstack.org/#/c/204105/, but they have to hardcode it to http/https
14:27:08 <flaper87> nikhil_k_: +2
14:27:09 <nikhil_k_> I agree to ftp store
14:27:31 <nikhil_k_> pound action to mfedosin :P
14:28:03 <nikhil_k_> mfedosin: yeah, I see what sigmavirus24 is saying
14:28:18 <nikhil_k_> the safer call is to see if a store type is supported or not
14:28:21 <sigmavirus24> mfedosin: the spec we're discussing wants to add a new endpoint /v2/available_schemes
14:28:47 <nikhil_k_> hmm, that looks more like a url string in the openstack lingo to me
14:28:56 <flaper87> :P
14:29:00 <mfedosin> sigmavirus24, I though it's a good solution
14:29:05 <sigmavirus24> Anyway we're almost out of time
14:29:06 <mfedosin> are there others?
14:29:12 <sigmavirus24> We can discuss this in -glance
14:29:18 <nikhil_k_> I always get confused when people refer to endpoint for non-keystone things
14:29:22 <nikhil_k_> sorry :P
14:29:58 <mfedosin> okay, thanks all for your attention
14:30:02 <nikhil_k_> okay, we are out of time now. Let's continue flaper87's spec first on -glance
14:30:06 <nikhil_k_> THanks mfedosin
14:30:08 <nikhil_k_> #endmeeting