14:01:07 <jokke_> #startmeeting glance
14:01:08 <openstack> Meeting started Thu Jan 17 14:01:07 2019 UTC and is due to finish in 60 minutes.  The chair is jokke_. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:12 <openstack> The meeting name has been set to 'glance'
14:01:15 <jokke_> #topic roll-call
14:01:16 <abhishekk> o/
14:01:17 <jokke_> o/
14:01:19 <rosmaita> o/
14:01:23 <LiangFang> o/
14:01:26 <GregWaines> o/
14:01:32 <smcginnis> o/
14:01:42 <jokke_> #link https://etherpad.openstack.org/p/glance-team-meeting-agenda
14:02:18 <jokke_> #topic updates
14:02:29 <jokke_> really quick two
14:02:41 <jokke_> first of all I have requested a room for us in PTG
14:03:12 <rosmaita> \o/
14:03:18 <jokke_> So hoping we can get interested parties joining and we can get some work planning moving on there
14:03:26 <abhishekk> yey
14:03:59 <jokke_> second thing I have in my mind. IIRC the submissions for talks for the summit is closing
14:04:18 <jokke_> so if you have something planned make sure you have submitted your proposals
14:04:46 <abhishekk> ack
14:04:58 <jokke_> #topic release updates
14:05:04 <jokke_> abhishekk stage is yours
14:05:14 <abhishekk> thanks
14:05:24 <abhishekk> we are in R-12 weeks
14:05:38 <abhishekk> S2 milestone has just passed
14:06:08 <abhishekk> It's time for us to be prepared for non-client library, i.e. glance-store release
14:06:36 <abhishekk> (We have one important patch up for glance-store, rosmaita added that to discussion)
14:06:57 <abhishekk> Periodic jobs, all is well
14:07:15 <abhishekk> everything is green at the moment
14:07:21 <jokke_> thanks
14:07:54 <jokke_> One more thing I almost forgot. You might have noticed we have RP column now in the gerrit
14:08:01 <abhishekk> over to you again
14:08:17 <rosmaita> saw that
14:08:34 <jokke_> So keep an eye on that and please review those pathes flagged with +votes as priority
14:08:59 <abhishekk> ack
14:09:32 <jokke_> and also cores should have rights to vote on that column so do not expect me being only one flagging those
14:09:33 <rosmaita> on that note, do either of you have a link to the default viz change patch?
14:09:46 <jokke_> I can dig it out for you
14:10:05 <rosmaita> looks like that will be up to me and sean since you and abhishek are co-authors
14:10:20 <abhishekk> https://review.openstack.org/#/c/628430/
14:10:26 <rosmaita> thanks
14:11:08 <rosmaita> i have to build a new devstack this weekend, i will test the migration on my old one, and then check the fresh install on the new one
14:11:25 <jokke_> cool
14:11:29 <abhishekk> great
14:11:38 <jokke_> ok lets move on
14:12:21 <jokke_> #topic unicode object TypeError from swift store hashing
14:12:28 <jokke_> rosmaita: you wanted to take this one
14:12:43 <rosmaita> yeah, just wanted to discuss the patch real quick
14:13:03 <rosmaita> i am wondering whether the fix should be in the readers instead of in the driver?
14:13:13 <rosmaita> also, i am worried about not having a test
14:13:43 <rosmaita> because i am confused about how this error happens in the first place
14:13:50 <jokke_> I'm even more worried about the fact that doing the encoding likely breaks anyone who upgrades to py3
14:13:53 <rosmaita> i thought we were dealing with byte streams everywhere
14:14:31 <jokke_> So py27 does not use utf-8 encoding and using that with py3 _will_ give different hash
14:15:21 <jokke_> which means that the hash verifications will fail once you have cross version platform or the python version changes over upgrade
14:16:06 <jokke_> rosmaita: yeah, with the md5, the problem is that it's hashing string, not bytes
14:17:48 <rosmaita> you'd think we'd see other problems, because a raw byte stream has to have all sorts of crap in it that would make it illegal utf-8
14:18:22 <abhishekk> I don't think so
14:18:59 <jokke_> well utf-8 uses the whole range of 8bits so it's always possible ot encode to that
14:19:29 <jokke_> I just have no idea how the md5 actually works on py2.7 which has default encoding of 'ascii' that is only 7 bits
14:19:56 <rosmaita> i thought it just worked on the raw bytes
14:19:57 <jokke_> and I think we need to verify that before jumping the gun and adding some encoding there that will produce different results
14:20:47 <rosmaita> see, the problem you mentioned earlier is that the string -> byte translation is different for py2 and py3
14:20:53 <rosmaita> but if we don't need to translate
14:21:01 <rosmaita> then we have the exact same bytes
14:21:25 <rosmaita> i am confused about what in our code thinks the image data is a unicode stream
14:21:44 <jokke_> as said, we need to figure this out before just going and starting to encode shit and breaking whole world with it
14:22:02 <rosmaita> right, i think we need to have an online meeting so we can talk it out
14:22:04 <jokke_> rosmaita: ++ and wy this seems to break only with the swift driver
14:22:15 <rosmaita> i admit, i am talking out of my butt here since i havne't had time to dig in
14:22:22 <jokke_> there must be something different there we're missing
14:22:23 <rosmaita> but the whole situation is a bit weird
14:22:31 <rosmaita> jokke_: ++
14:23:21 <jokke_> Who ever is interested to dig into this with us, please let me know
14:23:32 <rosmaita> yeah, i would feel much better if we knew why this is only the swift driver that's affected
14:23:33 <jokke_> I'll try to schedule bluejeans for us for tomorrow
14:23:49 <rosmaita> i'm on pto tomorrow, will be travelling
14:23:54 <jokke_> oh ok
14:24:02 <rosmaita> but you don't need me if you want to get started
14:24:12 <jokke_> for monday then unless we find the reason out before that
14:24:16 <abhishekk> because encoding is used in swift driver only
14:24:37 <jokke_> abhishekk: so our swift driver actually encodes the data?
14:24:39 <jokke_> :o
14:24:42 <rosmaita> i'm not clear why it is used there at all
14:24:56 <abhishekk> jokke_, I guess so
14:25:28 <rosmaita> this is a very weird situation
14:25:31 <jokke_> ok lets dig into this and get someone with swift/swiftclient knowledge involved
14:25:51 <jokke_> if there is no reason to encode it, we should get rid of that encoding in the driver
14:25:53 <rosmaita> i think it must be on our end, because swift itself is all py2
14:26:01 <rosmaita> and iirc, they have no plans to make it py3
14:27:09 <LiangFang> swift is not so popular now, is replaced by ceph
14:27:10 <rosmaita> anyway, schedule a monday meeting at a good time for abhishek, i can get up early to make up for not being available tomorrow
14:27:15 <jokke_> #action jokke will try to get hold of some swift folks to see if there is actual reason to do encoding on the data and we decide how we move on after
14:27:21 <abhishekk> I am confused, its not on the master
14:28:01 <jokke_> ok, lets not burn our meeting time for the investigation and find way forward offline
14:28:08 <abhishekk> https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/swift/store.py#L1629
14:28:13 <rosmaita> according to wxy's comment in the bug, it only shows up when we start chunking the object
14:28:55 <jokke_> #topic cache prefetcher redesing
14:29:00 <jokke_> abhishekk: this was your
14:29:54 <abhishekk> jokke_, ?
14:30:12 <jokke_> cache-prefetcher in agenda
14:30:13 <rosmaita> item #4 on agenda
14:30:22 <abhishekk> ohh, sorry
14:30:30 <abhishekk> Just to highlight
14:30:57 <abhishekk> cache-prefetcher is using registry which is deprecated and is in line to remove this cycle
14:31:01 <jokke_> I moved it up from the Open Discussion
14:31:06 <jokke_> yes
14:31:25 <abhishekk> so we need to change the entire design pf cache-prefetcher to use v2 api instead of registry
14:31:45 <jokke_> GR8 ... do we need actual api extensions for this?
14:32:12 <abhishekk> Earlier I have submitted a spec-lite for this but I guess it will require full spec now
14:32:17 <abhishekk> I guess yes
14:32:22 <rosmaita> bummer
14:32:25 <jokke_> abhishekk: ok do you have tomorrow free?
14:32:27 <rosmaita> how bad a change is it?
14:32:36 <abhishekk> Its a mess
14:32:42 <abhishekk> jokke_, I have
14:32:43 <rosmaita> :(
14:33:01 <jokke_> abhishekk: ok, lets get on bluejeans tomorrow and have a look on this
14:33:13 <abhishekk> Fixing cache-manage was a easy task, but cache-prefetcher will be tough one
14:33:23 <abhishekk> jokke_, ping me once you are online
14:33:45 <jokke_> abhishekk: will do ... I'll try to make it as early as possible to not burn your fri night
14:34:08 <jokke_> moving on
14:34:15 <abhishekk> thank you ;)
14:34:37 <jokke_> #topic Feature branch for edge image handling (namely metadata caching)
14:34:56 <GregWaines> :-)
14:34:56 <jokke_> So GregWaines and his guys have been working on this metadata caching spec
14:35:19 <GregWaines> Is this possible to move forward on at least a POC wrt the Glance Edge work ?
14:35:22 <jokke_> and their code is based on old glance which makes all kind of life around this difficult
14:35:25 <GregWaines> in the stein timeframe
14:35:59 <GregWaines> yeah we would .... minimally port our existing code to stein
14:36:14 <jokke_> So I was thinking of kicking up feature branch for it so that the code could be brought up to master while we keep working on the spec
14:36:29 <GregWaines> I like that idea
14:36:33 <abhishekk> +1
14:36:36 <jokke_> I wanted to bring this up to confirm that everyone would be on board with this
14:37:16 <rosmaita> i agree, it's a good idea, it will allow work to continue
14:37:48 <jokke_> I'm not expecting this to be released with Stein, but I do want to see the work moving on and us having common place to experiment and try this out
14:38:06 <abhishekk> sounds good to me
14:38:12 <jokke_> checking out and carrying gerrit reviews for long period of time is not really sustainable
14:38:22 <GregWaines> yeah understand that this will miss Stein wrt official glance branch
14:38:50 <GregWaines> but gets our forked code closer to real glance and somewhere where you guys can look at it more easily
14:39:00 <jokke_> exactly
14:39:01 <smcginnis> ++
14:39:03 <rosmaita> exactly
14:39:24 <abhishekk> ++
14:39:43 <LiangFang> ++
14:39:52 <jokke_> So what I'm expecting from our cores is, please lets not nitpick on the commits to the feature branch, lets do the experientation and reverse if we need to
14:40:04 <abhishekk> GregWaines, I will be happy to contribute as well
14:40:09 <abhishekk> ack
14:40:26 <GregWaines> excellent
14:40:32 <jokke_> and once we get the branch to the point we can agree it being sustainable we can then work on the merge between the branches
14:41:53 <jokke_> I do not want us wasting huge amount of time to try to finetune the code up to the highest standards while we are still working to even agree on the approach in the spec
14:42:02 <jokke_> but this allows the work to continue
14:42:11 <GregWaines> Agreed
14:42:45 <abhishekk> +1
14:43:35 <jokke_> GregWaines: this might mean that we need to spend quite some time to work out patch from the feature branch we can submit to master if we agree on the approach but I think it would be worth of the effort to keep this moving
14:43:58 <GregWaines> I am ok with that
14:44:08 <jokke_> as I expect the git history being a mess we don't want to just merge into master at the end :P
14:44:48 <jokke_> cool
14:45:32 <jokke_> #startvote Everyone ok with the very experimental feature branch for metadata caching? yes, no
14:45:34 <openstack> Begin voting on: Everyone ok with the very experimental feature branch for metadata caching? Valid vote options are yes, no.
14:45:35 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
14:45:39 <jokke_> #vote yes
14:45:47 <abhishekk> #vote yes
14:45:51 <GregWaines> #vote yes
14:45:57 <rosmaita> #vote yes
14:46:01 <LiangFang> #vote yes
14:46:04 <smcginnis> #vote yes
14:46:14 <LiangFang> I am not a core :)
14:46:18 <jokke_> I think that's all of us
14:46:25 <jokke_> LiangFang: this wasn't just for cores
14:46:44 <LiangFang> Ok Ok
14:46:48 <jokke_> I really want us all aboard as a team
14:46:54 <jokke_> #endvote
14:46:55 <openstack> Voted on "Everyone ok with the very experimental feature branch for metadata caching?" Results are
14:46:56 <openstack> yes (6): LiangFang, smcginnis, GregWaines, abhishekk, jokke_, rosmaita
14:47:17 <jokke_> thanks all ... smcginnis I'll put the release patch for that up ;)
14:47:29 <jokke_> #topic Open Discussion
14:47:43 <rosmaita> i was just looking at https://review.openstack.org/#/c/628430/ ... maybe you should tell ken'ichi that this restores visibility to its pre-ocata default, and since most people are on newton, they won't notice a difference when the upgrade to stein if we get back to viz==private on image creation
14:47:46 <jokke_> Anything else. We have generous 12 min ...
14:47:59 <lpetrut> hi, so since we've reached the open discussion, there's something that I'd like to bring up (sorry for not adding a topic to the meeting agenda).
14:48:04 <lpetrut> we'd like to run Glance on Windows. We have customers that are using Hyper-V and HA SMB shares on top of Storage Spaces Direct clusters.
14:48:10 <lpetrut> they'd like to leverage those storage backends for Glance images as well, but the linux cifs client doesn't provide proper support for HA SMB shares at the moment.
14:48:14 <lpetrut> I have a couple of small patches that will allow us to do so: https://review.openstack.org/#/c/630705/ https://review.openstack.org/#/c/630706/
14:48:16 <jokke_> rosmaita: thanks for that input
14:48:18 <lpetrut> we already have CIs voting on Nova and Cinder that test our Nova Hyper-V driver in conjunction with the Cinder SMB driver. We could easily test Glance on Windows along with it, also testing each patch that lands on Glance.
14:48:36 <abhishekk> I have some patches up for reviews, please have a look at those when all of you have some free time
14:48:48 <rosmaita> free time ... what's that?
14:48:56 <abhishekk> :D
14:49:01 <jokke_> lpetrut: having CI on that would be amazing!
14:49:11 <smcginnis> lpetrut: Doesn't look like the changes for that are too extensive. That's great.
14:49:55 <lpetrut> just wanted to know what's your input on this so that we may move forward. If that's acceptable for you, I can file a blueprint and start working on a CI
14:50:14 <rosmaita> the CI would be great, that addresses a lot of my reservations about this
14:50:17 <jokke_> lpetrut: I did notice your message in os-glance channel and the pathes but as they are flagged as WIP and we had some chaos here around the switf driver and good amount of discussion going on around the metadata caching I have kind of skipped those so far
14:50:50 <smcginnis> lpetrut: Was the "WIP" just because you were waiting on the OK to move ahead with these? Or is there more work to be done?
14:51:11 <lpetrut> sure, no worries. indeed, I wanted to have some input before removing the "WIP" tag :) that, and I need to add some tests
14:51:24 <abhishekk> lpetrut, I doubt those patch are tested as well
14:51:47 <abhishekk> https://review.openstack.org/#/c/630706/4/glance/common/wsgi.py@362
14:52:03 <jokke_> lpetrut: cool ... have you actually ran our tests on Win or are these just addressing some specific issues you stumbled upon?
14:52:12 <abhishekk> Here CONF.pipe_handle is used while option name defined as 'pipe-handle'
14:52:40 <lpetrut> abhishekk: that's intended
14:52:42 <smcginnis> Eagle eyes! :)
14:53:03 <abhishekk> :D
14:53:11 <lpetrut> jokke: did some minimal testing for now (uploading some images, spinning up some instances). I'll run the full tempest suite ASAP
14:53:34 <lpetrut> at this point, the only thing that I'm a bit concerned is the glance task feature
14:53:43 <rosmaita> aren't we all
14:53:48 <abhishekk> lpetrut, ohh, eager to know how it understands the difference
14:53:53 <jokke_> lpetrut: cool ... run the unit and the functional tests from the repo as well. I'm very interested to hear how it behaves
14:54:29 <lpetrut> the functional tests may need some changes in order to run on Windows but I think that's manageable
14:54:40 <jokke_> IIRC last person who tried this deemed it way too much of work to keep going, so I'd be very happy to hear if we can claim it actually working
14:55:18 <lpetrut> for now, we only support running under eventlet. I was aiming IIS as well, but had some issues because of chunked encoding and the fact that it was buffering the whole image
14:55:21 <jokke_> and if we get CI to back that up, I have no problems to support such efforts
14:55:38 <lpetrut> awesome, thanks a lot for feedback. I'll keep you posted.
14:55:48 <jokke_> lpetrut: we really only support running under eventlet for multiple reasons
14:56:32 <jokke_> one being the taskflow totally breaking or the full bufferinf of the images under fully fledged web servers
14:56:58 <jokke_> so don't spend too much time banging your head against the wall to get it working under iss
14:56:59 <lpetrut> jokke_: I was actually wondering if that's expected to change in the near future. Some people want to drop eventlet.
14:57:20 <jokke_> lpetrut: yeah they do, we have no intention to spend time on that
14:57:30 <lpetrut> got it
14:57:35 <jokke_> as it seems to be massive change for us
14:57:42 <jokke_> 2min
14:57:52 <lpetrut> would this require a spec?
14:57:52 <abhishekk> with minimum manpower atm
14:58:37 <jokke_> lpetrut: I'll need to look into the patches. At least spec-lite would be amazing just for documenting the work that is going on
14:58:57 <jokke_> we can see what needs to be done and if we want to expand it to full spec
14:59:03 <lpetrut> makes sense. we'll provide some proper documentation once we're there.
14:59:27 <jokke_> ok last minute on its way
14:59:31 <jokke_> anything else?
14:59:46 <abhishekk> nope, please review \0/
14:59:51 <rosmaita> ok
14:59:52 <jokke_> Thanks all!
14:59:57 <jokke_> #endmeeting