21:01:00 <notmyname> #startmeeting swift
21:01:00 <openstack> Meeting started Wed Jun  6 21:01:00 2018 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:01:01 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:01:03 <openstack> The meeting name has been set to 'swift'
21:01:05 <notmyname> who's here for the swift team meeting?
21:01:15 <m_kazuhiro> o/
21:01:17 <alecuyer> o/
21:01:18 <tdasilva> hi
21:01:19 <rledisez> o/
21:01:20 <kota_> hello
21:01:33 <acoles> hi
21:01:33 <mattoliverau> o/
21:01:48 <notmyname> welcome, all
21:01:59 <notmyname> I'm not sure if this meeting will be really short or really long
21:02:07 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift
21:02:15 <notmyname> there's nothing specific on the agenda for this week
21:02:19 <notmyname> so...
21:02:24 <notmyname> #topic open discussion
21:02:33 <notmyname> what's up? anything to bring up this week in the meeting?
21:03:02 <rledisez> LOSF, here it is: https://review.openstack.org/#/c/571763/ ;)
21:03:02 <kota_> lol, agenda is only open discussion
21:03:06 <notmyname> alecuyer: when can we see more LSOF code in gerrit?
21:03:09 <notmyname> oh nice!
21:03:26 <notmyname> that... is a really big patch ;-)
21:03:32 <alecuyer> it is here, Romain kicked me until I pushed it ;) I am not happy with everything in there.. but,
21:03:38 <tdasilva> passes probe tests ;)
21:03:49 <notmyname> tdasilva: lol
21:04:01 <alecuyer> it applies to master, and it does run
21:04:05 <rledisez> i guess something is wrong then ;)
21:04:15 <tdasilva> rledisez: lol
21:04:20 <alecuyer> the patch is not so big
21:04:31 <alecuyer> I mean, there are generated files there
21:04:45 <notmyname> it's only half as big as sharding. it's tiny! ;-)
21:04:48 <acoles> rledisez: alecuyer I started reviewing https://review.openstack.org/#/c/447129/10 today
21:05:02 <alecuyer> acoles: ok, thanks!
21:05:31 <rledisez> acoles: thx! it needs some more work, but an opinion on the way it does things is really welcomed
21:05:54 <alecuyer> notmyname: it touches much less existing files than before. And the kvfile.py is down from about 4000 lines (diskfile.py copy) to about a 1000 (inherits from classes in diskfile)
21:06:14 <acoles> rledisez: it seems heading the right way, I'll post some comments tomorrow I hope
21:07:19 <rledisez> acoles: if you see a better way to patch tests, i'll take it. for now i'm not really happy with the way I did it
21:07:32 <mattoliverau> Wow nice work, /me will definitely be a coffee before I starting tho
21:07:43 <mattoliverau> *need
21:07:54 <kota_> mattoliverau: +1, me too
21:09:47 <notmyname> rledisez: alecuyer: this is great to see. I know you have been working on the migration in your clusters, but you weren't planning on sharing that custom work
21:10:11 <notmyname> in general, how does this work in the sense of migrating to it (or away from it)
21:10:37 <rledisez> we only planned one way: to it ;)
21:10:40 <notmyname> heh
21:11:03 <notmyname> I've been thinking on the ordering of things, and LSOF vs getting rid of rsync is the big question. seems to me that with an rsync replacement, we may be able to more easily move to a different on-disk format
21:11:33 <rledisez> the migration is done in-place. sending new write to losf, but reading both from losf and filesystem. and a process scan the filesystem to write to losf
21:12:04 <rledisez> the downside of that migration process is that it disable the REPLICATE command, so no rebalance/reconstruction while a server is being converted
21:12:15 <rledisez> I let alecuyer talk more in details if needed
21:12:44 <alecuyer> well, what your said is correct ;)
21:12:47 <rledisez> alecuyer: do you think it would be possible to improve the in-place migration? notmyname: would you be interrested with that.
21:12:51 <rledisez> ?
21:13:33 <torgomatic_> I'm sort of surprised you don't just use replication to do it. Take a node out of the ring, reconfigure it for LOSF, put it back in, and wait...
21:14:53 <rledisez> torgomatic_: the issue is the time it would take. the concerned cluster basically does not work anymore, at least for anything related to replication. it is so slow that it timeout fro everytwhere (think 5, even 10 or 20 minutes for a REPLICATE request)
21:15:08 <notmyname> honestly, my personal concern is coordinating the work so that we don't do too much and once and not make progress on anything. we know the LSOF needs migration, but also that there's other planned work ("tsync" that may help). but LSOF is obviously further along
21:15:08 <alecuyer> Well it could be improved, probably REPLICATE could be supported but I don't know if it's a good general purpose tool
21:15:14 <alecuyer> for migration
21:16:02 <alecuyer> notmyname: yes. I would be happy to get feedback before I work further on this version, if we can agree on a way to do things
21:16:13 <alecuyer> (lots of work still, tests… etc)
21:16:31 <torgomatic_> I mean, in-place migration is definitely going to be better; I was just being lazy and considering ways to avoid work ;)
21:17:10 <torgomatic_> also, at some point I got the LOSF RPC server up and running, and I made a branch in vagrant-swift-all-in-one with the necessary dependencies, so that may be a good starting point for anyone else wanting to poke at it
21:17:13 <notmyname> alecuyer: that's why I'm glad you've pushed code before it's "perfect" so we can work on figuring it out together
21:17:14 <torgomatic_> #link https://github.com/swiftstack/vagrant-swift-all-in-one/tree/losf
21:17:28 <notmyname> torgomatic_: nice, thanks
21:18:24 <alecuyer> notmyname: yes, far from perfect which made it hard to push ;) but here it is and I'll work to make it better
21:18:25 <kota_> nice
21:18:44 <alecuyer> torgomatic_: there are some changes in the golang code in this version, but the way to build it has not changed
21:19:19 <alecuyer> and ill look at the vagrant :)
21:20:45 <rledisez> an other interesting question is "grpcio==1.3.3" (can't upgrade grpcio because of eventlet)
21:20:54 <notmyname> ah right
21:22:04 <torgomatic_> is there a library out there that only does the gRPC encoding and decoding, but leaves the IO up to the user?
21:22:04 <tdasilva> rledisez: do you know if the evenlet issue has been fixed in a recent version of grpcio?
21:22:11 <alecuyer> they have a patch to make grpc work with gevent https://github.com/grpc/grpc/issues/4629 but not eventlet
21:22:18 * torgomatic_ is not familiar with the gRPC ecosystem
21:23:00 <alecuyer> torgomatic_do you mean, use protobuf and something else for network communication ?
21:24:31 <torgomatic_> alecuyer: yes
21:24:45 <tdasilva> rledisez: nevermind, was under false impression...
21:25:33 <torgomatic_> like I said, though, I don't know much about this ecosystem, so that may be an absolutely terrible idea
21:26:10 <alecuyer> torgomatic_: we could do that, they're separate libraries. So it's not a terrible idea at all. I started with grpc at a time where it worked fine… then it broke after 1.3.1.
21:26:25 <alecuyer> On the other hand, I also wonder if we need eventlet on the object servers
21:26:36 <notmyname> alecuyer: eventually, we definitely want to remove that, too
21:26:47 <notmyname> again, just a matter of figuring out the right order to do stuff :-)
21:26:57 <alecuyer> right :)
21:28:12 <notmyname> is this a good summary? alecuyer has proposed the remainder of the LSOF code, but it's definitely a WIP. alecuyer will look at torgomatic_'s saio work to get it running. torgomatic_ will likely be looking at the patch, but others should too. especially to think about migrations
21:28:40 <notmyname> there's the higher level question of LSOF migrations and how that should be ordered against tsync work (ie remove rsync)
21:29:12 <alecuyer> I think it's a good summary. I would add, before thinking about migrations, that I would like to have input on the general design, if there are things you feel should be done differently
21:29:16 <kota_> sounds good to me.
21:29:28 <notmyname> alecuyer: right!
21:29:57 <alecuyer> I do have a few specific questions, but maybe I can put that in the review and we discuss this outside the meeting
21:30:15 <notmyname> yes
21:30:30 <notmyname> does anyone else have questions for alecuyer and rledisez about LSOF right now?
21:30:32 <rledisez> notmyname: what is the status of tsync? is there something in progress (eg: https://github.com/swiftstack/tsync). nothing started yet? how big do you think it will be? (tsync is quite small, but not complete)
21:31:40 <notmyname> rledisez: there's lots of good ideas for tsync, but nothing really done right now
21:31:42 <acoles> alecuyer: review discussion can be hard to follow - gerrit is not great for that. another idea might be an etherpad. also not great, but discussion persists there beyond each patchset.
21:31:43 <mattoliverau> Nice work guys, and thanks alecuyer  for pushing it up as it is!
21:32:32 <alecuyer> acoles: right, I will make an etherpad page
21:32:48 <rledisez> notmyname: ack
21:33:13 <notmyname> rledisez: see also https://etherpad.openstack.org/p/swift-rebalance from the ideas wiki
21:33:33 <notmyname> ideas wiki: https://wiki.openstack.org/wiki/Swift/ideas
21:34:09 <rledisez> yeah, I remember that, nicely analyzed :)
21:34:56 <tdasilva> acoles, alecuyer we could also have a video meetup if folks like that idea
21:35:02 <alecuyer> finally and I'll stop tallking about this unless you have questions :) there's a readme to get it running https://review.openstack.org/#/c/571763/2/README.LOSF
21:35:23 <notmyname> tdasilva: I feel like we should spend some time looking at the code first. but it's definitely a good idea
21:36:13 <notmyname> is there anything else to bring up in the meeting today?
21:36:20 <notmyname> or shall we call it done?
21:36:22 <tdasilva> notmyname: yep, agree.
21:36:24 <kota_> wait
21:36:30 <notmyname> kota_: what's up?
21:36:30 <kota_> just a small note
21:36:32 <kota_> https://review.openstack.org/#/c/564700/
21:36:38 <kota_> #link https://review.openstack.org/#/c/564700/
21:37:03 <kota_> i recently has been working on setting up s3api compatibility check with timburke
21:37:23 <kota_> and it's close to get successful with any gates.
21:37:38 <kota_> and I like to add it as the gate job (even it's non-voting)
21:37:56 <kota_> if you have any opinions to that, please let me know at the patch.
21:38:07 <kota_> it will creates like...
21:38:22 <kota_> http://logs.openstack.org/00/564700/27/check/swift-tox-func-s3api-ceph-s3tests-tempauth/2742951/output/ceph-s3-summary.log
21:39:25 <notmyname> I like it. thanks
21:39:34 <kota_> i think it's worth to know which s3 apis are supported at the commit.
21:39:48 <mattoliverau> +1 nice work
21:39:53 <kota_> just a notification from me :)
21:40:16 <notmyname> kota_: oh, I moved https://review.openstack.org/#/c/571561/ to swift from swift3. so it puts the s3 compat info in the docs
21:40:37 <kota_> yup
21:40:38 <timburke> ugh... that reminds me: i've got a whole bunch of patches i need to move...
21:40:45 <notmyname> speaking of patches (and our review change last week)...
21:40:47 <kota_> notmyname: is it ready to review?
21:40:59 <notmyname> does anyone have any patches that need extra eyes (ie more than 1 +2)?
21:41:01 <notmyname> kota_: yes
21:41:03 <kota_> notmyname: i think you added workflow -1 at the swift3 for some reasons.
21:41:19 <notmyname> kota_: correct. in part because of the planned move. in part because I was being silly
21:41:36 <notmyname> kota_: that is, I had reasons that I've changed my mind on since then
21:41:50 <kota_> got it. I'll take a look.
21:42:24 <notmyname> thanks
21:42:45 <notmyname> I'm not seeing anyone else raise other issues to discuss this week
21:42:50 <m_kazuhiro> I think patch 517389 is ready for review.
21:43:08 <notmyname> ah, sorry. I didnt' move patchbot in here
21:43:28 <m_kazuhiro> https://review.openstack.org/#/c/517389/
21:43:36 <m_kazuhiro> Oops, merge conflict...
21:44:03 <m_kazuhiro> Sorry I will fix it soon. But I think the patch can be reviewed.
21:44:13 <notmyname> thanks
21:44:45 <notmyname> I think we're done, then
21:44:59 <notmyname> thanks everyone for coming. and thanks for your work on swift
21:45:07 <notmyname> #endmeeting