20:59:56 #startmeeting swift 20:59:57 Meeting started Wed Oct 31 20:59:56 2018 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:59:58 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:00 The meeting name has been set to 'swift' 21:00:04 who's here for the swift team meeting? 21:00:08 o/ 21:00:11 o/ 21:00:13 hi 21:00:22 o/ 21:00:30 welcome zaneb 21:00:33 o/ 21:00:58 clayg: tdasilva: ping 21:01:09 ohai! 21:01:10 hello! 21:01:29 zaneb: glad you could make it. do you have any particular schedule restrictions? (leaving early, etc) 21:01:48 agenda this week is at... 21:01:49 #link https://wiki.openstack.org/wiki/Meetings/Swift 21:01:54 notmyname: nope, I can be here for the duration 21:02:24 "normal" stuff, but the main topic is the tc vision proposal that zane is championing 21:02:29 zaneb: great! 21:02:53 #topic tc vision document 21:02:55 #link http://lists.openstack.org/pipermail/openstack-dev/2018-October/136031.html 21:03:03 #link https://review.openstack.org/#/c/592205/ 21:03:04 patch 592205 - governance - [DRAFT] Add a Technical Vision statement - 4 patch sets 21:03:24 ok, so the first question is "who actually read it?" :-) 21:03:35 :D 21:03:42 I skimmed it - but then I wanted to start making comments - so I stopped 21:03:47 i read... some of it, anyway 21:03:53 :P 21:04:04 the patch seemed fine, but then people start talking - it makes me want to drink 21:04:09 heh 21:04:20 I think the frustrating part of this kind of thing is that it is very meta. it's not about solving problems; it's for solving problems about how we solve problems. 21:04:36 some people are better at that sort of thing 21:04:51 i remember thinking "event notifications where appropriate" could use some fleshing out -- that's definitely something that swift's been interested in, but it's hard 21:05:08 rledisez: tdasilva: kota_: did you get a chance to read it? 21:05:19 Bidirectional Compatibility seemed to be stepping outside of the service-only purview 21:05:27 it's on the client to figure out how to maintain compatibility with older services, just as it's on the services to figure out how to maintain compat with older clients 21:05:28 briefly, not get in detail. 21:05:36 notmyname: not really, I'm going through it right now briefly 21:05:39 ok 21:05:51 timburke: it does apply to some stuff like Heat and Mistral on the service side 21:06:00 here's my summary ( zaneb, please correct me if I get things grossly wrong) 21:06:03 tc want's to write down some guidelines used for defining the "in" and "out" for openstack. it's doing this by describing and defining basic things about cloud infrastructure and then declaring them to be in scope for openstack. so that if project x shows how they match one or more of these basic principles, they are then in scope for being in openstack 21:06:12 felt like fault-tolerance/high-availability might be another good pillar (and just so happens to be something we're good at ;-) 21:06:14 timburke: and arguably to e.g. microversions and stuff in APIs 21:06:15 swift often has a different perspective on openstack because we pioneered "independently deployable" and also have a data-centric API (instead of provisioning). this document describes several cloud principles that swift does, so zane says this should finally put that debate to bed 21:07:28 zaneb: is that about right? maybe my understanding of the overall purpose is more limited than you have in mind 21:07:36 notmyname: that's a fair summary, although there are other reasons for doing it also 21:07:46 yes. please share :-) 21:08:11 one is, as you said, that the TC currently has no standard for judging which projects should be allowed to join, and it's a pain point for us 21:08:37 but another is that we want to give projects something they can refer to when making design decisions 21:08:39 I've skimmed it, seems notmyname you've had some interesting discussions in gerrit about some definitions 21:09:05 mattoliverau: I tried to avoid the biggest traps, I think :-) 21:09:08 and a lot of that stuff have previously only been at the level of folklore that people picked up by osmosis, if at all 21:09:41 zaneb: meaning that if eg we have a new feature we are designing, we could compare it against this list to see if it's good or bad? 21:09:53 well, appropriate or not 21:10:12 and it'd be nice to have a kind of framework within which the services sit so that the people independently designing them eventually produce something coherent 21:10:54 that last thing sounds terrific. design a system so that independent parties converge on a cohesive whole. (also sounds really hard) 21:11:35 yes, really hard to get it happen on purpose, but absolutely not going to happen by accident :) 21:11:47 ^ that's rich 21:11:48 designing converging systems in general is hard -- see our troubles with replication, dark data, and ghost listings :P 21:12:13 zaneb: so are you thinking this document will guide (or advise) feature development? 21:12:15 it's even worse when the system's made up of people instead of programs 21:12:40 notmyname: to some extent, yes, but to different degrees for different projects 21:13:08 and a lot of it is stuff that we old-timers already know, but hadn't written down in one place 21:13:47 TBH I'm struggling with ... well, I'm struggling with how to express my thoughts via text chat :-) 21:14:06 meme? interpritive dance? 21:14:12 lol 21:14:41 lol 21:15:00 one thing I personally would love to get is events out of every service. imagine if we'd had Storlets but for everything 5 years ago - we'd have invented Lambda before AWS did 21:16:00 I'm very concerned when someone starts defining boundries in the openstack ecosystem, because swift has oftentimes been put in the "other" group. that's why I pay attention to these sorts for discussions. but on the other hand, it's hard to see how it matters on an ongoing day-to-day basis 21:16:59 I think that might be baggage - I feel more in-crowd cool kids these days 21:17:13 clayg: it totally could be 21:17:51 fwiw I think everyone else has a lot to learn from Swift, and allowing it to be defined as 'other' has prevented the rest of OpenStack from doing that as much as we should 21:18:45 so as I said in the email, I hope drawing the boundary where it is in this doc will help people to *stop* seeing Swift as 'other' 21:18:53 :hugs: 21:19:12 clayg: that's the emoji thing I'm looking for! :-) 21:19:15 zaneb: what would be most helpful from "us" (and by us I mostly mean other people besides me) 21:19:32 ... but especially from notmyname :P 21:19:40 :hugs: 21:19:50 heh 21:19:52 lol 21:20:02 mattoliverau! mattoliverau is helpful 21:20:06 lol 21:20:37 lol 21:20:52 * notmyname waits for zaneb before asking more questions 21:21:49 the most helpful thing for me is people engaging with the review, because in the end this will be successful if people feel like we all came up with it together, and it's not something handed down from the TC 21:22:17 zaneb: the outreach you're doing to specific projects in great. thank you 21:22:19 if I start typing gibberish, it's because a miniature human is attacking the keyboard 21:22:40 I mean, more gibberish than usual 21:22:48 clayg: timburke: you both said you had some comments but help back from posting them 21:23:04 were they about the doc in general, any specifics in it, or just the commentary about the doc? 21:23:05 lol 21:23:46 notmyname: honestly 9/10 of my "comments" were "hell, yeah!" the others were like "no YOU'RE WRONG, this is good." 21:24:17 I'm paraphrasing 21:24:33 lol 21:25:10 i think mostly just the comments i made here. seemed like actually posting would mostly add more noise (to what will definitely be a noisy review) rather than help, but i could be wrong 21:25:57 timburke: you mentioned fleshing out the 'event notifications where appropriate' - would be very interested in what sort of stuff you'd like to see there 21:26:17 mostly just what "where appropriate" means 21:26:38 because i honestly don't know where the appropriate line would be in swift 21:26:44 timburke: in my experience with openstack, it's very good to have noisy reviews early in order to get a right version of the doc from the start (but like clayg, that may be my own historic baggage) 21:26:52 that section is kind of toned down because that is TOTALLY MY HOBBY HORSE and I would write the whole thing about just that if I could ;) 21:27:42 "on every swift operation" feels *incredibly* noisy, like we'd almost certainly cause whatever message queue we're putting those notifications in to fall over 21:27:57 TIL https://en.wikipedia.org/wiki/Hobby_horse#Other_meanings 21:28:36 clayg: it's fortunate that didn't take a much darker turn :D 21:29:01 timburke: maybe we should let users filter at the source? 21:29:21 zaneb: do you think this document will take a turn towards more "provisioning-only" language? one commenter seemed to want to move it in that direction 21:29:48 notmyname: i was noticing that too... 21:30:27 notmyname: by that you mean just-Nova-providing-a-VPS? 21:30:49 aka the "free VMWare" model? 21:30:54 more generally, like "you ask openstack to give you resources that you can then use directly" 21:31:17 oh, ok 21:31:58 I have two general concerns with things that try to define openstack. the first is provisioning vs "data-plane" api disctinctions. the second is a focus on compute-only or compute-first language 21:33:04 ... so many half typed sentences ... 21:33:12 imo there's such a mixture of different things that it's never going to make sense to say openstack is control-plane-only 21:33:12 lol 21:33:31 I actually need to cut out (happy halloween everybody!) 21:33:42 lol 21:33:45 notmyname: yeah, i kinda want to point kevin at https://governance.openstack.org/tc/reference/principles.html#openstack-primarily-produces-software and say that while others may reimplement the swift protocol (we have no control over that, they can make their own decisions), that's not Swift 21:33:52 +1 to everything zaneb says 21:34:14 clayg: thanks for joining while you could. I definitely appreciate your comments and perspective 21:34:25 notmyname: was there anything else on the agenda? 21:34:30 rledisez: ohai!? 21:34:37 I'll read the logs 21:34:43 that comment from Kevin was interesting food for thought, but I don't think it's going to change the direction of the document in any major way 21:34:45 clayg: nah, jsut "normal" stuff like "go land the s3api patches" and review losf :-) 21:34:54 k 21:35:00 l8r 21:35:04 clayg: thanks o/ 21:35:19 clayg: ohai??? 21:35:50 kota_: rledisez: what do you think about this doc (or this discussion)? we haven't heard from you yet 21:36:26 rledisez: don't worry, i speak clayg. "oh, hi there!" and in this particular context, "sorry we couldn't chat" 21:37:04 so, i just read it. it is interesting, i'm not sure i see how it could drive everydays decissions about feature/implementation. if we need something, will we really restict ourself to what this document allow? 21:37:13 timburke: thx for the translation :D 21:37:21 notmyname: i have (probably) same concerns with you on the definition of the openstack because swift provides data-centric apis. 21:37:33 rledisez: yeah, I completely agree about the feature development influence 21:37:37 and also, Storlets does... 21:38:24 kota_: yeah. did you notice anything that gives you concern? 21:38:33 notmyname: but I like the turn it takes about the place of Swift inside Openstack 21:38:42 just a guideline could be fine but if it'll be restriction, I could not sure I could follow it, maybe? 21:39:26 kota_: from what I understand of it, and from what zaneb has said, I do not expect there to be any definitions that turn in to "data-centric is bad" 21:39:47 notmyname: ok, 21:40:00 as far as features in Swift go, I'd be most interested in hearing if the things documented are consistent with the decisions about features you've made in the *past*, so we can hand the knowledge on to some of the newer teams 21:40:05 thanks for clarification. 21:40:23 zaneb: ah, yes, that cuts to the chase 21:41:06 zaneb: that sounds nice 21:41:17 zaneb: I question some of the "hardware virtualization" section. it mentions storage, but what we do with object storage is sortof hard to fit into a "storage virtualization" world. at least much harder than cinder 21:41:19 notmyname: that reminds me, we should discuss you comment about regions, but not sure if this is the time 21:41:47 zaneb: sure. we can discuss the regions in the review, I think. it doesn't seem like a major issue to me 21:41:55 ok 21:42:57 notmyname: IMHO hardware virtualisation is necessary for a cloud, but it also tends to scale in coarse-grained ways (from the application perspective) 21:43:18 so we need both that and things like Swift that provide more fine-grained scaling 21:44:21 yeah, just the term regions means different things in Swift and keystone, they can fit together depending on your deployment topology but doesn't have to. We need a venn diagram :P Just like Swift containers and containers (though they don't overlap at all) :P 21:44:30 yeah. the HW virtualization works well in the "free vmware" mindset. but swift (object storage) is really more about not thinking about hardware at all (from the app perspective). and that normally means the service is runnign on bare metal 21:44:32 i think it depends on the definition. IMO Swift API virtualized actual hardware and data placement. 21:44:41 we could assume. 21:45:24 kota_: yeah :-). I think the definition is vague enough that we could make what swift does fit in to it. swift abstracts hard drives. you can swap them out and everything just keeps working. 21:45:34 notmyname: i think that whole section doesn't really apply, because we don't 21:45:50 have a service "provided by a specialised piece of hardware" 21:45:54 kota_: you could say that about almost anything ;) 21:45:57 "a hard drive" 21:46:47 Lol @ tburke is my opposite of Obama’s “Luther” 21:46:55 my 2c: they're separate things; we need both in OpenStack 21:47:19 zaneb: I'm not really sure what to do next. my summary of what we've said in here is that most people are mostly ok with it (or at least not too concerned about the current version). we have some smaller wording questions, but nothing that's very troubling 21:47:57 zaneb: so to repeat what clayg said earlier, what do you need from us now? have we given you anything that will help you? do we need to do something else? 21:48:36 based on the comments from the rest of the team, I'm ok with putting my own +1 on it as a swift representative. 21:48:45 comments on the review if you have changes to be addressed would be great 21:49:05 there will be another patchset coming soon, but +1s on that would be great 21:49:19 ok. I'll wait until the next patchset 21:49:58 we have a Forum session at the summit, so hoping to get folks along to that to discuss where we're at and what we need to do to finalise it 21:50:23 * kota_ will be there. 21:50:26 ok. I'll look for that on the schedule 21:50:33 (my time in berlin is a bit limited) 21:50:57 #link https://www.openstack.org/summit/berlin-2018/summit-schedule/events/22818/vision-for-openstack-clouds-discussion 21:51:26 ok, last few minutes (thanks for being patient with a longer meeting this week) 21:51:40 really appreciate everyone's input, thank you! 21:51:41 zaneb: thank you for taking the time to specifically ask us about this doc 21:51:50 #topic other topics 21:52:03 #link https://wiki.openstack.org/wiki/Swift/PriorityReviews 21:52:16 priority reviews page is looking pretty similar to where it's been 21:52:24 thanks zaneb for coming, listening and taking the time, great work on the vision work 21:53:01 timburke: I'm wondering about the s3api patches. I'd really like to land them soon. maybe if we threaten to jsut land them then maybe mattoliverau and kota_ will look at them? ;-) 21:53:11 mattoliverau: my pleasure, thanks! 21:53:50 rledisez: I saw new patches (or patch sets) from alex about losf? 21:53:59 timburke was doing such a good job adapting proxy that at one point got my manager to write down a working py3 proxy as a checkpoint on 10/10. But now he's away doing god knows what that Swiftstack needs more... Like completely random things. 21:54:00 it looks 3 patches are remaining 21:54:08 for s3api in prioriry 21:54:13 kota_: yeah 21:54:13 Yeah, sorry, I've joined a new virtual team (happens every 6 months or so) so been busy ramping up. So haven't been reviewing as much as I should. I'll make sure I find work time to devote to more reivews :) 21:54:20 notmyname: merge 'em all! 21:54:23 :-) 21:54:53 notmyname: maybe, tbh i was not focused on that. but as tim said, "merge 'em all!" 21:55:21 zaitcev: i think we *are* really close to a working py3 proxy (at least for some requests?) 21:55:45 having tests that verify that will be interesting, though :-/ 21:56:19 i think I could look at 2 patches w/o patch 592231 but only 592231 is still not sure it goes fine way... 21:56:20 https://review.openstack.org/#/c/592231/ - swift - s3api: Include '-' in S3 ETags of normal SLOs - 3 patch sets 21:56:41 i'd like to get another person's opinion for that 21:57:23 kota_: ok. could you leave a comment with your concerns or questions in gerrit? right now only tdasilva is on it 21:57:38 so between the two of you, maybe we can get it landed quickly 21:58:04 and sorry of my slow pace for the reviews, various things goes in work and private... 21:58:13 notmyname: ok 21:58:41 kota_: do not worry. I completely understand 21:59:35 zaitcev: I also wanted to say thanks for your py3 work. it's often tedious and thankless and hard to see progress, but as you know it's important. so thanks for working on it 21:59:41 zaitcev: also, good luck with IBM :-) 22:00:16 ^ +100 to both these things 22:00:17 notmyname: I'm quite interested in IBM's public cloud, they have some storage running in there. Not sure if it's Swift or not. 22:00:51 zaitcev: the softlayer stuff is still, I think. eveything else is cleversafe (spectrum scale is the name now, i think) 22:00:58 ok, we're at full time 22:00:58 and I guess thats time. 22:01:02 thanks for coming today 22:01:14 thanks for working through the tc doc 22:01:18 and thanks for your work on swift! 22:01:22 #endmeeting