Wednesday, 2017-03-15

*** gyee has joined #openstack-swift00:02
*** hoonetorg has quit IRC00:22
*** dja has joined #openstack-swift00:29
*** catintheroof has joined #openstack-swift00:32
*** hoonetorg has joined #openstack-swift00:35
*** Renich_ has joined #openstack-swift00:37
Renich_timburke: btw, thanks for the help00:39
*** Renich has quit IRC00:39
Renich_timburke: it didn't work... no idea what's up. Might try, later on, at #rackspace or something00:39
Renich_bye00:39
*** Renich_ has quit IRC00:39
*** Renich has joined #openstack-swift00:40
*** zhurong has joined #openstack-swift00:53
*** m_kazuhiro has joined #openstack-swift00:59
m_kazuhirogood morning01:05
*** Renich has quit IRC01:05
*** Renich has joined #openstack-swift01:06
notmynamemattoliverau: nice!01:08
mattoliveraum_kazuhiro: morning01:08
m_kazuhiromattoliverau: morning!01:14
*** hoonetorg has quit IRC01:16
*** Renich has quit IRC01:20
*** catintheroof has quit IRC01:24
*** vint_bra has joined #openstack-swift01:27
tone_zrtmorning!01:28
mattoliverautone_zrt: mornin01:30
*** hoonetorg has joined #openstack-swift01:30
*** catintheroof has joined #openstack-swift01:30
notmynameI'd like to welcome mahatic to swift core :-)01:31
mattoliverau\o/01:31
mattoliveraumahatic: congrats \o/01:31
tone_zrtmattoliverau: good afternoon :)01:31
mattoliverau^ when you wake up :)01:32
mahaticmattoliverau: thanks, I'm up for a (very) early morning meeting  internally :)01:32
notmynameyeah, it's pretty early for mahatic, but I wanted to say something for her morning and before tomorrow's team meeting when she'll be asleep01:32
*** catintheroof has quit IRC01:32
*** JimCheung has quit IRC01:33
m_kazuhirotone_zrt: morning!01:35
tone_zrtm_kazuhiro: morning!01:35
m_kazuhiromahatic: congrats!01:36
mahaticm_kazuhiro: thanks!01:38
*** vint_bra has quit IRC01:51
*** vint_bra has joined #openstack-swift01:54
*** aluria has quit IRC01:56
*** sams-gleb has joined #openstack-swift01:57
*** vint_bra has quit IRC02:01
*** sams-gleb has quit IRC02:01
jrichlimahatic: Congrats :-)02:04
openstackgerritThiago da Silva proposed openstack/swift master: Symlink implementation.  https://review.openstack.org/23216202:06
claygmahatic: welcome to teh suck02:07
mahaticclayg: lol, thanks?02:07
tdasilvamahatic: congratulations!02:11
*** tanee is now known as tanee_away02:14
*** dja has quit IRC02:14
*** tanee_away is now known as tanee02:14
*** dja has joined #openstack-swift02:16
*** sweeper has quit IRC02:40
*** dja has quit IRC02:40
*** dja has joined #openstack-swift02:42
*** SkyRocknRoll has quit IRC02:43
*** sweeper has joined #openstack-swift02:53
*** bkopilov has quit IRC02:58
*** gyee has quit IRC03:01
pdardeau_mahatic: congratulations!! :)03:07
*** JimCheung has joined #openstack-swift03:10
*** JimCheung has quit IRC03:15
*** gyee has joined #openstack-swift03:17
*** redbo has quit IRC03:19
openstackgerritJanie Richling proposed openstack/swift master: Log the correct request type of a subrequest downstream of copy mw  https://review.openstack.org/44460403:20
*** dmorita has quit IRC03:25
*** dmorita has joined #openstack-swift03:27
*** redbo has joined #openstack-swift03:27
*** ChanServ sets mode: +v redbo03:27
*** Jeffrey4l has quit IRC03:28
*** dmorita has quit IRC03:31
*** Jeffrey4l has joined #openstack-swift03:41
jrichlihow do you edit a commit message directly from gerrit?  I can never find the button for that ...03:41
claygjrichli: I always have to click on the commit message in the list of files03:42
claygthen next to the list of patch sets - there's a notepad/edit button03:42
jrichliclayg: thanks, i had done that, but wasn't on the latest patch. i see it now03:42
*** m_kazuhiro has quit IRC03:43
openstackgerritJanie Richling proposed openstack/swift master: Log the correct request type of a subrequest downstream of copy  https://review.openstack.org/44460403:44
*** dmorita has joined #openstack-swift03:50
*** dmorita has quit IRC03:54
*** links has joined #openstack-swift03:56
*** klrmn has quit IRC03:56
*** sams-gleb has joined #openstack-swift03:59
*** sams-gleb has quit IRC04:03
*** m_kazuhiro has joined #openstack-swift04:05
*** dmorita has joined #openstack-swift04:16
*** dmorita_ has joined #openstack-swift04:18
*** dmorita has quit IRC04:18
*** psachin has joined #openstack-swift04:20
*** dmorita_ has quit IRC04:22
*** dmorita has joined #openstack-swift04:23
*** dmorita_ has joined #openstack-swift04:25
*** dmorita has quit IRC04:27
openstackgerritOpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements  https://review.openstack.org/8873604:29
*** dmorita_ has quit IRC04:29
*** gkadam has joined #openstack-swift04:45
*** psachin has quit IRC05:00
*** zhurong has quit IRC05:00
*** bkopilov has joined #openstack-swift05:05
kota_good afternoon05:06
mattoliveraukota_: good arvo :)05:06
kota_mahatic: welcome to the swift core team :-D05:07
kota_mattoliverau: o/05:10
*** dja has quit IRC05:16
*** psachin has joined #openstack-swift05:17
*** JimCheung has joined #openstack-swift05:27
mahaticjrichli: tdasilva pdardeau_ kota_ thank you!05:28
*** psachin has quit IRC05:49
*** gyee has quit IRC05:51
*** JimCheung has quit IRC05:55
*** JimCheung has joined #openstack-swift05:55
*** psachin has joined #openstack-swift05:57
*** adriant has quit IRC05:59
*** JimCheung has quit IRC06:00
*** sams-gleb has joined #openstack-swift06:01
*** cshastri has joined #openstack-swift06:05
*** sams-gleb has quit IRC06:05
claygnotmyname: look I typed things!  Moar Better Faster Rebalance -- https://etherpad.openstack.org/p/swift-rebalance06:24
*** ChubYann has quit IRC06:28
*** dmorita has joined #openstack-swift06:43
mattoliverauclayg: nice.. but also your realise there is a high chance that _any_ temp name may end up being the actual name.. long live tsync ;P06:47
*** dmorita has quit IRC06:48
*** zhurong has joined #openstack-swift06:58
openstackgerritKota Tsuyuzaki proposed openstack/swift master: TestObjectController refactoring  https://review.openstack.org/44046607:07
*** m_kazuhiro has quit IRC07:07
openstackgerritKota Tsuyuzaki proposed openstack/swift master: Fix (un)patch_policies  https://review.openstack.org/44093607:10
kota_just rebased07:11
*** tonanhngo_ has joined #openstack-swift07:12
*** tonanhngo has quit IRC07:14
*** gkadam is now known as gkadam-lunch07:29
*** hseipp has joined #openstack-swift07:39
*** silor has joined #openstack-swift07:40
*** sams-gleb has joined #openstack-swift07:41
*** silor1 has joined #openstack-swift07:43
*** tesseract has joined #openstack-swift07:44
*** silor has quit IRC07:46
*** silor1 is now known as silor07:46
*** tonanhngo_ has quit IRC07:47
*** SkyRocknRoll has joined #openstack-swift07:50
*** silor has quit IRC07:50
*** tonanhngo has joined #openstack-swift07:53
*** tonanhngo has quit IRC07:54
*** SkyRocknRoll has quit IRC07:56
*** pcaruana has joined #openstack-swift08:05
*** jcaron has joined #openstack-swift08:15
*** zhurong has quit IRC08:28
*** d0ugal has joined #openstack-swift08:32
*** geaaru has joined #openstack-swift08:35
*** amoralej|off is now known as amoralej08:37
acolesgood morning08:43
acolesmattoliverau: great write up!08:45
acolesmahatic: congrats and welcome to the team :D08:45
acolesmattoliverau: I have a script that installs the swift test users into keystone. It may be a little dated but I used it recently. https://gist.github.com/alistairncoles/b5b905f7f68d82ae242e59982ed902bd08:47
acolestdasilva: did I promise you that link ^^ a long time ago ?08:48
*** dja has joined #openstack-swift08:55
*** tonanhngo has joined #openstack-swift08:56
*** tonanhngo has quit IRC08:57
*** kei_yama has quit IRC08:58
*** cbartz has joined #openstack-swift09:01
acoleskota_: thanks for your comments on the composite ring etherpad09:09
kota_acoles: yeah, i'm still now writing though09:09
mahaticacoles: thanks :D09:13
*** furlongm has joined #openstack-swift09:20
*** furlongm has quit IRC09:20
mattoliverauacoles: cool, mind if I steal that? ;)09:27
acolesmattoliverau: nope, go ahead09:27
*** zhurong has joined #openstack-swift09:28
acolesmattoliverau: there is also this which includes setting up the endpoints, services etc https://gist.github.com/alistairncoles/eace39b1e19c6ce708edd8cd9e951ff209:30
acolesbut not the keystone install/uwsgi goodness you have!09:30
acoleskota_: (brainstorming...) do we need a different ring builder file suffix/type for composite ring? could we add keys to the builder dict that is pickled and use those to differentiate composite from normal ring when loading the builder file?09:33
*** gkadam-lunch is now known as gkadam09:34
kota_acoles: making a brand-new builder file? ah maybe like as `swift-ring-builder composite-ring.builder zip composite-ring.json`09:35
kota_or, `swift-ring-builder composite-ring.builder add_ring <some ring>`?09:36
*** zhurong has quit IRC09:38
acoleskota_: yeah, something like that. TBH I'm not sure what clayg had in mind (he'll tell me I am talking nonsense!)09:39
acolesbut my thinking is that composite-ring.builder could maintain list of component rings09:40
kota_ah, so we will maintaine 2 types of builder file with same suffix09:40
*** tonanhngo has joined #openstack-swift09:41
acoleswell, we add keys to the dicts that are serialized to builder file. but yes, effectively two kinds of builder files.09:41
*** tonanhngo has quit IRC09:41
*** zhurong has joined #openstack-swift09:43
* kota_ is thinking09:43
acolesthen maybe decorate each builder command with something like @composite_unsupported which raises error if the new composite-related keys are found in the dict (hmmm, or something more elegant)09:44
kota_sounds nice09:46
kota_and some command should be also @normal_unsupported09:46
kota_currently, i'm thinking the entry point instaed of *create* command09:47
kota_ah...09:47
kota_it may be ok, we set create as the entry point even if it's composite09:47
kota_no09:48
kota_no?09:48
kota_part_power should be ok to specify because the rings in the composite ring should have same part power09:48
kota_min_part_hours too09:49
kota_ah, it may be ok if difference on min_part_hours exit09:49
kota_probably the pain point is *replicas* for the create command09:50
acolesor we use add_ring to bootstrap if builder file does not exist?09:50
acoleshmmm, not great because we might like add_ring to error if builder file does not exist :)09:51
kota_acoles: true09:52
*** zhurong has quit IRC09:53
kota_or, if we can take an option like --composite and then, skip <replicas> and <min_part_hours> for args?09:53
kota_`swift-ring-builder c.builder create --composite <part_power>`09:53
*** dmellado has quit IRC09:54
*** amoralej has quit IRC09:54
kota_or even, we don't need to take any args on --composite option09:55
kota_might be09:55
acoleskota_: we'll find a way09:57
acolesreplicas == 0 -> composite09:57
acolesmaybe not obvious enough09:57
acolesjust add a new command09:58
acolesor like you say, --composite09:58
kota_ok, let's write down it to the etherpad10:00
*** amoralej has joined #openstack-swift10:00
*** dmellado has joined #openstack-swift10:02
acoleskota_: thanks!10:08
kota_acoles: done to write as "possible solutions -> idea 1".10:18
*** openstackgerrit has quit IRC10:18
kota_acoles: if you find something different, please correct that directly10:18
kota_and I could start to modify my patch since tommorow morning if you and clayg confirmed that idea (and support it)10:19
acolesok kota_ have a good evening10:19
kota_acoles: yeah, thanks a lot for the discussion10:20
*** tonanhngo has joined #openstack-swift10:23
*** tonanhngo has quit IRC10:24
*** geaaru has quit IRC10:32
*** janonymous has joined #openstack-swift10:42
*** geaaru has joined #openstack-swift10:46
*** tonanhngo has joined #openstack-swift10:52
*** furlongm has joined #openstack-swift10:52
*** tonanhngo has quit IRC10:53
*** dmorita has joined #openstack-swift11:01
*** dmorita has quit IRC11:05
*** geaaru has quit IRC11:10
*** dmorita has joined #openstack-swift11:14
*** sams-gleb has quit IRC11:14
*** dmorita has quit IRC11:15
*** sams-gleb has joined #openstack-swift11:15
*** dmorita has joined #openstack-swift11:15
*** dmorita_ has joined #openstack-swift11:18
*** sams-gleb has quit IRC11:19
*** dmorita has quit IRC11:19
*** dmorita_ has quit IRC11:19
*** dmorita has joined #openstack-swift11:19
*** dmorita has quit IRC11:20
*** dmorita has joined #openstack-swift11:20
*** dmorita has quit IRC11:25
*** psachin has quit IRC11:30
*** psachin has joined #openstack-swift11:30
*** joeljwright has joined #openstack-swift11:34
*** ChanServ sets mode: +v joeljwright11:34
tdasilvaacoles: maybe??? ;)11:35
tdasilvamattoliverau, acoles now i'm really looking to put this stuff into an ansible script together with mathiasb scripts for barbican11:36
tdasilvait's coming all together :)11:36
acolestdasilva: :)11:41
*** gkadam has quit IRC11:42
*** dmorita has joined #openstack-swift11:48
*** dmorita has quit IRC11:52
*** jordanP has joined #openstack-swift11:52
*** tonanhngo has joined #openstack-swift11:55
*** tonanhngo has quit IRC11:56
*** dmorita has joined #openstack-swift11:57
*** sams-gleb has joined #openstack-swift11:58
*** dmorita has quit IRC12:02
*** bkopilov has quit IRC12:27
*** tonanhngo has joined #openstack-swift12:37
*** tonanhngo has quit IRC12:38
*** janonymous has quit IRC12:44
*** klamath has joined #openstack-swift12:45
*** klamath has joined #openstack-swift12:46
*** jaosorior has joined #openstack-swift12:47
*** catintheroof has joined #openstack-swift12:59
*** tonanhngo has joined #openstack-swift13:00
*** tonanhngo has quit IRC13:01
*** links has quit IRC13:02
*** geaaru has joined #openstack-swift13:06
*** cshastri has quit IRC13:09
*** tongli has joined #openstack-swift13:23
*** psachin has quit IRC13:24
*** winggundamth has joined #openstack-swift13:29
*** amoralej is now known as amoralej|lunch13:34
rledisezfor an object upload, the proxy is providing the account/container/container-servers+devices where to register the object13:49
rledisezfor the delete-at feature, the proxy is providing the account/container/container-servers+devices where to register the "task"13:49
rledisezis there like a rule that the object-server must be able to work without the container ring ?13:49
*** vint_bra has joined #openstack-swift13:50
acolesrledisez: I think the goal is to avoid every object server process needing to load the ring data13:54
*** winggundamth has quit IRC13:55
rledisezacoles: ok, it makes sense. thx for the info13:56
*** zhurong_ has joined #openstack-swift14:03
*** links has joined #openstack-swift14:05
*** zhurong_ has quit IRC14:06
*** bkopilov has joined #openstack-swift14:12
*** cbartz has left #openstack-swift14:18
*** tonanhngo has joined #openstack-swift14:23
*** tonanhngo has quit IRC14:25
*** cbartz has joined #openstack-swift14:29
*** jordanP has quit IRC14:39
*** jordanP has joined #openstack-swift14:39
*** d0ugal has quit IRC14:42
*** links has quit IRC14:42
*** d0ugal has joined #openstack-swift14:43
*** sams-gleb has quit IRC14:50
*** sams-gleb has joined #openstack-swift14:51
*** sams-gleb has quit IRC14:55
*** JimCheung has joined #openstack-swift14:57
notmynamegood morning15:01
*** JimCheung has quit IRC15:02
notmynameclayg: thanks for the writeup!15:02
*** amoralej|lunch is now known as amoralej15:02
*** sams-gleb has joined #openstack-swift15:04
timssclayg: read your rebalance etherpad and wondering if I should be scared now :o Have yet to do a big rebalance on my cluster15:07
*** cbartz has left #openstack-swift15:07
*** dja has quit IRC15:08
*** tongli has quit IRC15:13
thurloatmorning !15:14
notmynametimss: don't be too worried (probably ;-) )15:15
*** tonanhngo has joined #openstack-swift15:17
notmynametimss: but that being said, it's generally easier to add a little bit of capacity to swift frequently instead of "saving it up" and doubling your capacity all at once15:18
*** tonanhngo has quit IRC15:18
timssnotmyname: I'll put those worrying thoughts somewhere hidden until the day of doom then! :)15:22
*** klrmn has joined #openstack-swift15:22
notmynamehas anyone been following the "use etcd for storing configs" ML thread? (and has a short summary?)15:23
timssnotmyname: yup thanks for the tip, might not match with whatever hardware I'll get my hands on, but disks could be added in chunks using rings/weights I suppose15:23
*** Jeffrey4l has quit IRC15:38
*** Jeffrey4l has joined #openstack-swift15:39
timburkegood morning15:53
timburkewelcome and congratulations, mahatic! glad to have someone new to pester for reviews ;-)15:54
*** jaosorior has quit IRC16:02
*** silor has joined #openstack-swift16:05
*** silor1 has joined #openstack-swift16:10
timburketdasilva: i had a thought on symlinks this morning, although it doesn't need to be a part of the current patchset16:10
*** silor has quit IRC16:10
tdasilvatimburke: sure, no problem, i'm sure we will still require change16:10
*** silor1 is now known as silor16:10
timburkei wonder if slo should dereference symlinks during a manifest PUT16:10
tdasilvachanges16:10
timburkethis would let us avoid (at least) one extra backend request during GETs, and would play well with a symlink-based versioned_writes (the SLO would reference the current parts in the archive container rather than the symlink which could change)16:12
*** klrmn has quit IRC16:18
timburkeat the same time, i feel like it maintains the spirit of an slo being static16:19
tdasilvatimburke: just to make sure I understand. by " dereference symlinks during a manifest PUT" you mean actually write down the target path on the manifest we write to disk16:19
*** openstackgerrit has joined #openstack-swift16:20
openstackgerritAlistair Coles proposed openstack/swift master: WIP - Make EC GET defer nodes likely to have duplicate fragments  https://review.openstack.org/44605016:20
tdasilvatimburke: if that's the case, then my concern would be for users that are using symlinks to point changing targets and they used symlinks in the slo manifest for that reason16:22
tdasilvatimburke: I think versioning is a good example. One could argue that the user wants the most current version and not the archived version??? idk16:22
*** JimCheung has joined #openstack-swift16:22
timburkeyeah, that was my thinking. and this debate is why i'm partial toward it being a follow-up16:23
tdasilvatimburke: but then does that mean an api / behavior change that we never want to do it because of backwards compatibility?16:23
timburkebut having the manifest point to "whatever the current version is" will cause the slo to break whenever anything's updated16:23
timburkei would be content to describe it as a bug fix16:24
timburke(because of the aforementioned breakage)16:24
tdasilvahehehehe16:24
jrichliIt's been awhile since I thought about symlinks, but I thought we were restricting the interface so that the target could not change.  I realize that doesn't mesh with wanting to maybe use them for auto-tiering, however16:24
tdasilvajrichli: target doesn't change on POSTs16:25
tdasilvabut user can always do a new PUT16:25
jrichlitdasilva: ah, right.  thx16:25
tdasilvathat's how I envision we would do versioning for example16:25
timburkebingo16:26
timburkei should really try to find some time to take a stab at that (symlink-backed versioning)...16:26
tdasilvasometimes i really dislike the fact that we got the API so restricted and no new versions of the API. It forces us to get new APIs right the first time or we end up with bad apis and some weird hacks to fix things16:27
timburkeyup :-(16:28
tdasilvathat's why i'd prefer that we at least be more comfortable doing "beta" apis16:28
tdasilvaor alpha, whatever16:28
timburkethe only trouble is that the "alpha/beta" api inevitably ends up in production16:28
tdasilvaprobabably alpha is better cause it really tells operators/users that things are expected to change16:29
tdasilvayeah, but if somebody put something alpha in prod, then i'm sorry, but don't complain if there was a change16:29
notmynameturns out that people don't use it when you tell them it could break :-)16:30
tdasilvaI think, I'd prefer that situation then forcing us to get the API right the very first time it lands on master16:30
timburkeit's even worse when you're talking about new on-disk formats. an api could conceivably change; there'd be the period of pain as clients get fixed, but it eventually passes. we have no story for migrating on-disk formats, though16:30
timburkeand symlinks is both16:31
notmynamebut we *have* migrated on-disk formats. those are "easy" to change, compared to APIs. we've reverted several changes because some client somewhere got broken because we started following the RFC or something16:31
notmyname(easy in the sense that carrying away everest is easy compared to terraforming mars)16:32
tdasilvaheh16:32
*** joeljwright has quit IRC16:35
timburkenotmyname: when? and could we ever feel comfortable ripping out the support for the old format? how can you ever feel reasonably certain that you converted everything?16:35
notmynametimburke: ring format changes. .durable changes. storage policies. adding columns to DBs16:36
timburkei've been thinking about this a bit in the context of key rotation, or making the etag of a slo in container listings match the etag in a GET/HEAD response, or...16:36
timburkestorage policies don't have a migration story! we've been talking about how to handle that for ages!16:37
tdasilvatimburke: i think he means that we actually added storage policies16:38
notmynameoh I just mean that we used to have everything in objects/ and now we have objects-N/16:38
tdasilvaobjects objects-1 etc16:38
notmynameyeah, what tdasilva16:38
notmynamesaid16:38
*** SkyRocknRoll has joined #openstack-swift16:39
notmynametimburke: but I'll grant you that everything is terrible, and we can't change anything. we weren't smart enough back then to do the right thing, and we aren't smart enough now to fix it.16:39
notmyname;-)16:40
tdasilvanotmyname: but we keep not being smart thinking that we are smart to get new apis right this time16:40
tdasilvaread that really fast16:40
notmynamelol16:41
notmynameTHIS TIME IT'S DIFFERENT!16:41
tdasilvamore than fixing the broken apis, i wish we would change our process with the knowledge that we will probably mess up again on new apis16:41
tdasilvahehehehe16:42
tdasilvasorry, i'm done ranting, back to fixing symlinks16:42
notmynameyeah, I agree. the only thing I take issue with is that we haven't or can't change internal formats. we're pretty good on that front, considering that changing has to take into account enormous amounts of data that can't go away16:43
notmynameand I'm scared that if we finally arrive at the "one day we'll version the api...", then we'll go too far and have version 38 in six months16:44
tdasilvalol, that's terrible16:44
notmynamewe've had v1 of the API for 7 years (and it predates swift itself a couple of years). so if we have a v2, I'd love for it to last at least another 10 more years :-)16:44
notmynamebut maybe that's the wrong attitude16:44
notmynameidk16:44
tdasilvabut yeah, hence, my current naive thinking is that 'alpha apis' is a middle ground16:45
tdasilvabut i'm saying this more on the fact that others seem to be going that way rather than knowing from experience that it is a good thing16:47
timburkenotmyname: yeah, we'd totally hit some "large" version number pretty quickly. i'd wager if we ever wanted to go down the "v2" path, we'd need to throw it all on a (*very* long-lived) feature branch, do all of our breaking, let people run it in prod anyway, find all the *other places* we needed or wanted to break, and wait for it to get vaguely stable before brining it back in and calling it "v2"16:47
notmynameor we could do microversions! ;-)16:47
notmynameswift api v1, now with symlinks v2.13, tempurls v1.1, POST v3, crypto v1.0, ec v3, and staticweb v2.1716:48
notmynameswift api compatibility matrix v2.14!16:49
tdasilvalol, that sounds awful too, where did i see that recently?16:49
tdasilvaoh yeah, qnap nas16:49
jrichlithe problem then is regression testing with different combinations16:50
*** gyee has joined #openstack-swift16:50
jrichlii have worked in a world where we had hundreds of different services versioned separately.  it was craziness.16:51
jrichli(and yes, i did know you were joking)16:52
tdasilvagoing to do some more snow cleanup...brb...16:52
*** tesseract has quit IRC16:53
*** cbartz has joined #openstack-swift17:10
*** klrmn has joined #openstack-swift17:10
*** cbartz has left #openstack-swift17:19
*** pcaruana has quit IRC17:28
*** jordanP has quit IRC17:31
*** JimCheung has quit IRC17:36
acolesclayg: kota_ left some comments on the etherpad re builder files for composite rings, https://etherpad.openstack.org/p/composite_rings17:37
acolesclayg: we were chatting about it earlier today17:38
*** szaher has joined #openstack-swift17:39
*** JimCheung has joined #openstack-swift17:43
timburkedoes anyone remember why https://github.com/openstack/swift/blob/2.13.0/swift/common/middleware/versioned_writes.py#L473-L475 isn't a bug? tdasilva, maybe? i'm sure it must've come up in the middleware refactor...17:43
*** SkyRocknRoll has quit IRC17:44
tdasilvatimburke: if I remember correctly this was maintained for backwards compatibility "it's always been this way so let's not change it"17:44
timburkeclayg! i wanna change it!17:44
tdasilvaespecially during the refactor, we tried to maintain the behavior very much the same as pre-refactor17:45
timburketdasilva: yup... as torgomatic would say, it was for "hysterical raisins"17:46
tdasilvahhahahahaha17:46
tdasilvatimburke: as clayg would say, do you actually have a use case for wanting to change it17:47
tdasilva?17:47
notmynameto be discussed at today's meeting https://wiki.openstack.org/wiki/Swift/Fixing-rebalance-and-golang17:49
timburketdasilva: yup, related to history mode. i want to give a less-privileged user read/write access to a container with history-mode versioning enabled, and i don't want to ever have to worry about losing data as a result17:49
*** lespaul_ has quit IRC17:50
timburkethat non-account-owner can do whatever they want inside that container (including creating large objects) and currently if they create a DLO, they can overwrite an object without having an archive copy created17:51
timburkeSLO behavior is great (or, as great as can be expected). DLO behavior starts to feel like a security hole17:52
*** hseipp has quit IRC17:54
tdasilvatimburke: just trying to follow up: "if they create a DLO, they can overwrite..." but isn't that code preventing them from creating a DLO in a versioned container?17:57
*** SkyRocknRoll has joined #openstack-swift17:57
tdasilvaoh, it doesn't prevent, it just doesn't version what's there17:57
timburkeyep! just let it through!17:58
tdasilvaso they can overwrite without an archive copy, got it17:58
timburkethen there's also the awkward delete behavior acoles documented at https://bugs.launchpad.net/swift/+bug/1626989 ...17:58
openstackLaunchpad bug 1626989 in OpenStack Object Storage (swift) "versioning history mode treats DLO manifests inconsistently" [Undecided,New]17:58
tdasilvaI think we discussed that and were generally ok, like "naahh..it's just a header"17:58
tdasilvalol at that bug17:59
*** ChubYann has joined #openstack-swift18:00
tdasilvawe create delete markers but no versions18:00
timburkei wanna be able to roll back state. if we don't version dlos, it's impossible; there's no way of knowing where that dlo pointed, if we even know that it ever existed!18:01
acolestimburke: as I remember it, DLO manifest weren't versioned before the refactor to middleware, so we kept it that way, for better or worse18:01
tdasilvatimburke: yep, makes sense18:01
tdasilvaacoles: yes, that's how i remember too18:01
timburkeif we *do* version dlos (and whatever container(s) have the segments) there's a chance at rolling back. it'll be a bit painful, but doable18:01
timburkeso, any opposition to versioning dlos? notmyname, clayg?18:02
*** tonanhngo has joined #openstack-swift18:03
*** tonanhngo has quit IRC18:07
*** tonanhngo has joined #openstack-swift18:11
*** gyee has quit IRC18:18
notmynameI ... don't knwo18:21
*** shivanim has joined #openstack-swift18:24
shivanimHello, I am Shivani Mehendarge. I am interested in working on swift for the outreachy program. Can anyone please tell me how to start with it?18:29
notmynameshivanim: hello!18:29
*** ganders has joined #openstack-swift18:29
shivanimnotmyname, :)18:29
notmynameshivanim: I've been an outreachy mentor a couple of times in the past. what are you interested in for swift?18:29
shivanimI have cloned the project just now. I have very litte idea about it.18:30
notmynameshivanim: do you have any ideas of what you'd like to work on? what made you consider swift in the first place?18:31
shivanimI wanted to contribute to it. Maybe by fixing a bug or documentation perhaps. But I want to learn how swift works.18:31
notmynameshivanim: sure (and that's great). but why swift as opposed to some other project?18:32
shivanimActually I am interested in cloud computing and seeing that swift can help retrieve a lot of data through simple API, makes me interested. :)18:34
notmynamecool18:34
notmynameshivanim: have you looked through the outreachy site and talked to any of the coordinators yet?18:34
notmynameshivanim: what time zone are you in?18:36
shivanimnotmyname, I have looked through the site but haven't talked to any coordinators.18:36
notmynamehttps://wiki.openstack.org/wiki/Outreachy seems to be the openstack landing page for outreachy18:37
shivanimnotmyname, Iam in IST.18:37
notmynameshivanim: ok. mahatic would be a great person to talk to. she's in your time zone and a swift core and also one of the main outreachy coordinators for openstack18:38
shivanimnotmyname, I have visited the wiki page for Openstack :)18:38
shivanimnotmyname, ok :)18:38
shivanimnotmyname, After cloning the swift project, how can I go about it? What should I preferably do next?18:39
notmynameyou should set up a swift all in one (SAIO) https://docs.openstack.org/developer/swift/development_saio.html18:40
notmynameshivanim: we don't have any outreachy projects listed for swift right now. that's not to say you can't be helpful (quite the opposite)18:40
shivanimnotmyname, Oh, I didn't notice that.18:41
notmynameshivanim: we do have several significant things going on that will take a lot of work from a lot of people to do. these are some of the places where you could help out18:41
notmynameshivanim: what's your dev background and experience? are you just getting started or have you been doing dev work for a while?18:42
shivanimnotmyname, I am a beginner in open source contributions.18:42
notmynamethat's totally ok :-)18:43
shivanimI have made 2 contributions to pagure recently during pycon pune :)18:43
notmynamehave you done stuff on your own or for an employer that's not open source?18:43
notmynamepagure? /me goes to google18:43
shivanimnotmyname, yes I have :)18:43
notmynameshivanim: the first steps for you will be to set up the SAIO, start using the API, and understand how the code is put together. swift.openstack.org is where the dev docs are hosted, and of course you can always ask questions in here18:46
shivanimnotmyname, Thanks for your help :) I will surely go through the docs .18:46
shivanimnotmyname, If I want to participate in outreachy, I will have to contribute to one of the projects openstack that is participating in it?18:47
notmynameshivanim: great! you can also check out https://wiki.openstack.org/wiki/Meetings/Swift for a list of some stuff that's being worked on and https://wiki.openstack.org/wiki/Swift/ideas for some things that people have written related to some of their work18:47
notmynameshivanim: as I understand it, to participate in outreachy in openstack, you need a project and a mentor. and the mentor is the most important (ie the mentor helps define the project with you and says it's good or not)18:48
*** catinthe_ has joined #openstack-swift18:48
notmynameshivanim: but there's also an application process. I'm not as familiar with that step. mahatic would be the person to talk to18:49
shivanimnotmyname, okay , I will surely talk to her :)18:49
timburkeas i recall, it does involve having at least one proposed change, just to make sure you've gotten as far as checking out the repo, reading the contributor docs, etc.18:50
*** catintheroof has quit IRC18:51
shivanimtimburke, yes, you have to make atleast one contribution to the project you are opting for.18:52
*** SkyRocknRoll has quit IRC18:53
shivanimnotmyname, where can I find the list of projects in openstack that are there in outreachy?18:55
notmynameshivanim: all I've got is the wiki page I linked earlier18:55
shivanimI just checked this link https://wiki.gnome.org/Outreachy/2017/MayAugust#Participating_Organizations . Here there is a project for swift.18:55
timburkehttps://wiki.openstack.org/wiki/Internship_ideas maybe?18:56
notmynameyeah, there's one listed for swift3 (which is closely related to swift--swift3 is an external extension to swift, so you gotta know both)18:57
notmynameand to be clear, despite there not being a "dp project XYZ in swift" listed, there's a lot of work to do in swift. but in the short term there's probably not a lot of projects in swift that need to be done solo by an outreachy intern. but an outreachy intern could help with the other big stuff going on18:58
shivanimnotmyname, okay :)18:59
shivanimtimburke, notmyname  Thank you :)18:59
*** lespaul has quit IRC19:16
*** amoralej is now known as amoralej|off19:20
*** geaaru has quit IRC19:32
*** shivanim has quit IRC19:32
openstackgerritTim Burke proposed openstack/swift master: Version DLOs, just like every other type of object  https://review.openstack.org/44614219:38
*** ganders has quit IRC19:44
*** sgundur has quit IRC19:45
*** pumaranikar has quit IRC19:45
*** mmotiani has quit IRC19:45
*** ntata has quit IRC19:45
claygtimss: If you're going to be rebalancing a large replicated cluster don't worry too much - you're in good company - there's tons of stuff you can do to make that go well for you and you'll get to learn cool stuff along the way!19:48
claygtimss: if you're running a large EC cluster that needs to rebalance... well... me too!19:49
*** chsc has joined #openstack-swift19:57
claygDid we point shivanim at low-hanging-fruit?  https://bugs.launchpad.net/swift/+bugs?field.assignee=&field.tag=low-hanging-fruit&field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS20:05
*** silor has quit IRC20:07
timburkeclayg: did we ever come to a conclusion on https://bugs.launchpad.net/swift/+bug/1537811 ? should we just close that as Won't Fix / Invalid?20:07
openstackLaunchpad bug 1537811 in OpenStack Object Storage (swift) "204 No Content responses have Content-Length specified" [Low,In progress] - Assigned to Trevor McCasland (twm2016)20:07
*** sgundur has joined #openstack-swift20:10
*** sweeper has quit IRC20:11
claygtimburke: no decision was made to my knowledge - i belive it it is still low priority - but probably should be "not assigned"20:26
claygtimburke: re-reading the varnish ticket it's like reading my future - "we did the what spec said - pretty sure this will be fine" "yeah you broke me ... me too ... me as well" "gd, ^&*# you web - you are a cesspool of ill behaved clients - it's unfathomable anything anywhere works at all!" "everything was working fine till you read the damn spec and wanted to look smart" "touche; revert"20:31
timburkeright? so it seems like we probably want to keep Content-Length20:32
claygi'm ok with pulling it out if there's a good reason to?  So far all we've got is "it's not reliable"20:33
claygi.e. some proxies strip it out and some proxies add it in even when not there20:33
claygso there's a strong case that clients SHOULD not expect a content-length header on requests which don't have a body (esp. 204)20:33
claygour current respones don't make that especially apparent - so there's some risk we reenforce the cesspool of ill behaved clients20:35
claygin varnishes case they went with liberal what you accept and conservative of what you send20:36
claygthey don't get mad (anymore) if a backend *sends them* a 204 w/ content-length - but they also don't go spewing that back out into the world20:36
notmynameIMO we should strive for the same20:37
claygwe fixed tempest - which was the only client we *know of* that expected the header there20:37
claygI really do feel like we *could* pull it out20:37
claygi just wish we had *any other* demonstrable harm - but it's such a small innocuous thing20:38
claygso... I'm torn... good thing it's also a really low priority - helps significantly to minimzie the require soul searching20:39
claygmaybe at somepoint someone will bump into something annoying find that bug and ding us20:39
claygoh oh oh!  how bout this!  since 7230 *obsoletes* 2616 we should expect new proxies to implemnt 7230 - which *is* (surprisingly?) prescriptive about servers sending content-length with 204 responses20:41
mattoliverauMorning20:41
clayg... so if the assumption is that new proxies getting built will follow 7230 - we can also expect the mistake varnish made to be common going forward20:41
timburkefwiw, it looks like 204s aren't the only times proxies will mess with content-length; see https://bugs.launchpad.net/python-swiftclient/+bug/1586690 and https://bugs.launchpad.net/python-swiftclient/+bug/162158120:42
openstackLaunchpad bug 1586690 in python-swiftclient "Uploading empty(0 B) file fails" [Undecided,Incomplete] - Assigned to Uday Swami (swamius)20:42
timburkedoesn't really indicate (to me) which way we ought to go here, just putting it out there20:42
openstackLaunchpad bug 1621581 in python-swiftclient "swiftclient returns response headers without 'Content-Length' param, thus causing upload object to fail" [Undecided,In progress] - Assigned to Arun Mani (arun-mani)20:42
claygesentially swift (and other old school webservers like apache) will be the cesspool of clients that make it impossible to implement the spec20:42
clayg... until we 1) get someone to update 7230 or 2) stop sending content-length20:42
claygone of those two things seems significantly easier than the other ;)20:42
notmyname1) someone else 2) us20:43
notmyname;-)20:43
timburkealways be waiting on someone else ;-)20:43
*** sgundur has quit IRC20:43
notmynameor we could just stop sending it and think about reverting it someone complains20:44
* notmyname stepping out for 10 minutes. be back for the meeting20:44
claygtimburke: i'm not sure what either lp bug #1586690 or lp bug #1621581 where getting at exactly20:45
openstackLaunchpad bug 1586690 in python-swiftclient "Uploading empty(0 B) file fails" [Undecided,Incomplete] https://launchpad.net/bugs/1586690 - Assigned to Uday Swami (swamius)20:45
openstackLaunchpad bug 1621581 in python-swiftclient "swiftclient returns response headers without 'Content-Length' param, thus causing upload object to fail" [Undecided,In progress] https://launchpad.net/bugs/1621581 - Assigned to Arun Mani (arun-mani)20:45
claygI think there was also a bug about 304 returning content-length and some if-match response returning accept-ranges20:46
timburkemostly, i'm just sad that proxies keep mangling our headers :'(20:47
*** sgundur has joined #openstack-swift20:56
notmynameteam meeting in 2 minutes in #openstack-meeting20:58
*** dmorita has joined #openstack-swift20:59
*** m_kazuhiro has joined #openstack-swift20:59
kota_good morning20:59
m_kazuhirokota_: good morning21:00
kota_m_kazuhiro: o/21:00
*** sgundur has quit IRC21:04
*** ntata has joined #openstack-swift21:06
*** dja has joined #openstack-swift21:08
notmynamethurloat: talking about your patch in -meeting21:08
*** lespaul has joined #openstack-swift21:12
*** dja has quit IRC21:23
*** sgundur has joined #openstack-swift21:33
*** Jeffrey4l_ has joined #openstack-swift21:35
*** Jeffrey4l has quit IRC21:35
openstackgerritTim Burke proposed openstack/swift master: Version DLOs, just like every other type of object  https://review.openstack.org/44614221:36
*** sgundur has quit IRC21:41
*** sgundur has joined #openstack-swift21:41
openstackgerritTim Burke proposed openstack/swift master: Factor out a bunch of common testing setup  https://review.openstack.org/44618521:43
*** m_kazuhiro has quit IRC21:52
*** ntata has quit IRC21:53
*** jamielennox has quit IRC21:55
*** sgundur has quit IRC21:59
acoleskota_: before you head for breakfast...21:59
kota_acoles: ?21:59
notmynamehttps://www.openstack.org/blog/2017/03/helping-ptg-attendees-and-other-developers-get-to-the-openstack-summit/21:59
acoleskota_: it might be helpful to split the composite rings patch into two : 1. the functions to perform prevalidation and perform the zipping, along with necessary tests; followed by 2. some kind of CLI support.22:01
* tdasilva is sorry about missing swift meeting, catching up now22:01
acoleskota_: that way we can make progress on 1 while debating 2 :) and decouple testing of the functionality from any particular CLI interface22:02
notmynametdasilva: no worries22:02
notmynametdasilva: I think we only volunteered you for a few things22:02
tdasilvarofl22:02
kota_acoles: that's what I've been thinking since the last now > spliting patch as chain ;-)22:02
acoleskota_: two agree -> decision ! :D22:02
acoleskota_: let me know where I can help22:03
kota_acoles: let me make sure where you think as "functions"?22:04
acoleskota_: also FYI (still very much WIP) https://review.openstack.org/44605022:04
patchbotpatch 446050 - swift - WIP - Make EC GET defer nodes likely to have dupli...22:04
kota_acoles: the boader could not be concrete to me for now22:05
kota_thanks the GET thing!22:05
acoleskota_: so for example as clayg commented here https://review.openstack.org/#/c/441921/2/swift/cli/ringzipper.py@16722:06
patchbotpatch 441921 - swift - Composite Ring Zipper Implementation22:06
kota_oic22:06
kota_the *MAGIC* is the first functionality to implement22:07
acolesso split out the validation/zipping to separate module e.g. common/ring/composite_ring.py22:08
kota_from... builder file? ok, that's a difference22:08
kota_acoles: ok, and no new bin and cli tool22:08
acoleskota_: well, that would be patch 2, dependent.22:09
acolesand we can continue to debate new cli tool vs integrate into swift-ring-builder as we gather more opinions22:09
acolesbut make progress on composite_ring.py (or whatever its called)22:10
kota_sounds great plan22:10
kota_that works in parallel, make sense, it will make it speed up22:10
kota_let's do that22:11
acoleskota_: ok :)22:12
kota_thanks acoles to bring it up ;-D22:12
acoleskota_: NP have a good day, I'm away now.22:13
kota_acoles: good night, have a good sleep!22:13
* kota_ is going to find breakfirst22:13
*** adriant has joined #openstack-swift22:14
*** jamielennox has joined #openstack-swift22:15
*** m_kazuhiro has joined #openstack-swift22:16
*** sams-gleb has quit IRC22:24
*** dja has joined #openstack-swift22:26
timssclayg: Running replication, and that's reassuring to hear, thanks! Trying to pick up tips and tricks here and there, but I'll probably be around when the day arrives :)22:31
clayglol22:45
*** m_kazuhiro has quit IRC22:47
*** dja has quit IRC22:49
*** gyee has joined #openstack-swift22:52
*** klamath has quit IRC22:59
*** tonanhngo has quit IRC23:02
*** catinthe_ has quit IRC23:02
*** tonanhngo has joined #openstack-swift23:02
*** tonanhngo has quit IRC23:07
*** sams-gleb has joined #openstack-swift23:24
*** sams-gleb has quit IRC23:29
*** lespaul has quit IRC23:34
*** chsc has quit IRC23:36
*** kei_yama has joined #openstack-swift23:40
openstackgerritDaisuke Morita proposed openstack/swift master: WIP: Changing Policies  https://review.openstack.org/20932923:43
*** mmotiani has joined #openstack-swift23:49
*** ntata has joined #openstack-swift23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!