| *** mat128 has quit IRC | 00:12 | |
| tdasilva | kota_, timburke: hello, what's the update on swift3? | 00:29 |
|---|---|---|
| *** wes_dillingham has joined #openstack-swift | 00:33 | |
| clayg | UPDATE: swift3 is awesome (but not as awesome as kota_ or timburke) | 00:33 |
| timburke | tdasilva: i landed the last outstanding patch kota_ wanted before cutting a release earlier today. i left some comments on the authors/changelog patch he has up. i also proposed https://review.openstack.org/#/c/509321/ (which soft-deps on https://review.openstack.org/#/c/510715/ -- it doesn't really *work* until you have a swift that supports it, but it doesn't break any worse, either) because i've got internal pressures for support f | 00:43 |
| timburke | or long-running uploads and patches-against-swift3 seem as good a way as any to make sure that we *eventually* get patches-against-s3api-in-swift | 00:43 |
| patchbot | patch 509321 - swift3 - Support long-running multipart uploads | 00:43 |
| patchbot | patch 510715 - swift - Fix bulk heartbeating when emitting XML | 00:43 |
| timburke | er, no, not that swift patch... | 00:45 |
| timburke | https://review.openstack.org/#/c/509306/ | 00:46 |
| patchbot | patch 509306 - swift - Let clients request heartbeats during SLO PUTs | 00:46 |
| timburke | yeah, that's the one :-) | 00:46 |
| *** tovin07_ has joined #openstack-swift | 00:53 | |
| *** mingyu has joined #openstack-swift | 00:53 | |
| *** mingyu has quit IRC | 00:59 | |
| *** gyee has quit IRC | 01:08 | |
| *** mat128 has joined #openstack-swift | 01:11 | |
| *** abhitechie has quit IRC | 01:24 | |
| *** mat128 has quit IRC | 01:42 | |
| *** abhitechie has joined #openstack-swift | 01:47 | |
| *** psachin has joined #openstack-swift | 01:51 | |
| kota_ | thx timburke! I'll address the comments on the change log. | 02:10 |
| kota_ | soon | 02:10 |
| kota_ | tdasilva: btw, would you attend tomorrow meeting? I'd like to get updates (or collect past discussion) for symlinks. | 02:12 |
| *** Shatadru|Gone is now known as Shatadru | 02:13 | |
| *** Shatadru is now known as Shatadru|coffee| | 02:13 | |
| *** catintheroof has joined #openstack-swift | 02:20 | |
| *** jlvillal has quit IRC | 02:21 | |
| *** jlvillal has joined #openstack-swift | 02:21 | |
| *** catintheroof has quit IRC | 02:22 | |
| *** Shatadru|coffee| is now known as Shatadru | 02:23 | |
| *** mat128 has joined #openstack-swift | 02:42 | |
| *** Shatadru is now known as Shatadru|afk | 02:47 | |
| *** tovin07_ has quit IRC | 02:48 | |
| kota_ | timburke: I pushed the new version for swift3 changelog! | 02:53 |
| kota_ | timburke: https://review.openstack.org/504479 | 02:53 |
| patchbot | patch 504479 - swift3 - Change log updates for version 1.12 | 02:53 |
| *** mingyu has joined #openstack-swift | 02:56 | |
| *** mingyu has quit IRC | 02:56 | |
| *** links has joined #openstack-swift | 02:57 | |
| *** mingyu has joined #openstack-swift | 02:59 | |
| *** mingyu has quit IRC | 02:59 | |
| *** mingyu has joined #openstack-swift | 03:04 | |
| *** mingyu has quit IRC | 03:10 | |
| *** mingyu has joined #openstack-swift | 03:13 | |
| *** mat128 has quit IRC | 03:13 | |
| *** mingyu has quit IRC | 03:14 | |
| *** mingyu has joined #openstack-swift | 03:14 | |
| *** kei_yama has quit IRC | 03:38 | |
| *** abhitechie has quit IRC | 03:43 | |
| *** guimaluf has quit IRC | 03:46 | |
| *** guimaluf has joined #openstack-swift | 03:49 | |
| *** kei_yama has joined #openstack-swift | 04:01 | |
| *** tovin07_ has joined #openstack-swift | 04:07 | |
| *** mingyu has quit IRC | 04:09 | |
| *** mat128 has joined #openstack-swift | 04:12 | |
| *** guimaluf has quit IRC | 04:13 | |
| *** mingyu has joined #openstack-swift | 04:15 | |
| *** guimaluf has joined #openstack-swift | 04:19 | |
| *** Shatadru|afk is now known as Shatadru | 04:23 | |
| *** abhitechie has joined #openstack-swift | 04:24 | |
| *** mingyu has quit IRC | 04:32 | |
| *** mingyu has joined #openstack-swift | 04:32 | |
| *** gyee has joined #openstack-swift | 04:34 | |
| *** mat128 has quit IRC | 04:44 | |
| *** SkyRocknRoll has joined #openstack-swift | 04:49 | |
| *** wes_dillingham has quit IRC | 04:58 | |
| *** torgomatic has quit IRC | 05:07 | |
| *** gyee has quit IRC | 05:10 | |
| *** mingyu has quit IRC | 05:11 | |
| *** mingyu has joined #openstack-swift | 05:12 | |
| *** tone_zrt has joined #openstack-swift | 05:13 | |
| *** tone_z has quit IRC | 05:14 | |
| *** gkadam has joined #openstack-swift | 05:15 | |
| *** ChubYann has quit IRC | 05:18 | |
| *** HCLTech-SSW has joined #openstack-swift | 05:19 | |
| *** mingyu has quit IRC | 05:25 | |
| *** mingyu has joined #openstack-swift | 05:32 | |
| *** abhitechie has quit IRC | 05:34 | |
| *** mingyu has quit IRC | 05:36 | |
| *** abhitechie has joined #openstack-swift | 05:38 | |
| *** mat128 has joined #openstack-swift | 05:43 | |
| *** cshastri has joined #openstack-swift | 05:49 | |
| *** cbartz has joined #openstack-swift | 06:12 | |
| *** mat128 has quit IRC | 06:14 | |
| *** hoonetorg has quit IRC | 06:17 | |
| *** abhitechie has quit IRC | 06:18 | |
| *** abhinavtechie has joined #openstack-swift | 06:18 | |
| *** spectr has quit IRC | 06:27 | |
| *** klrmn has quit IRC | 06:28 | |
| *** spectr has joined #openstack-swift | 06:28 | |
| *** hoonetorg has joined #openstack-swift | 06:30 | |
| *** abhitechie has joined #openstack-swift | 06:32 | |
| *** mingyu has joined #openstack-swift | 06:33 | |
| *** abhinavtechie has quit IRC | 06:35 | |
| *** mingyu has quit IRC | 06:38 | |
| *** HCLTech-SSW has quit IRC | 06:39 | |
| *** hseipp has joined #openstack-swift | 06:55 | |
| *** openstackgerrit has quit IRC | 07:03 | |
| *** rcernin has joined #openstack-swift | 07:05 | |
| *** geaaru has joined #openstack-swift | 07:10 | |
| *** oshritf has joined #openstack-swift | 07:11 | |
| *** mat128 has joined #openstack-swift | 07:12 | |
| *** pcaruana has joined #openstack-swift | 07:13 | |
| *** oshritf has quit IRC | 07:17 | |
| *** tesseract has joined #openstack-swift | 07:19 | |
| *** abhitechie has quit IRC | 07:28 | |
| *** openstackgerrit has joined #openstack-swift | 07:41 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift master: Imported Translations from Zanata https://review.openstack.org/510000 | 07:41 |
| *** mat128 has quit IRC | 07:44 | |
| *** openstackgerrit has quit IRC | 07:48 | |
| *** itlinux has joined #openstack-swift | 07:52 | |
| acoles | clayg: torgomatic: hits all the right buttons for me ;) | 07:58 |
| acoles | good morning | 07:59 |
| acoles | timburke: mattoliverau thanks for your comments on shrinking idea | 07:59 |
| *** itlinux has quit IRC | 08:04 | |
| *** pcaruana has quit IRC | 08:27 | |
| *** abhitechie has joined #openstack-swift | 08:28 | |
| *** oshritf has joined #openstack-swift | 08:29 | |
| *** pcaruana has joined #openstack-swift | 08:31 | |
| *** m_kazuhiro has joined #openstack-swift | 08:35 | |
| *** kei_yama has quit IRC | 08:36 | |
| *** spectr has quit IRC | 08:40 | |
| *** ianychoi has quit IRC | 08:42 | |
| *** mat128 has joined #openstack-swift | 08:43 | |
| *** spectr has joined #openstack-swift | 08:50 | |
| *** oshritf has quit IRC | 08:58 | |
| *** oshritf has joined #openstack-swift | 09:00 | |
| *** ianychoi has joined #openstack-swift | 09:06 | |
| *** mat128 has quit IRC | 09:15 | |
| *** dr_gogeta86 has joined #openstack-swift | 09:16 | |
| *** dr_gogeta86 has quit IRC | 09:16 | |
| *** dr_gogeta86 has joined #openstack-swift | 09:16 | |
| *** abhitechie has quit IRC | 09:33 | |
| mattoliverau | acoles: np, I like the idea. I am thinking of another potential edge case.. but haven't finished thinking it through. But what happens if 1 primary shard is way off, so the sharder on the node it lives, finds it, sends a stats update for that shard range.. which is small enough (or somehow massively small, if that could happen) which could trigger a shrink. | 09:38 |
| mattoliverau | I guess it'll just mean the merged shards would cleave again.. which is fine | 09:39 |
| mattoliverau | So not really a show stopper.. but trying to think it through. I mean I expect it to happen but would assume containers are only out of sync a bit.. but is there a case when there way out, a failed cleave maybe. | 09:40 |
| acoles | mattoliverau: we could put back in some of the quorum stuff you had i.e. root fires off HEADs to all candidate shrink replicas before committing to shrink | 09:40 |
| mattoliverau | Yeah | 09:40 |
| mattoliverau | But maybe it doesn't matter | 09:40 |
| *** mvk has quit IRC | 09:40 | |
| acoles | mattoliverau: the quorum thing could be a sanity check after the root makes its decision based on local info | 09:41 |
| mattoliverau | I mean yeah we'll have some extra movement that maybe shouldn't have happened.. but is that so bad | 09:41 |
| mattoliverau | Yeah that's true, let the shards confirm maybe | 09:41 |
| mattoliverau | And delete the range if required | 09:42 |
| acoles | it's a valid concern. maybe for first cut we can ignore and accept a shrink/cleave might occur, then optimise | 09:42 |
| mattoliverau | Would the accepter be able to back out | 09:42 |
| mattoliverau | Yeah | 09:43 |
| mattoliverau | I think trying to back out and rollback add too much complexity, when in reality, how likely is it | 09:43 |
| acoles | backing out decisions at the root is something I'd like to ponder more | 09:43 |
| acoles | think I had some comments in the patch about *not* updating the shard timestamps but maybe having just a 'window' of time during which usage updates are frozen assuming a shrink is occurring , rather then permanently ignoring all updates from replicas that have not yet shrunk/merged...just in case the shrink/merge never happens | 09:45 |
| acoles | mattoliverau: the two win I was hoping for are 1. to be able to reason over the process more easily since it is driven from root rather than autonomous neighbour negotiations (imagine debugging multiple neighbour pairs shrinking concurrently) - timburke comment re probe test is also very pertinent and 2. to make shrinking actually be a cleave event -> less code | 09:48 |
| *** abhitechie has joined #openstack-swift | 09:48 | |
| mattoliverau | Yeah, I love both those reasons. Especially the cleave only.. shards can only do one thing, cleave. Smart and awesome idea. Much better then what I had! | 09:49 |
| acoles | mattoliverau: new ideas always benefit from the wealth of thinking that has gone before! | 09:52 |
| -openstackstatus- NOTICE: The CI system will be offline starting at 11:00 UTC (in just under an hour) for Zuul v3 rollout: http://lists.openstack.org/pipermail/openstack-dev/2017-October/123337.html | 10:09 | |
| *** mat128 has joined #openstack-swift | 10:14 | |
| *** tovin07_ has quit IRC | 10:14 | |
| *** MeltedLux has quit IRC | 10:16 | |
| *** m_kazuhiro has quit IRC | 10:18 | |
| *** oshritf has quit IRC | 10:19 | |
| *** oshritf has joined #openstack-swift | 10:21 | |
| *** MeltedLux has joined #openstack-swift | 10:21 | |
| *** oshritf has quit IRC | 10:22 | |
| *** early has quit IRC | 10:29 | |
| *** abhitechie has quit IRC | 10:31 | |
| *** early has joined #openstack-swift | 10:32 | |
| *** abhitechie has joined #openstack-swift | 10:32 | |
| *** openstackgerrit has joined #openstack-swift | 10:43 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift master: WIP: Move swift-drive-audit code to swift/cli/drive_audit https://review.openstack.org/511205 | 10:43 |
| openstackgerrit | HCLTech-SSW proposed openstack/swift master: Add ability to undelete an account. https://review.openstack.org/507808 | 10:43 |
| kota_ | I spent today as bug triage work but I did write tests and cleanup to reach out A BUG :/ | 10:43 |
| kota_ | and the work is still WIP at patch 511205 :/ | 10:44 |
| patchbot | https://review.openstack.org/#/c/511205/ - swift - WIP: Move swift-drive-audit code to swift/cli/driv... | 10:44 |
| *** mat128 has quit IRC | 10:45 | |
| *** abhitechie has quit IRC | 10:48 | |
| acoles | kota_: nice | 10:54 |
| *** oshritf has joined #openstack-swift | 10:58 | |
| *** skudlik has joined #openstack-swift | 10:59 | |
| cbartz | Hello, anybody here to take a look on bug 1712507 ? | 11:03 |
| openstack | bug 1712507 in python-swiftclient "add tempurl auth for object operations" [Undecided,New] https://launchpad.net/bugs/1712507 | 11:03 |
| *** itlinux has joined #openstack-swift | 11:05 | |
| *** mvk has joined #openstack-swift | 11:12 | |
| *** hoonetorg has quit IRC | 11:21 | |
| *** hoonetorg has joined #openstack-swift | 11:34 | |
| *** Shatadru is now known as Shatadru|Gone | 11:38 | |
| *** spectr has quit IRC | 11:44 | |
| *** spectr has joined #openstack-swift | 11:46 | |
| *** mat128 has joined #openstack-swift | 11:50 | |
| *** spectr-RH has joined #openstack-swift | 11:54 | |
| *** m_kazuhiro has joined #openstack-swift | 11:56 | |
| *** spectr has quit IRC | 11:57 | |
| m_kazuhiro | tdasilva: acoles: Are you here now? (If this message is duplicated, I'm sorry. I had trouble in my irc client.) | 11:58 |
| *** dancn has joined #openstack-swift | 12:08 | |
| *** itlinux has quit IRC | 12:10 | |
| *** spectr-RH has quit IRC | 12:13 | |
| dancn | hi, I here just to ping on https://review.openstack.org/#/c/473862/ because I saw remix_tj request of recheck. It was meant to be a really simple patch to review, every step (apart of users creation) require only a cut and paste without changing anything. The test should take no more than 3 minutes on a working OS installation. Feel free to suggest me a better aproach in patch submission. Tnx | 12:13 |
| patchbot | patch 473862 - swift - Add full working example of sharing a container wi... | 12:13 |
| *** spectr has joined #openstack-swift | 12:13 | |
| *** itlinux has joined #openstack-swift | 12:14 | |
| *** spectr-RH has joined #openstack-swift | 12:14 | |
| remix_tj | dancn: hi i've a fresh swift setup here, i'll try today | 12:15 |
| remix_tj | was a recheck only to trigger again jenkins jobs that seems to be failing without any apparent issue | 12:15 |
| dancn | remix_tj: tnx, feel free to ping if required! | 12:16 |
| remix_tj | sure | 12:16 |
| *** mat128_ has joined #openstack-swift | 12:17 | |
| *** spectr has quit IRC | 12:18 | |
| *** wes_dillingham has joined #openstack-swift | 12:18 | |
| *** openstackgerrit has quit IRC | 12:18 | |
| *** mat128 has quit IRC | 12:19 | |
| kota_ | acoles: thx | 12:33 |
| acoles | m_kazuhiro: hi | 12:34 |
| *** itlinux has quit IRC | 12:35 | |
| m_kazuhiro | acoles: hi | 12:35 |
| *** itlinux has joined #openstack-swift | 12:35 | |
| m_kazuhiro | acoles: I want to discuss about symlink behavior in container listing in tomorrow meeting. And I saw you discussed about that with kota in last week. | 12:36 |
| acoles | m_kazuhiro: ok, yes I remember discussing that with kota_ | 12:36 |
| kota_ | m_kazuhiro: hi | 12:36 |
| kota_ | and acoles ;) | 12:37 |
| m_kazuhiro | kota_: hi | 12:37 |
| acoles | m_kazuhiro: did you add it to the meeting agenda? | 12:37 |
| m_kazuhiro | acoles: yes. And I made etherpad page for it. I want you to read it and I want get your opinion. | 12:38 |
| m_kazuhiro | acoles: The etherpad page is https://etherpad.openstack.org/p/swift_symlink_remaining_discussion_points | 12:38 |
| acoles | m_kazuhiro: item 1 on that etherpad? | 12:39 |
| m_kazuhiro | acoles: Yes. | 12:40 |
| *** oshritf has quit IRC | 12:41 | |
| acoles | m_kazuhiro: is one of the strategies implemented in the current patch? | 12:41 |
| acoles | is it 'etag; symlink_path' ? | 12:41 |
| m_kazuhiro | acoles: Yes. In current implementation. symlink_path info is included in etag. | 12:42 |
| acoles | ok. yes I will try to read that before the meeting. thanks for writing it up. | 12:42 |
| remix_tj | dancn: ok, works! the only issue i'm experiencing is that openrc file generated by horizon has an incomplete OS_AUTH_URL (missing version) | 12:43 |
| remix_tj | dancn: but is not a swift issue :-) | 12:43 |
| *** catintheroof has joined #openstack-swift | 12:43 | |
| m_kazuhiro | acoles: Thank you! If you have questions, please write them on the etherpad. I will try to answer them. | 12:45 |
| acoles | m_kazuhiro: ok | 12:45 |
| kota_ | m_kazuhiro: thx for preparing the etherpad, I'll also read it. | 12:49 |
| m_kazuhiro | kota_: Thank you! | 12:49 |
| * kota_ is now at home so no time constraint to do anything | 12:49 | |
| m_kazuhiro | acoles: kota_: Now, I have to go home. See you next meeting. | 12:51 |
| kota_ | good night m_kazuhiro | 12:52 |
| m_kazuhiro | kota_: good night! | 12:52 |
| *** m_kazuhiro has quit IRC | 12:54 | |
| -openstackstatus- NOTICE: Due to unrelated emergencies, the Zuul v3 rollout has not started yet; stay tuned for further updates | 13:06 | |
| *** mat128_ has quit IRC | 13:14 | |
| tdasilva | good morning | 13:17 |
| acoles | tdasilva: good morning | 13:17 |
| kota_ | good morning tdasilva | 13:17 |
| tdasilva | acoles, kota_: hello! :) | 13:18 |
| tdasilva | looks like there was a good discussion on symlinks....catching up.... | 13:19 |
| kota_ | tdasilva: i also am catching up too, though :P | 13:21 |
| *** zhurong has joined #openstack-swift | 13:21 | |
| kota_ | tdasilva: will you attend the meeting today? m_kazuhiro will bring the symlink topic there. | 13:23 |
| tdasilva | kota_: yes, i'm planning to be there | 13:23 |
| kota_ | tdasilva: good to hear! | 13:24 |
| dancn | remix_tj: nice! thanks again | 13:27 |
| dancn | as5712 | 13:28 |
| dancn | sorry wrong focus :( | 13:28 |
| *** itlinux has quit IRC | 13:36 | |
| *** jistr is now known as jistr|mtg | 13:38 | |
| *** peluse has joined #openstack-swift | 13:45 | |
| peluse | hey swifters, anyone got their ears on? | 13:45 |
| tdasilva | peluse!!! | 13:46 |
| peluse | what up man?? | 13:46 |
| tdasilva | good to see you around here! | 13:46 |
| *** mat128 has joined #openstack-swift | 13:46 | |
| peluse | hope things are kicking ass there... I am working on http://spdk.io for the last few months and someone just proposed a Swift interaction! | 13:46 |
| acoles | peluse !?! | 13:46 |
| peluse | Here's their proposal: https://trello.com/c/6eCUC0zC/69-blobkv-uses-swift-as-a-target-system | 13:46 |
| peluse | acoles, howdy!! | 13:46 |
| kota_ | hey peluse! | 13:47 |
| peluse | Hi kota_ ! | 13:47 |
| * peluse missed these swift people :) | 13:47 | |
| *** mat128 has quit IRC | 13:48 | |
| peluse | I can sum up SPDK pretty quick, it's a whole bunch of user mode stuff meant to be used in a single storage node to basically make storage go faster by using userpsace polled mode drivers, mainly NVMe (there's a bunch of other misc stuff too but that's the main thing for this proposal) | 13:48 |
| acoles | peluse: that's interesting. it's likely that diskfile API may get some more attention/abstraction to support the lots-of-small-file work that's going on | 13:49 |
| *** mat128 has joined #openstack-swift | 13:49 | |
| peluse | hmmm... | 13:49 |
| peluse | also, here's a talk I did at SDC last month that's more about a different module in SPDK but has some high level pics and a few perf claims.. https://www.snia.org/sites/default/files/SDC/2017/presentations/Solid_State_Stor_NVM_PM_NVDIMM/Luse_Paul_Verma_Vishal_SPDK_Blobstore_A_Look_Inside_the_NVM_Optimized_Allocator.pdf | 13:50 |
| peluse | how much has diskfile evolved in the last few eyars? | 13:50 |
| acoles | peluse: since EC, not a huge amount IIRC | 13:52 |
| acoles | but we talked at last face to face about breaking out a better abstraction of the lower level read/write operations (to filesystem or to something else...) from the semantics of what is latest valid data, metadata etc | 13:53 |
| peluse | acoles, OK cool thanks. hard to dust off the cobwebs and get a feel for myself as to how feasible or attractive this might be to Swift. I'll go look at some code and see if that helps. In the meantime if you guys could think about that's be great. I'll check back in tonight or you can find me or a few others on #spdk here in freenode | 13:53 |
| acoles | peluse: oh, we did away with separate durable files to save inodes but that's not a huge change | 13:54 |
| peluse | yeah, that's what I can't picture off the top of my head, how low level the abstraction is/was and what kind of effort, beyond the py wrappers for this stuff, this would | 13:54 |
| acoles | peluse: you know the best way to blow away those cobwebs is to do some reviews :P | 13:54 |
| peluse | interesting.... that's cool | 13:55 |
| peluse | LOL! | 13:55 |
| peluse | I've been in C for the last 6 months and am still forgetting to free memory that I allocated ;) | 13:55 |
| *** links has quit IRC | 13:56 | |
| acoles | but you *want* people to buy more memory right? ;) | 13:57 |
| peluse | we want people to but the latest/greatest/fastest CPUs and SSDs for sure | 13:57 |
| peluse | oh, I get it now... | 13:58 |
| peluse | its still early in Phoenix :) | 13:58 |
| acoles | hehe | 13:58 |
| acoles | peluse: are you hosting the 1st spdk hackathon? | 13:59 |
| peluse | yup | 13:59 |
| peluse | Swift-style too! | 13:59 |
| *** cbartz has left #openstack-swift | 14:00 | |
| peluse | (minus the beer bus, no budget for that this year) | 14:00 |
| acoles | are you going to have a crazy bus with rocking horses ? | 14:00 |
| peluse | haha | 14:00 |
| acoles | oh, no beer bus :( | 14:00 |
| *** jistr|mtg is now known as jistr | 14:00 | |
| acoles | I was just about to check flights, no point now | 14:00 |
| peluse | that was fun for sure.... ahh, the good ole days | 14:00 |
| kota_ | yeah, that was funtastic | 14:02 |
| *** spectr-RH has quit IRC | 14:04 | |
| *** chlong has joined #openstack-swift | 14:04 | |
| *** spectr has joined #openstack-swift | 14:04 | |
| *** SkyRocknRoll has quit IRC | 14:06 | |
| peluse | and actually there are a few potential avenues for SPDK+Swift. One, the trello proposal above, is 100% relevant to the SDC preso link I pasted in as it would use blobstore in place of XFS along with poled mode NVMe | 14:16 |
| jrichli | sounds like I missed out on the rocking horses! Hey peluse! | 14:16 |
| peluse | a second, not mentioned in the proposal, would be just to use the polled mode NVMe driver | 14:16 |
| peluse | Hi jrichli! | 14:16 |
| peluse | was that a rocking horse or a saddle? I dunno, clayg would remember.... | 14:16 |
| jrichli | haha | 14:17 |
| peluse | hey, I gotta bolt. Will check in w/you guys later this week!! | 14:17 |
| jrichli | o/ | 14:17 |
| *** zhurong has quit IRC | 14:29 | |
| *** mvk has quit IRC | 14:39 | |
| *** spectr has quit IRC | 14:43 | |
| *** mweshi has joined #openstack-swift | 14:50 | |
| * kota_ is done to read the new symlink etherpad and added some comments and questions | 14:52 | |
| kota_ | and is heading for my bed to get good sleep for tomorrow morning meeting. | 14:53 |
| tdasilva | thanks kota_ ! | 14:55 |
| clayg | It was just saddles. There was no rockin horse on a bus... | 14:55 |
| *** spectr has joined #openstack-swift | 14:57 | |
| jrichli | awe ... | 14:57 |
| *** acoles has left #openstack-swift | 14:58 | |
| *** acoles has joined #openstack-swift | 14:58 | |
| *** ChanServ sets mode: +v acoles | 14:58 | |
| acoles | ok saddle it was | 14:59 |
| kota_ | https://photos.app.goo.gl/PrD8iKn7eNJswmg53 | 14:59 |
| kota_ | not sure, it was saddle or horse (looking at my photo album at google) | 14:59 |
| *** mat128 has quit IRC | 14:59 | |
| acoles | kota_: nice! | 15:00 |
| jrichli | omg, that ceiling is great! Thanks for sharing kota_ ! | 15:00 |
| acoles | peluse: ^^ | 15:00 |
| acoles | what needs to be pointed out is the lack of windows which is fun at 20mph and hair-raising at 60mph | 15:00 |
| *** cshastri has quit IRC | 15:01 | |
| kota_ | good night | 15:01 |
| jrichli | good night | 15:02 |
| *** psachin has quit IRC | 15:02 | |
| tdasilva | definitely a fun ride. | 15:02 |
| acoles | kota_: good night | 15:03 |
| *** chlong has quit IRC | 15:06 | |
| *** links has joined #openstack-swift | 15:11 | |
| tdasilva | acoles, kota_ I'm not sure I understand the concerns regarding adding symlink path to container listings | 15:13 |
| tdasilva | we did something that was very similar to what crypto did, because it had already been proven and simple to implement | 15:13 |
| tdasilva | we did discuss making bigger modification that would require a database change in the containers, but we opted for a simpler change for now | 15:14 |
| *** SkyRocknRoll has joined #openstack-swift | 15:15 | |
| tdasilva | going over kota_ comments on review now see if I can understand things better | 15:15 |
| *** klrmn has joined #openstack-swift | 15:18 | |
| *** gyee has joined #openstack-swift | 15:33 | |
| *** chlong has joined #openstack-swift | 15:37 | |
| *** mweshi has quit IRC | 15:39 | |
| acoles | tdasilva: I think kota_ is concerned about etags being returned to the client with the symlink path if symlink middleware is removed | 15:40 |
| acoles | whereas if it were in content-type it might be considered less of a problem for that to leak out ? IDK? | 15:41 |
| *** gkadam has quit IRC | 15:44 | |
| *** chlong has quit IRC | 15:51 | |
| *** pcaruana has quit IRC | 15:59 | |
| *** rcernin has quit IRC | 15:59 | |
| acoles | tdasilva: I think I prefer using etag if the problem of the symlink info leaking can be fixed | 15:59 |
| acoles | unless symlink target might ever change on a POST in which case it's messy wherever it goes | 16:01 |
| *** wes_dillingham has quit IRC | 16:06 | |
| *** klrmn has quit IRC | 16:07 | |
| *** openstackgerrit has joined #openstack-swift | 16:09 | |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep: Replicate deleted shard ranges https://review.openstack.org/511282 | 16:09 |
| acoles | timburke: mattoliverau ^^ might help with shrinking | 16:10 |
| *** chlong has joined #openstack-swift | 16:10 | |
| *** itlinux has joined #openstack-swift | 16:11 | |
| timburke | good morning | 16:17 |
| tdasilva | acoles: changing target with a POST is not allowed, user must issue a new PUT | 16:25 |
| acoles | tdasilva: yes, that's what I thought. I was just thinking out loud the implications of that ever changing. | 16:26 |
| * acoles afk | 16:27 | |
| *** hseipp has quit IRC | 16:28 | |
| timburke | do we generally think of it as a problem when SLOs turn into small(ish) JSON data when you take SLO out of the pipeline? it seems like trying to disable features after you've started using them is just generally a Bad Idea™ | 16:29 |
| *** dja has quit IRC | 16:32 | |
| *** d0ugal has quit IRC | 16:35 | |
| *** wes_dillingham has joined #openstack-swift | 16:37 | |
| tdasilva | timburke: agree | 16:37 |
| *** SkyRocknRoll has quit IRC | 16:38 | |
| tdasilva | tdasilva: same for crypto...the idea that we should protect ourselves from users adding and removing middleware at will is a bit of news to me... | 16:39 |
| * tdasilva is getting lunch, brb... | 16:40 | |
| *** SkyRocknRoll has joined #openstack-swift | 16:44 | |
| *** dja has joined #openstack-swift | 16:44 | |
| *** chlong has quit IRC | 16:48 | |
| *** links has quit IRC | 16:54 | |
| *** SkyRocknRoll has quit IRC | 16:57 | |
| *** d0ugal has joined #openstack-swift | 16:57 | |
| timburke | tdasilva: crypto's a bit of a special case though -- it's like adding a storage policy, we really *really* expect that you're never taking it out completely. you can disable encryption of *new* objects, but we have an assumption that you still want to read existing data | 17:01 |
| *** klrmn has joined #openstack-swift | 17:03 | |
| *** tesseract has quit IRC | 17:15 | |
| *** links has joined #openstack-swift | 17:27 | |
| openstackgerrit | Tim Burke proposed openstack/swift feature/deep: Replicate deleted shard ranges https://review.openstack.org/511282 | 17:35 |
| openstackgerrit | Tim Burke proposed openstack/swift feature/deep: Shard ranges don't need storage policies https://review.openstack.org/511310 | 17:35 |
| *** torgomatic has joined #openstack-swift | 17:38 | |
| *** ChanServ sets mode: +v torgomatic | 17:38 | |
| *** itlinux has quit IRC | 17:40 | |
| *** geaaru has quit IRC | 17:44 | |
| acoles | timburke: thanks for that fixup | 17:50 |
| timburke | acoles: i *think* it's fair? not really sure why it wasn't inheriting from the main broker class... and also not sure how/whether test_db_replicator should handle other_items... | 17:51 |
| *** links has quit IRC | 17:53 | |
| tdasilva | notmyname: just fyi, i've updated the launchpad bug trends graph... | 17:58 |
| notmyname | thanks | 17:59 |
| notmyname | ... where did I host that .... | 17:59 |
| tdasilva | heh | 17:59 |
| acoles | timburke: I promise I ran tox before I pushed that test. it seems I ignored the negative result though :/ | 18:01 |
| acoles | s/test/patch/ | 18:01 |
| timburke | heh. no worries | 18:02 |
| notmyname | tdasilva: since I've already got it running, what's the docker command to restart with the updated code? | 18:05 |
| tdasilva | notmyname: I believe you will need to rebuild your docker container, right? | 18:05 |
| tdasilva | i mean, i guess I could throw a container up on Docker registry and that would make things easier for you | 18:06 |
| notmyname | looks like I need to restart the box anyway | 18:06 |
| tdasilva | ok, i think you really just need to run this: docker build --rm -t lp_report:dev . | 18:07 |
| notmyname | tdasilva: I did that. and the run command. and `docker ps` shows nothing at all | 18:12 |
| tdasilva | docker ps --all | 18:13 |
| notmyname | ah. ImportError: No module named dateutil.relativedelta | 18:14 |
| tdasilva | ooops..sorry | 18:15 |
| tdasilva | one sec | 18:16 |
| notmyname | np | 18:16 |
| notmyname | https://pypi.python.org/pypi/python-dateutil/ | 18:16 |
| notmyname | install that one? | 18:16 |
| tdasilva | notmyname: yeah, need to add that to requirements.txt | 18:18 |
| tdasilva | just pushed the change to github | 18:18 |
| notmyname | nice! | 18:20 |
| notmyname | it looks good. and the very recent trend is down :-) | 18:20 |
| *** d0ugal has quit IRC | 18:24 | |
| *** gyee has quit IRC | 18:27 | |
| *** gyee has joined #openstack-swift | 18:29 | |
| timburke | i'm not sure it's calculating correctly... try switching to the six-month view, then just look at new bugs -- i really expected to see some dips :-/ | 18:36 |
| tdasilva | notmyname: after updating that I noticed that the number really apply to the time range. | 18:37 |
| tdasilva | timburke: maybe that's what you are seeing too ?? | 18:37 |
| *** gyee has quit IRC | 18:37 | |
| tdasilva | so for the 6 months, it will count new bugs that were opened in these 6 months | 18:38 |
| *** gyee has joined #openstack-swift | 18:38 | |
| tdasilva | we closed or confirmed a bunch of 'New' bugs that were older than 6 months, so the dip doesn't show up in the 6 months graph | 18:38 |
| *** gyee has quit IRC | 18:39 | |
| timburke | but even looking at all time, just New is (or seems to be) non-decreasing | 18:39 |
| timburke | as is just Incomplete | 18:40 |
| tdasilva | you can see that clearly in the graph below too...outoing for the week of 9/24 (6months graph) shows 11 | 18:40 |
| timburke | Open goes up and down... | 18:40 |
| tdasilva | but for all time it is 21 | 18:41 |
| tdasilva | timburke: oh i see what you are saying... | 18:42 |
| timburke | i wonder if it's looking at the *current* status only, not the historical statuses? | 18:42 |
| timburke | but then i don't know why open and in-progress go up & down | 18:43 |
| tdasilva | timburke: at least it does match the number here: https://snag.gy/cSX2MG.jpg | 18:44 |
| *** gyee has joined #openstack-swift | 18:45 | |
| tdasilva | timburke: ahh...I think you might have hinted at the right problem. I remember that I added New to the graph because it was not there before, so maybe the calculation is wrong for New (i.e., just lloking at 'current' value) | 18:45 |
| tdasilva | timburke: will look further | 18:45 |
| notmyname | tdasilva: tim and I were just saying that it would be really cool to put up this graph on the tv on the wall in the office | 18:49 |
| *** ChubYann has joined #openstack-swift | 18:50 | |
| tdasilva | heh, cool | 18:50 |
| notmyname | however, since that's a ... what's the opposite of a "headless" system? ... kiosk! ... since that's a kiosk, I can't do much with the display since they don't have linkable URLs | 18:50 |
| *** wes_dillingham has quit IRC | 18:53 | |
| *** wes_dillingham has joined #openstack-swift | 18:57 | |
| peluse | kota_, just checked out the photos, nice!! | 19:14 |
| *** mvk has joined #openstack-swift | 19:18 | |
| *** oshritf has joined #openstack-swift | 19:19 | |
| *** mat128 has joined #openstack-swift | 19:31 | |
| *** gyee has quit IRC | 19:31 | |
| *** gyee has joined #openstack-swift | 19:51 | |
| *** gyee has quit IRC | 19:57 | |
| *** itlinux has joined #openstack-swift | 20:03 | |
| *** wes_dillingham has quit IRC | 20:06 | |
| notmyname | anyone having issues setting up an all in one today? | 20:14 |
| notmyname | clarkb: do you know anything about any dependency changes that may have been upgraded in the last 24 hours that break things? I'm thinkin setuptools, pbr, disutils, etc | 20:20 |
| *** m_kazuhiro has joined #openstack-swift | 20:21 | |
| clarkb | I dont know of those changing recently | 20:22 |
| notmyname | ok | 20:22 |
| tdasilva | notmyname: testing ansible-saio to see if I run into any issues... | 20:26 |
| notmyname | "pkg_resources.RequirementParseError: Invalid requirement, parse error at "'; python'"" | 20:28 |
| notmyname | some requirements somewhere has "cffi (>=1.7); python_implementation != 'PyPy'" | 20:31 |
| notmyname | in a requirements file | 20:31 |
| notmyname | but not swift | 20:31 |
| notmyname | some dependency somewhere | 20:31 |
| notmyname | upgrading setuptools "fixes" it | 20:34 |
| notmyname | but | 20:34 |
| notmyname | it doesn't answer what's broken | 20:34 |
| notmyname | something somewhere added the "; ..." into a requirements file so it doesn't work with distro version of setuptools | 20:42 |
| notmyname | ah. looks like something related to cryptography | 20:45 |
| timburke | idk... they seem to do a pretty sane thing: https://github.com/pyca/cryptography/blob/2.1/setup.py#L38-L45 | 20:53 |
| kota_ | good morning | 20:57 |
| mattoliverau | Morning | 20:58 |
| notmyname | meeting time | 21:00 |
| notmyname | wooo | 21:00 |
| *** catintheroof has quit IRC | 21:02 | |
| kota_ | oops? I missed coloring at symlink etherpad? | 21:03 |
| clarkb | notmyname: ya the environment marker support is newer but not that new, couple years maybe | 21:08 |
| notmyname | clarkb: right, but in the last 24 hours some transitive dependency has broken stuff. the distro version of setuptools doesn't support the env marker | 21:09 |
| *** joeljwright has joined #openstack-swift | 21:15 | |
| *** ChanServ sets mode: +v joeljwright | 21:15 | |
| *** mat128 has quit IRC | 21:28 | |
| *** oshritf has quit IRC | 21:32 | |
| acoles | kota_: m_kazuhiro export/import of the etherpad loses the author colours :( | 21:35 |
| kota_ | acoles: yeah, i also tried that and sigh :/ | 21:36 |
| *** wes_dillingham has joined #openstack-swift | 21:45 | |
| tdasilva | acoles: maybe authors can add their name to the beginning where they commented | 21:50 |
| acoles | tdasilva: yes, I'll try to annotate my comments that way | 21:50 |
| tdasilva | kota_, m_kazuhiro if you have time, can you summarize the issue 1? Container listing with symlinks? | 21:51 |
| m_kazuhiro | tdasilva: OK | 21:51 |
| m_kazuhiro | In current symlink implementation, if clients GET container and the container contains symlinks, the container listing will include symlink_path information. | 21:53 |
| m_kazuhiro | About that, I think there are 2 discussion points. | 21:53 |
| m_kazuhiro | Discussion points 1 is "Is this function is needed or not?". I think we need to discuss use case for the function. | 21:54 |
| * kota_ is still here to join this discussion | 21:54 | |
| m_kazuhiro | Discussion point 2 is "If we implement the function, how to contain the symlink_path information in container DB?". | 21:55 |
| tdasilva | m_kazuhiro: I think you listed one use case which was auto-tiering. But I think another use case is that users that are doing container listing would like to know how to differentiate between a regular object and a symlink object | 21:55 |
| tdasilva | symlink objects are 'special' in that they are not regular objects, so it would be nice to be able to differentiate them in a container listing | 21:56 |
| kota_ | I think, the key issue is where we should save the symlink path info (we cannot find the design decision anywhare). For my eyes, the hash space in the object table looked painful because we won't be able to remove in the future forever. | 21:56 |
| kota_ | and back to the question "why we need this functionality (i.e. symlink path in the object listing)" | 21:57 |
| * acoles is hanging around for a while | 21:57 | |
| kota_ | the point 1 from m_kazuhiro is from the history. | 21:57 |
| tdasilva | kota_: I think the decision was really made over IRC, which I agree is not good. But I remember we discussed several options and ended up on that one for it's simplicity, since that's what we already do for crypto | 21:58 |
| tdasilva | so we know how it works | 21:58 |
| kota_ | in particular, a bunch of code is written for only for the object listing (e.g. SymlinkConatinerContext) ... | 21:58 |
| timburke | the converse to "we won't be able to remove in the future forever" is that if we don't do it now, we probably never will, because we'd never be able to rely on it being there | 21:59 |
| acoles | timburke: +1 | 21:59 |
| tdasilva | I don't understand 'won't be able to remove in the future forever? | 21:59 |
| kota_ | timburke: yup, that's why we're bringing to get consensus for the design (and future functionality) goal | 22:00 |
| kota_ | bringing the issue to here. | 22:00 |
| acoles | kota_: what is your concern with using the etag/hash field? is it the possibility of symlink middleware being removed and the etag returned to clients being invalid? | 22:01 |
| acoles | or other concern? | 22:01 |
| kota_ | acoles: a couple of things | 22:01 |
| kota_ | acoles: the first may be easy to solve, downgrade compatibility. | 22:02 |
| kota_ | as well as, removing symlink middleware, it will break any object listing once symlink deployed, create symlink, then downgraded. | 22:02 |
| kota_ | perhaps, just set "Upgrade Impact" tag in the commit message might be ok, though. | 22:03 |
| acoles | 'break object listing' because there is a weird hash value? | 22:04 |
| kota_ | yes | 22:04 |
| torgomatic | Is it necessary to know a symlink's target in the container listing, or is it enough to know that an object is a symlink but without knowing its target? | 22:05 |
| tdasilva | torgomatic: the decision to add the target was just a 'if we need to add a flag, let's add information that can be useful' | 22:06 |
| kota_ | torgomatic: idk, as m_kazuhiro said above, for now, we don't have special use case for now. | 22:07 |
| tdasilva | instead of just adding 'symlink=yes' | 22:07 |
| *** joeljwright has quit IRC | 22:07 | |
| tdasilva | I like the analogy of doing a file listing on my terminal | 22:08 |
| tdasilva | symlinks are treated special | 22:08 |
| torgomatic | tdasilva: fair enough. I'm questioning that first "if". Swift setting the content-type to "swift/symlink" (or something like that) would tell the user which objects are symlinks without also telling their targets, but I think (and I could be wrong) that it's possible to set the Content-Type without any changes to container listing code | 22:08 |
| timburke | kota_: what was our approach for slo? if you try to downgrade and remove slo from your pipeline, bad/weird things are going to happen if you created some SLOs in the meantime... | 22:08 |
| torgomatic | tdasilva: heh, I'm using that same analogy in my head... with Content-Type only, that gives us the equivalent of NFS's ReadDirPlus operation | 22:09 |
| timburke | torgomatic: content-type will change on POST | 22:09 |
| torgomatic | to determine a symlink's target, you lstat() it, and that's analogous in my mind to an object HEAD | 22:09 |
| acoles | but on POST the object stops being a symlink correct? | 22:09 |
| kota_ | timburke: it could be weird. perhaps, just fair reason enough to go the current way :/ | 22:10 |
| tdasilva | acoles: what do you mean? | 22:10 |
| acoles | what happens to the symlink target sysmeta when you POST to a symlink? | 22:11 |
| tdasilva | acoles: it remains, because it is a sysmeta | 22:11 |
| kota_ | acoles: the idaa is strategy 2 on the etherpad (about Line 59) | 22:11 |
| tdasilva | torgomatic: I was thinking more like a `ls -l` | 22:11 |
| timburke | acoles: my understanding was that on POST we apply whatever metadata we got (because we have to; we don't know whether there may have been another PUT we haven't heard about), but the symlink is still a symlink | 22:11 |
| tdasilva | timburke: correct | 22:12 |
| kota_ | use Content-Type and accept user updates Content-Type via POST, and it changes symlink to a normal object | 22:12 |
| acoles | tdasilva: timburke yep, got it, just catching up. | 22:12 |
| tdasilva | kota_: we wanted to avoid allowing user to change symlink with POST. if user wants to change to a normal object, then a new PUT is required | 22:13 |
| torgomatic | tdasilva: which calls getdents() and then lstat() on each entry :) | 22:13 |
| tdasilva | if they want to change target, a new PUT is required | 22:13 |
| tdasilva | torgomatic: oic | 22:14 |
| timburke | and everyone *loves* the behavior of `swift list --lh` :P | 22:14 |
| timburke | an extra request for each of 10k items is gonna suck | 22:15 |
| tdasilva | yep | 22:15 |
| acoles | kota_: updating content-type via post won't change symlink to normal object it would just lose the target info from container listing (without new code to fix it up like swift_bytes) | 22:16 |
| m_kazuhiro | acoles: correct. | 22:17 |
| kota_ | and strategy 3 is for the problem, m_kazuhiro? | 22:17 |
| timburke | i'm 99% sure we can lose swift_bytes. i should go write that probe test... | 22:18 |
| m_kazuhiro | kota_: Yes. | 22:18 |
| acoles | which seems undesirable - imagine writing the documentation for that | 22:18 |
| kota_ | anyway, we need to hack container db/replicator to do this. | 22:18 |
| mattoliverau | I kinda like the idea of having the target in the listing because like tdasilva reminds me of ls -l. But it seems the problem is we now have more and more middleware wanting to add more special metadata to the object lines in a container database. And being middleware, there optional, at can be removed. | 22:18 |
| kota_ | i know it's also impacted. | 22:18 |
| kota_ | ok, we shouldn't follow the swift_bytes impl. | 22:18 |
| timburke | one more reason to support https://review.openstack.org/#/c/504472/ :-) | 22:19 |
| patchbot | patch 504472 - swift - Shorten typical proxy pipeline. | 22:19 |
| kota_ | being back to my second concern, I was not sure *we can assume etag column for extra space to synchronized info with sysmeta". The question if from... | 22:19 |
| mattoliverau | So 1, we'd need a way for core swift to ignore anything after the hash or content_type that starts with a ; like its treated as a comment. | 22:19 |
| kota_ | can i add more info (outside from symlink) for future development? | 22:19 |
| kota_ | e.g. some object acl/owner info for s3api compatibility | 22:19 |
| mattoliverau | maybe we finally just need to try and add a metadata field (TEXT) per object row, that can simply store json, that middleware can utilise. Then it's up to them to use | 22:20 |
| acoles | ideally I would think of container server removing everything after ';' in a hash and parsing for key value pairs that it drops into the json listing | 22:20 |
| mattoliverau | with timburke's listing middleware, we can assume everything is json, so middlewares can append what they want themselves to the listing | 22:20 |
| acoles | or what mattoliverau says | 22:21 |
| kota_ | mattoliverau: yeah, the extra filed may be better idea except we need more code change, rolling schema upgrading, etc... | 22:21 |
| mattoliverau | I know | 22:21 |
| mattoliverau | it kinda sucks and adds more work | 22:21 |
| timburke | and the listing middleware would be a sensible place to drop | 22:22 |
| timburke | ';...' strings from hashes... | 22:22 |
| mattoliverau | but we've also have this discussion for what.. SLO, encryption and now symlinks.. we will need to do it again soon I'm sure | 22:22 |
| tdasilva | timburke: i was thinking that too | 22:22 |
| mattoliverau | so when do we bite the bullet? | 22:22 |
| acoles | timburke: wfm too | 22:22 |
| acoles | mattoliverau: never :) | 22:22 |
| mattoliverau | lol | 22:23 |
| kota_ | hehe | 22:23 |
| timburke | mattoliverau: we probably will have to eventually, but not for this. for the metedata lifecycle. we've got etag for object data/sysmeta lifecycle, content-type for its own particular lifecycle, but nothing for the rest of POST | 22:24 |
| timburke | and god help us if we ever make sticky object metadata | 22:25 |
| mattoliverau | lol | 22:25 |
| acoles | I am nervous about attaching data-related state (symlink target) to content-type. swift_bytes did that and it was a pain to handle for fast-POST container updates. So I prefer using the hash field, if we can find a way prevent it ever leaking to clients. | 22:25 |
| acoles | but that does not allow for kota_ 's downgrade scenario | 22:25 |
| tdasilva | acoles: for the downgrade scneario, we could have the code in listing middleware to remove whatever comes after ';...' no? | 22:27 |
| acoles | and if we add a field for this, then that field is associated with *data* timestamp not metadata | 22:27 |
| tdasilva | i guess it would need to be there today?? | 22:28 |
| acoles | tdasilva: I understood kota_ to mean rolling back swift version - did I misunderstand? dropping symlink middleware is not hard to handle - yeah, do it in listing mware. | 22:28 |
| kota_ | correct acoles | 22:28 |
| timburke | i'm not sure how a reasonable concern that is, though. again, just after slo was released, there was no sane way to downgrade if you already created slos | 22:29 |
| kota_ | but i know we had have some non-downgrad-able changes in the past development-cycle so that perhaps, what we should do may be to describe the note somewhere. | 22:29 |
| acoles | we also changed hashes.pkl in a non-downgradeable manner IIRC | 22:29 |
| timburke | i guess you could upgrade without enabling the new feature, then be able to roll back and be fine. but that's true here, too | 22:30 |
| mattoliverau | Well so long as something drops ;.. "today" or soon (maybe a point release). Then rollbacks should be ok if things are stored on the hash. I mean there could be symlinks objects in the cluster that don't work, but like timburke said, same with if you remove SLO middleware etc. | 22:31 |
| mattoliverau | and listing middleware sounds like the right place to append and strip to the listings. So cool? Unless we want a field, that sounds like the best approach. | 22:33 |
| kota_ | mattoliverau, notmyname: fortunately, we will have new release soon (queen-1 milestone)? not sure :P | 22:33 |
| timburke | :-/ worse, you won't be able to *tell* that it was a symlink | 22:33 |
| timburke | (as compared with slo, that is) | 22:33 |
| mattoliverau | timburke: unless you used a symlink content type, but yeah | 22:34 |
| timburke | even if you did, you won't know where it pointed. maybe stuffing it in the container listing *is* helping... | 22:34 |
| acoles | fixing up hash in the listing mware also avoid upgrade ordering problem I think - vs doing it in container server | 22:35 |
| timburke | (whereas with slo, it falls back to some strange format of json, but when you pick it apart you can start to figure out what requests you need to make client-side) | 22:35 |
| kota_ | so... | 22:40 |
| kota_ | it seems like we're feeling "we want symlink info for object listing" for the m_kazuhiro's first question? | 22:41 |
| mattoliverau | yup, well I do :P | 22:41 |
| kota_ | and then, using hash space to save it. | 22:41 |
| kota_ | then, the current issue, where we should extract the info, A) symlink middleware, B) another middleware (listing format, gate keeper, or whatever), C) container-server | 22:42 |
| tdasilva | i think we will keep symlink mw extracting the symlink info, but then listing format should just do a cleanup of whatever is lleft after ;... | 22:43 |
| timburke | i'm really leaning toward keeping it all in symlink, without even a generic stripping mechanism in listings_format. we can call out the awkwardness if you later disable symlink, but i don't know that we should prevent it -- i'm pretty sure that'll be the only client-accessible way to get that info at that point | 22:45 |
| mattoliverau | yeah, mw's probably need to ability to make decisions on how to use there data (ie grab the real hash in crypto) or append. | 22:45 |
| tdasilva | kota_, m_kazuhiro: I am sorry, but I need to leave as I have family waiting for me...I will try to log-in as early as I can tomorrow so we can continue | 22:45 |
| kota_ | tdasilva: np, thanks! | 22:46 |
| m_kazuhiro | tdasilva: OK. Thank you! | 22:46 |
| acoles | I'm out too. good night everyone. | 22:46 |
| m_kazuhiro | For downgrade issue, I think (C) container-server will be nice. Because, we can make etag cleanup patch for past releases (i.e. Pike, Ocata, Newton...). | 22:47 |
| mattoliverau | acoles, tdasilva: o/ | 22:47 |
| kota_ | acoles, tdasilva: good night! | 22:47 |
| m_kazuhiro | acoles: good night! | 22:48 |
| mattoliverau | but how does the container server know what to do with the data? or do you mean anything appended to hash gets appending to the listing json format? | 22:48 |
| m_kazuhiro | I mean that extra values in etag will be removed. | 22:50 |
| m_kazuhiro | In patch for past releases. | 22:50 |
| m_kazuhiro | So, after ';' in etags of container listing will be removed in my idea. | 22:52 |
| mattoliverau | but don't you need to know the target to append as the request comes back from the container-server, unless its all placed into a header as it comes back. | 22:53 |
| timburke | idk, if i'm worried about going from X.0 -> Y.0, i don't think my rollback choice will be Y.0 -> X.1 | 22:54 |
| kota_ | timburke: lol, it doesn't look rollback | 22:55 |
| timburke | exactly | 22:55 |
| m_kazuhiro | OK. I see. Please forget my idea. | 22:55 |
| mattoliverau | I'm leaning toward symlink with potentially stripping in listing middleware (can't use gate keeper cause then it'll need to know how to potenually strip ;.. from xml as well if symlinks are removed). | 22:57 |
| timburke | i mean, you could still reasonably do something like X.0 -> X.1 -> Y.0 with an option at any point of rolling back to the previous known-good state. but you've doubled the validations you'll need to do | 22:57 |
| kota_ | exatcly, backporting seems worse idea. | 22:58 |
| mattoliverau | maybe do things in symlink and then we can add stripping ;.. anytime in the future (though maybe decided before next release?) | 22:58 |
| mattoliverau | because I feel the current road block is what to do. stripping of ;.. is something we can continue to discuss, but this at least moves forward. | 22:59 |
| kota_ | k, so it seems accepting to give up the downgrade scenario and note it <- this seems reasonable to move forward. | 22:59 |
| mattoliverau | yup | 23:00 |
| kota_ | and keeping the strip/translation of hash field in symlink mw | 23:00 |
| kota_ | ok, sounds fine. | 23:00 |
| mattoliverau | I mean we can add stripping (if we decide) and we end up with weird objects that are only slightly worse then if you remove SLOs :) | 23:01 |
| mattoliverau | *SLO middleware. | 23:01 |
| mattoliverau | or we don't strip, but at least people know where things pointed (in the hash). | 23:01 |
| kota_ | *and also Encryption middleware | 23:01 |
| mattoliverau | +1 | 23:01 |
| m_kazuhiro | +1 | 23:02 |
| kota_ | m_kazuhiro: anyting else for now here? | 23:02 |
| mattoliverau | so stripping or not can be a continued discussion, whereas I think we want the target in listing, so do it, and append in symlink. | 23:03 |
| kota_ | my family is now waiting breakfast :P | 23:03 |
| m_kazuhiro | Umm. There are 3 remaining topics in the etherpad page. But we can discuss them in the etherpad. | 23:03 |
| m_kazuhiro | So I think there are no thing for now. If you have time, please read etherpad and add your opinions. | 23:04 |
| mattoliverau | will do. | 23:04 |
| kota_ | m_kazuhiro: could you explain briefly for the remaining topic? | 23:05 |
| mattoliverau | thanks m_kazuhiro for all the hard work | 23:05 |
| kota_ | ah, ok you want to keep the discussion on the etherpad. | 23:05 |
| m_kazuhiro | kota_: yes. because these topics are small. I think we can discuss them on etherpad. | 23:06 |
| kota_ | kk | 23:06 |
| m_kazuhiro | everyone: Thank you for your long discussion. | 23:07 |
| * kota_ noticed it was about 1 hour. | 23:08 | |
| *** vint_bra has quit IRC | 23:11 | |
| kota_ | poke timburke to head up for https://review.openstack.org/#/c/504479/ | 23:11 |
| patchbot | patch 504479 - swift3 - Change log updates for version 1.12 | 23:11 |
| kota_ | will be back a few hours later at my office | 23:12 |
| kota_ | see you | 23:12 |
| *** kei_yama has joined #openstack-swift | 23:33 | |
| *** m_kazuhiro has quit IRC | 23:38 | |
| timburke | kota_: when you get back, mind taking a look at my comments on https://review.openstack.org/#/c/345739/ ? S3 behaved the way i expected, at least in the us-west-2 bucket i usually use for testing such things... | 23:43 |
| patchbot | patch 345739 - swift3 - Add support for more characters in header keys | 23:43 |
| *** itlinux has quit IRC | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
