| *** haomaiwang has quit IRC | 00:01 | |
| *** haomaiwang has joined #openstack-swift | 00:01 | |
| *** daemontool_ has quit IRC | 00:04 | |
| ho_ | good morning! | 00:06 |
|---|---|---|
| *** daemontool_ has joined #openstack-swift | 00:06 | |
| *** blmartin has joined #openstack-swift | 00:13 | |
| mattoliverau | ho_: morning | 00:13 |
| *** daemontool_ has quit IRC | 00:13 | |
| ho_ | mattoliverau: morning! | 00:13 |
| *** tongli has quit IRC | 00:20 | |
| *** fbo has quit IRC | 00:20 | |
| *** fbo_ has joined #openstack-swift | 00:20 | |
| openstackgerrit | Merged openstack/swift-bench: Bump eventlet min version to 0.17.4 https://review.openstack.org/271487 | 00:25 |
| clayg | oh yeah, that reminds me - i wanted to check if i was already packaging psutil | 00:25 |
| clayg | hrmm... yeah but it's only 0.4.1 | 00:26 |
| clayg | whoa - latest from pypi is 3.4.2 (!?) | 00:27 |
| clayg | preciase has 1.2.1 | 00:28 |
| *** nadeem has quit IRC | 00:37 | |
| *** fthiagogv_ has joined #openstack-swift | 00:41 | |
| *** dmorita_ has quit IRC | 00:43 | |
| *** fthiagogv has quit IRC | 00:43 | |
| torgomatic | wow, swift-bench changes merge *fast* | 00:46 |
| torgomatic | I've gotten so used to things taking at least a few hours, but swift-bench changes take less than 2 minutes | 00:46 |
| kota_ | hello | 00:47 |
| torgomatic | hi | 00:47 |
| kota_ | torgomatic: o/ | 00:48 |
| kota_ | torgomatic: speaking of which, did you notice my ping for patch 271920? | 00:49 |
| patchbot | kota_: https://review.openstack.org/#/c/271920/ - swift - CompositeRing Support | 00:49 |
| torgomatic | kota_: yes, I saw that. I'm going to look at it once I'm done with the thing I'm currently working on | 00:50 |
| clayg | onovy: peterlisak: hey before we going making the auditors even slower by nicing them - do either you have any intrest in taking a stab a lp bug #1183656 | 00:50 |
| openstack | Launchpad bug 1183656 in OpenStack Object Storage (swift) "object auditors don't finish" [Undecided,Confirmed] https://launchpad.net/bugs/1183656 | 00:50 |
| kota_ | torgomatic: sounds cool, thanks :-) | 00:50 |
| kota_ | that one is still stuck on gerrit failure though :/ | 00:51 |
| *** m_kazuhiro has joined #openstack-swift | 00:52 | |
| clayg | onovy: peterlisak: I also think maybe psutil is a fairly meaty dependency - I wonder what it might look like to just make a quick little ctypes wrapper like we do for other things in utils? Did you try it to compare the two options? | 00:53 |
| mattoliverau | kota_: o/ | 00:54 |
| kota_ | mattoliverau: o/ | 00:54 |
| *** dmorita has joined #openstack-swift | 00:54 | |
| clayg | onovy: peterlisak: lastly I'm curious how much further you want to take the setpriority options? Do you have any reason to believe they make sense in the the proxy where there's hardly any file-system access? Does it make sense to document them everywhere if all we really want (right now) is to nice the auditors? | 00:54 |
| *** arch-nemesis has quit IRC | 00:57 | |
| *** haomaiwang has quit IRC | 01:01 | |
| *** haomaiwa_ has joined #openstack-swift | 01:01 | |
| *** fthiagogv__ has joined #openstack-swift | 01:02 | |
| *** fthiagogv_ has quit IRC | 01:04 | |
| *** mgarza has quit IRC | 01:10 | |
| *** zhill has quit IRC | 01:11 | |
| *** haomaiwa_ has quit IRC | 01:12 | |
| *** fthiagogv_ has joined #openstack-swift | 01:12 | |
| *** fthiagogv__ has quit IRC | 01:14 | |
| *** dmorita has quit IRC | 01:15 | |
| *** dmorita has joined #openstack-swift | 01:18 | |
| openstackgerrit | Merged openstack/swift: Add __next__() methods to utils iterators for py3 https://review.openstack.org/237040 | 01:19 |
| *** garthb_ has quit IRC | 01:20 | |
| clayg | mattoliverau: haypo's still at it!? | 01:22 |
| *** xiangxinyong has joined #openstack-swift | 01:24 | |
| *** rminmin has joined #openstack-swift | 01:25 | |
| *** Jeffrey4l has joined #openstack-swift | 01:28 | |
| *** david-lyle has quit IRC | 01:36 | |
| *** fthiagogv__ has joined #openstack-swift | 01:36 | |
| *** fthiagogv_ has quit IRC | 01:39 | |
| *** fthiagogv_ has joined #openstack-swift | 01:43 | |
| *** fthiagogv__ has quit IRC | 01:45 | |
| *** diogogmt has quit IRC | 01:48 | |
| *** dmorita has quit IRC | 01:48 | |
| *** dmorita has joined #openstack-swift | 01:53 | |
| *** jmccarthy has quit IRC | 01:54 | |
| *** jmccarthy has joined #openstack-swift | 01:55 | |
| *** haomaiwang has joined #openstack-swift | 01:57 | |
| mattoliverau | clayg: seems it never ends :) | 01:58 |
| *** haomaiwang has quit IRC | 02:01 | |
| *** haomaiwang has joined #openstack-swift | 02:01 | |
| *** xiangxinyong has quit IRC | 02:01 | |
| *** dmorita has quit IRC | 02:06 | |
| *** nexusz99 has joined #openstack-swift | 02:06 | |
| *** fthiagogv__ has joined #openstack-swift | 02:08 | |
| nexusz99 | Why swift using XFS filesystem? Is there any document about that reason? | 02:09 |
| *** fthiagogv_ has quit IRC | 02:10 | |
| clayg | cause like one time 5 years ago someone did some benchmark on file systems and liked XFS the best - and it hasn't really let us down yet? | 02:27 |
| clayg | Gluster likes XFS - so does redhat in general | 02:27 |
| clayg | I like XFS having worked with it awhile | 02:27 |
| clayg | why *not* use XFS!? | 02:27 |
| clayg | and Dave is cool guy | 02:28 |
| clayg | he's helped us out on the ML a couple times - shit liek that | 02:28 |
| clayg | also someone is probably redbo, he takes crap notes | 02:28 |
| *** daemontool has joined #openstack-swift | 02:31 | |
| *** diazjf has joined #openstack-swift | 02:33 | |
| *** fthiagogv_ has joined #openstack-swift | 02:34 | |
| *** dmorita has joined #openstack-swift | 02:34 | |
| nexusz99 | @clayg oh.. thanks. I need to find the benchmark result. | 02:35 |
| *** fthiagogv__ has quit IRC | 02:36 | |
| redbo | https://docs.google.com/spreadsheets/d/1cjMYx-wAqqgvj_pI8HMCpbEDmVGgeFbqVTVyHBgzncs/pubchart?oid=5&format=interactive | 02:36 |
| redbo | https://docs.google.com/spreadsheets/d/1cjMYx-wAqqgvj_pI8HMCpbEDmVGgeFbqVTVyHBgzncs/pubchart?oid=4&format=interactive | 02:37 |
| redbo | I have a lot of data, but it's all a few years old now. Supposedly ext4 has gotten better since then. | 02:38 |
| *** Jeffrey4l has quit IRC | 02:38 | |
| nexusz99 | what does 'FS Traversal Times' mean? | 02:39 |
| redbo | amount of time it takes to enumerate all the files on the filesystem | 02:39 |
| redbo | which is something I care about more than write speed, as long as write speed is reasonable | 02:40 |
| nexusz99 | A ha! Thanks! | 02:42 |
| *** Jeffrey4l has joined #openstack-swift | 02:48 | |
| *** forcebanana has quit IRC | 02:55 | |
| *** Jeffrey4l_ has joined #openstack-swift | 02:57 | |
| *** esker has quit IRC | 02:57 | |
| *** diogogmt has joined #openstack-swift | 02:58 | |
| *** venkat has joined #openstack-swift | 03:00 | |
| *** Jeffrey4l has quit IRC | 03:00 | |
| *** haomaiwang has quit IRC | 03:01 | |
| *** haomaiwa_ has joined #openstack-swift | 03:01 | |
| *** janonymous has joined #openstack-swift | 03:03 | |
| * clayg takes back about redbo's note-taking | 03:07 | |
| *** diazjf has quit IRC | 03:08 | |
| *** sanchitmalhotra has joined #openstack-swift | 03:10 | |
| redbo | Well, there's some crappiness. Like I didn't write down what "EXT4-tuned" means. | 03:12 |
| *** xiangxinyong has joined #openstack-swift | 03:16 | |
| *** nexusz99 has quit IRC | 03:20 | |
| *** nexusz99 has joined #openstack-swift | 03:32 | |
| *** dmorita has quit IRC | 03:48 | |
| *** xiangxinyong has quit IRC | 03:49 | |
| *** rminmin has quit IRC | 03:51 | |
| *** mingdang1 has joined #openstack-swift | 04:00 | |
| *** haomaiwa_ has quit IRC | 04:01 | |
| *** haomaiwang has joined #openstack-swift | 04:01 | |
| mahatic_ | good morning | 04:19 |
| kota_ | mahatic: good morning :-) | 04:20 |
| mahatic_ | kota_: hello | 04:20 |
| *** trifon has joined #openstack-swift | 04:21 | |
| *** dmorita has joined #openstack-swift | 04:21 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Add assertions to mem_server unit tests https://review.openstack.org/272893 | 04:23 |
| *** david-lyle has joined #openstack-swift | 04:25 | |
| *** david-lyle has quit IRC | 04:25 | |
| *** dmorita has quit IRC | 04:25 | |
| *** dmorita has joined #openstack-swift | 04:30 | |
| *** mragupat has joined #openstack-swift | 04:33 | |
| *** david-lyle has joined #openstack-swift | 04:38 | |
| *** trifon has quit IRC | 04:45 | |
| *** dmorita has quit IRC | 04:54 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Follow up mem_server diskfile fixes https://review.openstack.org/272893 | 04:54 |
| *** dmorita has joined #openstack-swift | 04:55 | |
| *** klrmn has quit IRC | 04:56 | |
| *** haomaiwang has quit IRC | 05:01 | |
| *** haomaiwang has joined #openstack-swift | 05:01 | |
| *** forcebanana has joined #openstack-swift | 05:06 | |
| *** ChubYann has quit IRC | 05:14 | |
| *** blmartin has quit IRC | 05:15 | |
| *** dmorita has quit IRC | 05:21 | |
| *** aparnav has joined #openstack-swift | 05:21 | |
| *** dmorita has joined #openstack-swift | 05:28 | |
| *** ChubYann has joined #openstack-swift | 05:29 | |
| *** dmorita has quit IRC | 05:33 | |
| *** dmorita has joined #openstack-swift | 05:33 | |
| *** km has quit IRC | 05:35 | |
| *** km has joined #openstack-swift | 05:39 | |
| notmyname | mattoliverau: still here? | 05:43 |
| notmyname | mattoliverau: note the last 2 tweets https://twitter.com/linuxconfau | 05:44 |
| mattoliverau | notmyname: still here | 05:45 |
| mattoliverau | Yay possible disruptions | 05:46 |
| notmyname | I'm still coming ;-) | 05:49 |
| notmyname | I just checked in for my flight | 05:49 |
| *** ChubYann has quit IRC | 05:49 | |
| mattoliverau | I'm sure it'll be fine, the LCA in Brisbane got washed out to venue changed, but was still awesome :) | 05:51 |
| notmyname | it seems like I do need to pack some rain gear | 05:52 |
| notmyname | http://forecast.io/#/f/-37.8175,144.9671 | 05:52 |
| *** haomaiwang has quit IRC | 06:01 | |
| *** silor has joined #openstack-swift | 06:01 | |
| *** haomaiwa_ has joined #openstack-swift | 06:01 | |
| *** trifon has joined #openstack-swift | 06:03 | |
| *** SkyRocknRoll has joined #openstack-swift | 06:09 | |
| *** silor1 has joined #openstack-swift | 06:10 | |
| mattoliverau | notmyname: and a life raft ;) | 06:11 |
| *** silor has quit IRC | 06:12 | |
| *** silor1 is now known as silor | 06:12 | |
| notmyname | heh | 06:12 |
| mattoliverau | OK, culled my slides down a little. looking a little better now. | 06:12 |
| notmyname | great | 06:12 |
| notmyname | I'm reviewing swift for genomics workloads right now... | 06:12 |
| mattoliverau | so you'll be at the bioinformatics mini conf then I guess :P | 06:13 |
| notmyname | it looked interesting :-) | 06:13 |
| mattoliverau | notmyname: plus, if we get rained out and need to evacualte to a boat, then our LCA tradition continues, notmyname and mattoliverau on a boat at LCAs ;) | 06:13 |
| notmyname | that's true! | 06:14 |
| notmyname | we've been on a boat together in 2 countries. we need to make it 3 | 06:14 |
| mattoliverau | +1 :) | 06:14 |
| *** mragupat has quit IRC | 06:28 | |
| *** nexusz99 has quit IRC | 06:31 | |
| *** ppai has joined #openstack-swift | 06:31 | |
| *** dmorita has quit IRC | 06:45 | |
| *** asettle is now known as asettle-afk | 06:51 | |
| *** haomaiwa_ has quit IRC | 07:01 | |
| *** haomaiwa_ has joined #openstack-swift | 07:01 | |
| *** dmorita has joined #openstack-swift | 07:06 | |
| *** nexusz99 has joined #openstack-swift | 07:07 | |
| *** dmorita has quit IRC | 07:11 | |
| openstackgerrit | Hisashi Osanai proposed openstack/swift: Add unit tests for checking responses in AccountController https://review.openstack.org/271142 | 07:13 |
| *** silor1 has joined #openstack-swift | 07:18 | |
| *** cbartz has joined #openstack-swift | 07:18 | |
| *** silor has quit IRC | 07:21 | |
| *** Jeffrey4l_ has quit IRC | 07:21 | |
| *** kei_yama has quit IRC | 07:21 | |
| *** kei_yama has joined #openstack-swift | 07:21 | |
| *** Jeffrey4l_ has joined #openstack-swift | 07:21 | |
| *** silor1 has quit IRC | 07:22 | |
| *** silor has joined #openstack-swift | 07:23 | |
| *** rcernin has joined #openstack-swift | 07:27 | |
| *** chlong_zzz is now known as chlong | 07:31 | |
| *** SkyRocknRoll has quit IRC | 07:37 | |
| *** zaitcev has quit IRC | 07:38 | |
| openstackgerrit | Hisashi Osanai proposed openstack/swift: Add unit tests for checking responses in ContainerController https://review.openstack.org/271160 | 07:41 |
| *** zhill has joined #openstack-swift | 07:44 | |
| *** fbo_ is now known as fbo | 07:45 | |
| *** SkyRocknRoll has joined #openstack-swift | 07:50 | |
| *** arnox has joined #openstack-swift | 07:58 | |
| *** haomaiwa_ has quit IRC | 08:01 | |
| *** haomaiwang has joined #openstack-swift | 08:01 | |
| *** dmorita has joined #openstack-swift | 08:05 | |
| *** dmorita has quit IRC | 08:09 | |
| *** rledisez has joined #openstack-swift | 08:22 | |
| *** geaaru has joined #openstack-swift | 08:29 | |
| *** asettle-afk has quit IRC | 08:33 | |
| *** km has quit IRC | 08:33 | |
| *** zhill has quit IRC | 08:55 | |
| *** haomaiwang has quit IRC | 09:01 | |
| *** haomaiwang has joined #openstack-swift | 09:01 | |
| *** vinsh has quit IRC | 09:04 | |
| *** asettle has joined #openstack-swift | 09:04 | |
| *** dmorita has joined #openstack-swift | 09:07 | |
| *** venkat has quit IRC | 09:07 | |
| *** nexusz99 has quit IRC | 09:09 | |
| *** acoles_ is now known as acoles | 09:10 | |
| *** dmorita has quit IRC | 09:12 | |
| acoles | good morning | 09:15 |
| *** joeljwright has joined #openstack-swift | 09:19 | |
| *** ChanServ sets mode: +v joeljwright | 09:19 | |
| *** jordanP has joined #openstack-swift | 09:20 | |
| *** nexusz99 has joined #openstack-swift | 09:20 | |
| mattoliverau | acoles: Evening! | 09:21 |
| *** venkat has joined #openstack-swift | 09:22 | |
| acoles | wow that was a short day, good night :) | 09:22 |
| *** takashi_ has joined #openstack-swift | 09:30 | |
| *** Jeffrey4l_ is now known as Jeffrey4l | 09:30 | |
| *** jordan_ has joined #openstack-swift | 09:34 | |
| *** jordanP has quit IRC | 09:34 | |
| *** permalac has joined #openstack-swift | 09:34 | |
| *** jordan_ has quit IRC | 09:34 | |
| *** jordanP has joined #openstack-swift | 09:35 | |
| *** asettle has quit IRC | 09:38 | |
| *** jistr has joined #openstack-swift | 09:43 | |
| *** takashi_ has quit IRC | 09:44 | |
| acoles | ho_: are you still here? | 09:50 |
| *** asettle-afk has joined #openstack-swift | 09:53 | |
| *** forcebanana has quit IRC | 09:56 | |
| *** forcebanana has joined #openstack-swift | 09:56 | |
| *** SkyRocknRoll has quit IRC | 09:57 | |
| *** forcebanana has quit IRC | 09:57 | |
| mingdang1 | @kota_ hello, how to see the swift log | 09:57 |
| *** aix has joined #openstack-swift | 09:57 | |
| mingdang1 | someone can help me ? | 09:58 |
| *** admin6 has joined #openstack-swift | 09:58 | |
| *** haomaiwang has quit IRC | 10:01 | |
| *** haomaiwa_ has joined #openstack-swift | 10:01 | |
| *** venkat has quit IRC | 10:09 | |
| *** m_kazuhiro has quit IRC | 10:09 | |
| *** SkyRocknRoll has joined #openstack-swift | 10:13 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Remove unnecessary code from test_account.py https://review.openstack.org/272979 | 10:20 |
| acoles | ^^ totally trivial review | 10:20 |
| *** haomaiwa_ has quit IRC | 11:01 | |
| *** haomaiwang has joined #openstack-swift | 11:01 | |
| ntt | HI, is there a way for create the thumbnail of an image in swift (eg some middleware)? | 11:05 |
| *** haomaiwang has quit IRC | 11:10 | |
| *** Razva has joined #openstack-swift | 11:10 | |
| Razva | hey! I'm trying to setup a Swift storage and I have a question: is there any way to make the swift machine use its own IP, instead of using the controller as a router? what I'm trying to achieve here is swift content -> net, not swift content -> controller -> net | 11:10 |
| tdasilva | ntt: not in swift core. there might be third-party middleware that does that...sounds pretty cool | 11:13 |
| tdasilva | Razva: I'm assuming by controller you mean Swift proxy, is that correct? if so, no, you have to go through the proxy | 11:14 |
| ntt | tdasilva: yes I found something and I will try to write a middleware. Another question: is there a way to do a kind of "bulk download": Download a list of files as a zip | 11:14 |
| *** mingdang1 has quit IRC | 11:14 | |
| tdasilva | ntt: there's bulk upload, but no download | 11:15 |
| ntt | tdasilva: and write a middleware is a bad idea in your opinion? | 11:15 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Copy more headers when copying object https://review.openstack.org/272724 | 11:15 |
| Razva | tdasilva: no, I'm talking about the OpenStack controller (we have all the services installed on a single machine, except Compute) | 11:16 |
| tdasilva | ntt: for bulk download? | 11:16 |
| ntt | yes | 11:16 |
| tdasilva | acoles: interesting | 11:16 |
| tdasilva | Razva: mmm...not sure what the openstack controller is, sorry | 11:17 |
| tdasilva | Razva: but generally speaking, the user can send a request directly to swift proxy | 11:17 |
| *** kei_yama has quit IRC | 11:18 | |
| tdasilva | ntt: I think you could pull that off with slo, but there's no compression | 11:18 |
| Razva | tdasilva oook so what do I need to do/install in order to achieve this? | 11:26 |
| tdasilva | Razva: you mentioned trying to run everything in one machine, so it sounds like you are in try-out mode. I'd start with setting up a SAIO VM: http://docs.openstack.org/developer/swift/development_saio.html | 11:27 |
| admin6 | Hi, I’m trying to test/validate/understand object replication by putting an object in a rep3 container, then deleting 2 of the 3 replicas directly on their disk location. Now I’m expecting to see something happening in the object replicator or updater logs of my storage nodes, at least during the next hour, but nothing special happens. Any idea? | 11:27 |
| tdasilva | admin6: are you running all swift services? | 11:28 |
| Razva | tdasilva everything except Swift. :| | 11:29 |
| *** ppai has quit IRC | 11:29 | |
| admin6 | it seems so : swift-object-server swift-object-replicator, swift-object-auditor and swift-object-updater are running on all my aco nodes | 11:30 |
| tdasilva | gotta run to a meeting, be back later | 11:30 |
| *** bkumar has joined #openstack-swift | 11:33 | |
| *** bkumar has quit IRC | 11:35 | |
| *** bkumar has joined #openstack-swift | 11:35 | |
| *** ho_ has quit IRC | 11:36 | |
| acoles | admin6: the replicator does not discover objects that have been manually deleted. it uses a cache of the state of the objects in a partition that is updated as objects are put and deleted via the object server API, and if that cache state does not change then the replicator does not find any work to do. | 11:38 |
| admin6 | acoles: ok, but what about a corrupted data file? is’nt supposed the replicator process to check md5sum of files and detect that type of corruption, then takes action to copy it back froma good replica ? I can modify some replica instead of deleting them. | 11:42 |
| *** Farzad has joined #openstack-swift | 11:43 | |
| *** aix has quit IRC | 11:43 | |
| Farzad | Hi there, | 11:43 |
| *** ppai has joined #openstack-swift | 11:43 | |
| *** pcaruana has joined #openstack-swift | 11:44 | |
| Farzad | How can I config swift for multi-connection and resume support downloading? | 11:44 |
| *** takashi_ has joined #openstack-swift | 11:44 | |
| *** asettle-afk has quit IRC | 11:45 | |
| acoles | admin6: the object-auditor process checks length and md5sum, and it it detects corruption it quarantines the file and updates the partition state, which the replicator would then discover has changed. so you'd need to corrupt/truncate a .data file, run auditor, then run replicators | 11:48 |
| *** Farzad has quit IRC | 11:49 | |
| admin6 | acoles: Ok so I should expect something in my object-auditor.log | 11:49 |
| admin6 | acoles: cool : ERROR Object /srv/node/staco1z4d02/objects/246773/923/f0fd7353eaccab3450da4cac094fb923 failed audit and was quarantined: Exception reading metadata | 11:50 |
| admin6 | acoles: and now, I should expect log from the replicator logs | 11:51 |
| acoles | admin6: should be | 11:55 |
| admin6 | acoles: the replicator is running every 30 seconds, but didn’t do anything for my quarantined object… | 11:56 |
| acoles | tdasilva: versioned_writes should be *after* slo and dlo, correct? thats what the sample config file says, and where I think it should be, but its not where it gets auto-inserted :/ | 11:58 |
| *** Razva has quit IRC | 12:05 | |
| *** bkumar has quit IRC | 12:06 | |
| mahatic_ | acoles: I removed it. I started my services, ran some tests, it's still not auto-inserted, should I do something else too? | 12:08 |
| acoles | admin6: when you didn't do anything, do you mean the object file is still corrupt, or there is nothing obvious in the logs? | 12:10 |
| acoles | admin6: also, the replicator needs to have run once after the object was written and before the auditor run (I just realised that while experimenting myself) | 12:11 |
| acoles | mahatic_: so when your proxy starts up it logs its pipeline 'Pipeline was modified.' - you don't see that? | 12:12 |
| *** mingdang1 has joined #openstack-swift | 12:14 | |
| mahatic_ | acoles: nope! | 12:16 |
| mahatic_ | acoles: I was on feature/crypto, but it shoudn't matter right. I tested on master again. I don't see it in the piple as well, nor the message | 12:17 |
| mahatic_ | pipeline* | 12:17 |
| acoles | mahatic_: in proxy.log? you'll only see it if you had a mandatory middleware missing from your config file pipeline | 12:18 |
| mahatic_ | acoles: sorry | 12:18 |
| mahatic_ | acoles: I do see the message | 12:18 |
| acoles | mahatic_: phew :) | 12:18 |
| mahatic_ | acoles: and yes it is before slo! | 12:18 |
| mahatic_ | acoles: :) | 12:18 |
| acoles | mahatic_: thanks for confirming that I am not insane | 12:18 |
| acoles | mahatic_: well, not more insane than usual | 12:19 |
| *** daemontool has quit IRC | 12:19 | |
| mahatic_ | acoles: lol. rather I confirmed, I am indeed running swift :P | 12:19 |
| admin6 | acoles: I got no logs from the replicator, however the 2 replicas file that I have corrupt disapeared, even their directory as disapear (f0fd7353eaccab3450da4cac094fb923) | 12:19 |
| admin6 | acoles: I’ve only one replica that is valid, ans nothing has been created / recreated either in the base devices nor in the handoff devices. | 12:20 |
| acoles | mahatic_: yeah, i was beginning to wonder if you were on havana release or somethinf :) | 12:22 |
| acoles | admin6: the auditor will remove the dirs for corrupt objects | 12:22 |
| acoles | admin6: if you look in the parent dir that remains do you see a hashes.pkl file? | 12:22 |
| acoles | admin6: i.e. <device>/objects/<partition> | 12:23 |
| admin6 | no, directory /srv/node/staco1z4d02/objects/246773/923 is empty | 12:23 |
| mahatic_ | acoles: lol. git told me I'm only 31 commits behind the master ;) | 12:23 |
| acoles | admin6: you need to have PUT object, run replicators, then corrupt object, then run auditor, then run replicator | 12:24 |
| admin6 | acoles: Ok, I’ll retry the process from the begining | 12:24 |
| *** aix has joined #openstack-swift | 12:25 | |
| acoles | admin6: the hashes.pkl would be in /srv/node/staco1z4d02/objects/246773, but now I think about it, that's not necessarily a great indicator | 12:26 |
| acoles | admin6: btw, fwiw the hashes.pkl file is where the replicator maintains its perspective of the partition's state. | 12:27 |
| admin6 | acoles: I’ve not a single hashses.pkl files anywhere on my devices… | 12:31 |
| acoles | admin6: and you have put object(s) and run the object-replicators? | 12:32 |
| admin6 | acoles: yes. | 12:32 |
| *** SkyRocknRoll has quit IRC | 12:33 | |
| acoles | admin6: weird. and you never see your corrupted object files restored? if sp, seems like there is some issue with replicator. | 12:36 |
| acoles | admin6: sorry, have to go for a while | 12:36 |
| admin6 | acoles: I’m wondering if tat could’nt come from a misconfiguration on my side regarding dedicated network replicas. I’ve declared two different networks in each devices such as : | 12:37 |
| admin6 | id region zone ip address port replication ip replication port name weight partitions balance meta | 12:37 |
| admin6 | 0 1 1 10.10.1.51 6000 10.10.2.51 6003 staco1z1d01 4000.00 54237 0.00 | 12:37 |
| *** pcaruana has quit IRC | 12:37 | |
| admin6 | but I did not configure specific things elsewhere in the conf files | 12:38 |
| *** daemontool has joined #openstack-swift | 12:39 | |
| *** ppai has quit IRC | 12:41 | |
| admin6 | acoles: nothing listen on port 6003. That should probably be part of my problem :-) | 12:45 |
| *** SkyRocknRoll has joined #openstack-swift | 12:49 | |
| *** takashi_ has quit IRC | 12:51 | |
| *** aparnav has quit IRC | 12:54 | |
| *** ppai has joined #openstack-swift | 12:55 | |
| *** SkyRocknRoll has quit IRC | 12:59 | |
| *** david-lyle has quit IRC | 13:08 | |
| *** haomaiwang has joined #openstack-swift | 13:14 | |
| *** chlong has quit IRC | 13:15 | |
| *** chlong has joined #openstack-swift | 13:28 | |
| *** ho_away has joined #openstack-swift | 13:32 | |
| openstackgerrit | Hisashi Osanai proposed openstack/swift: Add unit tests for checking responses in AccountController https://review.openstack.org/271142 | 13:36 |
| acoles | admin6: any luck changing your port? | 13:37 |
| admin6 | acoles: no, I’m still testing without succes. however I’m not sure that my swift-ring-builder object.builder —replication-port 6003 —change-replication-port 6000 worked becaus md5sul of object.ring.gz is still the same. | 13:41 |
| acoles | admin6: I think you need to do a write-ring after using set-info to change your replication ports | 13:47 |
| acoles | admin6: but I'm not hugely familiar with those commands | 13:48 |
| *** ho_away has quit IRC | 13:48 | |
| admin6 | acoles: that’s not things we change everyday ;-) | 13:49 |
| admin6 | acoles: now after applying write_rings, I see a lot of rsync logs in my object-replicator.log. Seems that it is applying all the backlogs :-) I’m confifent it will work better now | 13:53 |
| *** zul has joined #openstack-swift | 13:59 | |
| *** daemontool has quit IRC | 14:00 | |
| admin6 | acoles: Thanks for your help :-) | 14:00 |
| *** proteusguy has quit IRC | 14:00 | |
| *** haomaiwang has quit IRC | 14:01 | |
| acoles | admin6: np. glad to hear you got it sorted. remember not to manually delete diskfiles for fun though :) | 14:01 |
| *** haomaiwang has joined #openstack-swift | 14:01 | |
| *** diogogmt has quit IRC | 14:01 | |
| admin6 | acoles :-D I’ll try to remember | 14:01 |
| *** haomaiwang has quit IRC | 14:02 | |
| *** haomaiwa_ has joined #openstack-swift | 14:02 | |
| peterlisak | acoles, admin6: hi, i have one patch 253038 ... it compares ring-file and ring-builder (checks if ring file is up-to-date), IMO it could help in your issue, gives you info to do a write-ring. | 14:02 |
| patchbot | peterlisak: https://review.openstack.org/#/c/253038/ - swift - Add info about state of ring file to default command. | 14:02 |
| *** haomaiwa_ has quit IRC | 14:03 | |
| *** sanchitmalhotra has quit IRC | 14:03 | |
| *** haomaiwa_ has joined #openstack-swift | 14:03 | |
| *** diogogmt has joined #openstack-swift | 14:03 | |
| *** fthiagogv_ has quit IRC | 14:03 | |
| *** haomaiwa_ has quit IRC | 14:04 | |
| *** haomaiwang has joined #openstack-swift | 14:04 | |
| admin6 | peterlisak: Thanks | 14:04 |
| *** haomaiwang has quit IRC | 14:05 | |
| *** haomaiwa_ has joined #openstack-swift | 14:05 | |
| *** haomaiwa_ has quit IRC | 14:06 | |
| *** haomaiwa_ has joined #openstack-swift | 14:06 | |
| *** haomaiwa_ has quit IRC | 14:07 | |
| *** haomaiwa_ has joined #openstack-swift | 14:07 | |
| *** haomaiwa_ has quit IRC | 14:08 | |
| *** haomaiwang has joined #openstack-swift | 14:08 | |
| *** haomaiwang has quit IRC | 14:09 | |
| *** haomaiwang has joined #openstack-swift | 14:09 | |
| *** haomaiwang has quit IRC | 14:10 | |
| *** haomaiwang has joined #openstack-swift | 14:10 | |
| *** haomaiwang has quit IRC | 14:11 | |
| *** haomaiwang has joined #openstack-swift | 14:11 | |
| *** haomaiwang has quit IRC | 14:12 | |
| *** haomaiwa_ has joined #openstack-swift | 14:12 | |
| *** haomaiwa_ has quit IRC | 14:13 | |
| *** haomaiwang has joined #openstack-swift | 14:13 | |
| *** haomaiwang has quit IRC | 14:14 | |
| *** haomaiwang has joined #openstack-swift | 14:14 | |
| *** haomaiwang has quit IRC | 14:15 | |
| *** haomaiwa_ has joined #openstack-swift | 14:15 | |
| *** haomaiwa_ has quit IRC | 14:16 | |
| *** haomaiwa_ has joined #openstack-swift | 14:16 | |
| *** haomaiwa_ has quit IRC | 14:17 | |
| *** david-lyle has joined #openstack-swift | 14:17 | |
| *** haomaiwa_ has joined #openstack-swift | 14:17 | |
| *** proteusguy has joined #openstack-swift | 14:17 | |
| *** mingdang1 has quit IRC | 14:17 | |
| *** haomaiwa_ has quit IRC | 14:18 | |
| *** haomaiwang has joined #openstack-swift | 14:18 | |
| *** haomaiwang has quit IRC | 14:19 | |
| *** haomaiwa_ has joined #openstack-swift | 14:19 | |
| *** haomaiwa_ has quit IRC | 14:20 | |
| *** haomaiwa_ has joined #openstack-swift | 14:20 | |
| *** haomaiwa_ has quit IRC | 14:21 | |
| *** diogogmt has quit IRC | 14:21 | |
| *** haomaiwa_ has joined #openstack-swift | 14:21 | |
| *** haomaiwa_ has quit IRC | 14:22 | |
| *** haomaiwang has joined #openstack-swift | 14:22 | |
| *** haomaiwang has quit IRC | 14:23 | |
| *** haomaiwa_ has joined #openstack-swift | 14:23 | |
| *** haomaiwa_ has quit IRC | 14:24 | |
| *** haomaiwang has joined #openstack-swift | 14:24 | |
| *** haomaiwang has quit IRC | 14:25 | |
| *** 64MAAWZHI has joined #openstack-swift | 14:25 | |
| *** 64MAAWZHI has quit IRC | 14:26 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 14:26 |
| *** haomaiwa_ has joined #openstack-swift | 14:26 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Insert versioned_writes in correct pipeline position https://review.openstack.org/273073 | 14:31 |
| *** daemontool has joined #openstack-swift | 14:31 | |
| acoles | tdasilva: jrichli ^^ phew this copy/versions stuff is sooo complicated. turns out the bug 1537042 I saw was actually due to versioned_writes being auto-inserted ahead of SLO in pipeline :/ | 14:32 |
| openstack | bug 1537042 in OpenStack Object Storage (swift) "versioned_writes middleware is mis-placed in proxy pipeline" [Undecided,In progress] https://launchpad.net/bugs/1537042 - Assigned to Alistair Coles (alistair-coles) | 14:33 |
| acoles | when it is after SLO there are different bugs then jrichli was right and the slo copy_hook is not called, because the copy request uses a new environ and the copy_hook is not copied into it that environ. | 14:34 |
| acoles | but when versioned_writes is after SLO there are other bugs like bug 1365862 which maybe jrichli's patch fixes? | 14:35 |
| openstack | bug 1365862 in OpenStack Object Storage (swift) "Object Versioning for Large Object does not work" [Undecided,Confirmed] https://launchpad.net/bugs/1365862 - Assigned to Zhou Yuan (yuan-zhou) | 14:35 |
| acoles | notmyname: ^^ 'cost of complexity' !! | 14:36 |
| *** SkyRocknRoll has joined #openstack-swift | 14:37 | |
| *** ho_away has joined #openstack-swift | 14:41 | |
| *** david-lyle has quit IRC | 14:44 | |
| openstackgerrit | Hisashi Osanai proposed openstack/swift: Add unit tests for checking responses in ContainerController https://review.openstack.org/271160 | 14:53 |
| *** nadeem has joined #openstack-swift | 14:53 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Copy headers correctly when copying object https://review.openstack.org/272724 | 14:54 |
| acoles | jrichli: I think I may now be competing with your patch ^^ | 14:55 |
| *** nadeem has quit IRC | 14:57 | |
| *** nadeem has joined #openstack-swift | 14:58 | |
| *** haomaiwa_ has quit IRC | 15:01 | |
| *** haomaiwang has joined #openstack-swift | 15:01 | |
| *** zul has quit IRC | 15:03 | |
| jrichli | acoles: I had noticed you uploaded that patch yesterday, and was gonna chat with you about it this morning. | 15:08 |
| jrichli | I just got in and need to visit the caf to get sustenance - brb | 15:10 |
| acoles | jrichli: i am not *intentionally* competing with you. your patch helped me track down the real bug, cos your patch failed func tests on my saio | 15:10 |
| *** diazjf has joined #openstack-swift | 15:11 | |
| jrichli | acoles: oh geez, of course not. that is not why i wanted to chat! it did seem like you had figured some things out :-) | 15:11 |
| acoles | jrichli: lol. so not *that* kind of chat then :) | 15:12 |
| acoles | jrichli: so you were right about the copy_hook not being called | 15:13 |
| *** lcurtis has joined #openstack-swift | 15:13 | |
| *** blmartin has joined #openstack-swift | 15:14 | |
| *** artide has joined #openstack-swift | 15:15 | |
| *** bkumar has joined #openstack-swift | 15:17 | |
| *** mingdang1 has joined #openstack-swift | 15:18 | |
| *** bkumar has quit IRC | 15:20 | |
| pdardeau | good morning | 15:20 |
| *** mingdang1 has quit IRC | 15:23 | |
| *** ppai has quit IRC | 15:24 | |
| *** cschwede has quit IRC | 15:34 | |
| *** eranrom has joined #openstack-swift | 15:35 | |
| *** mragupat has joined #openstack-swift | 15:35 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Remove a print statement from tests https://review.openstack.org/273113 | 15:36 |
| acoles | ^^ another trivial review | 15:36 |
| *** artide has quit IRC | 15:38 | |
| tdasilva | acoles, jrichli: you guys are right on top of things | 15:38 |
| tdasilva | jrichli: do you still have that patch that fixes slo on patch 260179? | 15:39 |
| patchbot | tdasilva: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY | 15:39 |
| eranrom | acoles: Any place where I can comment on the fastpost spec? | 15:39 |
| eranrom | acoles: I understand you have deviated, but want to make sure I understand some details | 15:39 |
| acoles | eranrom: the spec has merged. you can propose changes, just like a code change, in the swift-specs repo | 15:40 |
| eranrom | acoles: alright | 15:40 |
| acoles | eranrom: or if you see where it needs clarification then propose an update | 15:40 |
| jrichli | tdasilva: yes. i am just catching up on scrollback. do you want me to upload that, or show you what I did? | 15:40 |
| eranrom | acoles,: ok, will do | 15:41 |
| tdasilva | jrichli: feel free to go ahead an upload, I was hoping to finish today, but I'm running out of time...flying back home today and need to finish packing | 15:41 |
| acoles | tdasilva: back to the snow? | 15:41 |
| *** cschwede has joined #openstack-swift | 15:42 | |
| tdasilva | acoles: yep! :( | 15:42 |
| *** cbartz has left #openstack-swift | 15:42 | |
| acoles | tdasilva: ouch | 15:43 |
| tdasilva | acoles: hehe...but that's ok..missing home by now | 15:43 |
| *** awelleck has joined #openstack-swift | 15:43 | |
| tdasilva | so I will be missing meeting later tonight | 15:44 |
| acoles | tdasilva: before you leave, where should versioned_writes should be w.r.t. slo? https://review.openstack.org/273073 | 15:44 |
| acoles | is the doc wrong or the code? | 15:44 |
| tdasilva | acoles: you are correct, it should be after slo | 15:44 |
| acoles | tdasilva: thanks | 15:44 |
| tdasilva | it was my mistake to not add it there, I was probably thinking that by just adding that entry after slo would be enough | 15:44 |
| jrichli | tdasilva: ok, thanks. Have a safe trip! | 15:46 |
| *** dmorita has joined #openstack-swift | 15:47 | |
| acoles | tdasilva: its not obvious! and I think it might work on master if the other middlewares are also auto-inserted, because of the auto-insert ordering. | 15:47 |
| tdasilva | acoles: yeah, spelling it out is the right way to go to avoid any mess up | 15:48 |
| *** nadeem has quit IRC | 15:49 | |
| *** ho_away has quit IRC | 15:51 | |
| *** dmorita has quit IRC | 15:51 | |
| *** dmorita has joined #openstack-swift | 15:55 | |
| *** dmorita has quit IRC | 16:00 | |
| *** haomaiwang has quit IRC | 16:01 | |
| *** 21WAAW3MN has joined #openstack-swift | 16:01 | |
| *** mgarza_ has joined #openstack-swift | 16:03 | |
| *** rcernin has quit IRC | 16:09 | |
| pchng | acoles: So if I understand your comments in patch 266545, the change is basically to ensure that in diskfile.BaseDiskFile._construct_from_data_file(), any sysmeta found in the .meta files should take precedence? This is what I gathered from the updates to the test_server.py you suggested. (I made these changes, and the updated UT passed) | 16:12 |
| patchbot | pchng: https://review.openstack.org/#/c/266545/ - swift (feature/crypto) - Ensure sysmeta is written/updated on (fast) POST | 16:12 |
| *** natarej_ has quit IRC | 16:13 | |
| *** natarej_ has joined #openstack-swift | 16:13 | |
| *** natarej_ has quit IRC | 16:13 | |
| *** natarej has joined #openstack-swift | 16:14 | |
| *** david-lyle has joined #openstack-swift | 16:14 | |
| *** klrmn has joined #openstack-swift | 16:16 | |
| *** diazjf has quit IRC | 16:17 | |
| *** tdasilva has quit IRC | 16:21 | |
| *** tdasilva has joined #openstack-swift | 16:26 | |
| timburke | good morning | 16:27 |
| *** diazjf has joined #openstack-swift | 16:28 | |
| *** forcebanana has joined #openstack-swift | 16:28 | |
| *** arch-nemesis has joined #openstack-swift | 16:33 | |
| acoles | timburke: good morning | 16:34 |
| acoles | pchng: see this line https://github.com/openstack/swift/blob/master/swift/obj/diskfile.py#L1831-1831 | 16:34 |
| acoles | pchng: that ^^ always replaces any sysmeta keys found in .meta with sysmeta from .data file | 16:35 |
| acoles | pchng: you'll need to change that so that the .data file sysmeta is added to metadata first and then .meta file sysmeta keys are added, taking precedence | 16:36 |
| *** Toni_ has quit IRC | 16:37 | |
| acoles | pchng: but note that we are talking about x-object-sysmeta-* keys only. its a little confusing because this is NOT the keys in DATAFILE_SYSTEM_METADATA. | 16:37 |
| *** david-lyle has quit IRC | 16:38 | |
| acoles | pchng: in fact DATAFILE_SYSTEM_METADATA might be better named IMMUTABLE_METADATA - those immutable metadata items should always take precedence over keys in .meta file, that should not be changed. | 16:38 |
| *** david-lyle has joined #openstack-swift | 16:40 | |
| openstackgerrit | Janie Richling proposed openstack/swift: WIP: Change POST-as-COPY to preserve the DLO manifest contents https://review.openstack.org/252791 | 16:41 |
| jrichli | acoles ^^ | 16:41 |
| jrichli | acoles: as if you didn't have enough on your mind ;-) | 16:41 |
| *** zaitcev has joined #openstack-swift | 16:52 | |
| *** ChanServ sets mode: +v zaitcev | 16:52 | |
| *** ho_away has joined #openstack-swift | 16:52 | |
| acoles | jrichli: could it be the test assertion is wrong on line 2456. you updated the x-object-manifest to point to the manifest itself i think. | 16:52 |
| acoles | so the contents of the large object would be the contents of the manifest object when the multipart-manifest=get is omitted from the POST | 16:53 |
| jrichli | acoles: oh geez! | 16:54 |
| pchng | acoles: Thanks, appreciate the explanation! | 16:54 |
| acoles | and in fact when the param is included the test proves a bug? which is that the POST fails to overwrite the original x-object-manifest | 16:55 |
| acoles | so its a neat test! | 16:55 |
| jrichli | another bug in large objects. ugh! | 16:56 |
| *** diogogmt has joined #openstack-swift | 16:56 | |
| jrichli | or is that overlapping with what will be fixed with your recent patch? | 16:56 |
| *** ho_away has quit IRC | 16:57 | |
| *** trifon has quit IRC | 16:58 | |
| *** jistr has quit IRC | 16:59 | |
| *** admin0 has joined #openstack-swift | 17:00 | |
| admin0 | hello all | 17:00 |
| acoles | idk, what should we expect when POSTing a new x-object-manifest header but also using multipart-manifest=get? for sure, we do not expect the manifest to have the large object copied into its body, which is the case, phew, but which x-object-manifest header should win? I would argue the new one in the POST, so that's a bug | 17:00 |
| acoles | jrichli: ^^ | 17:00 |
| admin0 | when new storage node is added, is it the objects that are moved, or partitions ? | 17:00 |
| *** 21WAAW3MN has quit IRC | 17:01 | |
| *** awelleck1 has joined #openstack-swift | 17:01 | |
| *** haomaiwang has joined #openstack-swift | 17:01 | |
| admin0 | nv | 17:02 |
| admin0 | nvm | 17:02 |
| acoles | pchng: i meant to say, i'd expect some unit tests to break when you change things there | 17:02 |
| acoles | pchng: so there'll be some more work to fix those | 17:03 |
| pchng | acoles: Yes, already seeing some of that with changes to diskfile.py | 17:03 |
| *** fthiagogv has joined #openstack-swift | 17:03 | |
| *** awelleck has quit IRC | 17:03 | |
| pchng | acoles: Haven't looked at the func, probe tests yet but will check/compare to see if any changes to make as well | 17:04 |
| admin0 | i have some ring related questions .. suppose i have 4 zones are 1000 containers .. do each zone get 250 containers ? | 17:07 |
| *** dmorita has joined #openstack-swift | 17:07 | |
| admin0 | and if data replication is = 3, does each conttainer in the same zone have 1/3 of the data ? | 17:07 |
| admin0 | if replicaiton = 3, is it 1 per zone, or is it 3 containers in the same zone ? | 17:07 |
| *** dmorita has quit IRC | 17:11 | |
| *** sanchitmalhotra has joined #openstack-swift | 17:15 | |
| *** klrmn has quit IRC | 17:17 | |
| *** rledisez has quit IRC | 17:17 | |
| *** arnox has quit IRC | 17:17 | |
| *** jmccarthy has quit IRC | 17:20 | |
| *** admin6 has quit IRC | 17:20 | |
| *** McMurlock has quit IRC | 17:22 | |
| *** McMurlock has joined #openstack-swift | 17:22 | |
| *** permalac has quit IRC | 17:22 | |
| *** McMurlock has quit IRC | 17:26 | |
| *** gmmaha has quit IRC | 17:28 | |
| openstackgerrit | Darrell Bishop proposed openstack/swift: Allow IPv6 addresses/hostnames in StatsD target https://review.openstack.org/270991 | 17:29 |
| *** McMurlock has joined #openstack-swift | 17:29 | |
| *** pgbridge_ has quit IRC | 17:33 | |
| *** dmorita has joined #openstack-swift | 17:34 | |
| *** dmorita has quit IRC | 17:35 | |
| *** Razva has joined #openstack-swift | 17:35 | |
| Razva | hey folks! how in the world can I find the Swift S3 "Access Key" and "Access Secret"? | 17:36 |
| *** aix has quit IRC | 17:36 | |
| *** pgbridge_ has joined #openstack-swift | 17:36 | |
| timburke | Razva: what auth system are you using? | 17:37 |
| timburke | keystone, tempauth, swauth, something else... | 17:37 |
| *** dmorita has joined #openstack-swift | 17:37 | |
| Razva | timburke keystone | 17:38 |
| *** dmorita has quit IRC | 17:38 | |
| *** dmorita has joined #openstack-swift | 17:39 | |
| timburke | Razva: `openstack ec2 credentials list` is probably a good place to start; if there aren't any credentials defined, `openstack ec2 credentials create` | 17:42 |
| timburke | it's been a while since i've had to do either of those, though; i don't remember the details real well | 17:43 |
| *** awelleck1 has left #openstack-swift | 17:46 | |
| Razva | timburke swift = s3 not ec2. should I search for ec2 or s3? | 17:49 |
| *** gmmaha has joined #openstack-swift | 17:51 | |
| admin0 | The minimum number of hours before a partition can be reassigned is 8 — | 17:51 |
| *** dmsimard has quit IRC | 17:51 | |
| admin0 | how to change that value to less ? | 17:51 |
| admin0 | reassigned = reshuffeled for re-balancing ? | 17:51 |
| timburke | Razva: still use the ec2 cred commands; i think they just lump all of the AWS-like credentials under one name. at any rate, it's what swift3's functional tests use: https://github.com/openstack/swift3/blob/v1.9/swift3/test/functional/setup_keystone#L56-L57 | 17:52 |
| *** ho_away has joined #openstack-swift | 17:54 | |
| *** garthb has joined #openstack-swift | 17:55 | |
| *** eranrom has quit IRC | 17:55 | |
| *** zhill has joined #openstack-swift | 17:59 | |
| timburke | 181 subcommands is *too many* subcommands... | 17:59 |
| *** ho_away has quit IRC | 17:59 | |
| *** rcernin has joined #openstack-swift | 17:59 | |
| *** haomaiwang has quit IRC | 18:01 | |
| *** haomaiwang has joined #openstack-swift | 18:01 | |
| *** Jeffrey4l has quit IRC | 18:02 | |
| admin0 | is there a tool to show/calculate total swift object size ? | 18:04 |
| admin0 | total swift usage ? | 18:04 |
| admin0 | in GB or TB | 18:04 |
| *** Microwise has joined #openstack-swift | 18:12 | |
| *** klrmn has joined #openstack-swift | 18:17 | |
| *** jordanP has quit IRC | 18:18 | |
| Razva | timburke when connecting with s3curl I get "<html><h1>Not Found</h1><p>The resource could not be found.</p></html>" | 18:22 |
| Razva | so it connects (I've tried with a dummy IP and it cont connection refused) | 18:22 |
| *** nadeem has joined #openstack-swift | 18:24 | |
| timburke | Razva: are you trying to list buckets/containers, list keys/objects, or get a particular object? what's your swift3 config look like? (maybe this would be better over in #swift3...) | 18:25 |
| Razva | timburke any command that - via s3curl - will show me that I can connect to the Swift instance | 18:29 |
| Razva | so this way I can see if it's a networking/firewall problem | 18:29 |
| *** vinsh has joined #openstack-swift | 18:41 | |
| *** cdelatte has joined #openstack-swift | 18:43 | |
| *** daemontool_ has joined #openstack-swift | 18:45 | |
| *** daemontool has quit IRC | 18:47 | |
| *** Microwise has quit IRC | 18:51 | |
| *** haomaiwang has quit IRC | 19:01 | |
| *** haomaiwa_ has joined #openstack-swift | 19:01 | |
| *** geaaru has quit IRC | 19:05 | |
| *** tongli has joined #openstack-swift | 19:10 | |
| *** mrmoje has joined #openstack-swift | 19:22 | |
| *** silor has quit IRC | 19:27 | |
| *** acoles is now known as acoles_ | 19:28 | |
| openstackgerrit | James Nzomo proposed openstack/python-swiftclient: Fix segmented upload to pseudo-dir via <container> https://review.openstack.org/271728 | 19:29 |
| admin0 | my dispersion-report does not work because my URL is like https://<project-name>.url/ — are there any developers around who can help me with this ? | 19:32 |
| *** klamath has joined #openstack-swift | 19:40 | |
| * clayg hides in the corner | 19:45 | |
| jrichli | hee hee, clayg: btw, love the blade runner reference in the docs. i didn't read that until notmyname tweeted about it | 19:46 |
| clayg | jrichli: oh did he - lol | 19:47 |
| ahale | oh its clayg :) | 19:47 |
| ahale | i was looking at that handoff thing i said about yesterday again, but with current ring builder built ring | 19:48 |
| ahale | im not convinced its better in that weird case - https://gist.github.com/ahale/b91fc9bc745e710e070a | 19:49 |
| openstackgerrit | Merged openstack/swift: _get_info_cache should encode unicode metadata https://review.openstack.org/267016 | 19:49 |
| *** ho_away has joined #openstack-swift | 19:57 | |
| *** admin0 has quit IRC | 19:58 | |
| *** haomaiwa_ has quit IRC | 20:01 | |
| *** haomaiwang has joined #openstack-swift | 20:01 | |
| *** ho_away has quit IRC | 20:01 | |
| *** ChubYann has joined #openstack-swift | 20:04 | |
| clayg | ahale: ok, good notes - will take me some time to digest | 20:06 |
| *** barra204 has quit IRC | 20:06 | |
| *** daemontool has joined #openstack-swift | 20:06 | |
| ahale | yeah no worries, it took me a little while to work out , its all a bit confusing | 20:07 |
| *** mrmoje has quit IRC | 20:07 | |
| ahale | and sorry for the sketchy code :) | 20:08 |
| Zyric | Good morning | 20:08 |
| clayg | ahale: looks fine | 20:08 |
| Razva | I might be to tired, but I swear that I cannot find any documentation on Liberty regarding S3 API | 20:09 |
| zaitcev | Why would you | 20:09 |
| *** daemontool_ has quit IRC | 20:09 | |
| clayg | ahale: FWIW, IME, everyone thinks their code looks like crap until they read a *bunch* of code - then they realize all code looks like crap, and start to idolize code that looks only marginally inscrutible vs. the piles of shit most of live our lives in | 20:09 |
| ahale | lol | 20:10 |
| zaitcev | Swift is one of the best codebases I hacked on for 30 years. | 20:10 |
| zaitcev | And surprisingly our acceptance of "cleanups" helps. | 20:10 |
| clayg | zaitcev: parts are better than others - but that's still high prise - she's one of my favorites too - but I've been in it to long - and the bad parts annoy the fuck out of me | 20:11 |
| ahale | yeah no clarity of swift code helped me a ton when i started doing this | 20:11 |
| ahale | but that was before diskfile ;) | 20:12 |
| zaitcev | I too think Peter took it a step too far with the Manager and Writer. | 20:13 |
| clayg | rofl | 20:13 |
| clayg | Manager might have been my fault - I know the router was my bad | 20:13 |
| zaitcev | There was a certain logic to it, because we needed Writer to avoid some kind of tricky fd leak and was difficult to solve with just the monolithic DiskFile class. | 20:14 |
| clayg | zaitcev: don't buy it - monolithic BaseDiskFile FTW | 20:14 |
| zaitcev | why didn't you -2 it then | 20:15 |
| zaitcev | you have all the moral authority | 20:15 |
| *** sanchitmalhotra has quit IRC | 20:15 | |
| pdardeau | clayg: is there anything that marks (or tracks) the bad parts and identifies what makes them bad? for education and later cleanup | 20:15 |
| clayg | zaitcev: i wasn't at the hack-a-thon when it got merged *acctually* | 20:16 |
| clayg | but it was fine, it had gone on long enough and there was more better than worse by far | 20:16 |
| clayg | I have a love hate with DiskFile - acoles_ has been in there like a boss - it's probably not the part that annoys me the most (although now I'm trying to think what *is*) | 20:16 |
| *** SkyRocknRoll has quit IRC | 20:16 | |
| clayg | pipelines? anything with auto in the name probably... | 20:17 |
| clayg | pdardeau: not for a long time - i've started to open wishlist's tho - people are even working on them - it's spooky | 20:17 |
| clayg | zaitcev: oh - TESTS - for the love of god every test that chuck ever wrote where he called a controller's VERB method directly instead of req.get_response(app) | 20:18 |
| clayg | test.unit.proxy.test_server | 20:19 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 20:21 |
| clayg | what time is the thing with the deal? | 20:21 |
| zaitcev | What thing with what deal? | 20:22 |
| *** blmartin has quit IRC | 20:26 | |
| openstackgerrit | Janie Richling proposed openstack/swift: Change POST-as-COPY to preserve the DLO manifest contents https://review.openstack.org/252791 | 20:26 |
| jrichli | acoles_: thanks for the help on the functest, that worked! | 20:26 |
| jrichli | acoles_: now i just need everyone to agree on the behavior of POST to a DLO manifest! | 20:27 |
| *** forcebanana has quit IRC | 20:28 | |
| clayg | jrichli: I think it should be traslated to a DELETE (after copying to a hidden version container) | 20:30 |
| jrichli | clayg: either that is a joke, or I am seriously never gonna get this stuff. | 20:31 |
| clayg | lol - it was joke! i was talking in another chan - sheez | 20:37 |
| clayg | ;) | 20:38 |
| jrichli | whew! | 20:38 |
| jrichli | if you wanna give your input: https://bugs.launchpad.net/swift/+bug/1487791 | 20:38 |
| openstack | Launchpad bug 1487791 in OpenStack Object Storage (swift) "POST to DLO squashes data without fast-POST" [Undecided,Confirmed] | 20:38 |
| jrichli | the patch i have out there is implementing what was deemed as "option #2" | 20:38 |
| *** diazjf has quit IRC | 20:41 | |
| *** dmorita has quit IRC | 20:42 | |
| *** awelleck has joined #openstack-swift | 20:42 | |
| *** dmorita has joined #openstack-swift | 20:46 | |
| notmyname | team meeting in 14 minutes. I may be a few minutes late | 20:46 |
| *** blmartin has joined #openstack-swift | 20:47 | |
| mattoliverau | morning | 20:48 |
| *** cbartz has joined #openstack-swift | 20:48 | |
| jrichli | mattoliverau: morning! so when does your conference start? | 20:48 |
| mattoliverau | jrichli: on Monday, so Sunday arvo/evening for you :) | 20:49 |
| jrichli | and i remember reading about arvo in channel :-) | 20:49 |
| *** dmorita has quit IRC | 20:50 | |
| notmyname | ...that feeling when you wake up at 4:30am with an upset stomach the day of an international flight... | 20:51 |
| jrichli | notmyname: hope you are feeling better now | 20:52 |
| notmyname | thanks. I'm not feeling worse :-/ | 20:52 |
| *** ho_away has joined #openstack-swift | 20:53 | |
| jrichli | oh, that doesnt sound too good, although I guess that is a the bright side | 20:53 |
| mattoliverau | jrichli: nice one, otherwise I'd need to give everyone an Aussie Slang lesson again | 20:54 |
| mattoliverau | notmyname: :( | 20:54 |
| *** acoles_ is now known as acoles | 20:57 | |
| kota_ | mornig | 20:57 |
| ho_away | good morning! | 20:58 |
| jrichli | mattoliverau: I think you could compile a whole series of lessons and we still wouldn't know all the slang. i remember that video you posted a while back. | 20:58 |
| *** dmorita has joined #openstack-swift | 20:58 | |
| notmyname | meeting time in #openstack-meeting | 20:59 |
| jlhinson | mattoliverau: jrichli: arvo? | 20:59 |
| *** haomaiwang has quit IRC | 21:01 | |
| mattoliverau | jlhinson: afternoon | 21:01 |
| *** haomaiwang has joined #openstack-swift | 21:01 | |
| mattoliverau | jrichli: this one: https://www.youtube.com/watch?v=yDb_WsAt_Z0 :) | 21:02 |
| *** forcebanana has joined #openstack-swift | 21:02 | |
| *** diazjf has joined #openstack-swift | 21:15 | |
| *** rcernin has quit IRC | 21:16 | |
| *** Razva has quit IRC | 21:20 | |
| *** bill_az has joined #openstack-swift | 21:21 | |
| *** mrmoje has joined #openstack-swift | 21:23 | |
| *** awelleck has quit IRC | 21:31 | |
| openstackgerrit | Jonathan Hinson proposed openstack/swift: Automatic refresh of memcache config settings https://review.openstack.org/218490 | 21:43 |
| *** mingdang1 has joined #openstack-swift | 21:46 | |
| *** mingdang1 has quit IRC | 21:50 | |
| *** diazjf has quit IRC | 21:51 | |
| *** lifeless has quit IRC | 21:53 | |
| hrou | kota_, hey did you have any idea what folks are currently doing for symlink, other than us of course ? :) | 21:53 |
| kota_ | well | 21:54 |
| *** lifeless has joined #openstack-swift | 21:55 | |
| kota_ | at first, i didn't catch up in detail for current status of symlinks but... | 21:55 |
| hrou | kota_, FYI you can see the conversation re: the problems with symlink here "https://etherpad.openstack.org/p/swift_symlink_post_strategy" | 21:55 |
| hrou | problems with post and symlinks I should say | 21:55 |
| kota_ | ah yeah, roughly | 21:55 |
| *** cbartz has left #openstack-swift | 21:56 | |
| onovy | patch 258158 // anyone have problem with this deprecation remove? | 21:56 |
| patchbot | onovy: https://review.openstack.org/#/c/258158/ - swift - Keystone middleware deprecated option is_admin rem... | 21:56 |
| joeljwright | timburke: looks like notmyname says no | 21:56 |
| kota_ | i should know also current problem on fast-post | 21:56 |
| clayg | kota_: I'm down with not waiting on fast-post for symlinks - but honestly I'm more interested in getting fast-post fixed than getting symlinks released? | 21:56 |
| timburke | clayg: yeah, the _RetryBody patch is just a follow-up | 21:56 |
| timburke | the unicode one is also quite small | 21:56 |
| kota_ | if we have a solvable issue, yes do it with post-as-copy. | 21:56 |
| kota_ | but... | 21:57 |
| clayg | timburke: I'm honestly not sure I caught all those links bro - do you acctually need some eyes or are you just waving your arms? | 21:57 |
| acoles | timburke: is the _RetryBody one your one? I'm already +2 on that :) | 21:57 |
| timburke | acoles: it is :) | 21:57 |
| acoles | timburke: so I'm done with all the good ones? :D :D | 21:57 |
| joeljwright | timburke: I can probably find time on Friday to spend some time on swiftclient review | 21:58 |
| hrou | clayg, kota_ well I believe we all established / agreed it works today as is with post-as-copy :) Not that I'm disagreeing that getting fast-post working (so it can be used throughout) is not more interesting / important | 21:58 |
| kota_ | clayg: wow, i think you can save us :P | 21:58 |
| joeljwright | timburke: unless the sky falls in on me again | 21:58 |
| timburke | clayg: i just know that if *i* don't point them out, they'll probably continue to be overlooked | 21:58 |
| timburke | joeljwright: thanks | 21:59 |
| clayg | i'm re-reading that pad - but I don't think i've missed anything - no progress? | 21:59 |
| *** haomaiwang has quit IRC | 22:01 | |
| kota_ | i'm also reading that pad to catch up | 22:01 |
| hrou | clayg, if that was in reference to symlink, nope no progress on where we left - we still believe fast-post has a problem with the existing symlink implementation. Haven't had too many resources to think that one through more as of late. | 22:01 |
| *** haomaiwang has joined #openstack-swift | 22:01 | |
| ho_away | acoles: onvoy: takashi: regarding keystone (deprecated issue) i have path 201461. just advertisement :-) | 22:02 |
| ho_away | acoles: onvoy: takashi: patch 201461 | 22:03 |
| patchbot | ho_away: https://review.openstack.org/#/c/201461/ - swift - Improve Keystone v3 token support | 22:03 |
| clayg | hrou: oh, ok, np | 22:03 |
| *** achhabra has joined #openstack-swift | 22:04 | |
| clayg | hrou: so who is m_kazuhiro - is *he* working on symlinks? | 22:04 |
| *** ho_away has quit IRC | 22:04 | |
| *** mrmoje has quit IRC | 22:04 | |
| hrou | clayg, I think so ;) Chatting with him now to figure that out actually | 22:04 |
| kota_ | clayg: my collegue | 22:04 |
| kota_ | clayg: he is working on swift tiering. | 22:05 |
| hrou | kota_, neat ! Yea I can see why this matters a lot for them in that case. | 22:05 |
| kota_ | and also wants to know he can use symlink earlier for tiering or not. | 22:05 |
| clayg | oic | 22:06 |
| acoles | hrou: that pad says "swift is missing documentation on dos and don't due to the eventual inconsistency nature" - you want me to try to write something, somewhere? | 22:06 |
| clayg | acoles: no! hrou said he would do it in Japan | 22:06 |
| hrou | acoles, I'd love that, we actually wrote a little in house too actually ;) But need to clean that up | 22:06 |
| *** mrmoje has joined #openstack-swift | 22:06 | |
| hrou | ^^ and see I'm not saying anything differently ;) | 22:06 |
| clayg | hrou: bah :shipit: | 22:06 |
| acoles | clayg: hrou i guess it'll be done...eventually :) | 22:07 |
| *** achhabra has left #openstack-swift | 22:07 | |
| clayg | acoles: !!!! oh oh oh - i get it! | 22:07 |
| hrou | acoles, pun intended there ? ;) | 22:07 |
| hrou | acoles, you know what let me take the first crack at it, I'm sure you'll have a lot to add either way ! | 22:07 |
| *** marcusvrn_ has quit IRC | 22:07 | |
| hrou | and we have notes. | 22:07 |
| acoles | actaull it wasn't intended pun, but in retrospect it's better that way | 22:08 |
| *** joeljwright has quit IRC | 22:09 | |
| kota_ | hrou: to collect actual problem on symlink with fast-post, could you help me to point out the paragrah in the pad? | 22:10 |
| *** ho_away has joined #openstack-swift | 22:10 | |
| acoles | hrou: basically, "don't ever take a piece of state that a client wrote at t1 and internally store it under time t2, or t0" | 22:10 |
| hrou | kota_, we don't actually go into detail on the problem there, let me write that up, most of the etherpad is concentrated on post-as-copy handling and how there is NOT a problem with it ;) | 22:10 |
| clayg | acoles: but moving it to t3 is just http://hqwallbase.pw/images/medium/a-tyrannosaurus_rex_drums_nuclear_explosions_orange_wallpaper-42087.jpg | 22:11 |
| kota_ | hrou: oic | 22:11 |
| hrou | acoles, way to go .. sum up 1 month of work in a single sentence .. ;) ... jk yea something like that but we go into a few use cases that are problematic (a lot of it centers around post-as-copy, which you can argue we can throw out eventually). | 22:11 |
| *** ho_away has quit IRC | 22:12 | |
| acoles | clayg: sure! t3 is a special case. anything can be made to be the real deal at t3. not t4 tho. | 22:12 |
| acoles | we're joking of course, in case anyone is listening | 22:12 |
| hrou | of course ;) | 22:13 |
| clayg | hrou: but... whe you say "NOT a problem [with post-as-copy]" - you mean "it's EVERY BIT AS MUCH OF A PROBLEM as post-as-copy" (right?) | 22:13 |
| clayg | I think post-as-copy was written under the facllacy that "that will rarely happen" is different from saying "that will happen" | 22:13 |
| hrou | clayg, hehe yes, so to date I've been very careful to say it the way you did above ... but got lazy above | 22:14 |
| hrou | clayg, 100% agreed | 22:14 |
| clayg | hrou: no problem - wanted to make sure I was following - yeah sux | 22:14 |
| clayg | hrou: the question was about if the API could make it so post updates wrote directly to the target (instead of returning a redirect) or something | 22:14 |
| hrou | clayg, right which leads to an interesting conversation of, does ever post now need a head to determine if it is a symlink | 22:16 |
| hrou | clayg, or is that something we do at the object server layer | 22:16 |
| hrou | kota_, acoles - just as a reminder the current implemtnation is simply: Post puts metadata on symlink directly, you do get/head - we get the metadata from the target and the symlnk, and return whatever is newer only. | 22:18 |
| acoles | hrou: but didn't we discuss that in tokyo, and that the result of a HEAD could be stale, so you may end up posting to the link anyway? (I confess I'm a little out of touch) | 22:18 |
| *** trifon has joined #openstack-swift | 22:19 | |
| hrou | acoles, yep! It may be, but that's Ok, we're operating in the context of the information that's known at the time (its really no different than if you make an application decision as a result of a head which does something to another object), we touch on that in the etherpad | 22:19 |
| kota_ | if we could have etag in symlink, could we use if-match for the GET/HEAD which might be stale? | 22:19 |
| *** daemontool has quit IRC | 22:19 | |
| hrou | kota_, that's interesting thought ! But I think the trick there would be getting the etag in the symlink there in the first place. | 22:21 |
| kota_ | ah, for now, seems talking about "making Symlink" process.... | 22:21 |
| Zyric | Is there a way to simulate a large and well populated cluster that doesn't involve slowly building a large and well populated cluster (or alternatively, a fast way to produce one)? | 22:21 |
| hrou | kota_, you could have a similar argument that the etag was stale at the time of the head we'd have to drive (to get the etag of the target object) | 22:21 |
| kota_ | hrou: yup, e.g. if-modified-since | 22:23 |
| acoles | hrou: i gotta go, i'll try to get up to speed with the ether pad | 22:23 |
| hrou | kota_, what about the case where the target doesn't exist first though ;) | 22:24 |
| kota_ | not completely if we have a lot of handoffs with different timesatmps. | 22:24 |
| hrou | acoles, sure ! But I'll deffinetly add more about fast-post overall going forward. | 22:24 |
| hrou | kota_, once you bring in the handoff situation, anything is possible, which is essentially what we get at in that ehterpad | 22:24 |
| *** acoles is now known as acoles_ | 22:24 | |
| kota_ | hrou:k | 22:25 |
| *** diazjf has joined #openstack-swift | 22:25 | |
| clayg | Zyric: I've made many an overnight run filling up develpoment/lab enivonrments of swift - let me know if you cook something up | 22:26 |
| hrou | kota_, now mind you, by adding the etag you are in my mind changing where the problem exists (lets ignore the 404 on symlink create ... :), so the problem in this situation is if the initial etag is accurate (it may not be due to handoffs) but assuming it is, you've solved the problem on the other end of the spectrum (i.e. at post time) | 22:26 |
| clayg | timburke: so patch 270991 looked super simple, and it's had your +2 on it for a week - thought I'd try to help out... | 22:26 |
| patchbot | clayg: https://review.openstack.org/#/c/270991/ - swift - Allow IPv6 addresses/hostnames in StatsD target | 22:26 |
| clayg | timburke: but when I checkout master the test fails - but with a KeyError instead of the decode error I was expecting? | 22:27 |
| clayg | timburke: well - i guess the lp bug #1532096 is a KeyError - but it looks like a *diferent* KeyError | 22:28 |
| openstack | Launchpad bug 1532096 in python-swiftclient "Error with uploading large object includes unicode path" [Undecided,In progress] https://launchpad.net/bugs/1532096 - Assigned to Jude Augustine Job (judeaugustine-j) | 22:28 |
| kota_ | hrou: i thought we could simlar semantics with slo, if an user set an etag to "make a symlink" request, middleware can accurate it (?) not sure i am addressing right problem : | 22:30 |
| kota_ | :/ | 22:30 |
| *** daemontool has joined #openstack-swift | 22:30 | |
| hrou | kota_, yes ! If you tell me the etag of the target object when you create the symlink you've just closed all sorts of holes ;) | 22:30 |
| timburke | clayg: :/ i know i saw it reproduce...maybe that was when i was mucking with my own tests... | 22:30 |
| clayg | timburke: i have no idea what this test is doing honestly | 22:31 |
| clayg | it mocks _upload_segment_job, then calls _upload_object_job - do our unittests maybe hide errors that the threads behind the service object are spitting out? | 22:31 |
| clayg | do you use ./run_tests.sh or ./.unittests or nosetests directly? | 22:32 |
| hrou | kota_, but I'm not sure if this would work for all applications of symlink, that's a really interesting thought though ! I wonder if in your tiering case that's sufficent. Mind you the problem you'll have with tiering is "who gives you the etag of the target" :) If its the user great, but that doesn't make sense in tiering, it'll be "swift it self" through a head/... getting you into a similar situation. | 22:32 |
| Zyric | clayg, that'd be great! I need something a bit more representative to test halt/restarting object and account audits. | 22:32 |
| Zyric | My development resources are a single i5 4200U 1.6-2.3GHz laptop with 8gb RAM, I'm running a Ubuntu 14.04 SAIO. | 22:32 |
| clayg | timburke: yeah the error i was looking for was being logged - i just needed to use nose | 22:32 |
| clayg | hurricanerix: aren't you somehow knowing something about the testr/subunit stuff? I *liked* nose. I know part of that was just having spent a lot of time with it - but Ned is cool guy! | 22:33 |
| kota_ | in my feeling, it could be because tiering agent should know which objects should be moved (i expect the agent also know the etag to prevent rollback come from eventual consistency...) | 22:34 |
| kota_ | not sure, need to talk with m_kazuhiro. | 22:34 |
| *** admin0 has joined #openstack-swift | 22:35 | |
| hrou | kota_, yep! It could be, so I think that'd be a neat discussion to have, it does close some holes but ... it does imply the object needs to exist first, I'm not sure if that'd be problematic for some use cases. As far as I know other than tiering, swift3 is the other context where this is useful, you probably know the details there though ? :) | 22:35 |
| timburke | clayg: would you feel better about it with something like http://paste.openstack.org/show/485203/ ? | 22:35 |
| timburke | then i see testtools.matchers._impl.MismatchError: None is not KeyError(u'\u30c6',) | 22:36 |
| kota_ | and if an user don't mind the etag (just want to link the eventual newest one), yup, symlink can set the etag from extra HEAD response (even if it's stale) for itself. | 22:37 |
| timburke | (plus the logging output, if run through nosetests) | 22:37 |
| Zyric | Could nyone give me an estimate of a good starting point for a large but manageable cluster with specs like mine? I haven't had need to make a large cluster before. | 22:37 |
| *** daemontool_ has joined #openstack-swift | 22:37 | |
| kota_ | that's same sematics wiht SLO | 22:37 |
| hrou | kota_, at symlink creation time we just drive a head always IFF we don't have an etag .. gotcha ! | 22:37 |
| kota_ | need more eyes, anyway :) | 22:38 |
| timburke | thanks clayg! | 22:39 |
| *** daemontool has quit IRC | 22:39 | |
| hrou | kota_, there are more complications like say I intentionally re-put the target, I'd have to update that somehow | 22:39 |
| *** trifon has quit IRC | 22:39 | |
| kota_ | hrou: how about to use X-Timestamp? | 22:39 |
| hrou | kota_, and if its stale re-drive the head to get a new etag ? | 22:40 |
| kota_ | maybe no, just put as old object though it might be removed soon. | 22:41 |
| kota_ | as eventual consitency | 22:41 |
| hrou | kota_, Right but I don't think that's accurate unless you operate under an assumption that customers never intentionally re-put the target (and expect the symlink to keep working after) | 22:42 |
| kota_ | you said about "un-intentionally re-put"? | 22:43 |
| clayg | timburke: ohhhhh - yeah problaby | 22:44 |
| *** asettle-afk has joined #openstack-swift | 22:44 | |
| hrou | kota_, ah no sorry I meant *intentionally* :) | 22:44 |
| timburke | clayg: some day when i have too much free time i should go add that to pretty much all of the tests... | 22:44 |
| clayg | timburke: so you really can't get the logging on faiure automatically from testr? | 22:44 |
| clayg | timburke: or more likely just no one knows *how* | 22:45 |
| timburke | more likely | 22:45 |
| clayg | timburke: hell i don't even know how to run a just a single test? | 22:45 |
| hrou | kota_, this is probably why at some point we just thought "failing post" would just be easier ;) | 22:45 |
| *** asettle-afk is now known as asettle | 22:45 | |
| clayg | where rcollins when you want to give him grief! | 22:45 |
| clayg | hrou: not "fail" so much as return a 3XX | 22:45 |
| hrou | clayg, or that ! Let the application handle it | 22:46 |
| kota_ | clayg: or 202(?) with stale X-Timestamp | 22:46 |
| clayg | hrou: besides doesn't tiering need PUT's to be poxied to the target too? why does it matter if the POST is proxied automatically if it's going to have to intercept PUT's anyway? | 22:46 |
| clayg | kota_: 2XX would only make sense to me if the transformation was persisted in the system in some way - I haven't been keeping up with the whole conversation | 22:47 |
| timburke | clayg: something like this mostly works: tox -e py27 -r '.*test_upload_object_job_file_with_unicode_path' | 22:47 |
| timburke | -r for regex | 22:47 |
| timburke | or something | 22:47 |
| clayg | maybe | 22:47 |
| hrou | clayg, that'd be an interesting question for m_kazuhiro i.e. how do they envision tiering working utilizing this symlink implemntation. I've had chats with folks where no, the idea was not to have PUTs hit the object on the colder tier, but rather operate almost akin to symlinks today | 22:47 |
| clayg | i best I still like nose better | 22:48 |
| clayg | hrou: oh right - no that makes sense | 22:48 |
| hrou | that is PUT overwrites symlink, and someone else cleansup the tier'd object at some point. | 22:48 |
| hrou | clayg, both can ! Really, that's why desinging symlink for all possible use cases gets complicated :) | 22:48 |
| timburke | clayg: ditto. i generally just use nose, then tox when i'm reasonably content that it'll actually behave as expected in all envs | 22:49 |
| hrou | kota_, yea I'm not sure I like 2xx really | 22:49 |
| hrou | kota_, because that's not what took place, the post is lost from the context of the target object, and remember that's all this application knew about in the first place. | 22:50 |
| kota_ | i like to bet the cleanup process in the out of symlink scope for simplyfying. | 22:50 |
| hrou | kota_, I'm liking 3xx more though now that I think about it, any thoughts on that ? | 22:50 |
| hrou | kota_, i.e. fail posts period, but give the application a chance to handle it. | 22:51 |
| hrou | 307 maybe | 22:52 |
| kota_ | interesting | 22:54 |
| kota_ | or 301? | 22:55 |
| hrou | kota_, maybe I was afraid that would cause the application to route all further requests (get/put/head) to that URI which isn't quite what we want. | 22:57 |
| timburke | kota_: 307. maybe 302, but i like that the method is (supposed to be) the same. 301 tends to be cached, potentially for a while | 22:57 |
| hrou | timburke, right exactly ! which takes care of my concern above. | 22:58 |
| kota_ | or just returning 404 with informational headers (e.g. was liked to xxxx)? | 22:58 |
| hrou | kota_, :) Sure that was the original plan actually, well some error / 404 code. | 22:58 |
| kota_ | (or informational bodies) | 22:58 |
| *** daemontool has joined #openstack-swift | 22:58 | |
| hrou | we were going to return the X-symlink-target-* headers yep | 22:59 |
| kota_ | hrou: sorry, i'm running out of time to be here, for now. | 22:59 |
| hrou | kota_, me too :) Its Ok, you're going to the hackathon right ? Unfortunetly I don't think I can make it, it'd be nice to touch base before so a conversation can be had about this up there (I think the "up" is applicable for most folks ;). | 23:00 |
| timburke | if we're going 4xx, i vote 409 or 400 | 23:00 |
| kota_ | hrou: i'll be online again in 2 hours, it would be nice to update the etherpad with tthis conversation (and your thought) :) | 23:00 |
| *** haomaiwang has quit IRC | 23:01 | |
| kota_ | hrou: yup, i'll be there. | 23:01 |
| *** haomaiwa_ has joined #openstack-swift | 23:01 | |
| *** daemontool_ has quit IRC | 23:01 | |
| *** diazjf has quit IRC | 23:01 | |
| hrou | kota_, awesome, Ok sounds good I'll update this again and we'll talk more about this; Thanks for all the input ! | 23:02 |
| *** km has joined #openstack-swift | 23:08 | |
| *** ho_away has joined #openstack-swift | 23:13 | |
| *** ho_away has quit IRC | 23:17 | |
| *** chlong has quit IRC | 23:18 | |
| *** admin0 has quit IRC | 23:21 | |
| *** mragupat has quit IRC | 23:22 | |
| *** kei_yama has joined #openstack-swift | 23:28 | |
| *** chlong has joined #openstack-swift | 23:30 | |
| *** dmorita has quit IRC | 23:31 | |
| *** asettle is now known as asettle-afk | 23:31 | |
| *** dmorita has joined #openstack-swift | 23:32 | |
| *** nadeem has quit IRC | 23:36 | |
| *** blmartin has quit IRC | 23:41 | |
| *** mingdang1 has joined #openstack-swift | 23:47 | |
| *** klamath has quit IRC | 23:53 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
