| *** fragiletag has quit IRC | 00:11 | |
| torgomatic | tsg_: ping | 00:23 |
|---|---|---|
| *** tsg_ has quit IRC | 00:26 | |
| torgomatic | guess I must smell bad or something | 00:27 |
| *** shri has quit IRC | 01:07 | |
| *** rdaly2 has joined #openstack-swift | 01:08 | |
| *** rdaly2 has quit IRC | 01:12 | |
| *** nosnos has joined #openstack-swift | 01:18 | |
| *** tsg has joined #openstack-swift | 01:31 | |
| *** kyles_ne has quit IRC | 02:05 | |
| *** kopparam has joined #openstack-swift | 03:03 | |
| *** kopparam has quit IRC | 03:07 | |
| *** rdaly2 has joined #openstack-swift | 03:10 | |
| *** rdaly2 has quit IRC | 03:14 | |
| *** nosnos has quit IRC | 03:25 | |
| *** rmcall has quit IRC | 03:27 | |
| *** bkopilov has quit IRC | 03:35 | |
| *** mrsnivvel has joined #openstack-swift | 04:08 | |
| *** nosnos has joined #openstack-swift | 04:11 | |
| *** kopparam has joined #openstack-swift | 04:13 | |
| *** kopparam has quit IRC | 04:19 | |
| *** ppai has joined #openstack-swift | 04:26 | |
| *** rmcall has joined #openstack-swift | 04:37 | |
| *** abhirc has quit IRC | 04:40 | |
| *** bkopilov has joined #openstack-swift | 05:06 | |
| *** kyles_ne has joined #openstack-swift | 05:06 | |
| *** kyles_ne has quit IRC | 05:11 | |
| *** rdaly2 has joined #openstack-swift | 05:11 | |
| *** rdaly2 has quit IRC | 05:16 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:20 | |
| *** kota_ has joined #openstack-swift | 05:21 | |
| *** nshaikh has joined #openstack-swift | 05:25 | |
| *** kopparam has joined #openstack-swift | 05:44 | |
| *** zaitcev has quit IRC | 05:59 | |
| *** exploreshaifali has joined #openstack-swift | 06:01 | |
| *** rmcall has quit IRC | 06:24 | |
| *** sungju has quit IRC | 06:28 | |
| *** sungju has joined #openstack-swift | 06:28 | |
| *** ekarlso has quit IRC | 06:37 | |
| *** kota_ has quit IRC | 06:37 | |
| *** exploreshaifali has quit IRC | 06:42 | |
| *** ekarlso has joined #openstack-swift | 06:43 | |
| *** ppai has quit IRC | 06:44 | |
| *** ppai has joined #openstack-swift | 06:56 | |
| *** rdaly2 has joined #openstack-swift | 07:12 | |
| *** rdaly2 has quit IRC | 07:17 | |
| *** sungju has quit IRC | 07:18 | |
| *** jyoti-ranjan has joined #openstack-swift | 07:23 | |
| *** jamespage_ has joined #openstack-swift | 07:23 | |
| *** jamespage_ has quit IRC | 07:28 | |
| *** ppai has quit IRC | 07:56 | |
| *** ppai has joined #openstack-swift | 07:59 | |
| *** kota_ has joined #openstack-swift | 08:05 | |
| *** wer has quit IRC | 08:14 | |
| *** jyoti-ranjan has quit IRC | 08:18 | |
| *** haomaiwang has joined #openstack-swift | 08:34 | |
| *** geaaru has joined #openstack-swift | 08:36 | |
| *** haomaiwang has quit IRC | 08:38 | |
| *** zul has joined #openstack-swift | 08:52 | |
| *** Tyger_ has joined #openstack-swift | 08:54 | |
| *** rdaly2 has joined #openstack-swift | 09:14 | |
| *** kopparam has quit IRC | 09:15 | |
| *** kopparam has joined #openstack-swift | 09:15 | |
| *** rdaly2 has quit IRC | 09:18 | |
| *** haomaiwa_ has joined #openstack-swift | 09:19 | |
| *** kopparam has quit IRC | 09:20 | |
| *** kopparam has joined #openstack-swift | 09:26 | |
| *** nshaikh has quit IRC | 09:29 | |
| *** Dafna has joined #openstack-swift | 09:30 | |
| *** tsg has quit IRC | 09:33 | |
| *** nshaikh has joined #openstack-swift | 09:35 | |
| *** ppai has quit IRC | 09:37 | |
| *** Tyger_ has quit IRC | 09:41 | |
| *** kota_ has quit IRC | 09:49 | |
| *** ppai has joined #openstack-swift | 09:50 | |
| *** zul has quit IRC | 09:52 | |
| *** kota_ has joined #openstack-swift | 09:55 | |
| *** Tyger_ has joined #openstack-swift | 10:05 | |
| *** leopoldj has joined #openstack-swift | 10:10 | |
| *** ppai has quit IRC | 10:14 | |
| mattoliverau | notmyname: https://wiki.openstack.org/wiki/Summit/Kilo/Etherpads#Swift | 10:19 |
| *** foexle has joined #openstack-swift | 10:23 | |
| *** zul has joined #openstack-swift | 10:24 | |
| *** ppai has joined #openstack-swift | 10:26 | |
| *** Tyger_ has quit IRC | 10:28 | |
| *** haomaiwa_ has quit IRC | 10:30 | |
| *** foexle has quit IRC | 10:36 | |
| *** zul has quit IRC | 10:36 | |
| *** nellysmitt has joined #openstack-swift | 10:37 | |
| *** Tyger_ has joined #openstack-swift | 10:37 | |
| *** openstackgerrit has joined #openstack-swift | 10:40 | |
| *** nellysmitt has quit IRC | 10:41 | |
| *** kota_ has quit IRC | 10:47 | |
| *** kota_ has joined #openstack-swift | 10:47 | |
| *** kota_ has quit IRC | 10:49 | |
| *** ppai has quit IRC | 10:51 | |
| *** nshaikh has left #openstack-swift | 10:56 | |
| *** kne has joined #openstack-swift | 10:56 | |
| *** brnelson has quit IRC | 10:59 | |
| *** fbo has quit IRC | 10:59 | |
| *** brnelson has joined #openstack-swift | 10:59 | |
| *** fbo has joined #openstack-swift | 11:01 | |
| *** aix has joined #openstack-swift | 11:04 | |
| *** haomaiwa_ has joined #openstack-swift | 11:04 | |
| *** ppai has joined #openstack-swift | 11:05 | |
| *** jamespage_ has joined #openstack-swift | 11:09 | |
| *** jamespage_ has quit IRC | 11:09 | |
| *** haomaiwa_ has quit IRC | 11:14 | |
| *** rdaly2 has joined #openstack-swift | 11:16 | |
| *** nosnos has quit IRC | 11:18 | |
| *** nosnos has joined #openstack-swift | 11:19 | |
| *** rdaly2 has quit IRC | 11:20 | |
| kne | I don't want to stall a nice discussion, but I see that EC will have a etag that is not md5sum of the object data | 11:24 |
| kne | since there was sth similar mentioned for the encryption, will there be some general approach for all these addendums to plain swift, so that we could actually rely on etag to be sth that a client could calculate and verify? | 11:25 |
| Tyger_ | By the time the client sees it, it should be the etag of the object, regardless of what is done to it, which would be an md5 of the unencoded, unencrypted data. | 11:26 |
| Tyger_ | The fact that it's ECd (Or encrypted) should be transparent to the client. | 11:27 |
| Tyger_ | (Someone feel free to correct me if I'm wrong) | 11:27 |
| *** nellysmitt has joined #openstack-swift | 11:27 | |
| kne | from the document: | 11:28 |
| kne | Object metadata changes: for ‘etag’, we store the md5sum of the EC archive object, as opposed to the non-EC case where we store md5sum of the entire object | 11:28 |
| Tyger_ | Look under 3.3.5 | 11:29 |
| Tyger_ | The real object etag goes into the container metadata | 11:29 |
| *** exploreshaifali has joined #openstack-swift | 11:30 | |
| kne | demo time, will have a look l8r, tx Tyger_ | 11:30 |
| Tyger_ | It was one of the things touched on very briefly towards the end of the talk | 11:31 |
| *** zul has joined #openstack-swift | 11:33 | |
| *** kne has quit IRC | 11:34 | |
| *** Tyger_ has quit IRC | 11:35 | |
| *** zul has quit IRC | 11:41 | |
| *** openstackgerrit has quit IRC | 11:42 | |
| *** mahatic has joined #openstack-swift | 11:46 | |
| *** nshaikh has joined #openstack-swift | 11:49 | |
| *** nosnos has quit IRC | 11:51 | |
| *** ppai has quit IRC | 12:14 | |
| *** kopparam_ has joined #openstack-swift | 12:20 | |
| *** kopparam has quit IRC | 12:23 | |
| *** ppai has joined #openstack-swift | 12:28 | |
| *** ppai has quit IRC | 12:29 | |
| *** kne has joined #openstack-swift | 12:30 | |
| *** Tyger_ has joined #openstack-swift | 12:31 | |
| *** nellysmitt has quit IRC | 12:32 | |
| *** kopparam_ has quit IRC | 12:32 | |
| *** mkollaro has joined #openstack-swift | 12:34 | |
| *** kne_ has joined #openstack-swift | 12:36 | |
| *** kne has left #openstack-swift | 12:38 | |
| Tyger_ | kne: I was just looking closer at the EC docs, and it shows the object server storing Etag as the fragment archive etag, and X-Object-Sysmeta-EC-Etag as the object ETag. | 12:39 |
| Tyger_ | So that wouldn't be a generic solution for other things like encryption which would alter the bytes on disk. | 12:40 |
| *** keving has joined #openstack-swift | 12:51 | |
| *** exploreshaifali has quit IRC | 12:55 | |
| *** zul has joined #openstack-swift | 12:59 | |
| *** aix has quit IRC | 13:00 | |
| kne_ | Tyger_: it might be generic enough - in both cases, the etag would store the md5sum of the data on the disk, while X-Object-Sysmeta-XXX-Etag would hold the actual etag | 13:00 |
| kne_ | however, I don’t see how to make encryption and EC composable | 13:01 |
| *** kne_ is now known as kne | 13:01 | |
| *** bkopilov has quit IRC | 13:04 | |
| Tyger_ | I presume encryption would be middleware? | 13:04 |
| Tyger_ | As long as it's proxy middleware, I don't see why it wouldn't be composable. But maybe I'm oversimplifying what encryption would need to do in my mind. | 13:06 |
| *** aix has joined #openstack-swift | 13:07 | |
| kne | it would be a middleware on proxy, possibly only on proxy | 13:07 |
| kne | actually, you’re right, there shouldn’t be any problem | 13:07 |
| *** zul has quit IRC | 13:08 | |
| Tyger_ | And having X-Object-Sysmeta-XXX-Etag for both separately would probably be good. Then EC gets the fragments, decodes, and puts the EC Etag as the resulting Etag. | 13:08 |
| Tyger_ | Then it passes it to encryption which can verify the encrypted data against the Etag, decrypt, then replace the Etag with the encryption etag. | 13:09 |
| Tyger_ | So there would be an Etag for every form of the data, with the client seeing the original object etag. | 13:09 |
| kne | exactly | 13:09 |
| Tyger_ | Though it seems like the encryption level should encrypt the etag; otherwise there's a small leak where you could look to see if a specific known document exists without knowing the encryption. | 13:11 |
| *** kne has quit IRC | 13:12 | |
| *** keving has left #openstack-swift | 13:12 | |
| *** kne has joined #openstack-swift | 13:16 | |
| *** rdaly2 has joined #openstack-swift | 13:17 | |
| *** kajinamit has joined #openstack-swift | 13:19 | |
| *** bkopilov has joined #openstack-swift | 13:20 | |
| *** rdaly2 has quit IRC | 13:21 | |
| *** tsg_ has joined #openstack-swift | 13:25 | |
| *** nshaikh has quit IRC | 13:30 | |
| *** ondergetekende has joined #openstack-swift | 13:31 | |
| kne | VERIFY REGISTER kne icwvwhpnyspa | 13:34 |
| kne | wrong window... | 13:35 |
| Tyger_ | Seriously | 13:35 |
| kne | yeah… | 13:35 |
| *** zul has joined #openstack-swift | 13:38 | |
| *** miqui has joined #openstack-swift | 13:40 | |
| *** mg has joined #openstack-swift | 13:42 | |
| mg | Hi guys, is there a good place to announce a JavaScript interface for Swift? | 13:45 |
| mg | I've been working on it for some months and just packaged it as a proper release: https://github.com/zerovm/swift-browser/releases/tag/0.1.0 | 13:45 |
| *** bkopilov has quit IRC | 13:45 | |
| mg | I was expecting to find a mailing list for Swift, but that doesn't seem to exist? | 13:46 |
| notmyname | mg: the openstack mailing list is an ok place. also, add it to http://docs.openstack.org/developer/swift/associated_projects.html | 13:47 |
| mg | (I cannot find anything about "mail" on http://docs.openstack.org/developer/swift/ and http://www.openstack.org/community/ only lists very general lists) | 13:47 |
| mg | notmyname: ah, okay | 13:47 |
| mg | thanks :) | 13:47 |
| notmyname | all of the openstack projects use the same mailing list | 13:48 |
| mg | okay, that explains it | 13:50 |
| mg | notmyname: review submitted: https://review.openstack.org/133038 -- when you get the time, please let me know what you think :) | 13:56 |
| *** vr2 has joined #openstack-swift | 14:01 | |
| *** aix has quit IRC | 14:03 | |
| *** kne has quit IRC | 14:12 | |
| *** wer has joined #openstack-swift | 14:14 | |
| *** vr2 has quit IRC | 14:15 | |
| *** aix has joined #openstack-swift | 14:16 | |
| *** kne has joined #openstack-swift | 14:16 | |
| *** vr2 has joined #openstack-swift | 14:17 | |
| *** vr2 has quit IRC | 14:18 | |
| *** vr2 has joined #openstack-swift | 14:21 | |
| *** mkollaro has quit IRC | 14:23 | |
| Tyger_ | "Keymaster" in addition to "Gatekeeper"? There's a Ghostbusters reference here somewhere. | 14:30 |
| *** zul has quit IRC | 14:52 | |
| *** kne has quit IRC | 14:53 | |
| *** openstackstatus has quit IRC | 14:58 | |
| *** openstack has joined #openstack-swift | 15:00 | |
| *** openstackstatus has joined #openstack-swift | 15:00 | |
| *** ChanServ sets mode: +v openstackstatus | 15:00 | |
| *** kne has joined #openstack-swift | 15:01 | |
| *** geaaru has quit IRC | 15:02 | |
| *** foexle has joined #openstack-swift | 15:08 | |
| *** fbo has quit IRC | 15:10 | |
| *** fbo has joined #openstack-swift | 15:11 | |
| *** kne has quit IRC | 15:12 | |
| *** tsg_ has quit IRC | 15:14 | |
| *** tsg_ has joined #openstack-swift | 15:18 | |
| *** rdaly2 has joined #openstack-swift | 15:18 | |
| wer | weird. I have a cluster that is %63 full consisting of 8 nodes. I had a node down yesterday. 5 other nodes began filling up their disks. The cluster is taking zero traffic. I didn't anticipate things re-replicating without a ring change. I got the downed node back in, and space is beginning to free up again. | 15:21 |
| *** rdaly2 has quit IRC | 15:23 | |
| wer | The replication to the 5 nodes was aggressive. So had this been an actual outage I may have needed to react much faster, especially if I was closer to being full. What is the mechanism that discovered and replicated the data on the missing node? Was it the various auditors? How should I throttle this so only new incoming requests are handed off? | 15:24 |
| *** rdaly2 has joined #openstack-swift | 15:25 | |
| wer | I honestly felt I was safe leaving a node out for a night, especially since the cluster wasn't taking any write traffic. but I was wrong. I also don't understand why two nodes didn't join the replication party :/ | 15:26 |
| wer | the answer has to be cause math :) | 15:31 |
| *** kne has joined #openstack-swift | 15:34 | |
| *** mkollaro has joined #openstack-swift | 15:34 | |
| glange | wer_ -- replication is push, not pull -- so a system walks the file system and finds things on the disk and pushes them to other places if it needs to | 15:35 |
| glange | sometimes it will delete what it finds after it pushes (if it's not deleted on the system) | 15:36 |
| wer | glange: so in this case when a third replicant couldn't be found a copy was pushed elsewhere? | 15:38 |
| glange | I mistyped that last part in ()'s it should have said "if it's not needed on the system" | 15:39 |
| glange | wer_ -- I'm not sure how that works, if a system can't push to all the 3 of the primary nodes, it probably just keeps what it has and tries again later | 15:40 |
| glange | you'd have to read the code to db check that, or talk to something who knows (but they are all in Paris :) | 15:40 |
| *** tsg_ has quit IRC | 15:40 | |
| glange | double check | 15:41 |
| *** bkopilov has joined #openstack-swift | 15:41 | |
| glange | push makes replication work with ring changes and with handoff nodes, swift is actively trying to get 3 copies of what it has to the right places all the time | 15:42 |
| wer | yeah so in this case it wouldn't have been able to verify copies on the node that was out, and replicated them elsewhere. But that copy would have been in a handoff location (ie not the first choice from the ring). :/ glange do you think it's the auditors that were responsible for this? | 15:45 |
| *** SkyRocknRoll has quit IRC | 15:45 | |
| *** vr2 has quit IRC | 15:45 | |
| *** nshaikh has joined #openstack-swift | 15:46 | |
| glange | the auditors don't move data, the replicators do, so maybe they caused what you saw | 15:47 |
| *** vr2 has joined #openstack-swift | 15:48 | |
| *** nellysmitt has joined #openstack-swift | 15:49 | |
| *** lcurtis has joined #openstack-swift | 15:51 | |
| wer | yeah was thinking the same thing actually. ok I'll poke at those. thanks. I no longer feel safe leaving 1 nodes worth of free space in the cluster :/ Unless I push the missing node out immediately. which is something I was hoping I could avoid doing immediately. If this had been my production cluster I might have really screwed myself. I really only anticipated handoff, and possibly pushing a node completely out of the ring to be t | 15:54 |
| wer | he things that would start using space. And the replication in this scenario wasn't balanced awesome at all. | 15:54 |
| wer | thanks for the help glange. be careful out there ;) | 15:55 |
| glange | haha | 15:55 |
| wer | :) | 15:55 |
| *** foexle has quit IRC | 15:57 | |
| *** zul has joined #openstack-swift | 15:58 | |
| *** lcurtis has quit IRC | 16:00 | |
| *** nellysmitt has quit IRC | 16:02 | |
| *** kne has quit IRC | 16:06 | |
| *** zul has quit IRC | 16:09 | |
| *** lcurtis has joined #openstack-swift | 16:12 | |
| *** zul has joined #openstack-swift | 16:20 | |
| *** tsg_ has joined #openstack-swift | 16:22 | |
| *** zul has quit IRC | 16:31 | |
| *** zul has joined #openstack-swift | 16:37 | |
| *** kyles_ne has joined #openstack-swift | 16:39 | |
| *** wer has quit IRC | 16:42 | |
| *** zul has quit IRC | 16:43 | |
| *** SkyRocknRoll has joined #openstack-swift | 16:53 | |
| *** nellysmitt has joined #openstack-swift | 16:54 | |
| *** wer has joined #openstack-swift | 16:54 | |
| *** exploreshaifali has joined #openstack-swift | 16:55 | |
| *** tsg_ has quit IRC | 17:02 | |
| *** tgohad has joined #openstack-swift | 17:02 | |
| *** SkyRocknRoll has quit IRC | 17:02 | |
| *** rmcall has joined #openstack-swift | 17:03 | |
| *** leopoldj has quit IRC | 17:05 | |
| *** mkerrin has joined #openstack-swift | 17:06 | |
| *** rmcall has quit IRC | 17:08 | |
| *** rmcall has joined #openstack-swift | 17:08 | |
| *** tgohad has quit IRC | 17:10 | |
| *** kajinamit has quit IRC | 17:10 | |
| *** Tyger_ has quit IRC | 17:11 | |
| *** foexle has joined #openstack-swift | 17:13 | |
| *** mahatic has quit IRC | 17:15 | |
| *** vr2 has left #openstack-swift | 17:16 | |
| *** slDabbler has joined #openstack-swift | 17:17 | |
| *** foexle_ has joined #openstack-swift | 17:21 | |
| *** rdaly2 has quit IRC | 17:23 | |
| *** rdaly2 has joined #openstack-swift | 17:23 | |
| *** SkyRocknRoll has joined #openstack-swift | 17:38 | |
| *** SkyRocknRoll has quit IRC | 17:43 | |
| *** nshaikh has quit IRC | 17:50 | |
| *** nomad411 has joined #openstack-swift | 17:53 | |
| *** tsg_ has joined #openstack-swift | 17:57 | |
| *** nomad411 has quit IRC | 18:04 | |
| *** miqui has quit IRC | 18:12 | |
| *** mahatic has joined #openstack-swift | 18:18 | |
| *** nshaikh has joined #openstack-swift | 18:24 | |
| *** mkollaro has quit IRC | 18:24 | |
| *** abhirc has joined #openstack-swift | 18:32 | |
| *** kyles_ne has quit IRC | 18:32 | |
| *** kyles_ne has joined #openstack-swift | 18:32 | |
| *** lcurtis has quit IRC | 18:35 | |
| *** SkyRocknRoll__ has joined #openstack-swift | 18:36 | |
| *** rmcall has quit IRC | 18:37 | |
| *** kyles_ne has quit IRC | 18:37 | |
| *** kyles_ne has joined #openstack-swift | 18:39 | |
| *** lcurtis has joined #openstack-swift | 18:51 | |
| *** mkollaro has joined #openstack-swift | 19:00 | |
| *** lcurtis has quit IRC | 19:01 | |
| *** slDabbler has quit IRC | 19:28 | |
| *** zaitcev has joined #openstack-swift | 19:56 | |
| *** ChanServ sets mode: +v zaitcev | 19:56 | |
| *** miqui has joined #openstack-swift | 19:58 | |
| *** aix has quit IRC | 20:05 | |
| *** foexle_ has quit IRC | 20:06 | |
| *** exploreshaifali has quit IRC | 20:08 | |
| *** kyles_ne has quit IRC | 20:15 | |
| *** Nadeem has joined #openstack-swift | 20:18 | |
| *** SkyRocknRoll__ has quit IRC | 20:42 | |
| *** lcurtis has joined #openstack-swift | 20:48 | |
| *** haomaiwang has joined #openstack-swift | 20:53 | |
| *** exploreshaifali has joined #openstack-swift | 20:56 | |
| *** haomaiwang has quit IRC | 21:08 | |
| *** kyles_ne has joined #openstack-swift | 21:11 | |
| *** mkollaro has quit IRC | 21:12 | |
| *** mahatic has quit IRC | 21:18 | |
| *** slDabbler has joined #openstack-swift | 21:30 | |
| *** foexle has quit IRC | 21:38 | |
| *** nellysmitt has quit IRC | 21:39 | |
| *** slDabbler1 has joined #openstack-swift | 21:50 | |
| *** slDabbler has quit IRC | 21:52 | |
| *** shri has joined #openstack-swift | 22:06 | |
| *** bkopilov has quit IRC | 22:14 | |
| *** occupant has quit IRC | 22:54 | |
| *** Nadeem has quit IRC | 23:02 | |
| *** rdaly2 has quit IRC | 23:13 | |
| *** rmcall has joined #openstack-swift | 23:14 | |
| *** rdaly2 has joined #openstack-swift | 23:33 | |
| *** rdaly2 has quit IRC | 23:34 | |
| *** abhirc has quit IRC | 23:39 | |
| *** lcurtis has quit IRC | 23:41 | |
| *** sungju has joined #openstack-swift | 23:44 | |
| *** exploreshaifali has quit IRC | 23:45 | |
| *** slDabbler1 has left #openstack-swift | 23:46 | |
| *** nshaikh has quit IRC | 23:54 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!