| *** vinsh has joined #openstack-swift | 00:00 | |
| *** annegentle has quit IRC | 00:02 | |
| *** vinsh has quit IRC | 00:04 | |
| *** zhill has quit IRC | 00:06 | |
| *** thumpba has quit IRC | 00:08 | |
| *** kota_ has joined #openstack-swift | 00:14 | |
| *** reed has joined #openstack-swift | 00:14 | |
| *** dmorita has joined #openstack-swift | 00:15 | |
| *** kota_ has quit IRC | 00:28 | |
| *** ho has joined #openstack-swift | 00:33 | |
| ho | morning | 00:35 |
|---|---|---|
| mattoliverau | ho: morning | 00:54 |
| peluse | yo | 00:56 |
| *** annegentle has joined #openstack-swift | 00:58 | |
| ho | mattoliverau: morning! thanks. | 00:58 |
| *** annegentle has quit IRC | 01:03 | |
| notmyname | peluse: back online if you need anything | 01:34 |
| openstackgerrit | paul luse proposed openstack/swift: Erasure Code Reconstructor https://review.openstack.org/131872 | 01:41 |
| openstackgerrit | paul luse proposed openstack/swift: Multiple fragment Archive Index Support https://review.openstack.org/159637 | 01:41 |
| openstackgerrit | paul luse proposed openstack/swift: Prefer policy to policy_index https://review.openstack.org/163690 | 01:41 |
| peluse | notmyname, I'm good... off for some grub here shortly | 01:46 |
| peluse | thanks though | 01:46 |
| openstackgerrit | Merged openstack/swift: fsync() on directories https://review.openstack.org/126923 | 01:55 |
| *** tsg has quit IRC | 01:57 | |
| *** annegentle has joined #openstack-swift | 01:59 | |
| *** annegentle has quit IRC | 02:04 | |
| *** jrichli has joined #openstack-swift | 02:16 | |
| *** annegentle has joined #openstack-swift | 02:19 | |
| *** annegentle has quit IRC | 02:25 | |
| *** annegentle has joined #openstack-swift | 02:26 | |
| *** reed has quit IRC | 02:33 | |
| openstackgerrit | paul luse proposed openstack/swift: Prefer policy to policy_index https://review.openstack.org/163690 | 02:33 |
| *** haomaiwang has joined #openstack-swift | 02:38 | |
| *** devlaps has quit IRC | 02:43 | |
| *** annegentle has quit IRC | 02:46 | |
| *** tsg has joined #openstack-swift | 03:22 | |
| *** gyee has quit IRC | 03:23 | |
| *** earthrocker has joined #openstack-swift | 03:43 | |
| *** earthrocker has quit IRC | 03:43 | |
| *** ppai has joined #openstack-swift | 03:43 | |
| *** zaitcev has quit IRC | 03:58 | |
| *** delattec has quit IRC | 04:14 | |
| *** cdelatte has quit IRC | 04:14 | |
| *** cdelatte has joined #openstack-swift | 04:15 | |
| *** delattec has joined #openstack-swift | 04:21 | |
| *** cdelatte has quit IRC | 04:21 | |
| *** delatte has joined #openstack-swift | 04:21 | |
| *** ahonda has quit IRC | 04:24 | |
| *** jrichli has quit IRC | 05:03 | |
| ho | os1: Please use this for your double-check of recon. http://paste.openstack.org/show/192018/ | 05:30 |
| *** delattec has quit IRC | 05:36 | |
| *** delatte has quit IRC | 05:36 | |
| *** cdelatte has joined #openstack-swift | 05:37 | |
| *** delattec has joined #openstack-swift | 05:37 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:40 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Imported Translations from Transifex https://review.openstack.org/164086 | 06:12 |
| *** nshaikh has joined #openstack-swift | 06:55 | |
| *** bkopilov has quit IRC | 07:08 | |
| mattoliverau | Night all, have a great weekend | 07:16 |
| *** bkopilov has joined #openstack-swift | 07:28 | |
| *** donagh has quit IRC | 07:31 | |
| *** jamielennox has quit IRC | 07:31 | |
| *** bkopilov has quit IRC | 07:32 | |
| *** jamielennox has joined #openstack-swift | 07:34 | |
| *** donagh has joined #openstack-swift | 07:34 | |
| *** donagh has quit IRC | 07:39 | |
| *** jamielennox has quit IRC | 07:39 | |
| *** mmcardle has joined #openstack-swift | 07:43 | |
| *** donagh has joined #openstack-swift | 07:53 | |
| *** jamielennox|away has joined #openstack-swift | 07:55 | |
| *** jamielennox|away is now known as jamielennox | 07:55 | |
| *** chlong has quit IRC | 08:09 | |
| *** sc has joined #openstack-swift | 08:12 | |
| *** jistr has joined #openstack-swift | 08:12 | |
| *** openstackgerrit has quit IRC | 08:21 | |
| *** openstackgerrit has joined #openstack-swift | 08:22 | |
| *** rledisez has joined #openstack-swift | 08:22 | |
| openstackgerrit | Tushar Gohad proposed openstack/swift: Bump eventlet version to 0.16.1 https://review.openstack.org/164108 | 08:38 |
| *** kei_yama has quit IRC | 08:44 | |
| *** jistr has quit IRC | 08:46 | |
| *** jistr has joined #openstack-swift | 09:04 | |
| *** jordanP has joined #openstack-swift | 09:16 | |
| openstackgerrit | Lorcan Browne proposed openstack/swift: Add swift-recon feature to track swift-drive-audit error count https://review.openstack.org/163889 | 09:17 |
| *** NM has joined #openstack-swift | 09:43 | |
| *** NM has quit IRC | 09:45 | |
| *** bkopilov has joined #openstack-swift | 09:49 | |
| openstackgerrit | Prashanth Pai proposed openstack/swift: Make object creation more atomic in Linux https://review.openstack.org/162243 | 09:54 |
| *** bkopilov has quit IRC | 09:55 | |
| *** sluo_wfh has quit IRC | 10:08 | |
| *** nellysmitt has joined #openstack-swift | 10:17 | |
| *** delattec has quit IRC | 10:24 | |
| *** cdelatte has quit IRC | 10:24 | |
| *** aix has joined #openstack-swift | 10:44 | |
| *** jamielennox has quit IRC | 10:55 | |
| *** dmorita has quit IRC | 10:55 | |
| *** donagh has quit IRC | 11:01 | |
| *** erlon has joined #openstack-swift | 11:01 | |
| *** jamielennox|away has joined #openstack-swift | 11:02 | |
| *** jamielennox|away is now known as jamielennox | 11:02 | |
| *** donagh has joined #openstack-swift | 11:08 | |
| *** vinsh has joined #openstack-swift | 11:12 | |
| *** jamielennox has quit IRC | 11:14 | |
| *** jamielennox|away has joined #openstack-swift | 11:24 | |
| *** cdelatte has joined #openstack-swift | 11:24 | |
| *** jamielennox|away is now known as jamielennox | 11:24 | |
| *** vinsh has quit IRC | 11:32 | |
| *** haomaiwang has quit IRC | 11:37 | |
| *** ho has quit IRC | 11:46 | |
| *** bkopilov has joined #openstack-swift | 11:53 | |
| *** NM has joined #openstack-swift | 11:57 | |
| *** vinsh has joined #openstack-swift | 12:17 | |
| *** geaaru has joined #openstack-swift | 12:19 | |
| *** SkyRocknRoll has quit IRC | 12:20 | |
| *** bkopilov has quit IRC | 12:20 | |
| *** vinsh has quit IRC | 12:22 | |
| *** chlong has joined #openstack-swift | 12:24 | |
| *** cca has joined #openstack-swift | 12:36 | |
| *** vinsh has joined #openstack-swift | 12:37 | |
| *** cca has left #openstack-swift | 12:37 | |
| *** cca has joined #openstack-swift | 12:37 | |
| *** cca has quit IRC | 12:47 | |
| *** jkugel has joined #openstack-swift | 13:01 | |
| cschwede | hmm, looks like a few patches on the ec branch need an update to reflect the change in 1.0.1 of pyeclib (missing ECPyECLibException)? | 13:15 |
| cschwede | nevermind, patches only need to be rebased, it’s done in 719483a81f83391ee8471ea3dc50ffce933cfc27 | 13:25 |
| openstackgerrit | Christian Schwede proposed openstack/swift: Bump PyECLib version to >= 1.0.1 https://review.openstack.org/164172 | 13:28 |
| openstackgerrit | paul luse proposed openstack/swift: Multiple fragment Archive Index Support https://review.openstack.org/159637 | 13:35 |
| peluse | cschwede, was just looking at your comment on policy names being empty.... | 13:37 |
| cschwede | peluse: yeah, should probably never happen, but i’m wondering if a logging message would be appropriate. just in case. wdyt? | 13:40 |
| *** ppai has quit IRC | 13:41 | |
| peluse | cschwede, last time I checked we didn't allow empty strings when we parsed swift.conf but unless I'm seeing things (or not) I guess we do | 13:41 |
| peluse | if that made any sense :) | 13:42 |
| cschwede | hmm, or it’s only this function that allows empty strings, but never gets an empty string as arg. which is fine then (if the error is already raised earlier) | 13:42 |
| peluse | yeah, OK maybe the check is still there. Look in the constructor for class StoragePolicy(object) | 13:43 |
| cschwede | i’m currently reviewing patch 159637, looks like there is some interesting stuff that could be used outside EC as well | 13:44 |
| patchbot | cschwede: https://review.openstack.org/#/c/159637/ | 13:44 |
| peluse | cschwede, yeah, that's a key one for EC though and the focus of a lot of attention right now (I'm in San Fran today) | 13:45 |
| openstackgerrit | paul luse proposed openstack/swift: Erasure Code Reconstructor https://review.openstack.org/131872 | 13:46 |
| cschwede | peluse: ah nice! you should talk to elon musk to get a direct hyperloop connection to SF ;) | 13:46 |
| peluse | I don't think that's available until Q3 this year :) | 13:47 |
| *** mikehn_ has quit IRC | 13:48 | |
| *** mikehn has joined #openstack-swift | 13:49 | |
| *** bkopilov has joined #openstack-swift | 14:21 | |
| openstackgerrit | paul luse proposed openstack/swift: Rename RUGGEDIZE to SSYNC https://review.openstack.org/164194 | 14:23 |
| eikke | I was wondering: how are Swift developer meetups announced? | 14:28 |
| *** doxavore has joined #openstack-swift | 14:29 | |
| *** wer has joined #openstack-swift | 14:32 | |
| *** CrackerJackMack has quit IRC | 14:33 | |
| *** panbalag has joined #openstack-swift | 14:33 | |
| doxavore | clayg: thanks for the help yesterday - i still must not understand the zero_byte_files_per_second setting in our use case, but taking it from the default down to 20/sec has taken I/O util down from 75% to 50%. (previously going with files_per_second=20, MB/sec=5 took it from 100% to 75%) | 14:35 |
| doxavore | if the kernel supports it, is anyone aware of a reason you would _not_ want to use spice=yes in swift-2.2.2? | 14:36 |
| *** CrackerJackMack has joined #openstack-swift | 14:38 | |
| *** jrichli has joined #openstack-swift | 14:40 | |
| tdasilva | eikke: typically notmyname announces it on irc meetings | 14:40 |
| eikke | tdasilva: ok. should start attending those as well | 14:41 |
| *** silor has joined #openstack-swift | 14:47 | |
| tdasilva | eikke: https://wiki.openstack.org/wiki/Meetings/Swift contains the meeting agenda | 14:47 |
| *** thumpba has joined #openstack-swift | 14:53 | |
| *** thumpba has quit IRC | 14:53 | |
| eikke | tdasilva: thanks. might not be able to make it next week though, @ customer | 14:54 |
| *** thumpba has joined #openstack-swift | 14:55 | |
| *** doxavore has quit IRC | 14:55 | |
| *** thumpba_ has joined #openstack-swift | 15:02 | |
| *** lcurtis has joined #openstack-swift | 15:04 | |
| *** thumpba has quit IRC | 15:05 | |
| *** erlon has quit IRC | 15:11 | |
| *** nellysmitt has quit IRC | 15:12 | |
| *** silor has quit IRC | 15:13 | |
| *** erlon has joined #openstack-swift | 15:29 | |
| *** tsg has quit IRC | 15:30 | |
| *** nshaikh has left #openstack-swift | 15:32 | |
| *** jrichli has quit IRC | 15:32 | |
| *** david-lyle_afk is now known as david-lyle | 15:33 | |
| *** mmcardle has left #openstack-swift | 15:35 | |
| notmyname | good morning | 15:38 |
| tdasilva | notmyname: hello | 15:39 |
| *** reed has joined #openstack-swift | 15:40 | |
| notmyname | eikke: the mid-cycle/hackthon/whatevers? ya, I intentionally don't publicize them super widely (they're intended as a small, focused dev time), but I do my best to make them known to the contributor community. so ya, they are mentioned in the meetings and in here | 15:41 |
| *** jrichli has joined #openstack-swift | 15:41 | |
| *** mahatic has joined #openstack-swift | 15:42 | |
| *** logan2 has joined #openstack-swift | 15:54 | |
| *** takotuesday has joined #openstack-swift | 15:55 | |
| takotuesday | Hey guys, what do you think about using a swift container as a storage device for static html files for a heroku application? for static html files? | 15:57 |
| notmyname | I think that's a great idea | 15:57 |
| takotuesday | cool | 15:58 |
| openstackgerrit | paul luse proposed openstack/swift: Rename RUGGEDIZE to SSYNC https://review.openstack.org/164194 | 15:59 |
| takotuesday | so far Ive gotten my heroku app (it generates static html files and then hosts them) to generate the html and write to my swift container | 15:59 |
| takotuesday | the interesting thing about swift containers is there isnt really a directory structure | 15:59 |
| takotuesday | so when I route to the static files I have to figure out a way to access just the string | 16:00 |
| notmyname | takotuesday: http://docs.openstack.org/developer/swift/middleware.html#staticweb | 16:00 |
| takotuesday | notmyname: thanks! | 16:01 |
| openstackgerrit | Thiago da Silva proposed openstack/swift: versioned writes middleware https://review.openstack.org/134347 | 16:03 |
| takotuesday | notmyname: do you think this works with multiple index files? | 16:04 |
| notmyname | takotuesday: what do you mean? | 16:04 |
| notmyname | oh, IIRC if you have a path that ends in a /, staticweb will look for the configured index page | 16:05 |
| takotuesday | ahh ok | 16:05 |
| takotuesday | yeah so my app generates web content for each user which is globally accessible. Im using swift to maintain persistent storage for these sites. So user A could have a generated site, and user B could as well. These both need to be accessible from my app by all users | 16:06 |
| notmyname | takotuesday: sounds like a basic "global read" permission | 16:12 |
| eikke | notmyname: thanks | 16:12 |
| eikke | I should start working on the 'community interaction' part of my role :P | 16:12 |
| notmyname | eikke: :-) | 16:14 |
| m_han | hi! newbie question - if i have an object stored in my swift-cluster,which is a media file, how can i play it on my VLC ? | 16:19 |
| openstackgerrit | paul luse proposed openstack/swift: Multiple fragment Archive Index Support https://review.openstack.org/159637 | 16:23 |
| notmyname | m_han: if VLC does html5 media (ie pseudo streaming with range requests), then I'd expect it to work out of the box with no problem | 16:24 |
| openstackgerrit | paul luse proposed openstack/swift: Multiple fragment Archive Index Support https://review.openstack.org/159637 | 16:33 |
| *** panbalag has quit IRC | 16:40 | |
| *** panbalag has joined #openstack-swift | 16:41 | |
| openstackgerrit | paul luse proposed openstack/swift: Diskfile decides if durable is written based on policy https://review.openstack.org/162717 | 16:43 |
| openstackgerrit | paul luse proposed openstack/swift: Erasure Code Reconstructor https://review.openstack.org/131872 | 16:50 |
| *** takotuesday has quit IRC | 17:00 | |
| *** rledisez has quit IRC | 17:14 | |
| *** jordanP has quit IRC | 17:18 | |
| *** tsg_ has joined #openstack-swift | 17:19 | |
| notmyname | tdasilva: got a few minutes? I wanted to talk about you middleware/refactor patches | 17:20 |
| tdasilva | sure | 17:20 |
| notmyname | peluse is here in the office with torgomatic and clayg yesterday and today. we've been talking about ec stuff, and your refactors came up | 17:22 |
| tdasilva | notmyname: not sure that's a good thing or bad :P | 17:22 |
| notmyname | tdasilva: https://review.openstack.org/#/c/134347/ and https://review.openstack.org/#/c/156825/ and (maybe) https://review.openstack.org/#/c/159610/2 | 17:22 |
| notmyname | tdasilva: so in general and if possible, getting that stuff landed soon to help with the EC code in the proxy server would be good. in part, just to help with the EC merge to master | 17:23 |
| notmyname | so, what's the status on those, what do you thinks necessary to get them landed? just waiting on reviews? or is there other stuff too? | 17:25 |
| *** delattec has joined #openstack-swift | 17:25 | |
| tdasilva | notmyname: so for obj. versioning. I think it's just a matter of reviews...clayg has been very helpful, but so far the only one to comment | 17:26 |
| *** zhill has joined #openstack-swift | 17:26 | |
| tdasilva | I have to answer some comments he left there, but I think overall it is ready... | 17:26 |
| notmyname | ok | 17:27 |
| tdasilva | reed: 156825 - while talking to clayg, I think we agreed to limit the scope of that one | 17:27 |
| tdasilva | s/reed/re: | 17:27 |
| reed | uhm | 17:28 |
| reed | ah :) | 17:28 |
| notmyname | reed: get in theose swift code reviews, reed! ;-) | 17:28 |
| tdasilva | lol, soory reed | 17:28 |
| *** cdelatte has quit IRC | 17:28 | |
| tdasilva | can't type! | 17:28 |
| reed | aye, captain notmyname | 17:28 |
| notmyname | lol | 17:28 |
| tdasilva | anyway...I hope we can make good progress on that too | 17:29 |
| tdasilva | then the last one, it's about defining the scope, maybe we can just limit to the PUT verb or go refactor the other verbs too | 17:30 |
| *** erlon has quit IRC | 17:31 | |
| *** jistr has quit IRC | 17:31 | |
| *** m_han has quit IRC | 17:31 | |
| tdasilva | notmyname: was that helpful? :\ | 17:32 |
| *** bkopilov has quit IRC | 17:34 | |
| *** bkopilov has joined #openstack-swift | 17:35 | |
| *** rdaly2 has joined #openstack-swift | 17:39 | |
| *** doxavore has joined #openstack-swift | 17:40 | |
| *** bkopilov has quit IRC | 17:43 | |
| *** bkopilov has joined #openstack-swift | 17:44 | |
| openstackgerrit | Thiago da Silva proposed openstack/swift: versioned writes middleware https://review.openstack.org/134347 | 17:48 |
| *** bkopilov has quit IRC | 17:50 | |
| *** bkopilov has joined #openstack-swift | 17:51 | |
| *** bkopilov has quit IRC | 17:58 | |
| *** bkopilov has joined #openstack-swift | 17:59 | |
| *** doxavore has quit IRC | 18:00 | |
| openstackgerrit | David Goetz proposed openstack/swift: make swift-init skip things if config does not have section https://review.openstack.org/164285 | 18:05 |
| openstackgerrit | David Goetz proposed openstack/swift: make swift-init skip things if config does not have section https://review.openstack.org/164285 | 18:10 |
| notmyname | dfg: fun with pep8? ;-) | 18:10 |
| *** straycat is now known as sadcat_ | 18:11 | |
| dfg | notmyname: ya- forgot to run it. its been a while i guess... | 18:11 |
| notmyname | tdasilva: ya, limiting to PUT is definitely a possibility too | 18:13 |
| notmyname | tdasilva: and some of that maybe we could proposed to feature/ec (since it also touches a lot of the PUT path) so that we spread out the merge conflicts (ie land it on feature/ec and get it in master when feature/ec lands in master) | 18:14 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: wip: ec probe test https://review.openstack.org/164291 | 18:15 |
| tdasilva | notmyname: yes! I was about to say that for the last patch is where it makes the most sense to try to merge as much of the effort with whatever is going on in the EC branch | 18:15 |
| clayg | peluse: https://review.openstack.org/164291 | 18:15 |
| clayg | fix the required replica count shiz and then try to make fill in the comments to rmtree on some datadirs | 18:16 |
| notmyname | tdasilva: ya, so getting it proposed to feature/ec instead of on master would be great | 18:16 |
| *** delatte has joined #openstack-swift | 18:16 | |
| clayg | dfg was into pep8 before they were popular, now that everyone is using it; gerrit probably hasn't heard of the tools dfg uses now | 18:19 |
| *** delattec has quit IRC | 18:20 | |
| *** fandi has joined #openstack-swift | 18:20 | |
| *** fandi has quit IRC | 18:21 | |
| *** bsdkurt has quit IRC | 18:22 | |
| *** bkopilov has quit IRC | 18:25 | |
| *** bkopilov has joined #openstack-swift | 18:25 | |
| openstackgerrit | David Goetz proposed openstack/swift: make swift-init skip things if config does not have section https://review.openstack.org/164285 | 18:32 |
| *** bkopilov has quit IRC | 18:32 | |
| dfg | ok- now there's a unit test for it too. | 18:32 |
| *** bkopilov has joined #openstack-swift | 18:32 | |
| dfg | and i ran pep8. even though i'm way too cool for that | 18:32 |
| notmyname | :-) | 18:34 |
| tdasilva | notmyname: to get patch 159610 proposed to feature/ec branch we would need 134347 and 156825 to land on master first, no? | 18:37 |
| patchbot | tdasilva: https://review.openstack.org/#/c/159610/ | 18:37 |
| notmyname | whoa! thanks patchbot | 18:38 |
| * notmyname didn't remember that it did that | 18:38 | |
| notmyname | tdasilva: what about getting https://review.openstack.org/#/c/156825 on feature/ec | 18:39 |
| tdasilva | notmyname: mmm..guess we could try that | 18:41 |
| openstackgerrit | paul luse proposed openstack/swift: wip: ec probe test https://review.openstack.org/164291 | 18:51 |
| clayg | cschwede: I don't think an empty *base* string is really invalid | 18:54 |
| clayg | cschwede: we use that get_policy_string and split_policy_string to convert a base (say "tmp", or "async" or "objects") | 18:54 |
| clayg | to a stirn like "tmp-3", "async" (for Policy-0) or "objects-1" | 18:55 |
| clayg | so if you give get_policy_string and empty base get_policy_string('', Policy-1) - it will return (correctly?) '-1' meaning I added the trailer for Policy-1 to '' | 18:55 |
| clayg | then if you split_policy_string('-1') - you get back '', Policy-1 | 18:56 |
| clayg | so it seems reasonable for the inverse of get_policy_string('', Policy-0) == '' and split_policy_string('') == '', Policy-0 | 18:57 |
| *** zaitcev has joined #openstack-swift | 18:57 | |
| *** ChanServ sets mode: +v zaitcev | 18:57 | |
| *** jordanP has joined #openstack-swift | 19:03 | |
| *** jordanP has quit IRC | 19:03 | |
| *** vinsh has quit IRC | 19:11 | |
| *** vinsh has joined #openstack-swift | 19:12 | |
| *** geaaru has quit IRC | 19:14 | |
| *** vinsh has quit IRC | 19:16 | |
| *** bsdkurt has joined #openstack-swift | 19:16 | |
| *** zul has quit IRC | 19:19 | |
| *** vinsh has joined #openstack-swift | 19:29 | |
| notmyname | peluse: torgomatic: clayg: jrichli is looking in to getting the functests to run against a specific storage policy | 19:31 |
| notmyname | FYI | 19:31 |
| *** thumpba has joined #openstack-swift | 19:36 | |
| notmyname | zaitcev: I think you wrote a guest blog post, cause it sounds exactly like what you'd say http://www.vitavonni.de/blog/201503/2015031201-the-sad-state-of-sysadmin-in-the-age-of-containers.html | 19:38 |
| openstackgerrit | paul luse proposed openstack/swift: wip: ec probe test https://review.openstack.org/164291 | 19:38 |
| zaitcev | notmyname: you sound omnious | 19:38 |
| *** thumpba_ has quit IRC | 19:40 | |
| *** thumpba has quit IRC | 19:41 | |
| zaitcev | notmyname: I'm sorry to sound like luddite, but downloading random shit at build time is a problem. Even if you pull from a trusted source like pypi and check SSL. | 19:45 |
| *** bkopilov has quit IRC | 19:45 | |
| *** bkopilov has joined #openstack-swift | 19:48 | |
| *** zhill has quit IRC | 19:54 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: wip: ec probe test https://review.openstack.org/164291 | 19:59 |
| clayg | zaitcev: curl http://random.org/install | bash wfm | 19:59 |
| notmyname | zaitcev: oh, I completely agree. I wasn't complaining. I jsut think you'd really sympathize with what was there | 20:00 |
| *** vinsh has quit IRC | 20:01 | |
| *** bkopilov has quit IRC | 20:02 | |
| *** bkopilov has joined #openstack-swift | 20:04 | |
| *** zhill has joined #openstack-swift | 20:07 | |
| *** panbalag has quit IRC | 20:16 | |
| openstackgerrit | Merged openstack/swift: Prefer policy to policy_index https://review.openstack.org/163690 | 20:16 |
| *** delatte has quit IRC | 20:20 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: wip: ec probe test https://review.openstack.org/164291 | 20:22 |
| *** rdaly2 has quit IRC | 20:32 | |
| dfg | notmyname: you around? | 20:52 |
| notmyname | dfg: yo | 20:53 |
| notmyname | dfg: what's up? | 20:53 |
| dfg | so- i'm trying to decide something wanted your opinion. you think anybody would ever want container metadata that would be added to every object request | 20:53 |
| dfg | like you'd set x-container-meta-public-hello:there on the container and then every GET to an object in the container would have a response header of Hello:there on it | 20:54 |
| dfg | it sounds lame right? nobody would want that. just use object metadata | 20:55 |
| notmyname | wouldn't it be easier to add once to a container instead of to the millions of objects in the container (including all the existing ones there)? | 20:56 |
| notmyname | dfg: do you have a customer asking for it or some other new feature that would use it? | 20:56 |
| dfg | a well ya. and a more useful case would be x-container-meta-public-cache-control: public, whatevrer | 20:56 |
| dfg | like for static web public containers | 20:57 |
| notmyname | ya, cache control headers would be interesting, especially when plumbing through different caching stuff (cdns, varnish, etc) | 20:57 |
| notmyname | so you're wanting to keep it on a per-object basis? | 20:58 |
| dfg | i basicly need to write something that does this and keep going back and forth about special casing it and adding into swift. | 20:58 |
| dfg | the thing is if I special case it and then somebody later goes and adds it to swift it'll be pretty ugly | 20:58 |
| notmyname | right. or when the next thing needs to be added for every object in a container | 20:59 |
| dfg | like if you wanted to use swift, public containers as an origin for a CDN it would be nice to do x-container-meta-public-cache-control: public, max-age:900 or whatever and then every origin gets a automatic ttl set for the cdn. | 21:02 |
| dfg | i feel like somebody eventually will want this. | 21:02 |
| notmyname | right. that seems like an awesome feature | 21:03 |
| dfg | but you could just tell everbody to just set cache-control: public, ... on every object | 21:04 |
| notmyname | I haven't heard from anyone needing that (container meta applied to every object), but it also sounds like a relatively minor addition | 21:04 |
| dfg | ya... i'm just trying to decide what to do. | 21:05 |
| notmyname | dfg: I think it's a great feature for swift. if... | 21:11 |
| notmyname | dfg: if you start it like the allowed_headers in the object server. I think the hard part is what to do when the object server already has some metadata. so eg you don't want to override the content-type or date headers based on the container metadata | 21:11 |
| notmyname | so if there were a config list of "these are the headers you can set", maybe that makes it safer | 21:12 |
| notmyname | I don't know | 21:12 |
| notmyname | what do you do when you have a cache control header set on the object? use that I'd guess. but are there some headers I can't override? or maybe only some that I can? | 21:13 |
| dfg | ya- thats why i kinda just want to special case it. you start getting these ugly edge cases and the solution is worse than the problem | 21:14 |
| notmyname | ya | 21:14 |
| dfg | i mena write a custom middleware | 21:14 |
| notmyname | dfg: your example had -public-. would you only want it for public data? or maybe have x-container-response-meta-foo: bar (and later x-container-request-meta-foo) | 21:15 |
| notmyname | then configure only what's allowed to be set | 21:15 |
| notmyname | an example of the x-container-request-meta might be auto-adding an expiry time (maybe x-delete-after: 86400) for every object added to the container | 21:17 |
| notmyname | and the example for x-container-response-meta is your's with the cache control headers | 21:17 |
| dfg | oh- now you're just going crazy | 21:17 |
| notmyname | heh. that's the one I've heard of here. not the response headers | 21:17 |
| dfg | i've heard of people wanting that too | 21:18 |
| dfg | hmph. i don't want to wade into all that though. | 21:18 |
| notmyname | so it seems like your cache control header idea could be pretty close to the other one too. so add the response header one. and let someone else do the request one later | 21:18 |
| notmyname | make it simple. wouldn't be too much to wade through. config variable to set the explicit values that are allowed. keeps the surface space small | 21:19 |
| clayg | torgomatic: peluse: notmyname: patch 163209 | 21:24 |
| patchbot | clayg: https://review.openstack.org/#/c/163209/ | 21:24 |
| *** patchbot has quit IRC | 21:25 | |
| *** patchbot has joined #openstack-swift | 21:25 | |
| dfg | notmyname: alright- i'll play around with it for a bit and see how well it works. sounds like an ok solution. thx. | 21:25 |
| notmyname | dfg: no. thank you. I think it sounds pretty cool | 21:25 |
| peluse | patch 164112 | 21:51 |
| patchbot | peluse: https://review.openstack.org/#/c/164112/ | 21:51 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Rename RUGGEDIZE to SSYNC https://review.openstack.org/164194 | 21:52 |
| notmyname | wow, there's a lot to get done for EC in the next 2 weeks! | 21:56 |
| *** echevemaster has joined #openstack-swift | 22:00 | |
| *** NM has quit IRC | 22:10 | |
| *** jkugel has quit IRC | 22:12 | |
| *** jrichli_ has joined #openstack-swift | 22:14 | |
| *** jrichli has quit IRC | 22:18 | |
| *** mahatic has quit IRC | 22:20 | |
| *** mahatic has joined #openstack-swift | 22:21 | |
| *** zhill has quit IRC | 22:21 | |
| notmyname | tsg_: still around? | 22:24 |
| tsg_ | notmyname: yep | 22:24 |
| notmyname | tsg_: can you help jrichli_? she's getting her saio for ec set up and having an issue with getting pyeclib installed | 22:25 |
| tsg_ | notmyname: sure let talk to her | 22:25 |
| jrichli_ | I got an error when I tried to pip install: | 22:25 |
| jrichli_ | Error: liberasurecode-1.0.5 install failed : Record file /tmp/pip-oHkCsm-record/install-record.txt not found | 22:25 |
| openstackgerrit | John Dickinson proposed openstack/python-swiftclient: Reduce memory usage for download/delete and add --no-shuffle option to st_download https://review.openstack.org/130339 | 22:26 |
| tsg_ | jrichli_: for starters, can you paste your pip install command line | 22:26 |
| jrichli_ | pip install pyeclib | 22:27 |
| swifterdarrell | tsg_: I'm about to package pyeclib... anything I should know? | 22:27 |
| tsg_ | swifterdarrell: package as a deb? | 22:27 |
| swifterdarrell | tsg_: mostly interested in where to get the appropriate code (pypi?) | 22:27 |
| swifterdarrell | tsg_: deb & rpm | 22:28 |
| tsg_ | swifterdarrell: zigo is working on updating the debian package and I have written to zaitcev about rpm | 22:28 |
| swifterdarrell | tsg_: (for our product--the actual packaging process is quite wacky, but I've got a handle on that) | 22:28 |
| swifterdarrell | tsg_: I just need the (right) source; but an existing deb or rpm package always speeds things up | 22:28 |
| zaitcev | tsg_: Looks like I missed it, unless it was just now. | 22:28 |
| tsg_ | zaitcev: it was last night I believe | 22:29 |
| tsg_ | let me forward you again | 22:29 |
| swifterdarrell | tsg_: is the latest on PyPi good? | 22:29 |
| swifterdarrell | tsg_: (which appears to be "1.0.4"?) | 22:29 |
| tsg_ | swifterdarrell: that's a special version (with other packages integrated) | 22:30 |
| swifterdarrell | tsg_: also appears to be super-fresh: 2015-03-13 | 22:30 |
| tsg_ | let me point you in the right direction, one moment | 22:30 |
| swifterdarrell | tsg_: k | 22:30 |
| tsg_ | swifterdarrell: yes, I pushed an update last night with some error reporting changes | 22:30 |
| tsg_ | swifterdarrell: pyeclib tag "v1.0.5m" off of https://bitbucket.org/kmgreen2/pyeclib | 22:32 |
| tsg_ | swifterdarrell: liberasurecode has been packaged already for Debian/Ubuntu | 22:32 |
| tsg_ | http://packages.ubuntu.com/search?suite=trusty-backports&searchon=names&keywords=liberasurecode | 22:32 |
| tsg_ | swifterdarrell: zigo asked last night and I have sent him the new tags .. so he may already be in the middle of packaging the latest pyeclib and liberasurecode | 22:33 |
| tsg_ | perhaps I should add you to that thread | 22:33 |
| tsg_ | zaitcev: email sent to 'Pete Zaitcev' <zaitcev@redhat.com>, subject line: "PyECLib/liberasurecode releases" | 22:34 |
| swifterdarrell | tsg_: cool, I got the 1.0.5m code tarball successfully; and that liberasurecode link should give me a leg up packaging that as well | 22:35 |
| tsg_ | yes | 22:35 |
| tsg_ | swifterdarrell: there is already a pyeclib debian version as well, in the process of being accepted into Ubuntu - you might be able to use those control files for pyeclib | 22:36 |
| tsg_ | https://packages.debian.org/unstable/main/python-pyeclib | 22:37 |
| swifterdarrell | tsg_: sweet! | 22:37 |
| tsg_ | swifterdarrell: you will need these 2 packages for Jerasure EC backend - http://packages.ubuntu.com/search?keywords=jerasure&searchon=names&suite=trusty-backports§ion=all | 22:38 |
| tsg_ | (libjerasure2, libgf-complete1) | 22:39 |
| tsg_ | those two have been backported to trusty and are fairly up to date | 22:39 |
| tsg_ | so you should be able to use those debs as-is | 22:40 |
| openstackgerrit | John Dickinson proposed openstack/swift: List valid EC types in error https://review.openstack.org/142311 | 22:40 |
| tsg_ | jrichli_: try removing "/tmp/pip-*" and retry the "pip install" perhaps? | 22:43 |
| jrichli_ | i'll try it | 22:43 |
| jrichli_ | there was nothing to remove: I ran again, and same error | 22:45 |
| notmyname | jrichli_: what os? | 22:47 |
| notmyname | I just upgraded it on my saio with no problem (`pip install -U pyeclib`) | 22:47 |
| jrichli_ | Ubuntu 14.04.1 LTS | 22:47 |
| jrichli_ | i will try that | 22:48 |
| zigo | tsg_: Indeed, I'll work on this very soon. | 22:49 |
| jrichli_ | same thing. | 22:49 |
| jrichli_ | I had tried running python setup.py install before | 22:49 |
| jrichli_ | and had got this: | 22:49 |
| *** cdelatte has joined #openstack-swift | 22:49 | |
| jrichli_ | src/c/pyeclib_c/pyeclib_c.c:32:25: fatal error: erasurecode.h: No such file or directory | 22:49 |
| jrichli_ | #include <erasurecode.h> | 22:49 |
| *** delattec has joined #openstack-swift | 22:49 | |
| jrichli_ | that was using this: | 22:50 |
| jrichli_ | Downloading https://pypi.python.org/packages/source/P/PyECLib/PyECLib- | 22:50 |
| jrichli_ | 1.0.4.tar.gz#md5=9886dd9a4b3d66fa40a642945fd70077 | 22:50 |
| tsg_ | jrichli_: I assume you have not installed liberasurecode or other deps before, correct? | 22:52 |
| tsg_ | zigo: many thanks! :) | 22:52 |
| jrichli_ | nothing for EC until today, that is right | 22:52 |
| zigo | tsg_: Is there a chance that SSE is detected and use at runtime only? | 22:53 |
| zigo | tsg_: Because I had to disable it, unfortunately... :( | 22:53 |
| tsg_ | zigo: that's possible with ISA-L :) | 22:53 |
| zigo | What's ISA-L ? | 22:53 |
| tsg_ | zigo: we'll recommend folks to build Jerasure/gf-complete locally if they need performance | 22:53 |
| tsg_ | zigo: https://01.org/intel%C2%AE-storage-acceleration-library-open-source-version | 22:53 |
| *** delattec has quit IRC | 22:53 | |
| *** cdelatte has quit IRC | 22:53 | |
| tsg_ | zigo: storage SIMD primitives library | 22:54 |
| tsg_ | zigo: open source version supports Erasure Coding - something that can use a Debian package for sure | 22:54 |
| zigo | tsg_: Did you bump sonames? | 22:55 |
| tsg_ | zigo: https://bitbucket.org/tsg-/liberasurecode/commits/4e1290ea61e5dba3e782155f426e4e492664d525 | 22:56 |
| zigo | I'll have to rename liberasurecode0 into liberasurecode1, which is annoying (it's going to have to go through the NEW queue...). | 22:56 |
| tsg_ | zigo: ack, but that's what we get by going to the stable(r) 1.x series :( | 22:58 |
| zigo | tsg_: SONAME has nothing to do with moving from beta to stable. | 22:59 |
| zigo | You'd bump it if there's an ABI breakage. | 22:59 |
| zigo | And eventually, we'll make a transition so that the lib and its dependencies are moving at once from SID to Testing. | 23:00 |
| tsg_ | zigo: I get the point .. what is your suggestion on SONAME? keep it at 0 for now, I can make that change if you feel necessary but that might break stuff for folks that are already using liberasurecode.so.1? | 23:02 |
| tsg_ | s/keep it at 0 for now/keep it at 0 for now?/ | 23:03 |
| zigo | tsg_: If there's already some users for .1, then yes, it's too late. | 23:03 |
| zaitcev | tsg_: thanks, I got the message. Indeed, I only packaged jerasure and gf-complete. Haven't touched PyECLib yet. | 23:03 |
| tsg_ | zigo: mainly folks that pulled off of pypi in the last few days only, though | 23:03 |
| zigo | tsg_: In the future, just keep in mind that the SONAME has nothing to do with the version number of your software. | 23:03 |
| zigo | tsg_: Look at MySQL for example... | 23:04 |
| zigo | We're at libmysqlclient18 for version 5.5 of mysql. | 23:04 |
| tsg_ | zigo: agreed - I read your comment differently to start with, but I do see the point | 23:04 |
| tsg_ | zigo: acknowledged | 23:04 |
| eikke | do you aim for full EC support in Kilo? | 23:05 |
| zigo | tsg_: Oh, and one more thing, SONAME bumps are a major pain to deal with in Debian, if there's many reverse dependencies. | 23:05 |
| zigo | Got to coordinate the transition with the release team, etc. | 23:05 |
| tsg_ | zigo: since most folks are using pypi version which has liberasurecode integrated for now, I can change liberasurecode to use the old SONAME | 23:05 |
| tsg_ | zigo: I am glad you brought this up .. we sure don't want to add to the pain :) | 23:06 |
| zigo | tsg_: For me, this has very little consequences, since liberasurecode was never in Testing. | 23:06 |
| zigo | tsg_: So, no transition, etc. | 23:06 |
| zigo | (thanks to the freeze of Jessie, and the fact it wasn't there, somehow...) | 23:07 |
| notmyname | eikke: the current plan is that kilo will have an EC beta. we'll begin the merge to master is about 2 weeks, based on what's done between now and then | 23:07 |
| zigo | tsg_: Just to give you an example. I also maintain miniupnp in Debian. I had to fix 4 or 5 software because of some API change (including bitcoin, warzone2000, and some other which I don't remember). | 23:08 |
| zigo | Just because upstream decided to break stuff... | 23:08 |
| zigo | It took about 2 or 3 months to do s. | 23:08 |
| tsg_ | zigo: :) that sucks - feel your pain! | 23:08 |
| notmyname | eikke: we expect that nearly all functionality will be in there, but there will be room to improve (filling in the final gaps) | 23:09 |
| tsg_ | zigo: in the current situation, are you suggesting to move ahead with liberasurecode1? | 23:09 |
| zigo | The process is: upload the new lib to experimental. Try to build all reverse dependencies to see if something breaks. File bugs for the breakage, preferably with patches if possible. Then upload to Sid, raise the bugs severity from important to serious, wait a few days, then do NMU (Non-maintainer-upload) if the maintainer of the package doesn't do his homework... | 23:10 |
| zigo | tsg_: In the current situation, don't do anything, I don't care. | 23:10 |
| zigo | I'm just warning you for the future, in order to avoid pain... :) | 23:10 |
| tsg_ | zigo: gotcha :) | 23:10 |
| mattoliverau | Wow, you all talk a lot, its Saturday here and thought I'd read some scroll back... Wow.. I guess it is a Friday for y'all. :p | 23:11 |
| tsg_ | zigo: and I did learn a lot for sure going just through the pain of getting the stuff into Ubuntu :) | 23:11 |
| tsg_ | zigo: points taken on SONAMEs and ABI compatibility | 23:11 |
| tsg_ | zigo: thank you for doing the heavy lifting here! | 23:11 |
| notmyname | mattoliverau: tons of stuff going on. IT'S SO EXCITING!!! | 23:12 |
| mattoliverau | notmyname: now I feel like I'm missing out :( | 23:12 |
| tsg_ | jrichli_, assume you've already attempted "sudo pip install pyeclib"? | 23:13 |
| mattoliverau | tdasilva: I'll take a look at your patches Monday my time (Sunday yours) | 23:13 |
| notmyname | mattoliverau: if you feel that way, download some EC patches and get to reviewing!! (also, fixing merge conflicts between the versioned writes middleware and the ec branch would be helpful) | 23:13 |
| zigo | tsg_: Well, thanks to you guys for doing the erasurecode stuff, which is a great feature! | 23:14 |
| tsg_ | zigo: thanks! :) | 23:14 |
| jrichli_ | tsg_ oh geez, I can't believe I didn't run as sudo. That was it! I would have expected a permission denied error instead | 23:15 |
| tsg_ | jrichli_, I am so relieved! ;) | 23:15 |
| jrichli_ | thanks so much! | 23:15 |
| tsg_ | jrichli_, it has been a pain to keep pypi, jenkins and co happy | 23:15 |
| tsg_ | jrichli_, thought we ran into something new! and that we might need a new tag :) | 23:15 |
| notmyname | jrichli_: lol | 23:15 |
| tsg_ | jrichli_, glad it worked | 23:15 |
| mattoliverau | notmyname: maybe I will... Tho the wife may kill me for working on weekends :p | 23:16 |
| jrichli_ | me too! | 23:16 |
| jrichli_ | ^^ re: glad it worked, not that my wife may kill me :-) | 23:16 |
| eikke | notmyname: interesting, thanks | 23:17 |
| *** tsg_ has quit IRC | 23:19 | |
| notmyname | eikke: are you in france or the bay area? I thought you were in france, but if so you're on IRC pretty late on a friday... :-) | 23:20 |
| eikke | notmyname: now in France. Just got back home from an Internations party (though its only 00:21) | 23:21 |
| notmyname | "only" | 23:21 |
| mattoliverau | Lol | 23:21 |
| mattoliverau | Thanks jrichli_ | 23:22 |
| eikke | my days are mostly 9AM-2AM | 23:22 |
| *** lcurtis has quit IRC | 23:27 | |
| zigo | liberasurecode 1.0.5 and pyeclib 1.0.5m uploaded to Sid. :P | 23:28 |
| *** david-lyle is now known as david-lyle_afk | 23:29 | |
| *** jrichli_ has quit IRC | 23:29 | |
| eikke | what kind of EC is used? plain RS? | 23:32 |
| *** keving has joined #openstack-swift | 23:32 | |
| notmyname | eikke: it's quite extensible. | 23:32 |
| notmyname | and keving just joined | 23:32 |
| notmyname | eikke: keving is the EC expert | 23:32 |
| keving | whats up? | 23:33 |
| notmyname | keving: eikke was just asking which ec algos are supported in pyeclib/liberasurecode | 23:33 |
| eikke | keving: just asked which kind of EC is being used | 23:33 |
| keving | reed-solomon mostly | 23:33 |
| eikke | plain RS or something different | 23:33 |
| eikke | right | 23:33 |
| keving | what do you mean by "plain"? | 23:33 |
| keving | the underlying libraries use pretty sophisticated optimizations | 23:34 |
| eikke | surue | 23:34 |
| keving | there is also a class for XOR-based codes available | 23:34 |
| eikke | I meant,using the general idea of RS, or another form of EC | 23:34 |
| keving | both | 23:35 |
| keving | i assume most people will just use RS | 23:35 |
| eikke | i see | 23:35 |
| * eikke has done quite some EC research in the past | 23:35 | |
| notmyname | eikke: keving maintains jerasure these days | 23:35 |
| eikke | mostly non-RS (though cant talk too much about that, not even in current job, because of things) | 23:35 |
| notmyname | eikke: also note that pyeclib (or liberasurecode) have a pretty small interface that allows for any other EC library to be used. (details left to the reader?) | 23:36 |
| keving | sure. so, the XOR-based codes are based on a paper i co-authored in 2010 | 23:36 |
| keving | eikke: if you have something, we could make a backend in liberasurecode | 23:36 |
| keving | eikke: i assume you are talking about some form of regenerative code? | 23:37 |
| eikke | keving: rateless codes, yes | 23:37 |
| keving | oh rateless | 23:37 |
| keving | for storage? | 23:37 |
| eikke | now I should STFU :) | 23:38 |
| keving | ok | 23:38 |
| keving | (i am also doing a little work in rateless codes) | 23:38 |
| eikke | keving: you at intel? | 23:38 |
| keving | Box | 23:38 |
| eikke | oh, ok | 23:38 |
| keving | i would like to add regenerative codes to liberasurecodce, but the current API does not fit rateless codes | 23:39 |
| notmyname | keving: typey, typey | 23:39 |
| keving | haha | 23:40 |
| * eikke doesnt get that joke? | 23:40 | |
| keving | eikke: gotta run to another meeting. feel free to drop my an email (kmgreen2@gmail.com) if you wanna talk more | 23:40 |
| notmyname | eikke: something I picked up from some australians. basically, "get to typing!" | 23:41 |
| eikke | keving: would love to, but Im afraid I better don't due to corporate BS (<- which doesnt refer to my current employer, because of the very same corporate BS I can't touch any of the EC stuff there either) | 23:42 |
| keving | eikke: heh. i get it. in any case, the offer is still on the table. | 23:42 |
| keving | notmyname: let me know if there are any other issues on the library side of things. AFAIK, tushar and the packaging folks have things worked out. | 23:43 |
| mattoliverau | bloody Australians | 23:43 |
| eikke | will remember (and dlook into the libs you've been working on with quite some interest soon) | 23:43 |
| keving | eikke: cool. | 23:44 |
| notmyname | keving: ok. will do. and tsg has been doing great tracking down the packaging stuff. Ithink we're good for know | 23:44 |
| keving | notmyname: good to hear! | 23:44 |
| keving | ok, gtg | 23:44 |
| notmyname | keving: thanks for stopping by | 23:44 |
| keving | notmyname: sure. i should do it more often ;) | 23:44 |
| *** keving has left #openstack-swift | 23:45 | |
| notmyname | yes you should :-) | 23:45 |
| *** mahatic has quit IRC | 23:46 | |
| notmyname | clayg: torgomatic: how can I help you this weekend? | 23:49 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!