Friday, 2015-01-23

*** abhirc has joined #openstack-swift00:00
*** jasondotstar has quit IRC00:03
*** ho has joined #openstack-swift00:09
*** nellysmitt has quit IRC00:10
openstackgerritSamuel Merritt proposed openstack/swift: Allow per-policy overrides in object replicator.  https://review.openstack.org/14944100:11
hogood morning!00:14
*** oomichi has joined #openstack-swift00:22
*** dmorita has joined #openstack-swift00:32
*** tdasilva has quit IRC00:39
*** jasondotstar has joined #openstack-swift00:40
*** k69 has joined #openstack-swift00:44
k69hi, are these values ok for swift-ring-builder command ? (http://paste.ubuntu.com/9827340/) the metadata for accound is fine?00:53
k69account.builder*00:53
k69sry i meant " balance meta " values00:54
*** jasondotstar has quit IRC00:58
*** yuanz has joined #openstack-swift01:03
*** yuan has quit IRC01:03
*** peluse_ has joined #openstack-swift01:03
*** tsg has quit IRC01:03
*** ahonda has joined #openstack-swift01:04
*** peluse has quit IRC01:06
*** gyee has quit IRC01:07
mattoliverauk69: have you done a rebalance on account.buider? post to adding sdc1 that is? cause devices id 2 and 3 have no partitions01:14
k69mattoliverau, actually i couldnt rebalance them, coz i added them after rebalanced sdb1 so it gave me error of "minimum_hour "which  i needed to wait for an hour, is it going to make problems if i rebalance it later?01:18
k69mattoliverau, ty for the reply01:19
mattoliverauk69: is this on a live cluster or a test one?01:21
k69test01:21
mattoliverauk69: then you can run: swift-ring-builder account.builder pretend_min_part_hours_passed01:22
mattoliverauk69: so you don't have to wait min-part-hours01:22
k69oh thanks01:23
mattoliverauoh and morning ho :) Sorry been in and out today and missed you coming online :)01:24
k69thats great, and one more problem i have faced is that at the part of restarting services, "sudo service memcached restart" works fine but "sudo service swift-proxy restart" shows "stop : unknown instances" and "start :job failed ro start"01:24
k69on all nodes01:25
mattoliverauk69: you can replace the ring without restarting services. The services wil automatically look for an updated ring. Also I find 'swift-init proxy restart' works better01:26
mattoliverauk69: just dump the ring in /etc/swift on your storage nodes and that's it :)01:26
k69mattoliverau, oh so u mean i dont need to restart them at all as (http://docs.openstack.org/juno/install-guide/install/apt/content/swift-finalize-installation.html) has said01:28
mattoliverauk69: if you are just updating the ring files, then no. If you modify the proxy configuration, then yes. You just rebalanced account.builder, building a new account ring, so assumed that is the only thing that has changed01:30
k69mattoliverau, oh, ok, but i have also modified proxy, and i have inserted "sudo chown -R swift:swift /etc/swift" but on controller and object nodes, all cannot recognize the service01:32
k69mattoliverau, when i want to restart them01:32
mattoliverauk69: does swift-init work? i.e: swft-init proxy restart01:34
mattoliverauSorry I mean swift-init proxy restart (you need to type correctly)01:35
k69mattoliverau, http://paste.ubuntu.com/9827768/01:36
mattoliverauk69: looks like you have a configuration issue in proxy-server.conf01:37
k69mattoliverau, oh yes it has error, thanks01:39
homattoliverau: morning! i'm looking forward to meeting you online. i recoginzed that hotels in sanfrancisco are expensive. :)01:40
mattoliverauho: yeah me too. are you going to be staying at the pickwick? the one suggested in eventbrite?01:41
homattoliverau: not yet decided.01:42
homattoliverau: I checked the cost of the hotel.01:42
homattoliverau: now i'm checking the cost of hotels around pickwick.01:44
mattoliverauho: well that's where I'll be :) Work is sending me, so I didn't check, just put down the recommended hotel and they booked it :)01:44
*** dmsimard_away is now known as dmsimard01:46
*** tellesnobrega_ has joined #openstack-swift01:47
openstackgerritDaisuke Morita proposed openstack/swift: Output logs of policy index  https://review.openstack.org/13699501:48
homattoliverau: i will let you know the info (biginning of next week).01:49
mattoliverauho: cool, glad you can come :)01:50
homattoliverau: welcom :)01:51
*** dmsimard is now known as dmsimard_away01:55
*** abhirc has quit IRC02:03
*** abhirc has joined #openstack-swift02:04
*** jasondotstar has joined #openstack-swift02:05
*** jasondotstar has quit IRC02:05
k69 mattoliverau: umm - still cant restart it swift.conf > ( http://paste.openstack.org/show/160567/ ) & proxy-server.conf > (http://paste.openstack.org/show/160569/)02:11
*** guest10101010 has joined #openstack-swift02:13
*** haomaiwang has joined #openstack-swift02:14
*** addnull has joined #openstack-swift02:21
mattoliverauk69: I just did a quick test, you need to remove the whitespace at the start of 'paste.filter_factory = keystonemiddleware.auth_token:filter_factory' in the [filter:authtoken] section02:22
k69<mattoliverau>, wow thanks for the test, the "swift-init proxy restart" works but still "service swift-proxy restart" doesnt02:26
*** tsg has joined #openstack-swift02:26
k69oh no wait, after that i also worked02:27
k69thanks alot it is solved :)02:27
*** fandi has quit IRC02:27
mattoliverauk69: cool! well happy testing and enjoy swift :)02:28
k69<mattoliverau>, :-)02:29
k69sure !02:29
*** guest10101010 has quit IRC02:33
*** hugespoon has quit IRC02:41
*** hugespoon has joined #openstack-swift02:41
*** dmsimard_away is now known as dmsimard02:57
*** tellesnobrega_ has quit IRC02:58
*** tellesnobrega_ has joined #openstack-swift03:04
*** tellesnobrega_ has quit IRC03:09
*** tellesnobrega_ has joined #openstack-swift03:18
*** addnull has quit IRC03:21
*** fandi has joined #openstack-swift03:22
*** bill_az has quit IRC03:25
*** lpabon has quit IRC03:26
k69hey guys, how can i rebuild a base file by "swift-ring-builder account.builder create 10 3 1" command coz i have made mistakes for the current account.builder and it shows false information by "swift-ring-builder account.builder " command03:28
*** hugespoon has quit IRC03:30
*** dmsimard is now known as dmsimard_away03:30
*** hugespoon has joined #openstack-swift03:44
*** hugespoon has left #openstack-swift03:44
openstackgerritYuan Zhou proposed openstack/swift: Fix container deletion synchronizing with non_zero SP  https://review.openstack.org/14946903:53
mattoliverauk69: what do you mean by rebuild a base file? if you want to recreate from scratch then mv the account.builder out of the way and run create again. If it is just the drives then you can use the add/remove03:56
k69mattoliverau, oo ok, can i delete account.builder and account.ring.gz, and run the swift builder add command thing ?03:59
*** jrichli has joined #openstack-swift03:59
openstackgerritYuan Zhou proposed openstack/swift: Fix container deletion synchronizing with non_zero SP  https://review.openstack.org/14946904:01
k69mattoliverau, nvm IT WORKS WONDER !!! thanks alot, swift is fun :) (with ur help)04:05
*** k69 has quit IRC04:05
*** lcurtis has joined #openstack-swift04:06
notmynameprint "hello, world"04:07
notmynamewhat did I miss today?04:07
notmyname(my flight home is delayed nearly 2 hours)04:07
mattoliverauk69: yes.. but if it was a real cluster I wouldn't, you'd modify the exiting build file as otherwise the partitions (depending on the seed used) could be distributed to other drives causing the cluster to move potentially all objects (not good)... but seeing as your testing, its fine :)04:17
mattoliveraunotmyname: hey! how's seattle?04:18
mattoliveraunotmyname: I've written a distributed prefix tree, it can search for large sub trees, I can split them out, join them back together and changes in data is timestamped.. now I just need to hook it into container sharded v2 and see how well it works ;)04:21
notmynamemattoliverau: seattle is nice (as always). pretty cloudy today. kinda colder than I expected. and seems to have a problem with a certain plane to my home. but they still have great fish for dinner!04:21
notmynamemattoliverau: nice! (wrt prefix trees)04:22
mattoliverauoh and finish the update code. So far everything stays in order... #famous last words.04:22
mattoliveraunotmyname: nice :) Presentation went well then?04:23
notmynameya, I think so. unfortunately, due to some schedule changes, there weren't many people there04:23
notmynamehowever, it was fun talking to people running Swift and hearing about what their experiences are04:24
mattoliveraunotmyname: bugger, yeah, so long as people (or even you) got something out of it, then it's a success :)04:24
notmynameMoz specifically said that things work pretty well and they like the performance and the ease of operations (manageability). they're pretty happy with it and building a new, larger swift cluster04:25
mattoliveraunotmyname: so this Monday is Australia day, so long weekend here.. so probably wont be around Monday, just a heads up.. but seeing as it's your Sunday you probably wont notice ;P04:25
notmynameand someone from oracle was there and talked a little about how they use swift (upstream swift!) internally for storage (backups, social media storage, etc)04:26
notmynamemattoliverau: :-)04:26
mattoliveraunotmyname: awesome!04:26
mattoliverauGo upstream swift!04:26
notmynameit's sortof well-known that oracle's public cloud, which purports a swift API, is a reimplementation that they did internally on their own stuff. but today I learned that they are using upstream swift internally04:27
*** panbalag has quit IRC04:28
notmynameI should wander over to my gate. if my guess is correct, boarding should start in about 15 minutes04:29
notmynametalk to everyone tomorrow04:30
mattoliveraunotmyname: k, have a great flight! Get some safe04:31
mattoliverau*get home safe04:31
*** jrichli has quit IRC04:37
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873604:38
openstackgerritTushar Gohad proposed openstack/swift: Bump eventlet version to 0.16.1  https://review.openstack.org/14540304:53
*** abhirc has quit IRC05:03
*** abhirc has joined #openstack-swift05:05
*** lcurtis has quit IRC05:08
*** nshaikh has joined #openstack-swift05:11
*** ppai has joined #openstack-swift05:22
*** lpabon has joined #openstack-swift05:27
*** lpabon has quit IRC05:29
*** nshaikh has quit IRC05:35
*** abhirc has quit IRC05:46
openstackgerritKota Tsuyuzaki proposed openstack/swift: Efficient Replication for Distributed Regions  https://review.openstack.org/9982405:51
*** chlong has quit IRC05:54
*** chlong has joined #openstack-swift06:01
*** addnull has joined #openstack-swift06:01
hoI was wondering about a way how to tox test with external package (only provided at git: https://github.com/rodrigods/oslo.policy). how to configure this for testing?06:05
hos/about a way//06:05
*** bpap has quit IRC06:16
hoi added oslo.policy entry in test-requirements.txt but it doesn't work (there is no module in pypi)06:21
*** fandi has quit IRC06:21
*** jyoti-ranjan has joined #openstack-swift06:22
ho-e git://github.com/rodrigods/oslo.policy.git#egg=oslo_policy in test-requirements.txt works. unnn...06:27
*** jyoti-ranjan has quit IRC06:31
*** silor has joined #openstack-swift06:44
*** fandi has joined #openstack-swift06:45
*** echevemaster has quit IRC06:46
*** wasmum has quit IRC06:51
*** wasmum has joined #openstack-swift06:51
*** tellesnobrega_ has quit IRC07:00
*** fandi has quit IRC07:23
*** fandi has joined #openstack-swift07:30
*** fandi has quit IRC07:36
*** jyoti-ranjan has joined #openstack-swift07:47
*** chlong has quit IRC08:14
*** fandi has joined #openstack-swift08:14
*** rledisez has joined #openstack-swift08:18
*** fandi has quit IRC08:20
*** geaaru has joined #openstack-swift08:21
*** fandi has joined #openstack-swift08:23
*** jyoti-ranjan has quit IRC08:23
*** jyoti-ranjan has joined #openstack-swift08:39
*** addnull has quit IRC08:39
*** addnull_ has joined #openstack-swift08:41
*** addnull_ has quit IRC08:42
*** addnull has joined #openstack-swift08:42
*** addnull has quit IRC08:42
*** acoles_away is now known as acoles08:43
*** addnull has joined #openstack-swift08:43
*** addnull has quit IRC08:44
*** addnull has joined #openstack-swift08:44
*** addnull has quit IRC08:45
*** addnull has joined #openstack-swift08:45
*** addnull has quit IRC08:46
*** addnull has joined #openstack-swift08:46
*** addnull has quit IRC08:47
*** addnull has joined #openstack-swift08:47
*** addnull has quit IRC08:48
*** addnull has joined #openstack-swift08:48
*** addnull has quit IRC08:49
*** addnull has joined #openstack-swift08:49
*** addnull has quit IRC08:50
*** addnull has joined #openstack-swift08:50
*** addnull has quit IRC08:51
*** addnull_ has joined #openstack-swift08:51
*** addnull_ has quit IRC08:52
*** addnull has joined #openstack-swift08:53
*** addnull has quit IRC08:54
*** addnull has joined #openstack-swift08:54
*** addnull has quit IRC08:55
*** addnull has joined #openstack-swift08:55
*** addnull has quit IRC08:56
*** addnull has joined #openstack-swift08:56
*** addnull has quit IRC08:57
*** addnull has joined #openstack-swift08:57
*** nellysmitt has joined #openstack-swift08:57
*** addnull has quit IRC08:58
*** addnull has joined #openstack-swift08:58
*** addnull_ has joined #openstack-swift09:00
*** addnull has quit IRC09:00
*** addnull has joined #openstack-swift09:01
*** addnull_ has quit IRC09:01
*** addnull has quit IRC09:02
*** addnull has joined #openstack-swift09:02
*** addnull has quit IRC09:02
*** addnull has joined #openstack-swift09:03
*** addnull has quit IRC09:04
*** addnull has joined #openstack-swift09:04
*** addnull has quit IRC09:05
*** addnull has joined #openstack-swift09:05
*** addnull has quit IRC09:06
*** addnull has joined #openstack-swift09:06
*** addnull has quit IRC09:07
*** addnull has joined #openstack-swift09:07
*** addnull has quit IRC09:08
*** addnull has joined #openstack-swift09:08
*** addnull has quit IRC09:09
*** addnull has joined #openstack-swift09:10
*** addnull has quit IRC09:10
*** addnull has joined #openstack-swift09:11
*** addnull has quit IRC09:12
*** addnull has joined #openstack-swift09:12
*** addnull has quit IRC09:13
*** addnull has joined #openstack-swift09:13
*** addnull has quit IRC09:14
*** addnull has joined #openstack-swift09:14
*** addnull has quit IRC09:15
*** jistr has joined #openstack-swift09:15
*** addnull has joined #openstack-swift09:15
*** addnull has quit IRC09:16
*** addnull has joined #openstack-swift09:16
*** addnull has quit IRC09:17
*** addnull has joined #openstack-swift09:17
*** addnull has quit IRC09:18
*** addnull has joined #openstack-swift09:19
*** addnull has quit IRC09:19
*** addnull has joined #openstack-swift09:20
*** addnull has quit IRC09:20
*** addnull has joined #openstack-swift09:21
*** addnull_ has joined #openstack-swift09:22
*** addnull has quit IRC09:22
*** addnull_ has quit IRC09:23
*** addnull has joined #openstack-swift09:23
*** addnull has quit IRC09:24
*** addnull has joined #openstack-swift09:24
*** addnull has quit IRC09:25
*** addnull has joined #openstack-swift09:25
*** addnull has quit IRC09:26
*** addnull has joined #openstack-swift09:26
*** addnull has quit IRC09:27
*** addnull has joined #openstack-swift09:28
*** addnull_ has joined #openstack-swift09:29
*** addnull has quit IRC09:29
*** addnull_ has quit IRC09:29
*** addnull has joined #openstack-swift09:30
*** addnull has quit IRC09:34
*** jordanP has joined #openstack-swift09:39
openstackgerritMerged openstack/swift: dlo: Update doc about manifest containing data  https://review.openstack.org/14639009:41
*** addnull has joined #openstack-swift09:50
*** addnull has quit IRC09:52
*** addnull has joined #openstack-swift09:52
openstackgerritMerged openstack/swift: Make ThreadPools deallocatable.  https://review.openstack.org/14564709:57
*** ppai has quit IRC09:58
*** tsg has quit IRC10:01
*** ppai has joined #openstack-swift10:12
*** jyoti-ranjan has quit IRC10:15
*** ho has quit IRC10:23
*** aix has joined #openstack-swift10:29
*** dmorita has quit IRC10:31
*** tellesnobrega_ has joined #openstack-swift10:36
*** nellysmitt has quit IRC10:43
*** jyoti-ranjan has joined #openstack-swift10:46
*** addnull has quit IRC10:56
*** ppai has quit IRC11:00
*** addnull has joined #openstack-swift11:00
*** addnull has quit IRC11:04
*** haomaiwang has quit IRC11:05
*** tellesnobrega_ has quit IRC11:05
*** addnull has joined #openstack-swift11:09
*** addnull has quit IRC11:13
*** ppai has joined #openstack-swift11:13
openstackgerritTakashi Kajinami proposed openstack/swift: Remove redundant container updating after rsync  https://review.openstack.org/14930811:21
*** jyoti-ranjan has quit IRC11:28
*** addnull has joined #openstack-swift11:29
*** jyoti-ranjan has joined #openstack-swift11:35
*** mahatic has joined #openstack-swift11:37
*** panbalag has joined #openstack-swift11:38
*** fandi has quit IRC11:42
openstackgerritJoel Wright proposed openstack/python-swiftclient: This patch fixes downloading files to stdout.  https://review.openstack.org/14489911:46
*** chlong has joined #openstack-swift11:50
*** nellysmitt has joined #openstack-swift12:10
*** tellesnobrega_ has joined #openstack-swift12:20
*** tellesnobrega_ has quit IRC12:35
*** gvernik has joined #openstack-swift12:42
*** tellesnobrega_ has joined #openstack-swift12:43
*** addnull has quit IRC12:50
*** gvernik has quit IRC12:57
*** jyoti-ranjan has quit IRC13:02
*** bill_az has joined #openstack-swift13:26
*** ppai has quit IRC13:49
*** mtreinish has quit IRC14:08
*** abhirc has joined #openstack-swift14:16
*** lcurtis has joined #openstack-swift14:19
*** joeljwright has joined #openstack-swift14:25
*** tellesnobrega_ has quit IRC14:27
*** tellesnobrega_ has joined #openstack-swift14:27
*** tellesnobrega_ has quit IRC14:52
*** chlong has quit IRC14:54
*** abhirc has quit IRC15:01
*** thebloggu has joined #openstack-swift15:06
*** jasondotstar has joined #openstack-swift15:10
*** abhirc has joined #openstack-swift15:14
theblogguIs there any way for me to specify the swift.conf to use when starting a server in Icehouse? I'm having this (https://bugs.launchpad.net/swift/+bug/1091007) issue because I'm trying to run swift in a user directory for testing and everytime I try to run the proxy server I get an error saying I don't have the /etc/swift/swift.conf file15:17
portanteHave folks heard about Espresso - "LinkedIn's hot new distributed document store"?  http://getprismatic.com/story/142187186659615:28
portanteThe are positioning it between Oracle and Voldemort (http://www.project-voldemort.com/voldemort/, which is apparently an opensource version of Amazon's dynamo)15:30
portantes/The/They/15:30
*** tsg has joined #openstack-swift15:31
*** abhirc has quit IRC15:45
*** jrichli has joined #openstack-swift15:45
*** dmsimard_away is now known as dmsimard15:51
*** abhirc has joined #openstack-swift15:55
*** abhirc has quit IRC16:03
*** booly-yam-9117 has joined #openstack-swift16:07
*** tdasilva has joined #openstack-swift16:09
*** jordanP has quit IRC16:19
*** tellesnobrega_ has joined #openstack-swift16:28
*** david-lyle_afk is now known as david-lyle16:35
*** geaaru has quit IRC16:49
*** Nadeem_ has joined #openstack-swift17:00
*** Nadeem_ has quit IRC17:00
*** abhirc has joined #openstack-swift17:02
*** gvernik has joined #openstack-swift17:11
*** gvernik has quit IRC17:16
*** jistr has quit IRC17:16
*** zul has quit IRC17:17
notmynamegood morning, world17:18
*** zul has joined #openstack-swift17:19
notmynameportante: interesting17:20
notmynamemahatic: congrats on having the OPTIONS patch land!17:20
mahaticnotmyname, thank you! that was quick :)17:21
mahaticpatch landing17:21
mahaticnotmyname, for the server type, won't Host header do?17:25
notmynametsg: hello. I've got a question about eventlet versions17:25
mahatichttp://tools.ietf.org/html/rfc4229#section-2.1.5117:25
notmynamemahatic: isnt' the host header set on the request? is there a def....oh, thanks. /me reads17:25
tsgnotmyname: good morning17:26
tsgnotmyname: eventlet versions seems to be a hot topic on all channels (nova, infra, swift!) :)17:26
notmynametsg: is 0.16.1 the min version? or just the recommended one?17:27
tsgnotmyname: for us 0.16.0 is the minimum17:27
notmynametsg: ah? what else? is it the nova breakage thing? I talked to infra people last week about it17:27
notmynametsg: hmm...what doe 0.16.1 specifically give us that 0.16.0 doesn't have (pure swift perspective)17:27
tsgnotmyname: yes, clarkb seemed to indicate yesterday that the nova gate breakage wasn't an issue at this point17:27
tsgnotmyname: there were some inconsistencies in the files included in the eventlet 0.16.0 source tarball and pip blob17:28
notmynameya, the cause is that stable release testing didn't freeze requirements. so it was tested with new versions of the code (which removed deprecated functionality) and thus broke in the gate17:28
notmynametsg: that's the only difference in 16 and 16.1?17:29
tsgso we suggested the author create a 0.16.1 point fix - so that's the only difference17:29
tsgnotmyname: correct17:29
tsgthere was an issue a couple of days ago where bandersnatch would not pick up 0.16.1 but that seems to have been resolved by clarkb17:29
notmynametsg: ok. so IMO then the only real consideration is whatever distros find easier to package. whatever they want is what we should have in the requirements17:30
notmynameie it's very likely that whatever we have in global requirements is what gets packaged in the next ubuntu/rhel/cent repos17:30
tsgnotmyname: yes, I have a pending review for global-requirements master to get 0.16.1 in (https://review.openstack.org/#/c/145816/) that clarkb said he would look at soon (next week)17:31
tsgwe can then match the same in swift requirements on feature/ec17:31
notmynameok. sounds like 16.1 is the way to go17:31
notmynametsg: what's it blocking for us wrt EC work?17:32
tsgnotmyname: yes .. the < 0.16 version freeze applied only to stable/juno17:32
*** cutforth has quit IRC17:33
tsgsince jenkins is pulling 0.16.1 now, there is no blocker for us17:33
tsg(it was a blocker earlier for the 2-phase commit stuff I have pending)17:33
notmynameok17:33
notmynametsg: thanks for working on it!17:33
tsgnotmyname: sure.  I will post the change today after some cleanup17:33
notmynamegreat17:34
notmynamemahatic: I don't think Host sounds right. (http://tools.ietf.org/html/rfc7230#section-5.4)17:36
notmynamemahatic: perhaps Server is better (http://tools.ietf.org/html/rfc7231#section-7.4.2)17:36
tsgnotmyname, peluse_: we just got a +2 for 0.16.1 on global-requirements.  so that's another thing out of the way17:36
notmynameyay17:36
*** gvernik has joined #openstack-swift17:40
*** panbalag1 has joined #openstack-swift17:44
mahaticnotmyname, oh, yes. My bad. Thanks. But er what should I be returning in the header? What all information?17:45
*** acoles is now known as acoles_away17:45
notmynamemahatic: what do you think?17:45
*** gvernik has quit IRC17:45
*** booly-yam-9117_ has joined #openstack-swift17:46
*** booly-yam-9117 has quit IRC17:47
*** panbalag has quit IRC17:48
*** jyoti-ranjan has joined #openstack-swift17:50
peluse_tsg:  nice job!!!!17:50
mahaticnotmyname, each server's details?17:51
mahaticnotmyname, ip, port, device?17:52
*** jasondotstar has quit IRC17:52
tsgpeluse_: should have the rest of the change up there today after we chat17:52
notmynamemahatic: we'd already know the IP and port (ie we just talked on that very ip and port to get this info). and there could be multiple devices17:52
mahaticnotmyname, ah, sorry, ring details17:56
notmynamemahatic: I'd expect somehting like "%s (%s)" % (server_name, swift_version)17:56
mahaticnotmyname, oh that's it?17:56
*** tsg_ has joined #openstack-swift17:57
*** tsg has quit IRC17:57
mahaticokay. I got the recon mixed up. I'm out of my mind.17:58
notmynamemahatic: ya, I think so. then recon can ping the servers in the ring with OPTIONS and then aggregate and report on the results17:59
*** rledisez has quit IRC18:00
mahaticnotmyname, hmm, okay. I need to understand ring a bit more I think.18:03
*** joeljwright has left #openstack-swift18:03
mahaticnotmyname, the aim is to only validate the server name and version? Or is it like, once we have them, more information on server will be followed?18:05
*** jasondotstar has joined #openstack-swift18:08
*** jrichli has quit IRC18:16
*** silor has quit IRC18:21
notmynamemahatic: I think to start with we only want to validate the name. having the version too offers some opportunity for new tools later. but for now, validating the name is sufficient I think18:28
*** kevinc_ has joined #openstack-swift18:29
mahaticnotmyname, ah okay18:31
*** Nadeem has joined #openstack-swift18:35
*** tsg_ has quit IRC18:39
*** david-lyle has quit IRC18:46
*** tdasilva has quit IRC18:46
theblogguIs there any way for me to specify the swift.conf to use when starting a server in Icehouse? I'm having this (https://bugs.launchpad.net/swift/+bug/1091007) issue because I'm trying to run swift in a user directory for testing and everytime I try to run the proxy server I get an error saying I don't have the /etc/swift/swift.conf file18:46
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873618:50
*** panbalag1 has quit IRC18:52
*** nshaikh has joined #openstack-swift19:00
*** zaitcev has joined #openstack-swift19:01
*** ChanServ sets mode: +v zaitcev19:01
*** tdasilva has joined #openstack-swift19:01
*** thebloggu has quit IRC19:05
*** jyoti-ranjan has quit IRC19:06
*** reed_ has joined #openstack-swift19:06
*** jrichli has joined #openstack-swift19:10
*** lcurtis has quit IRC19:10
*** jasondotstar has quit IRC19:15
*** jasondotstar has joined #openstack-swift19:18
*** Nadeem has quit IRC19:22
*** booly-yam-9117_ has quit IRC19:36
*** tsg has joined #openstack-swift19:38
*** mahatic has quit IRC19:41
*** tellesnobrega_ has quit IRC19:44
abhircnewbie question : Do Swift and Ceph go well together20:06
notmynameabhirc: define "together" ;-)20:06
*** bill_az has quit IRC20:07
*** jdprax has joined #openstack-swift20:08
*** reed_ is now known as reed20:09
*** reed has joined #openstack-swift20:09
jdpraxI've created a blueprint for something that I would love to see come to swift. Thoughts? https://blueprints.launchpad.net/swift/+spec/object-versioning-rsync20:10
abhircnotmyname: I have access to an OpenStack cluster we have Ceph as the backend and it was deployed in a 3 node cluster with object storage devices co-located20:12
notmynameabhirc: ah20:13
*** nshaikh has left #openstack-swift20:13
notmynameabhirc: I'd recommend giving swift and ceph separate drives (if not separate servers). mixing the two in a deployment would likely result in a lot of hardware contention20:14
abhircnotmyname: When I access the OpenStack dashboard I see that on accessing the Containers tab under Object Storage gives me a Server Error , on further trouble shooting , it seems Swift has not been integrated  and was just curious as to knowing if there are complications on having them both together, the cluster is an experimental one.20:15
notmynameabhirc: are you sure that "containers" tab in horizon isn't for the docker-like containers? ie instead of object storage things20:16
notmynameabhirc: horizon does support reading and writing data with swift20:16
*** gvernik has joined #openstack-swift20:19
*** kevinc_ has quit IRC20:21
abhircnotmyname: I have a RedHat OpenStack flavor deployed, I might be going in the right direction , my understanding is if I create a Project and access that , I can see Object Storage Panel and under that Containers tab , am I looking at the wrong tab20:21
notmynameabhirc: I'm not too familiar with horizon, actually. I'd ask david lyle, but I don't see him online right now. or maybe someone else in here has used horizon and can offer some guidance20:23
abhircnotmyname: thanks so much , really appreciate it!20:29
*** gvernik has quit IRC20:30
*** kevinc_ has joined #openstack-swift20:43
*** bpap has joined #openstack-swift20:46
*** jdprax has quit IRC20:52
claygso what's the priority review for the day?  I might be able to get one in.20:54
claygreuse-port?  concurrent-requests?  fix swift download container object -o -?20:54
claygmaybe I should try to rewrite the set_overload patch to rip out the calculator sillyness and just do the change display to and set to use %?20:55
notmynameclayg: Joel was asking about https://review.openstack.org/#/c/130339/7 this morning20:56
notmynameand it seems to be an internal blocker for him20:57
notmynameclayg: and for swift, the global cluster replication improvements could take an extra eye20:57
zaitcevGuys, do we know of any off-line or batch tools that verify consistency checking of Swift? I tried to do that with swift-report, but there wasn't any particular urgency, so I never finished it beyond basic matching of accounts against Keystone.20:58
notmynamezaitcev: consistency checking? like the dispersion report? or like auditor stuff?20:58
claygis that the shuffle thing?20:59
zaitcevnotmyname: I mean objects that do not belong into any containers, and generally mismatch between container DBs and objects.20:59
notmynameclayg: ya20:59
claygI thought I already tried to +2 that and then it got even more complicated with more scary edge cases that I don't really have a good way to functionally test :\21:00
notmynamezaitcev: ah. umm...nothing springs to mind, but I'm hoping that because I've not looked recently21:01
claygzaitcev: swifterdarrell had something like that once - you should try to weasel it out of him21:02
zaitcevnotmyname: okay, thought so, thanks21:02
*** bill_az has joined #openstack-swift21:02
zaitcevclayg: that is because of your argument, "if we don't do X, then an enhanced possiblity to get dark data exist (per current code)". So I thought, why not do this: don't do X, but instead re-animate an effort for "swift fsck". Currently we don't even _know_ if dark data exists, and how much.21:04
zaitcevWhat if RAX could save a million a year just by cleansing dark data21:04
claygwhen was I arguing that we need to do X to prevent dark data?21:04
claygthe async_update behavior on 404's?  That was the last time I remember talking abou ti...21:05
zaitcevyes, https://review.openstack.org/9959821:05
zaitcevwell, I'm going through review list from bottom up21:05
claygzaitcev: but yeah the last time we needed to fsck (I like that term) a swift cluster was cause it got >98% full21:05
swifterdarrellzaitcev: clayg; I *do* have something like that, but it's ugly21:05
claygswifterdarrell: zaitcev doesn't care if it's ugly!21:06
zaitcevswifterdarrell: it's okay, just lemme see even a draft21:06
zaitcevswifterdarrell: more importantly, did you apply it to any live clusters and did you find any interesting dark data and other screw-ups?21:06
zaitcev(that auditors cannot identify)21:07
claygzaitcev: oh, i don't know that i was arguing that Takashi's patch should get merged just that I still think it's a problem - briancline too.21:07
claygzaitcev: there was a bunch of hubub about trying test it, but no one did21:08
zaitcevclayg: okay, so I keep -1 for now. It may be not a problem for anyone who knows Swift at all, but I'm inundated by reports of stuck updaters on clusters that random people install to test RDO. Usually it's "list does not show my objects I HATE YOUR SHITWARE"21:09
claygbut ultimately the containers are where we list where the objects are, there's pleanty of code that currently deals with a replication re-animaging a container (half-deleted container) and I think that is definately perferable than loosing an object update, but again, i don't recall the specifics of the implementation that Takashi offered21:10
zaitcevso I'm touchy about updaters21:10
claygstuck updaters?  wow that's curious...21:10
zaitcevwell usually just overloaded VMs21:10
clayghrmmm... overloaded like out of disk space, or just slow cpu?21:11
zaitcevslow21:11
zaitcevthey get even servers timing out, so... But still...21:11
ahalewell we know if dark data exists from the difference between df and the account dbs - but yeah if RAX could save a million a year just by cleansing dark data I could buy a boat21:12
zaitcevRe. specifics, he only removed the check for 404. It's 1-liner.21:12
*** booly-yam-3388 has joined #openstack-swift21:13
zaitcevokay. I'll put fsck on todo list then21:13
notmynameahale: have you tracked any of that yet? I remember there being some delta just from fs-metadata overhead, so there osn21:14
ahaleyeah we have21:14
notmynameahale: have you tracked any of that yet? I remember there being some delta just from fs-metadata overhead, so they won't match exactly21:14
notmynameah. what delta are you seeing?21:14
ahalewell, a theory - and we found and deleted a bunch21:14
notmynameie what's acceptable21:14
ahaleoh i dont have numbers handy, it was a horrible long running thing I tried to have as little to do with as I could21:15
zaitcevfigures21:15
ahaleand only looking at files > 1000MB (not 1024, thanks glance)21:15
notmynameahale: sounds like a gholt project ;-)21:15
zaitcevwas it worthwhile though21:15
ahaleyeah it was/is21:15
notmynamezaitcev: I wonder how you'd track async pendings (or generally the eventual consistency of listings). maybe checking objects that are over a certain age?21:19
zaitcevnotmyname: I suppose that works... There must be a certain churn at all times that fsck cannot keep up with.21:22
*** booly-yam-3388 has quit IRC21:27
notmynameto quote from a presentation given by some people running swift: "maintenance is easy. [swift is] made of unicorn farts"21:27
notmynamethat's from http://moz.com21:28
ahaleyah its unicorn herding thats the hard bit21:28
notmyname:-)21:28
notmynameahale: ya, that was under the "short term maintenance" category21:29
notmynameahale: they acknowledged that long term maintenance (large containers, capacity planning, etc) is harder21:29
*** kevinc_ has quit IRC21:30
notmynamehttp://d.not.mn/swift_unicorn_farts.JPG21:32
*** geaaru has joined #openstack-swift21:34
ahaleheres some idea of what we got on some drives we cleared some darkdata from http://img.cfil.es/0a05bc52-c62c-4f43-b118-d77d0d333da7.jpg21:36
*** abhirc has quit IRC21:36
*** abhirc has joined #openstack-swift21:36
notmynameahale: half of what's interesting there is the domain name ;-)21:37
ahalelol21:37
ahaleits just mine not a rax thing21:37
notmynameah. like my d.not.mn21:37
ahaleyup21:37
notmynamea quarter is the number of graphed lines ;-)21:37
ahaleit goes to 105% cos we overclock our drives21:38
notmynameand then the 15% drop in used capacity (if I'm reading it right)21:38
ahale~ yah21:38
notmynameahale: you must put racing stripes and a spoiler on the drives!21:38
*** jasondotstar has quit IRC21:38
redbohigh flow cats21:38
*** kevinc_ has joined #openstack-swift21:39
notmyname"it helps airflow in the dc"21:39
ahalealso turns out if you flip half the drives the spin balances better21:39
notmynamedo you use these drives? http://www.damngeeky.com/2013/11/14/15498/f1-race-car-replica-assembled-entirely-wd-hard-drive-parts.html21:40
redboit was tough when we opened sydney, we had to find drives that spin counterclockwise21:41
notmynamelol21:41
redboI have this plan to make a bloom filter of all the objects in all the containers in the world, then just delete any objects that aren't in the bloom filter.  But for some reason that scares people.21:43
*** abhirc_ has joined #openstack-swift21:44
*** abhirc has quit IRC21:47
ahalei know it should work cos its maths, but its the just delete bit thats scary21:47
*** tsg has quit IRC21:49
claygredbo: http://www.somethingsimilar.com/2012/05/21/the-opposite-of-a-bloom-filter/21:49
notmynameswifterdarrell thinks bloom filters are cool. he'd be all over that21:49
*** tsg has joined #openstack-swift21:51
notmynameahale: when you're cleaning up the data, how'd you know to delete it instead of add it to the listing?21:52
notmynamewas there another index of data somewhere?21:52
redboclayg: That's a neat and simple idea21:53
ahalenope there wasn't, you just have to go on the listing21:53
notmynameahale: so why'd you delete data? ie why is the object there wrong instead of the listing being wrong?21:54
ahalesomething to do with tombstones expiring faster than asyncs and/or something else21:56
*** tsg has quit IRC21:56
ahalethat cluster had been at 100% as well btw21:57
notmynameat least it wasn't 105%. then youd have to deal with full cluster problems21:58
ahalehehe true21:58
*** lcurtis has joined #openstack-swift21:59
redboI checked logs for about a zillion of those objects and they all had DELETEs issued last.  Objects being uplaoded without the containers being updated doesn't really seem to happen.21:59
notmynameI don't doubt you handled it. just curious about how you did it22:00
redbobut objects hiding on some handoff server until the tombstones get removed is totally a thing that happens22:00
ahaleahh that was it22:00
notmynamecleanup interval longer than the replication cycle?22:01
claygnotmyname: it'd have to be on the handoff longer than reclaim age22:01
claygredbo: I'm pretty sure I've seen create container get pushed to 3 handoffs because of 507's - container existence still works - but async get's deleted without updating the container because it doesn't look at handoffs22:02
*** kevinc_ has quit IRC22:03
redboI don't think we've seen that, but we've had a problem where asyncs come around and create an object in the container after the tombstone entry in the container has been removed.22:06
claygyuk, good to know22:07
claygwas the async just hanging around the object tombstone had already claned up the .data too?22:07
*** gyee has joined #openstack-swift22:07
claygthis is very therapeutic22:08
redboyeah.  Either someone put a drive back into commission after it was out for a week, or someone was really hammering a container and the asyncs couldn't clear for a week or whatever.22:09
ahaleif i had a dollar for every async i'd be a billionaire..22:09
*** kevinc_ has joined #openstack-swift22:10
claygmaybe if we fixed the thing where tombstones don't get reaped from inactive suffixes we could run with a higher reclaim age... idk, maybe dynamic reclaim age with backpressure from how healthy things are (replication cycle time, async count) - not sure how to solve an old disk coming into the fix - mtime on the file > reclaim age brings great skepticism?22:11
claygany one doing reviews on swiftclient -> this one is super annoying for me: https://review.openstack.org/#/c/149108/22:13
redboyeah, maybe.  I've sat down to write a disk indexing/hashing thing that's better than hashes.pkl like 8 times now, but I haven't finished one yet.22:13
redbobut part of that should be tracking tombstone files22:16
claygrunning tox on pythonswiftclient is so freaking useless, I don't even understand, where is the summary of the failed tests - why is pypy in the test set by default - does anyone understand how testr failing works?22:17
clayglike what was wrong with nose?  I liked the pretty list of dots...22:19
*** tdasilva has quit IRC22:23
peluse_redbo:  fyi as part of the EC work I have a WIP progress that will refactor the whole hashes.pkl so likely more easily allow for replacing the whole scheme with really anything on a per policy basis...22:23
peluse_redbo, it is here if you're curious what its looking like now: https://review.openstack.org/#/c/131872/22:23
*** chlong has joined #openstack-swift22:24
*** jrichli has quit IRC22:33
*** peluse_ has quit IRC22:34
*** peluse has joined #openstack-swift22:44
redbopeluse: The API still seems pretty geared toward invalidating a whole suffix hash at a time.22:45
redbopeluse: I'd kind of like a class that's an abstract "partition index" or whatever, that you tell when something is added or removed to the partition, so a fancy new hashing strategy isn't forced to walk the suffix directory to find out what's changed.22:45
*** ChanServ sets mode: +v peluse22:46
peluseredbo: yeah, that's true as the intent here is to enable a new format for what we need for EC... I'm hesitant of course to expand the scope any further but maybe it makes the next project easier - maybe not :)22:48
redbofair enough22:49
redbobut I don't think it'd be able to do what I want (or not well)22:52
peluse:(22:54
redbohaha.. well, constantly invalidating a hash of a chunk of the filesystem then walking to rebuild it wasn't the most inspired thing.  When we generally have the information that something is added or removed and could update a hash.23:00
redboor do something even cleverer than hashing with that information23:03
*** david-ly_ has joined #openstack-swift23:07
torgomaticredbo: you mean chexor?23:07
*** dmsimard is now known as dmsimard_away23:07
*** abhirc_ has quit IRC23:08
*** david-ly_ is now known as david-lyle23:11
redboobviously I mean chexor23:12
*** bpap_r has joined #openstack-swift23:17
*** bpap has quit IRC23:19
claygpeluse: the thing is that interface needs to be moved up a level for the object-server so it's just .delete_object(**useful_info) and .add_object(**useful_info) then the implementation can decide if that means invalidate the whole suffix or update something smarter - the backend interfaces for the replicators matter less, if you didn't do per suffix syncing the replicators would look a lot different anyway23:19
*** gyee has quit IRC23:32
*** lcurtis has quit IRC23:51
*** abhirc has joined #openstack-swift23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!