Monday, 2014-04-07

*** RockKuo has joined #openstack-swift00:11
*** matsuhashi has joined #openstack-swift00:18
*** zackf has joined #openstack-swift00:58
*** fifieldt has joined #openstack-swift01:19
*** nosnos has joined #openstack-swift01:30
*** zackf has quit IRC02:13
*** mlipchuk has quit IRC02:24
*** chandan_kumar has joined #openstack-swift02:29
*** chandan_kumar has quit IRC02:30
*** matsuhashi has quit IRC02:47
*** nosnos has quit IRC03:02
*** matsuhashi has joined #openstack-swift03:04
*** nosnos has joined #openstack-swift03:20
*** matsuhashi has quit IRC03:29
*** chandan_kumar has joined #openstack-swift03:30
*** haomai___ has quit IRC03:30
*** chandan_kumar has quit IRC03:31
*** chandan_kumar has joined #openstack-swift03:31
*** cheri has joined #openstack-swift03:39
*** chandan_kumar has quit IRC03:48
*** zhiyan_ is now known as zhiyan03:50
*** nosnos has quit IRC03:51
*** sfineberg has quit IRC04:02
*** madhuri has quit IRC04:03
*** zackf has joined #openstack-swift04:03
*** madhuri has joined #openstack-swift04:03
*** sfineberg has joined #openstack-swift04:04
*** zackf has quit IRC04:07
*** chandan_kumar has joined #openstack-swift04:08
*** zackf has joined #openstack-swift04:33
*** zhiyan is now known as zhiyan_04:39
*** chandan_kumar has quit IRC05:02
*** Midnightmyth has joined #openstack-swift05:03
*** ppai has joined #openstack-swift05:08
*** saju_m has joined #openstack-swift05:13
*** chandan_kumar has joined #openstack-swift05:27
*** StevenK has quit IRC05:29
*** wkelly has quit IRC05:29
*** StevenK has joined #openstack-swift05:30
*** wkelly has joined #openstack-swift05:30
*** nshaikh has joined #openstack-swift05:33
*** scohen has joined #openstack-swift05:40
*** scohen has quit IRC05:43
*** Midnightmyth has quit IRC05:51
*** cheri has quit IRC06:18
*** cheri has joined #openstack-swift06:25
*** cheri has quit IRC06:29
*** scohen has joined #openstack-swift06:36
*** saju_m has quit IRC06:41
*** psharma has joined #openstack-swift06:50
*** RockKuo has quit IRC06:51
*** saju_m has joined #openstack-swift06:54
openstackgerritGil Vernik proposed a change to openstack/swift: Support for content-length in the upload object method for internal client.  https://review.openstack.org/8565006:59
*** chandan_kumar has quit IRC07:02
*** h6w has left #openstack-swift07:03
*** chandan_kumar has joined #openstack-swift07:07
*** mlipchuk has joined #openstack-swift07:27
*** Longgeek_ has joined #openstack-swift07:29
*** cheri has joined #openstack-swift07:30
*** Longgeek_ has quit IRC07:34
*** Longgeek_ has joined #openstack-swift07:38
*** mmcardle has joined #openstack-swift08:00
*** nacim has joined #openstack-swift08:00
*** mlipchuk has quit IRC08:02
*** foexle has joined #openstack-swift08:03
*** gvernik has joined #openstack-swift08:09
gvernikI uploaded a patch that failed with some error in Jenkis, i never saw before. Can someone explain me what is wrong with it? https://review.openstack.org/#/c/85650/08:10
*** tanee is now known as tanee-away08:13
*** tanee-away is now known as tanee08:14
*** mkerrin has joined #openstack-swift08:14
*** RockKuo has joined #openstack-swift08:15
*** [1]gvernik has joined #openstack-swift08:16
*** gvernik has quit IRC08:16
*** [1]gvernik is now known as gvernik08:16
*** mlipchuk has joined #openstack-swift08:20
openstackgerritVictor Stinner proposed a change to openstack/python-swiftclient: Fix swiftclient.client.quote() for Python 3  https://review.openstack.org/8410208:21
*** mlipchuk has quit IRC08:25
*** sungju_ has quit IRC08:29
openstackgerritVictor Stinner proposed a change to openstack/python-swiftclient: Fix swiftclient.client.quote() for Python 3  https://review.openstack.org/8410208:30
*** mlipchuk has joined #openstack-swift08:34
*** gadb has joined #openstack-swift08:35
*** jamie_h has joined #openstack-swift08:35
*** matsuhashi has joined #openstack-swift08:35
*** nosnos has joined #openstack-swift08:40
gadbhugokuo : hi08:47
hugokuogadb: hi08:47
gadbhow are u today?08:48
hugokuogadb: good thx/08:48
gadbi still test about delete .ts file ;(08:48
gadbi have some question.08:49
gadbdo you have some time for me?08:49
hugokuogadb: yes08:51
gadbwhen i delete hashes.pki file all object dir, delete ts file and remake hashes.pki consisit "<80>^B}q^A." text08:51
hugokuogadb: https://bugs.launchpad.net/swift/+bug/130172808:51
gadbin my think, that text means "empty" partition08:51
gadbright?08:51
gadbthx. that was my problam08:52
hugokuogadb: you can load the pickle for human readable format08:53
gadbi make scirpt for delete all object hashes.pki file.08:53
gadbso, swift delete all .ts file.08:53
gadbswift can not find hashes.pki file. then try reconstitution hashes.pki file.08:54
hugokuosure... if the .ts already over the reclaim_age and the hash_suffix is None.08:54
gadbyeah.08:54
*** d89 has joined #openstack-swift08:54
hugokuoAlmost right.08:54
gadbbut, i wonder when delete empty partition08:54
gadbpartitions are never delete? swift only create?08:55
gadbmy goal is clear all swift needless i-node08:56
gadbhugokuo : your bug report so cool :)09:00
gadbthat is my problum. exactly09:01
hugokuogadb: Perhaps someone will review it later.09:04
hugokuogadb: anyhow, I think the partition directory will not be removed.  But it doesn't matter from my point of view.09:05
gadbhugokuo : thx. i will review too. in my think. partitions are reuse. but i wondoer how many partition can create by swfit. and what logic09:09
hugokuogadb: easy.... that's the basic of Swift. You can have a look at the ring mechanism.09:10
gadbokey. i well check doc.09:10
gadbanyway have a good dinner and good time09:10
hugokuogadb: For the total partitions in a cluster, it will be 3*2**part_power .09:11
*** sungju_ has joined #openstack-swift09:14
*** sungju_ has quit IRC09:14
*** gvernik has quit IRC09:16
gadbhugokuo : yes. i know it. but i concern that swift much many partition.09:17
gadbsorry. i must  go out. have a good day.09:17
*** ashish_ has joined #openstack-swift09:19
*** gadb has quit IRC09:21
*** mlipchuk has quit IRC09:25
*** mkollaro has joined #openstack-swift09:27
*** mlipchuk has joined #openstack-swift09:27
*** saju_m has quit IRC09:27
*** ashish_ has quit IRC09:30
*** mlipchuk1 has joined #openstack-swift09:32
*** mkollaro has quit IRC09:35
*** mlipchuk has quit IRC09:35
*** mkollaro has joined #openstack-swift09:42
openstackgerritA change was merged to openstack/swift: Move module level setup under setup_package method  https://review.openstack.org/8433909:44
openstackgerritA change was merged to openstack/python-swiftclient: Make bin/swift testable part 2  https://review.openstack.org/7648909:45
*** Trixboxer has joined #openstack-swift09:52
*** tovecs has quit IRC10:03
*** gvernik has joined #openstack-swift10:08
*** matsuhashi has quit IRC10:16
*** Longgeek_ has quit IRC10:16
*** matsuhashi has joined #openstack-swift10:16
*** matsuhashi has quit IRC10:20
*** saju_m has joined #openstack-swift10:32
*** [1]gvernik has joined #openstack-swift10:42
*** gvernik has quit IRC10:44
*** [1]gvernik is now known as gvernik10:44
*** saju_m has quit IRC10:49
*** jamie_h_ has joined #openstack-swift10:55
*** [1]gvernik has joined #openstack-swift10:57
*** jamie_h has quit IRC10:57
*** gvernik has quit IRC10:58
*** [1]gvernik is now known as gvernik10:58
*** [1]gvernik has joined #openstack-swift11:02
*** gvernik has quit IRC11:03
*** tdasilva has joined #openstack-swift11:05
*** [1]gvernik has quit IRC11:07
*** scohen has left #openstack-swift11:07
*** gvernik has joined #openstack-swift11:08
*** RockKuo has quit IRC11:14
*** jamie_h_ has quit IRC11:21
*** jamie_h has joined #openstack-swift11:21
*** saju_m has joined #openstack-swift11:24
*** mlipchuk has joined #openstack-swift11:26
*** mlipchuk1 has quit IRC11:26
*** matsuhashi has joined #openstack-swift11:26
*** saschpe has joined #openstack-swift11:32
*** foexle has quit IRC11:40
*** annegentle has quit IRC11:43
*** matsuhashi has quit IRC11:49
*** matsuhashi has joined #openstack-swift11:50
*** foexle has joined #openstack-swift11:52
*** matsuhas_ has joined #openstack-swift11:53
*** matsuhashi has quit IRC11:54
*** mmcardle has quit IRC11:57
*** nosnos has quit IRC11:59
*** dmsimard has joined #openstack-swift12:00
*** cheri has quit IRC12:03
*** ppai has quit IRC12:06
*** JuanManuelOlle has joined #openstack-swift12:10
*** mlipchuk1 has joined #openstack-swift12:13
*** mlipchuk has quit IRC12:15
*** mlipchuk1 has quit IRC12:18
*** mlipchuk has joined #openstack-swift12:18
*** mmcardle has joined #openstack-swift12:20
*** cheri has joined #openstack-swift12:21
*** ppai has joined #openstack-swift12:23
*** zigo has quit IRC12:26
*** tdasilva has left #openstack-swift12:37
*** cheri has quit IRC12:42
*** ppai has quit IRC12:43
*** cheri has joined #openstack-swift12:55
*** rustlebee is now known as russellb13:08
*** changbl has quit IRC13:13
*** RockKuo has joined #openstack-swift13:18
*** cheri has quit IRC13:21
*** matsuhas_ has quit IRC13:27
*** matsuhashi has joined #openstack-swift13:28
*** ashish_ has joined #openstack-swift13:28
*** matsuhas_ has joined #openstack-swift13:31
*** matsuhashi has quit IRC13:32
*** matsuhas_ has quit IRC13:34
*** tanee is now known as tanee-away13:42
*** tdasilva has joined #openstack-swift13:52
*** piyush has joined #openstack-swift13:54
*** chandan_kumar has quit IRC13:56
*** zackf has quit IRC13:59
*** dmsimard1 has joined #openstack-swift14:02
*** zackf1 has joined #openstack-swift14:03
*** dmsimard2 has joined #openstack-swift14:03
*** dmsimard has quit IRC14:04
*** dmsimard1 has quit IRC14:06
*** changbl has joined #openstack-swift14:11
*** zackf1 has quit IRC14:12
*** psharma has quit IRC14:17
*** zackf has joined #openstack-swift14:23
*** zackf has quit IRC14:26
*** bill_az has joined #openstack-swift14:32
*** changbl has quit IRC14:34
*** tellesnobrega has joined #openstack-swift14:45
tellesnobregahi, im starting to use sahara and it uses swift for storage, im trying to start a devstack installation with swift, but im getting the following error http://paste.openstack.org/show/75216/14:46
tellesnobregacan anyone give me a hint how to fix it14:46
tellesnobrega?14:46
cschwedetellesnobrega: try to set the tenant name, for example: export OS_TENANT_NAME=mytenantname14:48
tellesnobregacschwede: doesn't devstack set tenant names when installing?14:49
*** saju_m has quit IRC14:50
cschwedeafaik not automatically. you can set them using „source ~/devstack/openrc“ (given that ~/devstack is the cloned repo). have a look at http://devstack.org/openrc.html14:50
tellesnobregacschwede: i see. i will try that14:53
*** nshaikh has quit IRC14:54
*** zaitcev has joined #openstack-swift15:02
*** ChanServ sets mode: +v zaitcev15:02
*** bada has joined #openstack-swift15:09
*** nacim has quit IRC15:15
*** Guest_ has joined #openstack-swift15:16
*** cschwede has quit IRC15:16
*** cschwede has joined #openstack-swift15:17
*** gyee has joined #openstack-swift15:20
wermy delete performance is not that great. I am not sure where the bottleneck is.15:23
*** zackf has joined #openstack-swift15:24
*** mwstorer has joined #openstack-swift15:28
*** zul has quit IRC15:28
*** zul has joined #openstack-swift15:33
werlarge batches of deletes begin to take about 3 seconds to complete... and I get ERROR container update failed with 10.9.2.133:6001/d69  timeouts.  It seems limited to a 3 drives that timeout....  But they appear healthy.15:36
creihtwer: delete performance has a similar path to small object puts15:37
werhard on container performance?15:37
creihtthe delete's have to be sent to all replicas, and all containers have to be updated15:37
openstackgerritGreg Lange proposed a change to openstack/swift: Unify backend logging  https://review.openstack.org/8323715:38
creihtyeah the deletes taking about 3 seconds are waiting on the container update15:38
creihtmost likely15:38
werah! so the container is stored on those three drives probably?15:38
creihtjust like if you tried to put a bunch of 0-byte files, you would probably see similar behavior15:38
creihtyeah15:38
wershit.  So I probably should have sharded this.15:38
creihtheh15:38
werok that makes some sense at least.... hrm.15:39
werThe other thing I noted is that the proxy server doesn't log the deletes.  Which is driving me a bit batty.  It logs all the puts but I only get timing info for the deletes.15:40
*** dmsimard2 has quit IRC15:41
werty creiht.  That bottleneck was staring me right in the face :/15:42
*** dmsimard has joined #openstack-swift15:42
*** foexle has quit IRC15:43
werI don't have a lot of containers.  I have a lot of objects though.  I used the same ring layout for both.  So maybe a different layout for the containers is justified.  If I could say move them to where there is less io until I can fix this?15:44
notmynamegood morning world15:47
portantenotmyname: good morning15:50
portanteany swiftclient folks hanging around in here?15:50
portantegot a question about swiftclient and threads15:51
portanteor rather swiftclient with threads vs eventlet15:51
swifterdarrellportante: maybe kinda...15:51
notmynameI was disconnected all weekend (camping in a redwood forest). what did I miss?15:51
portantenotmyname: did you camp up in a tree at all?15:52
swifterdarrellportante: I heard swiftclient w/requests doesn't eventlet properly with SSL15:52
notmynamethat wood (sic) have been cool15:52
portanteugh15:52
swifterdarrelllol15:52
swifterdarrellportante: otoh, I haven't used the new v2+ swiftclient much15:52
notmynameportante: creiht is working on that, I think15:52
portantenotmyname: does that sound like a manager joke to anybody? ;)15:52
creihtportante: yeah I've been working on that15:52
notmynameportante: lol15:52
portante;)15:52
creihtso monkey patching socket works reasonably15:52
portantecreiht: so I am running into a problem with the in-process functional tests15:53
creihtbut swiftclient doesn't use a session, so it recreates connections every time15:53
portantecreiht: ugh15:53
portanteso that is what you are fixing?15:53
swifterdarrellcreiht: oh yeah, I saw a crapton of auth requests w/the new swiftclient but haven't had a chance to look into it15:53
creihtportante: well at this point I'm just trying to completly understand the problem15:53
swifterdarrellcreiht: would that be a symptom?  (was straight http for me, not ssl)15:54
portantecreiht: cool15:54
creihthttps://review.openstack.org/#/c/74444/15:54
creihtlunch time but will bbl15:54
swifterdarrellcreiht: i mean, WHO CAN TRUST THAT DIRTY AUTH TOKEN, ANYWAY?!15:54
creihtswifterdarrell: oh dunno15:55
portantecreiht: thanks15:55
swifterdarrellcreiht: aight, I'll keep an eye out for it, and if it's still around next time swiftclient hits my plate, I'll fix it15:55
creihtswifterdarrell: my main motivation right now is fixing swift-bench performance :)15:56
swifterdarrellcreiht: *nod*15:56
creihtok really going to lunch now :)15:56
portantecreiht, swifterdarrell: what I am trying to do with https://review.openstack.org/#/c/66108/ is allow the functional tests to run in-process when requested or no-external config found so that we can run coverage with them and debug problems easier15:57
swifterdarrellcreiht: authing every time instead of re-using a token seems like a good way to b0rk performance...15:57
swifterdarrellportante: what's the issue you're seeing there w/swiftclient?15:58
portantecreiht, swifterdarrell: what is happening is that the "testFileSizeLimit" functional test creates a separate thread for running the PUT method, where it has a content-length header, but sends no body15:58
portantein the presence of eventlet, that method call fails to actually send any headers, so that in-process server never responds15:59
portantewhat I want to do is change the functional tests to either:15:59
swifterdarrellportante: could that be running afoul of requests' lack of support for expect-100-continue?15:59
swifterdarrellportante: (nevermind, sounds different)16:00
portanteswifterdarrell: I don't think it is sending that header by default, but not sure16:00
portante1. always use eventlet for the timing of that method16:00
portante2. run the in-process servers in a separate thread so that the main thread can behave normally (monkey patching sockets needed)16:01
swifterdarrellportante: so the test used to run w/OS threads w/o eventlet monkeypatchy, but now there's OS threads + monkeypatchy and it doesn't work in that new state?16:01
portanteswifterdarrell: when it is run in the in-process mode of operation proposed by the above patch16:02
portanteand only that test, all other tests work as is16:02
portante#1 would mean a change to how the existing functional tests run, which may not be desirable, hence the question about swiftclient and eventlet vs threads16:03
portante#2 would mean more complicated mechanism for the in-process tests, which is what I am leaning towards, but looking for feedback16:04
swifterdarrellportante: is that the only test using OS threads? or are there other tests doing that which still work w/the in-process biz for some reason?16:04
portantethe only test16:04
swifterdarrellah16:04
*** changbl has joined #openstack-swift16:04
portantefor a while I failed to even notice the problem because of the way the in-process functional tests constructed their configuration, the test would be skipped16:05
portantewhich I thought was normal16:05
portantethe test avoids sending any data because we don't want to generate 5GB of data 6 times to test the boundary conditions around the max_file_size setting16:06
swifterdarrellportante: which functional test file is that test in?16:06
portantetest/functional/tests.py:TestFile.testFileSizeLimit16:06
portanteIt is actually run twice, with TestFileUTF8 as well16:07
portanteswifterdarrell: I've gotta grab lunch before it gets too late, will be back in a bit16:08
swifterdarrellportante: k16:08
*** Longgeek_ has joined #openstack-swift16:10
*** saju_m has joined #openstack-swift16:12
*** tellesnobrega has left #openstack-swift16:15
swifterdarrellportante: you might try turning that "t.join(seconds)" call in timeout() into a loop whose runtime is bounded by the "seconds" value, but in which a smaller (0.1s?) value is passed into t.join() and also eventlet.sleep() is called to trampoline16:15
*** chandan_kumar has joined #openstack-swift16:16
swifterdarrellportante: I'm guessing that t.join(seconds) is causing the main thread to starve the eventlet hub, making your in-proc servers not read what the child thread's writing16:16
swifterdarrellportante: that seems like a pretty safe change that might clear up the problem.16:16
*** RockKuo has quit IRC16:19
zackmdavishttp://zackmdavis.net/blog/2014/04/ode-to-swift/16:19
*** ashish_ has quit IRC16:20
*** dmsimard1 has joined #openstack-swift16:25
*** Guest_ has quit IRC16:25
*** Guest_ has joined #openstack-swift16:25
*** dmsimard has quit IRC16:26
*** zigo has joined #openstack-swift16:28
*** zigo has quit IRC16:33
*** madhuri_ has joined #openstack-swift16:33
*** Longgeek_ has quit IRC16:34
*** ashish_ has joined #openstack-swift16:34
*** zigo has joined #openstack-swift16:34
*** tanee-away is now known as tanee16:35
*** saju_m has quit IRC16:35
*** Guest_ has quit IRC16:36
*** Guest_ has joined #openstack-swift16:36
portanteswifterdarrell: I thought a t.join() would block the main thread, allowing the created thread running the method to proceed16:40
swifterdarrellportante: do your in-proc servers run int he main thread?16:40
portanteyes16:41
*** zigo has quit IRC16:41
portanteI believe so16:41
swifterdarrellportante: then how will they do anything while the main thread is sitting in a t.join(seconds)?16:41
swifterdarrellportante: ...like read their sockets16:41
portanteI think that is what is happenning16:41
swifterdarrellportante: I think you need to simultaneously await that thread /and/ service eventlet in that main thread16:42
swifterdarrellportante: inside the definition of timeout()16:42
swifterdarrellportante: timeout() may just need a tweak to be "eventlet-friendly" while still successfully doing his job16:42
swifterdarrellportante: that's my hypothesis, anyway16:42
portanteswifterdarrell: yes, that is what I was thinking of doing with #1 above16:43
swifterdarrellportante: I read your #1 as getting rid of the OS thread16:44
portantethe other way to consider this is to create one thread that represents the in-process servers, where it is running an eventlet hub there, and the main thread and other threads can use the normal swiftclient interfaces with threads, no code changes to timeout()16:45
portantethat is what I meant by #216:45
*** madhuri_ has quit IRC16:45
portanteit sounds easier the way you describe #1, so I might try that first16:45
portanteswifterdarrell: :)16:45
*** zigo has joined #openstack-swift16:46
*** shri has joined #openstack-swift16:46
swifterdarrellportante: hrm, the only part of swiftclient which uses OS threads is bin/swift, right?16:46
portanteI am hoping, but I have not investigated it that deeply to be sure16:47
swifterdarrellportante: iirc, the python libarary part was agnostic... it was "eventlet safe" but could be used w/OS threads (a la bin/swift) or eventlet (a la swift-bench)16:47
portantethat is how I understood it16:47
swifterdarrellportante: i think just allowing timeout() to service an eventlet hub instead of going into a long/full t.join() will fix the prob w/little downside16:48
swifterdarrellportante: you shoudl probably use wall-time to exit the loop though16:48
portanteswifterdarrell: thanks, I'll try that and see how it works16:48
*** mmcardle has quit IRC16:48
portanteswifterdarrell: agreed regarding wall-time16:48
swifterdarrellportante: since with eventlet, your t.join() and evenetlet.sleep() will probably both eat wall time16:48
swifterdarrellportante: and w/o eventlet active, I think the eventlet.sleep() may NOP16:49
swifterdarrellportante: but giving t.join() and eventlet.sleep() each a chance to eat 0.1s in a wall-time-bounded-loop should be more than sufficient precision for this style of testing.16:49
portanteswifterdarrell: agreed16:50
swifterdarrellportante: but I wouldn't go so far as to remove the OS threading entirely (which is how I read your #1)16:50
portanteI was thinking of doing just that because testFileSizeLimit is the only test that uses it, and it *should* not affect the test16:51
*** Guest_ has quit IRC16:52
swifterdarrellportante: I guess eventlet is already a pre-req of running the functional tests?  could one theoretically run just the func tests on a server pointed at a swift cluster w/o eventlet installed and have it work?16:52
*** Guest_ has joined #openstack-swift16:52
portanteso usually, you do a python setup.py develop to get the test environment setup properly16:53
portanteso in that case, one would have eventlet16:53
portanteI am not sure if one could take a plain, non-eventlet install system and run the functional tests only16:54
portanteI know that eventlet is required for running the unit tests16:54
swifterdarrellportante: *nod*16:54
portantetest/unit/proxy/test_server.py requires it for the same reason16:54
swifterdarrellportante: ya, functional tests would be the only ones able to maybe possibly escape a req. on eventlet because they could talk to Swift on a different host (unlike the unit tests)16:55
swifterdarrellportante: and even if that were the case, adding eventlet as a req for running func tests seems ok16:55
swifterdarrellportante: so i'm indifferent to a full eventlet replacement or a eventlet-threading play-nice loop16:56
portanteI'll see how it works out using eventlet with that testFileSizeLimit test and propose that as a separate independent patch for review and consideration16:56
swifterdarrellportante: slight pref for the play-nice, but it's probably more overall complexity for the dubious gain of less-code-change16:56
swifterdarrellportante: k16:56
portantethanks16:57
*** Guest_ has quit IRC16:58
*** Guest_ has joined #openstack-swift16:58
*** Guest_ has quit IRC16:59
*** Guest_ has joined #openstack-swift16:59
*** tanee is now known as tanee-away17:01
*** piyush has quit IRC17:03
openstackgerritPeter Portante proposed a change to openstack/swift: Move module level func to unit test that uses it  https://review.openstack.org/8575717:07
*** ashish_ has quit IRC17:13
*** ashish_ has joined #openstack-swift17:14
openstackgerritPeter Portante proposed a change to openstack/swift: Move module level funcs to unit tests that use 'em  https://review.openstack.org/8575717:18
*** piousbox has joined #openstack-swift17:23
*** d89 has quit IRC17:25
*** saschpe has quit IRC17:34
notmynameseagate announced 6TB drives today17:35
openstackgerritJenkins proposed a change to openstack/swift: Updated from global requirements  https://review.openstack.org/8120717:38
*** saschpe has joined #openstack-swift17:41
*** Guest_ has quit IRC17:42
*** Guest_ has joined #openstack-swift17:43
*** dmsimard has joined #openstack-swift17:46
*** dmsimard1 has quit IRC17:46
*** dmsimard1 has joined #openstack-swift17:47
creihtzackmdavis: nice! :)17:47
*** piyush has joined #openstack-swift17:49
*** Guest_ has quit IRC17:49
*** mmcardle has joined #openstack-swift17:49
*** dmsimard has quit IRC17:50
*** piyush1 has joined #openstack-swift17:50
*** piyush has quit IRC17:53
*** mmcardle has quit IRC17:54
*** ashish_ has quit IRC17:54
*** mlipchuk has quit IRC17:55
*** ashish_ has joined #openstack-swift17:55
*** cheri has joined #openstack-swift17:57
*** annegentle_ has joined #openstack-swift17:58
ashish_Hey everyone.Could anyone tell me where in the ssbench is the scenrio directory located.17:58
notmynameashish_: I think swifterdarrell is on the bus right now. let me talk a look18:05
swifterdarrellashish_: I usually "sudo updatedb; locate scenarios"18:10
notmynameashish_: do you mean in the codebase or looking for some other directory somewhere18:10
notmynameswifterdarrell: :-)18:10
swifterdarrellashish_: notmyname: that's not even a joke response, that's literally what I do and I wrote ssbench18:11
*** jamie_h has quit IRC18:11
claygglange: you still poking at unify backend logging (saw the rebase) - if not I'll go ahead and stick my +2 back on it18:15
glangeclayg: yeah, I'm going to do 2 of the 3 things that notmyname wanted18:17
portanteclayg: seems like notmyname had a few comments that could go in future patches?18:17
portantewould be nice to get those changes in18:17
glangeclayg: update the docs and put a '-' in no additional information18:17
claygglange: yeah both of those are bonus - thanks!18:17
notmynameglange: thanks!18:18
notmynameI was talking to clayg about a complete overhaul of the storage node log format, but he wisely said that the time to do that is when we (re)address some of the proxy server logging, and thus add some sort of toggle, and then unify everything so it's all the same log format18:19
claygwhere wise ~= lazy18:19
portanteah grasshoppah18:19
notmyname:-)18:19
portanteboston accent applied18:20
claygpeluse: is 83840 ready to go?18:21
claygpatchbot: 8384018:21
patchbotclayg: Error: "83840" is not a valid command.18:21
claygaccahpachaphcpahc18:21
claygpatchbot: p 8384018:21
patchbotclayg: https://review.openstack.org/#/c/83840/18:21
claygnotmyname: patchbot should include the subject18:21
notmynameclayg: https://github.com/notmyname/Patches/pulls18:21
openstackgerritPeter Portante proposed a change to openstack/swift: Use eventlet instead of threading for timeout  https://review.openstack.org/8578218:26
portanteswifterdarrell: ^^^18:26
portantegotta head out for a bit, back in a couple to tree hours18:26
swifterdarrellportante: lgtmbidari (looks good to be but i didn't actually run it)18:27
*** dmsimard1 has quit IRC18:28
*** dmsimard has joined #openstack-swift18:29
openstackgerritGreg Lange proposed a change to openstack/swift: Unify backend logging  https://review.openstack.org/8323718:30
*** judd7 has joined #openstack-swift18:30
*** patchbot has quit IRC18:32
*** patchbot has joined #openstack-swift18:32
ashish_swiftdarrel thanks could you also tell me what is the process of  creation of a JSOn file in ssbench18:33
swifterdarrellashish_: sure, the "Scenarios" section at https://pypi.python.org/pypi/ssbench/0.2.23 has all the details of what can be defined in a scenario file and their semantics18:34
claygyuan: i was poking at 82515 and the unittest still seems to pass even w/o the new code?18:34
notmynameglange: you didn't add additional_info to the end of line 102 in logs.rst18:35
swifterdarrellashish_: note that many of the things defined in teh scenario file may also be overridden on the command-line of ssbench-master18:35
*** csd has joined #openstack-swift18:35
glangenotmyname: ok, I'll fix that18:35
notmynameglange: thanks. otherwise, +218:35
openstackgerritGreg Lange proposed a change to openstack/swift: Unify backend logging  https://review.openstack.org/8323718:37
glangethat was just a test of the swift review process, if that had been an actual mistake it would have been harder to spot :)18:38
claygglange: olo18:38
ashish_swiftdarrel  I am now able to locate my scenarios directory.Thanks18:38
ashish_I also want to know can I create my own JSON file.18:39
ashish_swiftdarrel I also want to know can I create my own JSON file.18:39
*** dmsimard_ has joined #openstack-swift18:43
*** cheri has quit IRC18:43
*** jamie_h has joined #openstack-swift18:49
*** mmcardle has joined #openstack-swift18:50
*** saju_m has joined #openstack-swift18:51
*** patchbot has quit IRC18:51
swifterdarrellashish_: any text editor should do, just make sure it's valid JSON18:52
swifterdarrellashish_: sometimes I like to leave trailing commas in lists and dictionaries but JSON doesn't like that18:52
*** patchbot has joined #openstack-swift18:52
ashish_Thanks swiftdarrel.Could you tell me the various parameters on the basis of which I can carry my test cases.18:54
*** mmcardle has quit IRC18:54
ashish_swiftdarrel.one parameter I have chosen is to test ssbench performance when I change the number of replication.Could you tell me other such factors.18:55
notmynameashish_: there are a lot of factors that can affect performance in swift. most of them should be orthogonal to the replica count. ie choose the replica count based on your durability and dispersion requirements, not performance18:57
*** Guest_ has joined #openstack-swift18:58
notmynameashish_: you shoudl read through http://docs.openstack.org/developer/swift/deployment_guide.html and http://docs.openstack.org/developer/swift/admin_guide.html for info on some general things that you can tune and how to monitor it18:58
ashish_notmyname Thanks.But my changing the replicas i want to see the change in the iops.18:59
notmynameashish_: you want to confirm that 3 replicas has 50% more disk requirements than 2 replicas?19:00
ashish_notmyname What I want to see is how the number of replicas affect my CRUD profile.19:02
peluseclayg:  83840, yes19:04
claygpeluse: sweet - I'll take a look19:04
*** Guest_ has quit IRC19:09
*** csd has quit IRC19:10
*** joeljwright2 has joined #openstack-swift19:13
claygpeluse: seems on the right track, don't like how the policy_stat seed happens on the container table migration instead of the policy_stat table creation...  hrm.... they're in the same method - maybe you can't get half upgraded there...19:13
peluseclayg:  I could move that SQL udpate into create_policy_stat table I guess19:16
peluseSQL statement I mean :)19:16
claygpeluse: i'm trying one other thing too19:16
peluseclayg:  K, BRB19:17
*** mmcardle has joined #openstack-swift19:26
*** dmsimard has quit IRC19:29
*** dmsimard_ is now known as dmsimard19:29
*** dmsimard1 has joined #openstack-swift19:30
notmynameopenssl security bug announced today. seems that ubuntu precise is affected and doesn't yet have a new package available http://www.openssl.org/news/secadv_20140407.txt19:32
*** dmsimard has quit IRC19:36
*** dmsimard1 is now known as dmsimard19:37
*** dmsimard has quit IRC19:38
*** dmsimard has joined #openstack-swift19:38
joeljwright2Hoping for a bit of help with the gerrit workflow: I submitted a patch to python-swiftclient on Friday last week, but it has failed the check-grenade-dsvm test. It seems to fail when running bin/swift because it fails to import the concurrent.futures module, but futures>=2.1.3 is specified in requirements.txt, so I'm at a bit of a loss.19:39
joeljwright2https://review.openstack.org/8545319:39
notmynamejoeljwright2: futures isn't in the swiftclient [test-]requirements.txt19:42
*** jamie_h has quit IRC19:43
notmynamejoeljwright2: you're saying that it's in the global requirements file?19:44
joeljwright2yes19:44
joeljwright2it's added to requirements.txt alongside requests, simplejson and six19:44
notmynamejoeljwright2: so while I'm a little surprised that it fails (I figured it would have been imported other places in devstack), I'm glad it does. looks like your patch needs to add it to requirements.txt19:45
notmynamejoeljwright2: ah. and I see you already have :-)19:45
joeljwright2:)19:45
* notmyname is unfamiliar with futures19:46
notmynamejoeljwright2: does futures provide the concurrent namespace?19:46
notmynamehmm19:47
joeljwright2notmyname: yes it should do - it's a backport of concurrent.futures from python >=3.219:47
notmynamewfm on my SAIO https://gist.github.com/notmyname/679a08aa0515d0c1552719:47
notmynamejoeljwright2: so that would seem that the problem is with getting it installed19:47
notmynameclarkb: maybe you can help make some searching shorter. do you know why a package added to requirements.txt in a patch would fail grenade?19:50
clarkbnotmyname: if there is a non backward compatible path from old version to new version with that requirement I think that may happen19:50
clarkbold version -> new version being of the running openstack cloud19:50
notmynameclarkb: no, this is a net new in this patch19:51
clarkbin that case no I don't know why that may happen19:51
*** csd has joined #openstack-swift19:54
*** Longgeek_ has joined #openstack-swift19:56
peluseclayg:  I'm back, let me know if you want me to udpate 83840 to move the policy_table population w/legacy data to the table creation routine...19:57
*** Longgeek_ has quit IRC19:57
*** Longgee__ has joined #openstack-swift19:57
*** Longgee__ has quit IRC19:58
*** Longgeek_ has joined #openstack-swift19:58
*** Longgeek_ has quit IRC19:59
*** Longgeek_ has joined #openstack-swift19:59
*** Longgeek_ has quit IRC20:00
*** judd7 has quit IRC20:01
*** dfg_ has joined #openstack-swift20:05
*** Anticime1 has joined #openstack-swift20:06
*** ashish_ has quit IRC20:06
*** mhu1 has joined #openstack-swift20:09
*** wayneseguin has joined #openstack-swift20:09
*** russellb_ has joined #openstack-swift20:09
*** fbo has joined #openstack-swift20:09
*** wayneeseguin has quit IRC20:10
*** russellb has quit IRC20:10
*** leews has quit IRC20:10
*** dfg has quit IRC20:10
*** fbo_away has quit IRC20:10
*** Anticimex has quit IRC20:10
*** mhu has quit IRC20:10
*** russellb_ is now known as russellb20:10
*** mhu1 is now known as mhu20:10
*** wayneseguin is now known as wayneeseguin20:10
*** Longgeek_ has joined #openstack-swift20:12
*** lpabon has joined #openstack-swift20:13
*** Longgeek_ has quit IRC20:14
*** mmcardle has quit IRC20:15
*** Longgeek_ has joined #openstack-swift20:16
*** Longgeek_ has quit IRC20:17
*** Longgeek_ has joined #openstack-swift20:17
*** Longgeek_ has quit IRC20:18
*** Longgeek_ has joined #openstack-swift20:18
*** Longgeek_ has quit IRC20:21
*** Longgeek_ has joined #openstack-swift20:23
*** Longgeek_ has quit IRC20:23
*** Longgeek_ has joined #openstack-swift20:24
*** Longgeek_ has quit IRC20:24
*** Longgeek_ has joined #openstack-swift20:25
*** Longgeek_ has quit IRC20:25
*** Longgeek_ has joined #openstack-swift20:26
*** Longgeek_ has quit IRC20:26
*** occupant has joined #openstack-swift20:27
*** lpabon has quit IRC20:28
*** joeljwright2 has quit IRC20:28
*** csd has quit IRC20:28
*** joeljwright has joined #openstack-swift20:33
*** saju_m has quit IRC20:41
*** Longgeek_ has joined #openstack-swift20:42
*** Longgeek_ has quit IRC20:43
*** Longgeek_ has joined #openstack-swift20:44
*** Longgee__ has joined #openstack-swift20:44
*** tdasilva has left #openstack-swift20:45
*** Longgee__ has quit IRC20:45
*** Longgee__ has joined #openstack-swift20:45
*** Longgee__ has quit IRC20:46
*** ChanServ sets mode: +v dfg_20:47
*** dfg_ is now known as dfg20:47
*** Longgeek_ has quit IRC20:48
*** csd has joined #openstack-swift20:49
*** Longgeek_ has joined #openstack-swift20:50
*** Longgeek_ has quit IRC20:50
*** Longgeek_ has joined #openstack-swift20:51
*** Longgeek_ has quit IRC20:51
*** Longgeek_ has joined #openstack-swift20:52
*** Longgeek_ has quit IRC20:52
*** Longgeek_ has joined #openstack-swift20:53
*** Longgeek_ has quit IRC20:53
*** Longgeek_ has joined #openstack-swift20:55
*** Anticime1 is now known as Anticimex20:55
*** Longgeek_ has quit IRC20:56
*** Longgee__ has joined #openstack-swift20:56
*** Longgee__ has quit IRC20:56
*** Longgeek_ has joined #openstack-swift20:57
*** Longgeek_ has quit IRC20:57
*** Longgeek_ has joined #openstack-swift20:58
*** Longgeek_ has quit IRC20:59
*** Longgeek_ has joined #openstack-swift21:01
*** Longgeek_ has quit IRC21:01
*** Longgeek_ has joined #openstack-swift21:02
*** Longgeek_ has quit IRC21:02
*** Longgeek_ has joined #openstack-swift21:03
*** JuanManuelOlle has quit IRC21:03
*** Longgeek_ has quit IRC21:03
*** Longgeek_ has joined #openstack-swift21:04
*** Longgeek_ has quit IRC21:05
*** joeljwright has quit IRC21:10
*** Longgeek_ has joined #openstack-swift21:11
*** Longgeek_ has quit IRC21:12
*** Longgeek_ has joined #openstack-swift21:12
*** Longgeek_ has quit IRC21:13
*** Longgeek_ has joined #openstack-swift21:13
openstackgerritpaul luse proposed a change to openstack/swift: Add Storage Policy Documentation  https://review.openstack.org/8582421:13
*** Longgeek_ has quit IRC21:14
claygpeluse: nice21:18
*** mmcardle has joined #openstack-swift21:19
*** Longgeek_ has joined #openstack-swift21:21
*** Longgee__ has joined #openstack-swift21:22
*** Longgeek_ has quit IRC21:22
*** Longgeek_ has joined #openstack-swift21:23
*** mmcardle has quit IRC21:24
*** Longgeek_ has quit IRC21:24
*** Longgee__ has quit IRC21:27
peluseclayg:  just a start :)21:39
openstackgerritClay Gerrard proposed a change to openstack/swift: Fix issue with account broker and pre-SPI databases  https://review.openstack.org/8384021:39
*** joeljwright has joined #openstack-swift21:40
peluse^ way cool clayg21:41
claygi thought it turned out ok - can you push it through the paces and I'll get notmyname to merge it?21:42
claygpeluse: when you get a chance21:42
claygpeluse: I think i also got him to commit to looking at the policy validation patch and merging master back onto feature/ec21:42
*** Longgeek_ has joined #openstack-swift21:42
notmynamedoing it now21:42
*** joeljwright1 has joined #openstack-swift21:42
notmynameLonggeek_: you keep leaving and joining (and filling the channel with connect/disconnect messages)21:42
*** Longgeek_ has quit IRC21:43
peluseclayg:  yup, I'll do a little more manual functinoal testing real quick and make sure I can't break it21:43
claygpeluse: even better if you can - then we'll fix THAT and know we have one less way to wreck things!21:44
*** joeljwright has quit IRC21:45
pelusewell, we could add a delete at the end of the unit test and assert correct rollups in policy_stat... about to check that manually21:45
*** Longgeek_ has joined #openstack-swift21:46
*** Longgeek_ has quit IRC21:46
*** Longgeek_ has joined #openstack-swift21:47
claygpeluse: oh yeah that'd be nice21:47
*** Longgeek_ has quit IRC21:47
*** Longgeek_ has joined #openstack-swift21:48
claygpeluse: I was also thinking the policy_stat's trigger should go ahead and count containers - at least I *think* that would be useful right?21:48
peluseI asked torgomatic about that a while back and he didn't think we should include it right now21:49
*** Longgee__ has joined #openstack-swift21:49
pelusebut he's at home watching cartoons so its up to you I guess :)21:49
claygpeluse: maybe?  i think it'll be harder to add later...21:49
*** piyush1 has left #openstack-swift21:49
clayglol21:49
* peluse wonders if torgomatic is lurking...21:49
*** Longgee__ has quit IRC21:49
*** joeljwright1 has quit IRC21:50
*** Longgeek_ has quit IRC21:53
*** changbl has quit IRC21:54
*** Longgeek_ has joined #openstack-swift22:03
*** Longgeek_ has quit IRC22:04
*** ChanServ sets mode: -b godb!*@*22:07
*** joeljwright has joined #openstack-swift22:09
*** Longgeek_ has joined #openstack-swift22:14
*** joeljwright has quit IRC22:14
*** ChanServ sets mode: +b Longgeek_!*@*22:14
*** Longgeek_ was kicked by ChanServ (Banned: continual join/leave spam !T 18h)22:14
*** mmcardle has joined #openstack-swift22:15
*** ChanServ sets mode: -b Longgeek_!*@*22:15
*** mmcardle has quit IRC22:17
*** mmcardle1 has joined #openstack-swift22:17
*** mmcardle1 has quit IRC22:21
openstackgerritpaul luse proposed a change to openstack/swift: Fix issue with account broker and pre-SPI databases  https://review.openstack.org/8384022:23
*** Longgeek_ has joined #openstack-swift22:26
*** ChanServ sets mode: +b Longgeek_!*@*22:26
*** Longgeek_ was kicked by ChanServ (Banned: continual join/leave spam)22:26
peluseclayg:  will be in transit for a bit.  wrt adding container count, say the word and I'll add it in.  Just added the extra delete test with the above patch change22:26
claygpeluse: we should track per policy container_count, and also include storage_policy_index in the container listings - but we an merge this as is I think...22:31
*** Longgee__ has joined #openstack-swift22:34
*** zhiyan_ has quit IRC22:34
*** Longgee__ has quit IRC22:35
*** zhiyan_ has joined #openstack-swift22:35
*** early has quit IRC22:42
*** Trixboxer has quit IRC22:43
*** early has joined #openstack-swift22:46
*** saschpe has quit IRC23:00
*** joeljwright has joined #openstack-swift23:10
*** mkollaro has quit IRC23:14
*** joeljwright has quit IRC23:14
*** mmcardle has joined #openstack-swift23:18
*** gyee has quit IRC23:21
*** mmcardle has quit IRC23:22
openstackgerritPeter Portante proposed a change to openstack/swift: Use eventlet instead of threading for timeout  https://review.openstack.org/8578223:28
portanteswifterdarrell: ^^^ the above uses eventlet instead of threads for that one test timeout, and it seems like it is a bit less code.23:29
notmynamethat sound you hear is all the sysadmins updating their boxes right now (BTW canonical pushed a patch for precise, so go update things)23:38
*** joearnold has quit IRC23:40
*** mlanner has quit IRC23:40
*** otherjon has quit IRC23:40
*** swifterdarrell has quit IRC23:40
*** alpha_ori has quit IRC23:40
*** minnear has quit IRC23:40
*** hugokuo has quit IRC23:40
*** ctennis has quit IRC23:40
*** anderstj has quit IRC23:40
*** amandap has quit IRC23:40
*** acorwin has quit IRC23:40
*** zackmdavis has quit IRC23:40
*** acorwin has joined #openstack-swift23:43
*** minnear has joined #openstack-swift23:43
*** ctennis has joined #openstack-swift23:43
*** swifterdarrell has joined #openstack-swift23:45
*** ChanServ sets mode: +v swifterdarrell23:45
*** hugokuo has joined #openstack-swift23:47
*** joearnold has joined #openstack-swift23:47
*** mlanner has joined #openstack-swift23:48
*** alpha_ori has joined #openstack-swift23:50
*** otherjon has joined #openstack-swift23:50
*** zackmdavis has joined #openstack-swift23:51
*** amandap has joined #openstack-swift23:52
*** anderstj has joined #openstack-swift23:53
*** gyee has joined #openstack-swift23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!