Thursday, 2014-11-20

peluseOK, thanks.  I'll shoot him a note00:00
notmynamepeluse: http://web.archive.org/web/20121231002445/http://greg.brim.net/page/building_a_consistent_hashing_ring.html00:01
notmynamebecause nothing is every really deleted from the internet00:01
peluse:)  thanks!00:01
*** oomichi_ has joined #openstack-swift00:09
*** sungju has quit IRC00:21
*** dmorita has joined #openstack-swift00:32
*** tkay has left #openstack-swift00:36
*** nomad411 has joined #openstack-swift00:38
*** zul has quit IRC00:44
*** jasondotstar has quit IRC00:48
*** rmcall has quit IRC00:50
*** sungju has joined #openstack-swift01:03
*** gyee has quit IRC01:18
openstackgerritNicolas Trangez proposed openstack/swift: Rework `splice` and `tee`  https://review.openstack.org/13531901:28
*** tamizh_geek has joined #openstack-swift01:28
openstackgerritNicolas Trangez proposed openstack/swift: Rework `splice` and `tee`  https://review.openstack.org/13531901:31
*** NM has joined #openstack-swift01:35
*** jasondotstar has joined #openstack-swift01:44
*** haomaiwang has quit IRC01:47
*** NM has quit IRC01:50
*** nosnos has joined #openstack-swift01:59
*** NM has joined #openstack-swift02:06
*** haomaiwang has joined #openstack-swift02:07
openstackgerritEohyung Lee proposed openstack/swift: fix example config typo  https://review.openstack.org/13581802:14
*** dmsimard_away is now known as dmsimard02:18
*** NM has quit IRC02:21
*** dmsimard is now known as dmsimard_away02:29
*** X019 has quit IRC02:33
openstackgerritEohyung Lee proposed openstack/swift: fix example typo  https://review.openstack.org/13582402:39
*** oomichi_ has quit IRC02:41
*** X019 has joined #openstack-swift02:46
*** atan8 has quit IRC03:03
*** rmcall has joined #openstack-swift03:04
*** tellesnobrega has joined #openstack-swift03:05
*** HenryG has quit IRC03:15
*** HenryG has joined #openstack-swift03:18
*** nosnos has quit IRC03:24
*** tellesnobrega has quit IRC03:45
*** tamizh_geek has quit IRC03:46
*** nosnos has joined #openstack-swift03:49
*** lpabon has quit IRC04:08
*** nomad411 has quit IRC04:16
*** tellesnobrega has joined #openstack-swift04:20
*** sandywalsh has quit IRC04:22
*** sandywalsh has joined #openstack-swift04:23
*** sungju has quit IRC04:33
*** tkay has joined #openstack-swift04:43
*** tkay has left #openstack-swift04:44
*** abhirc_ has joined #openstack-swift05:12
*** tellesnobrega has quit IRC05:12
*** abhirc has quit IRC05:12
*** zaitcev has quit IRC05:32
*** ppai has joined #openstack-swift05:48
*** addnull has joined #openstack-swift05:48
*** addnull has quit IRC05:51
*** addnull has joined #openstack-swift05:51
*** addnull has left #openstack-swift05:53
*** bkopilov has joined #openstack-swift05:53
*** addnull has joined #openstack-swift05:59
*** addnull has quit IRC06:03
*** addnull has joined #openstack-swift06:04
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Imported Translations from Transifex  https://review.openstack.org/13585606:11
*** bkopilov has quit IRC06:12
*** haomaiwang has quit IRC06:17
*** oomichi has joined #openstack-swift06:22
fifieldt_notmyname, patch is here at https://review.openstack.org/#/c/135856/06:31
*** k4n0 has joined #openstack-swift06:33
openstackgerritTom Fifield proposed openstack/swift: Imported Translations from Transifex  https://review.openstack.org/13585606:35
*** oomichi__ has joined #openstack-swift07:01
*** oomichi has quit IRC07:02
*** rmcall has quit IRC07:05
*** occupant has quit IRC07:10
*** occupant has joined #openstack-swift07:11
*** bkopilov has joined #openstack-swift07:12
*** bkopilov has quit IRC07:13
*** bkopilov has joined #openstack-swift07:42
*** addnull has quit IRC07:44
*** addnull has joined #openstack-swift07:45
*** SkyRocknRoll has joined #openstack-swift07:49
*** TaiSHi has quit IRC08:01
*** TaiSHi has joined #openstack-swift08:02
*** oomichi_ has joined #openstack-swift08:12
*** oomichi__ has quit IRC08:12
*** oomichi_ has quit IRC08:14
*** addnull has quit IRC08:17
*** addnull has joined #openstack-swift08:23
*** geaaru has joined #openstack-swift08:30
*** dmsimard_away is now known as dmsimard08:57
*** jordanP has joined #openstack-swift08:58
*** acoles_away is now known as acoles09:02
*** NM has joined #openstack-swift09:10
*** dmsimard is now known as dmsimard_away09:20
*** oomichi has joined #openstack-swift09:27
*** foexle has joined #openstack-swift09:32
*** addnull has quit IRC09:37
*** mmcardle has joined #openstack-swift09:41
*** ppai has quit IRC09:51
*** ppai has joined #openstack-swift09:54
*** NM has quit IRC10:00
*** aix has joined #openstack-swift10:08
*** joeljwright has joined #openstack-swift10:09
*** dmsimard_away is now known as dmsimard10:19
*** jordanP has quit IRC10:31
*** nellysmitt has joined #openstack-swift10:36
*** addnull has joined #openstack-swift10:37
*** tamizh_geek has joined #openstack-swift10:38
*** jordanP has joined #openstack-swift10:39
*** addnull has quit IRC10:42
*** tamizh_geek has quit IRC10:45
*** ppai has quit IRC10:47
*** ppai has joined #openstack-swift11:01
*** tamizh_geek has joined #openstack-swift11:03
*** aix has quit IRC11:11
openstackgerritMerged openstack/swift: Imported Translations from Transifex  https://review.openstack.org/13585611:15
*** addnull has joined #openstack-swift11:17
*** addnull has quit IRC11:17
*** addnull has joined #openstack-swift11:18
*** addnull has quit IRC11:34
*** flwang has quit IRC11:36
*** tamizh_g_ has joined #openstack-swift11:37
*** tamizh_geek has quit IRC11:37
*** aix has joined #openstack-swift11:42
*** flwang has joined #openstack-swift11:43
*** geaaru has quit IRC11:53
*** mahatic has joined #openstack-swift12:02
*** geaaru has joined #openstack-swift12:06
*** dmorita has quit IRC12:23
*** mmcardle has quit IRC12:23
*** joeljwright has quit IRC12:24
*** mmcardle has joined #openstack-swift12:25
*** oomichi has quit IRC12:31
*** SkyRocknRoll has quit IRC12:34
*** mmcardle has joined #openstack-swift12:46
*** tamizh_g_ has quit IRC12:50
*** tdasilva has joined #openstack-swift12:52
*** tamizh_geek has joined #openstack-swift12:57
*** tamizh_geek has quit IRC12:57
*** haomaiwa_ has joined #openstack-swift13:00
*** lpabon has joined #openstack-swift13:04
*** nosnos has quit IRC13:05
*** tamizh_geek has joined #openstack-swift13:05
*** nosnos has joined #openstack-swift13:06
*** X019 has quit IRC13:14
*** nellysmitt has quit IRC13:16
*** mmcardle has joined #openstack-swift13:16
*** nellysmitt has joined #openstack-swift13:17
*** joeljwright has joined #openstack-swift13:18
*** mahatic has quit IRC13:22
*** mmcardle1 has joined #openstack-swift13:27
*** mmcardle has quit IRC13:27
*** nellysmitt has quit IRC13:28
*** nellysmitt has joined #openstack-swift13:32
*** k4n0 has quit IRC13:38
*** bkopilov has quit IRC13:41
*** nellysmitt has quit IRC13:43
*** nellysmitt has joined #openstack-swift13:46
*** miqui has joined #openstack-swift13:46
*** ppai has quit IRC13:47
*** tdasilva has quit IRC13:48
*** miqui_ has joined #openstack-swift13:50
*** miqui__ has joined #openstack-swift13:52
*** miqui___ has joined #openstack-swift13:54
*** miqui____ has joined #openstack-swift13:55
*** miqui_____ has joined #openstack-swift13:55
*** miqui___ has quit IRC13:56
*** miqui___ has joined #openstack-swift13:57
*** ppai has joined #openstack-swift13:57
*** miqui__ has quit IRC13:58
*** miqui has quit IRC13:58
*** miqui____ has quit IRC13:58
*** miqui_ has quit IRC13:58
*** miqui___ has quit IRC13:58
*** miqui has joined #openstack-swift13:59
*** ppai has quit IRC14:00
*** mahatic has joined #openstack-swift14:01
*** jistr has joined #openstack-swift14:02
*** jistr is now known as jistr|mtg14:04
*** nellysmitt has quit IRC14:06
*** mkollaro has joined #openstack-swift14:11
*** zul has joined #openstack-swift14:11
*** joseff has joined #openstack-swift14:13
joseffhey all14:13
*** miqui has quit IRC14:14
joseffwhats the method to install swiftly again?14:14
*** miqui has joined #openstack-swift14:14
*** tamizh_geek has quit IRC14:14
joseffgot it actually14:15
*** miqui_ has joined #openstack-swift14:15
*** miqui_ has quit IRC14:17
*** nellysmitt has joined #openstack-swift14:17
*** nosnos has quit IRC14:34
openstackgerritDaniel Wakefield proposed openstack/python-swiftclient: Fix misnamed dictionary key.  https://review.openstack.org/12957414:44
openstackgerritDaniel Wakefield proposed openstack/python-swiftclient: Capture test output better  https://review.openstack.org/13123814:44
*** tdasilva has joined #openstack-swift14:51
*** judd7 has joined #openstack-swift14:56
*** miqui has quit IRC15:00
*** miqui has joined #openstack-swift15:05
*** nellysmitt has quit IRC15:17
*** miqui_____ has quit IRC15:18
*** bkopilov has joined #openstack-swift15:21
*** miqui_ has joined #openstack-swift15:22
*** miqui has quit IRC15:25
*** atan8 has joined #openstack-swift15:28
*** miqui has joined #openstack-swift15:31
*** miqui has quit IRC15:32
*** miqui has joined #openstack-swift15:34
*** miqui has quit IRC15:35
*** atan8 has quit IRC15:36
*** X019 has joined #openstack-swift15:37
*** zul has quit IRC15:43
*** Anticimex has quit IRC15:44
*** zul has joined #openstack-swift15:45
*** jordanP has quit IRC15:48
*** Anticimex has joined #openstack-swift15:51
*** tamizh_geek has joined #openstack-swift15:51
*** miqui has joined #openstack-swift15:54
*** Anticimex has quit IRC16:00
*** tsg_ has joined #openstack-swift16:03
*** rmcall has joined #openstack-swift16:03
*** Anticimex has joined #openstack-swift16:07
*** nellysmitt has joined #openstack-swift16:26
*** foexle has quit IRC16:26
*** jordanP has joined #openstack-swift16:42
*** anna_ has joined #openstack-swift16:57
*** X019 has quit IRC16:59
*** tsg_ has quit IRC17:04
*** gustavo has joined #openstack-swift17:11
*** tsg_ has joined #openstack-swift17:12
joseffHey all is there a way to have Swiftly reply back TRUE after it moves a file?17:13
joseffor false if it couldn't?17:14
*** tgohad has joined #openstack-swift17:14
*** exploreshaifali has joined #openstack-swift17:14
*** zul has quit IRC17:16
*** tsg_ has quit IRC17:17
*** zul has joined #openstack-swift17:18
*** nellysmitt has quit IRC17:19
*** gyee has joined #openstack-swift17:20
*** zaitcev has joined #openstack-swift17:21
*** ChanServ sets mode: +v zaitcev17:21
gustavoHi guys, We are experiencing a high I/O usage in a data node due to contaniner's processes. We have 4 data nodes, 2 of them have identical hardware configuration and have identical weight in containers ring. The problem is that one node has 99% IO usage and the other only 60%. Can anyone give me an advice on how to determine the root cause of this difference?17:24
*** gyee has quit IRC17:24
*** gyee has joined #openstack-swift17:24
*** Viswanath has joined #openstack-swift17:24
notmynamegood morning, world17:25
*** anna_ is now known as X01917:25
*** gyee has quit IRC17:25
notmynamefifieldt_: acoles: thanks for taking care of the translation commit and the co-authored-by17:25
*** gyee has joined #openstack-swift17:26
*** gyee has joined #openstack-swift17:26
notmynamegustavo: first steps would be standard admin type things like looking at top and lsof and etc17:26
*** Viswanath has quit IRC17:27
gustavonotmyname,  first, thank you for reply. Top shows similar cpu usage difference than iotop.17:33
gustavocontainer-server and container-replicator are the processes having different cpu and IO usage.17:34
notmynamegustavo: is this on an active cluster (clients still doing stuff?)17:42
*** aix has quit IRC17:51
*** rmcall has quit IRC17:57
*** rmcall has joined #openstack-swift17:58
acolesnotmyname: np18:01
*** abhirc_ has quit IRC18:11
*** jordanP has quit IRC18:15
*** mmcardle1 has quit IRC18:16
*** joeljwright has quit IRC18:17
*** tamizh_geek has quit IRC18:18
openstackgerritJohn Dickinson proposed openstack/swift: update AUTHORS  https://review.openstack.org/13608518:22
acolespeluse: you here?18:30
*** mkollaro has quit IRC18:39
*** rdaly2 has joined #openstack-swift18:41
gustavonotmyname, yes, this is a production cluster.18:41
*** jdaggett_ is now known as jdaggett18:47
*** zul has quit IRC18:51
*** mkollaro has joined #openstack-swift18:53
*** acoles is now known as acoles_away18:56
*** Viswanath has joined #openstack-swift19:00
*** rmcall has quit IRC19:03
*** abhirc has joined #openstack-swift19:03
*** rmcall has joined #openstack-swift19:05
*** Viswanath has quit IRC19:07
*** rmcall has quit IRC19:08
*** rmcall has joined #openstack-swift19:13
*** geaaru has quit IRC19:16
vishychmouel: commented https://bugs.launchpad.net/python-swiftclient/+bug/138395019:17
*** abhirc has quit IRC19:17
notmynamegustavo: ah ok. so maybe there's a container replica on the busy one that isn't on the less busy one? is there client activity to a particular container that is higher than to another container?19:17
clayganyone want a cheap merge -> https://review.openstack.org/#/c/135824/  numbers all check out, the default was correct (5 * 2 ** 20) in obj/diskfile.py19:24
*** exploreshaifali has quit IRC19:27
notmynameclayg: done19:27
*** IRTermite has quit IRC19:30
*** IRTermite has joined #openstack-swift19:34
gustavonotmyname, I don't know how to get that information. How can I get the container activity ranking  ?19:34
claygman i keep bouncing back and forth from my ec setup and regular swift reviews - and my regular swift is always blowing up on my "Invalid option 'policy_type' in storage-policy section 'storage-policy:1'"19:39
gustavonotmyname, Our cluster has a lot of PUT operations to store small objects19:39
claygi wish there was a way to somehow make swift master cut me some slack there - but I'm not sure exactly what the patch would look like :\19:39
notmynameclayg: "[proxy-server:app]\nmake_clayg_happy=True"19:40
*** dencaval has joined #openstack-swift19:41
claygi saw the api docs stuff landed - i wish there was a way to make the api overview toc [1] show up on every api page (1. http://docs.openstack.org/developer/swift/#object-storage-v1-rest-api-documentation)19:42
claygnotmyname: do you know what the sphinx config section for the make_clayg_happy option is?19:42
notmynamenot off the top of my head19:42
claygi guess the toc in the side bar is just the HEADINGS for the page, the only higher order links you get are next and previous topic19:44
notmynamegustavo: one way to look for the problematic container (after you've ruled out bad drives and other hardware) is to inspect the container server on the busy server. see where it's spending it's time (eg strace, lsof). you can also look at the proxy server logs to see if there's anything there that sticks out. check that it's not replication or auditing that's causing the load (instead of the container server). use recon to see if this19:47
notmyname is causing async pendings to pile up on the object servers (ie is this actually causing a problem?)19:47
*** zul has joined #openstack-swift19:48
gustavonotmyname, I've checked updater and auditor processes and they have the same usage behaviour on both data nodes, but container-replicator and container-server not.20:03
openstackgerritJay Bryant proposed openstack/swift: Handle os.listdir failures in object-updater  https://review.openstack.org/12574620:03
gustavonotmyname, thank you so much, I'm going to try your advices20:04
peluseacoles_away, guess I missed you :(20:12
*** mvenesio has joined #openstack-swift20:13
peluseclayg, yeah I get that damned thing all the time too jumping back and forth but I think you probably bounce more frequently than I...20:13
*** joseff has left #openstack-swift20:14
notmynamegustavo: let us know what you find20:18
claygit's starting to get on my nerves all of the skips in functional tests - like in-process can't do cross-policy tests OR v3 auth tests, and tempauth doesn't want to do v3 auth tests... now there's the concurrent read test that does a bunch of work to monkey with the proxy setup -> https://review.openstack.org/#/c/117710/12/test/functional/tests.py,unified20:20
clayg[func_test]make_clayg_happy=True didn't help20:21
openstackgerritpaul luse proposed openstack/swift: SAIO updates to support EC development environment.  https://review.openstack.org/12815420:24
peluseclayg, yeah, the concurrent read will skip like a million tests if not in-process....  back in a bit20:26
*** mvenesio has quit IRC20:26
*** abhirc has joined #openstack-swift20:27
*** gustavo has quit IRC20:31
*** gustavo has joined #openstack-swift20:31
*** exploreshaifali has joined #openstack-swift20:33
*** MasterPiece has joined #openstack-swift20:50
*** cpen has joined #openstack-swift20:58
*** shri has joined #openstack-swift21:15
*** david-lyle has joined #openstack-swift21:16
*** lpabon has quit IRC21:19
*** exploreshaifali has quit IRC21:20
*** abhirc has quit IRC21:21
*** abhirc has joined #openstack-swift21:21
*** mkollaro has quit IRC21:24
portanteanybody here have any experience with triple booting mac os x, win 7, linux?21:26
claygportante: one time I had a machine that could boot into windows and linux - then I installed virtualbox?21:29
portanteyeah, that is an option, but I want the ability to run it baremetal and vm, so having a hard partition allows that21:30
*** fifieldt__ has joined #openstack-swift21:31
*** fifieldt_ has quit IRC21:34
tdasilvaeikke: hi, not sure if my comment was more helpful or confusing, but if you have a chance to verify that would be nice21:48
eikketdasilva: I wont have time to look into the FS code anytime soon21:49
tdasilvaeikke: hehe, ok21:50
eikkebut IMHO it wouldn't hurt to always sync the tree upon creation, it won't hurt21:50
eikkeespecially when first creating all dirs, then walking it and syncing the dirents21:50
eikkeif the first sync takes the whole tree with it, then the others wont be dirty anymore, and an fsync on them should be a no-op21:50
eikkeso all you lose is a syscall, and I'd imagine dir trees aren't created all the time21:51
tdasilvaeikke: it's definitely a safer approach :-) it was your last statement that I was concerned about mostly for performance reasons21:57
tdasilvaeikke: anyway...thanks for taking a look at that...gotta go21:58
eikkethanks for the info as well21:58
*** tdasilva has quit IRC21:59
eikkethe information online about this kind of stuff is very fragmented and very contradicting, in my experience :-/21:59
notmynamenice shoutout for Swift from eHarmony in https://gigaom.com/2014/11/19/why-eharmony-is-rebuilding-itself-atop-openstack-and-hadoop/22:01
openstackgerritJay Bryant proposed openstack/swift: mock out os.listdir to return a list  https://review.openstack.org/13613822:01
*** mahatic has quit IRC22:04
*** bkopilov has quit IRC22:09
*** jistr|mtg has quit IRC22:27
*** rdaly2 has quit IRC22:35
*** NM has joined #openstack-swift22:36
*** MasterPiece has quit IRC22:37
*** nomad411 has joined #openstack-swift22:38
*** dmsimard is now known as dmsimard_away22:40
openstackgerritJay Bryant proposed openstack/swift: mock out os.listdir to return a list  https://review.openstack.org/13613822:45
*** NM has quit IRC22:47
openstackgerritJay Bryant proposed openstack/swift: Handle os.listdir failures in object-updater  https://review.openstack.org/12574622:48
*** rmcall has quit IRC22:52
*** rmcall has joined #openstack-swift22:54
*** zul has quit IRC22:56
claygtorgomatic: what are you doing!?  aren't you on vacation?22:58
torgomaticclayg: nope, back in the bay area now22:58
*** NM has joined #openstack-swift23:16
*** mkollaro has joined #openstack-swift23:26
*** dmsimard_away is now known as dmsimard23:36
*** haomaiwa_ has quit IRC23:46

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!