Tuesday, 2015-12-08

*** pgbridge has quit IRC00:00
*** jerrygb has quit IRC00:00
*** jerrygb has joined #openstack-swift00:02
*** jamielennox|away is now known as jamielennox00:05
notmynameho: thanks for the ring patch review and checking on it with your prod clsuters!00:09
honotmyname: you are welcome!00:11
dfgclayg: notmyname - no i like the new change. thats just what i found. the partitions all being frmo the same set was a little weird. but ya- if those are distributed i don't knwo how much it matters00:12
dfgbut over all it seems like a big improvement00:12
claygdfg: phew00:12
dfgand after the initial change it was great. it was kinda scary tht the old code just left those old nodes out of balance. sorry if i sounded negative- it was exhausting looking at that stuff :p00:13
notmynameoh yeah, I'll agree to that ;-)00:14
notmynamedfg: thanks for the affirmation. and yeah. current master is a little scary when you get down to it.00:15
notmynameso the plan as we said last week is to merge the ring patch tomorrow00:15
dfgclayg: you didn't that they pulling out the same partitions over and over again would be bad? the addition i made was relatively small so if i made a bigger cahnge it might have spread out more.00:15
notmynamedfg: do you have more stuff you want to make sure is looked at or done before that happens?00:15
claygdfg: I don't think so - but the *next* change should pick up where it left off00:16
*** Jeffrey4l has quit IRC00:17
claygi need a whiteboard00:17
dfgclayg: but i don't think it did. when i made that second change00:17
claygdfg: yeah00:17
claygthinking00:17
dfganywya- i don't know enough about this to aruge about it.00:17
claygdfg: i'll look at it - I think you're right to raise the flag - i'm not sure yet00:17
*** jerrygb has quit IRC00:18
*** jerrygb has joined #openstack-swift00:18
*** ianbrown has quit IRC00:20
*** ianbrown has joined #openstack-swift00:20
*** jerrygb has quit IRC00:23
*** Jeffrey4l has joined #openstack-swift00:23
*** ianbrown has quit IRC00:25
*** ianbrown_ has joined #openstack-swift00:25
*** Trixboxer has quit IRC00:32
openstackgerritClay Gerrard proposed openstack/swift: Put part-replicas where they go  https://review.openstack.org/24157100:40
clayg^ just some small updates for kota00:40
openstackgerritClay Gerrard proposed openstack/swift: Ring Overview updates  https://review.openstack.org/24441200:42
*** alejandrito has quit IRC00:46
*** Trixboxer has joined #openstack-swift00:46
kota_morning00:50
kota_clayg: thanks! i am going to continue to look at :-)00:50
*** Guest84101 has quit IRC00:50
*** km has joined #openstack-swift00:51
*** km is now known as Guest6412300:51
*** kei_yama has quit IRC00:51
*** kei_yama has joined #openstack-swift00:52
*** changbl has quit IRC01:00
claygjson is such a crappy format01:09
*** changbl has joined #openstack-swift01:13
*** changbl has quit IRC01:14
*** dmorita has joined #openstack-swift01:18
*** ianbrown_ has quit IRC01:20
*** rohit_ has joined #openstack-swift01:21
*** onovy has quit IRC01:22
*** peterlisak has quit IRC01:23
*** rohit_ has quit IRC01:23
*** rohit_ has joined #openstack-swift01:23
*** peterlisak has joined #openstack-swift01:24
*** onovy has joined #openstack-swift01:24
*** links has joined #openstack-swift01:27
*** lyrrad_ has quit IRC01:27
*** garthb has quit IRC01:31
*** changbl has joined #openstack-swift01:34
*** badari has quit IRC01:42
*** changbl has quit IRC01:47
*** ianbrown_ has joined #openstack-swift01:48
*** 7F1AA32U1 has joined #openstack-swift01:58
*** changbl has joined #openstack-swift02:00
*** 7F1AA32U1 has quit IRC02:01
*** haomaiwang has joined #openstack-swift02:01
*** janonymous_ has quit IRC02:01
*** Jeffrey4l has quit IRC02:13
*** Jeffrey4l has joined #openstack-swift02:14
*** jerrygb has joined #openstack-swift02:14
*** klrmn1 has quit IRC02:16
*** jerrygb has quit IRC02:16
*** km has joined #openstack-swift02:53
*** Guest64123 has quit IRC02:53
*** km is now known as Guest3315502:54
*** kei_yama has quit IRC02:54
*** kei_yama has joined #openstack-swift02:55
*** ianbrown_ has quit IRC02:57
*** sanchitmalhotra has joined #openstack-swift03:00
*** haomaiwang has quit IRC03:01
*** haomaiwang has joined #openstack-swift03:01
*** haomaiwang has quit IRC03:15
*** haomaiwang has joined #openstack-swift03:15
*** haomaiwang has quit IRC03:16
*** haomaiwang has joined #openstack-swift03:16
*** haomaiwang has quit IRC03:17
*** haomaiwang has joined #openstack-swift03:18
*** haomaiwang has quit IRC03:19
*** 17WAA5E9R has joined #openstack-swift03:19
*** 17WAA5E9R has quit IRC03:20
*** haomaiwang has joined #openstack-swift03:20
*** haomaiwang has quit IRC03:21
*** dmorita has quit IRC03:24
openstackgerritpei ran wu proposed openstack/swift: Fix POST-as-COPY mess up the DLO manifest  https://review.openstack.org/25279103:24
*** haomaiwang has joined #openstack-swift03:24
*** haomaiwang has quit IRC03:25
*** haomaiwang has joined #openstack-swift03:25
*** haomaiwang has quit IRC03:26
*** gyee has quit IRC03:26
*** haomaiwang has joined #openstack-swift03:26
*** haomaiwang has quit IRC03:27
*** dmorita has joined #openstack-swift03:27
*** haomaiwang has joined #openstack-swift03:27
*** haomaiwang has quit IRC03:28
*** haomaiwang has joined #openstack-swift03:28
*** haomaiwang has quit IRC03:29
*** haomaiwang has joined #openstack-swift03:29
*** haomaiwang has quit IRC03:30
*** 7F1AA34GJ has joined #openstack-swift03:30
*** 7F1AA34GJ has quit IRC03:31
*** haomaiwang has joined #openstack-swift03:31
*** haomaiwang has quit IRC03:32
*** dmorita has quit IRC03:32
*** haomaiwa_ has joined #openstack-swift03:33
*** haomaiwa_ has quit IRC03:34
*** haomaiwang has joined #openstack-swift03:34
*** klrmn1 has joined #openstack-swift03:34
*** haomaiwang has quit IRC03:35
*** jerrygb has joined #openstack-swift03:35
*** haomaiwa_ has joined #openstack-swift03:38
gmmahahey mattoliverau03:41
gmmahayeah, going to start looking into the gobal clusters card..03:42
*** blmartin has quit IRC03:42
gmmahaneed to get enough background to figure out whats the low hanging fruit that i can knock out03:42
*** silor has joined #openstack-swift03:47
*** silor1 has joined #openstack-swift03:54
*** klrmn1 has quit IRC03:54
*** dmorita has joined #openstack-swift03:55
*** ianbrown has joined #openstack-swift03:55
*** silor has quit IRC03:56
*** silor1 is now known as silor03:56
*** dmorita has quit IRC03:59
*** haomaiwa_ has quit IRC04:01
*** haomaiwang has joined #openstack-swift04:01
*** trifon has joined #openstack-swift04:03
*** darrenc is now known as darrenc_afk04:06
*** Jeffrey4l has quit IRC04:14
mattoliveraugmmaha: yeah, I suspect you'll need to do a bit of R&D for that one, setup a global cluster (or simulate one [there is a slowdown middleware you could use on a "remote" region]), profile/benchmark it with different settings, starting with the defaults.04:21
*** wupeiran has joined #openstack-swift04:27
gmmahamattoliverau: yeah.. spoke to peluse this morning and i have quite a bit of homework to do before i can work on anyting w.r.t to the clusters04:28
gmmahathanks for the tip.. didnt know about the middleware..04:28
*** dmorita has joined #openstack-swift04:29
mattoliveraugmmaha: yeah you will I suspect :) If there is anything I can do to help, just ask.04:29
gmmahawill not hesitate.. ;)04:29
notmynamegood evening04:30
gmmahagood evening notmyname04:30
*** rohit_ has quit IRC04:30
mattoliveraunotmyname: evening04:32
notmynamemattoliverau: is the osic trello board public?04:32
mattoliveraunotmyname: not at the moment, still private, I think it should be, cause the idea is to do it in the open. But I'll bug people about it ;)04:34
notmynameok :-)04:34
mattoliveraunotmyname: do you have access to it? Did peluse add you?04:35
*** ianbrown has quit IRC04:35
notmynamemattoliverau: no, I haven't been added04:35
notmynamebut I'd like to be ;-)04:36
*** trifon has quit IRC04:37
notmynamemattoliverau: what's your trello id?04:39
notmynamefound it04:39
mattoliveraumattoliverau04:39
mattoliverau:)04:39
notmynamemattoliverau: ok, you (and peluse) can now deal with boards under "openstack swift"04:40
notmynameyou probably just got an email04:40
mattoliveraunotmyname: \o/ thanks04:41
notmynameeg you could add your osic swift board to the swift group ;-)04:42
*** darrenc_afk is now known as darrenc04:42
*** proteusguy_ has quit IRC04:43
mattoliveraunice, maybe I'll add the container sharding board there too ;)04:44
notmynamemattoliverau: do you know flanders who works at uni melb?04:45
mattoliveraunotmyname: no I don't, I think I've talked to him briefly online (well talked to someone from the uni of melb), but I should meet him seeing as he is so close ;)04:47
notmynameheh, ok04:47
mattoliveraunotmyname: is he online much?04:47
notmynameno idea ;-)04:47
notmynameanyway, I'm talking to him (ie sent some emails) about meeting some people there to talk about swift during my trip for LCA04:47
notmynameso I'm considering getting to MEL perhaps on friday morning before LCA.04:48
notmynameotherwise, probably getting there sat/sun and headed to geelong on sunday. perhaps trying to find a way back to MEL during the week if there are people interested in swift ther04:49
notmynamepoint is, I'd love to see you in melb when possible :-)04:49
mattoliverauOh cool! You should come, and yes, we can do some Melbourne things :)04:50
*** SkyRocknRoll has joined #openstack-swift04:50
mattoliverauAlso I'm keen to do any swift meet ups with others if your doing community wise :)04:50
notmynameyou'll be down in geelong all week for lca, right?04:50
mattoliverauyup, I head down on Sunday head back up on Sat04:50
notmynamemattoliverau: well to be fair, it will be part community and probably part "let me talk about swiftstack" too.04:51
notmynamebut yeah, if there's any community focused stuff (meetups, coffee meetings, etc), I definitely would include you04:51
mattoliveraunotmyname: yeah, I suspected as much, but that's ok, if its "work" then that's understandable. Besides you might like melbourne so much you'd need to find some clients so you can move down here ;)04:52
notmynameheh04:52
notmynamemy wife and I actually briefly considered a 4 to 6 month home swap with someone there04:53
notmynamenothign came of it, for various reasons, but we'd still love to visit04:55
*** proteusguy_ has joined #openstack-swift04:56
mattoliverauWell I'll just have to do my best to drum up some business for you here then.. don't tell RS :P04:58
mahaticgood morning05:00
*** ppai has joined #openstack-swift05:00
*** haomaiwang has quit IRC05:01
*** haomaiwang has joined #openstack-swift05:01
mattoliveraumahatic: morning05:02
*** ianbrown has joined #openstack-swift05:02
mahaticmattoliverau: hello05:02
notmynamewow. "Ran 4344 tests..."05:04
notmynameI remember when we broke 300005:04
*** klrmn1 has joined #openstack-swift05:05
mattoliveraunext goal 10k :)05:05
notmynameoh you mean land ho's patch? ;-)05:06
mattoliveraulol05:06
kota_lol05:13
kota_mahatic: morning05:13
kota_and notmyname: evening05:14
mahatickota_: hi05:14
*** km has joined #openstack-swift05:23
*** Guest33155 has quit IRC05:24
*** km is now known as Guest6785705:24
*** asettle is now known as asettle-afk05:28
*** klrmn1 has quit IRC05:29
*** trifon has joined #openstack-swift05:42
notmynameZyric: some comments incoming on patch 24197805:42
patchbotnotmyname: https://review.openstack.org/#/c/241978/ - Add support for storage policies to have more than...05:42
honotmyname: mattoliverau: kota_: let's go ahead to 10k :p05:43
*** jerrygb has quit IRC05:50
*** ianbrown has quit IRC05:57
*** ianbrown has joined #openstack-swift05:57
*** ianbrown_ has joined #openstack-swift05:57
*** ianbrown_ has quit IRC05:57
ZyricAh neat, thanks notmyname - just got the review comments.05:58
notmynameZyric: oh, and FYI: a nit comment. normally if you get a "nit" comment, it means "something that maybe should change if you end up doing a different patch set for other reasons but don't do one just for this"05:59
notmynameeg the trailing whitespace05:59
openstackgerritpei ran wu proposed openstack/swift: Fix POST-as-COPY mess up the DLO manifest  https://review.openstack.org/25279106:00
*** ianbrown_ has joined #openstack-swift06:00
*** ianbrown has quit IRC06:00
*** haomaiwang has quit IRC06:01
*** haomaiwang has joined #openstack-swift06:01
*** wolsen has quit IRC06:04
*** wolsen has joined #openstack-swift06:05
*** donghao has joined #openstack-swift06:10
*** ianbrown_ has quit IRC06:17
*** eranrom has joined #openstack-swift06:22
*** ianbrown has joined #openstack-swift06:23
*** haomaiwang has quit IRC06:32
*** asettle-afk is now known as asettle06:33
*** 18WABB396 has joined #openstack-swift06:33
*** donghao has quit IRC06:37
*** Jeffrey4l has joined #openstack-swift06:39
*** ianbrown has quit IRC06:40
*** ianbrown has joined #openstack-swift06:40
*** ianbrown has quit IRC06:41
*** ianbrown_ has joined #openstack-swift06:41
*** ianbrown__ has joined #openstack-swift06:43
*** ianbrown_ has quit IRC06:43
*** ChubYann has quit IRC06:47
*** dmorita has quit IRC06:56
*** 18WABB396 has quit IRC07:01
*** haomaiwang has joined #openstack-swift07:01
*** zaitcev has quit IRC07:04
*** haomaiwang has quit IRC07:09
*** haomaiwang has joined #openstack-swift07:09
*** silor has quit IRC07:17
*** haomaiwang has quit IRC07:28
*** haomaiwa_ has joined #openstack-swift07:28
*** dmorita has joined #openstack-swift07:41
*** arnox has joined #openstack-swift08:04
*** jerrygb has joined #openstack-swift08:06
*** rledisez has joined #openstack-swift08:07
*** jerrygb has quit IRC08:12
openstackgerritpei ran wu proposed openstack/swift: Fix POST-as-COPY mess up the DLO manifest  https://review.openstack.org/25279108:17
*** dmorita has quit IRC08:18
*** hseipp has joined #openstack-swift08:47
*** hseipp has quit IRC08:47
*** hseipp has joined #openstack-swift08:47
*** links has quit IRC08:55
*** links has joined #openstack-swift09:09
*** shakamunyi has joined #openstack-swift09:13
openstackgerritPrashanth Pai proposed openstack/swift: Make object creation more atomic in Linux  https://review.openstack.org/16224309:16
openstackgerritBéla Vancsics proposed openstack/swift: Not used parameter  https://review.openstack.org/25462609:21
*** shakamunyi has quit IRC09:23
*** ho has quit IRC09:38
*** Lickitysplitted_ has quit IRC09:41
*** jordanP has joined #openstack-swift09:46
*** wupeiran has quit IRC09:46
*** Lickitysplitted has joined #openstack-swift09:47
*** torgomatic has quit IRC09:54
*** StevenK has quit IRC09:56
*** StevenK has joined #openstack-swift09:57
*** torgomatic has joined #openstack-swift09:57
*** ChanServ sets mode: +v torgomatic09:57
*** aix has joined #openstack-swift10:01
*** jistr has joined #openstack-swift10:14
*** bwall has quit IRC10:21
*** aix has quit IRC10:21
*** usdsd has joined #openstack-swift10:22
*** bwall has joined #openstack-swift10:23
*** bwall has joined #openstack-swift10:23
*** usdsd has left #openstack-swift10:28
*** aix has joined #openstack-swift10:33
*** jmccarthy has quit IRC10:33
*** jmccarthy has joined #openstack-swift10:34
*** links has quit IRC10:37
*** kei_yama has quit IRC10:40
*** janonymous has quit IRC10:49
*** Guest67857 has quit IRC10:50
*** Lickitysplitted has quit IRC10:53
*** links has joined #openstack-swift10:57
*** openstackgerrit has quit IRC11:02
*** Lickitysplitted has joined #openstack-swift11:03
*** openstackgerrit has joined #openstack-swift11:03
*** haomaiwa_ has quit IRC11:06
*** Jeffrey4l has quit IRC11:15
*** janonymous has joined #openstack-swift11:15
openstackgerritjanonymous proposed openstack/swift: remove all pyc before starting unit test  https://review.openstack.org/25427611:32
*** szaher_ has joined #openstack-swift11:34
*** szaher_ has quit IRC11:34
*** ppai has quit IRC11:50
*** SkyRocknRoll has quit IRC11:52
*** ppai has joined #openstack-swift12:00
*** SkyRocknRoll has joined #openstack-swift12:05
*** jerrygb has joined #openstack-swift12:09
*** jerrygb has quit IRC12:13
*** arnox has quit IRC12:18
*** rledisez has quit IRC12:19
*** rledisez has joined #openstack-swift12:32
*** arnox has joined #openstack-swift12:33
*** silor has joined #openstack-swift12:33
*** links has quit IRC12:36
*** portante has quit IRC12:42
*** ndk has quit IRC12:42
*** ppai has quit IRC12:44
*** ppai has joined #openstack-swift12:55
*** portante has joined #openstack-swift13:06
*** CaioBrentano has joined #openstack-swift13:06
*** ianbrown__ has quit IRC13:22
*** ianbrown_ has joined #openstack-swift13:22
*** SkyRocknRoll has quit IRC13:33
*** ppai has quit IRC13:39
*** jerrygb has joined #openstack-swift13:44
*** diogogmt has quit IRC13:58
*** diogogmt has joined #openstack-swift13:58
*** janonymous_ has joined #openstack-swift14:02
*** sanchitmalhotra has quit IRC14:04
*** changbl has quit IRC14:10
*** breitz has quit IRC14:18
*** breitz has joined #openstack-swift14:19
*** petertr7_away is now known as petertr714:21
*** diogogmt has quit IRC14:25
*** badari has joined #openstack-swift14:43
*** hseipp has quit IRC14:43
*** ndk has joined #openstack-swift14:44
*** hseipp has joined #openstack-swift14:48
*** blmartin has joined #openstack-swift15:00
*** dustins|gone is now known as dustins15:01
*** Jeffrey4l has joined #openstack-swift15:03
*** jerrygb has quit IRC15:09
*** jerrygb has joined #openstack-swift15:10
*** chsc has joined #openstack-swift15:11
*** janonymous_ has quit IRC15:12
*** yarkot has joined #openstack-swift15:13
*** siva_krishnan has left #openstack-swift15:13
*** jerrygb has quit IRC15:15
*** blmartin has quit IRC15:16
*** sgundur has joined #openstack-swift15:19
*** Jeffrey4l has quit IRC15:20
*** blmartin has joined #openstack-swift15:22
*** jerrygb has joined #openstack-swift15:24
*** noark9 has joined #openstack-swift15:26
*** diogogmt has joined #openstack-swift15:36
*** trifon has quit IRC15:39
*** thumpba has joined #openstack-swift15:39
*** esker has joined #openstack-swift15:40
*** siva_krishnan has joined #openstack-swift15:40
*** jistr is now known as jistr|mtg15:43
*** patchbot has quit IRC15:48
*** patchbot has joined #openstack-swift15:48
*** pgbridge has joined #openstack-swift15:55
*** garthb has joined #openstack-swift15:58
*** petertr7 is now known as petertr7_away16:06
*** mariusv has joined #openstack-swift16:07
*** mariusv has quit IRC16:07
*** mariusv has joined #openstack-swift16:07
*** aix has quit IRC16:08
*** janonymous_ has joined #openstack-swift16:13
*** changbl has joined #openstack-swift16:15
*** ianbrown_ has quit IRC16:17
*** klrmn1 has joined #openstack-swift16:17
*** noark9 has quit IRC16:24
*** changbl has quit IRC16:25
*** changbl has joined #openstack-swift16:26
*** ianbrown_ has joined #openstack-swift16:29
*** mariusv_ has joined #openstack-swift16:29
*** changbl has quit IRC16:30
*** mariusv has quit IRC16:31
*** kairat has joined #openstack-swift16:34
kairatHi guys16:34
kairatCould you please help with the question: when chunked transfer encoding used in swiftclient16:35
kairatI noticed if we don't specify content-length in glance and provides just an iterator to swift16:35
kairat*swiftclient16:35
kairatthen swiftclient also do not specify it in request16:38
kairatit causes radosgw to fail16:38
*** arnox has quit IRC16:39
kairatbecause the requests seems not in accordance with HTTP specification16:39
notmynamedoes radosgw support chunked transfer encoding?16:39
notmynamedo you have a dump of the headers or other request info that are being sent?16:39
kairatJust a sec16:39
kairatnotmyname, https://bugs.launchpad.net/glance/+bug/151843116:40
openstackLaunchpad bug 1518431 in Glance "Glance failed to upload image to swift storage" [Undecided,Confirmed]16:40
kairathere is the bug16:40
kairatnotmyname, this is paste: http://paste.openstack.org/show/479621/16:41
kairatIIUC, there is no content-length nor transfer-encoding here16:41
kairatI actually don't know if RadosGW support chunked-encoding but it seems that requests are not satisfy HTTP spec requirements16:42
kairathttps://github.com/openstack/glance_store/blob/master/glance_store/_drivers/swift/store.py#L550 - here is request from glance16:42
kairatcontent_length here is None16:43
*** changbl has joined #openstack-swift16:43
*** noark9 has joined #openstack-swift16:43
kairatI reviewed all requests from client to swift and it seems that no content-length is specified there if glance do not provide this16:45
notmynamekairat: where's the definition of ChunkReader?16:45
notmynameah. there at the bottom16:46
kairatnotmyname, https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/swift/store.py#L95116:46
kairatyep16:46
*** yarkot has quit IRC16:46
kairatI stopped in python requests lib16:47
kairatbecause I think no headers can be specified here16:47
*** openstackgerrit has quit IRC16:47
kairatBut I am not sure16:47
*** openstackgerrit has joined #openstack-swift16:47
*** sgundur has quit IRC16:49
*** yarkot has joined #openstack-swift16:49
*** garthb_ has joined #openstack-swift16:51
*** garthb has quit IRC16:51
*** nadeem has joined #openstack-swift16:53
*** nadeem has quit IRC16:55
*** ianbrown__ has joined #openstack-swift16:55
*** alejandrito has joined #openstack-swift16:55
*** nadeem has joined #openstack-swift16:56
*** ianbrown_ has quit IRC16:57
*** gyee has joined #openstack-swift17:00
notmynamekairat: I traced throught he swiftclient code until I got to the requests library17:02
kairatnotmyname, I consulted with glance folks also17:02
notmynamefrom what I can tell, we're relying on requests to set transfer-encoding: chunked if there's no content-length17:03
notmynameI haven't validated that yet17:03
kairatIt seems that requests library should add chunked if swiftclient provides a generated17:03
kairat* generator17:03
notmynamekairat: what would be super helpful is if you could trace the request between swift and glance and dump the requests on the wire, including the headers17:03
kairatyep17:04
kairatasking guys about that17:04
notmynameI think that's the next step here17:04
kairatyep17:04
notmynamebut I've got to run for the moment. I'll be back in an hour-ish17:04
kairatnotmyname, thanks for help17:04
*** noark9 has quit IRC17:06
*** janonymous__ has joined #openstack-swift17:07
*** noark9 has joined #openstack-swift17:08
*** petertr7_away is now known as petertr717:08
*** janonymous_ has quit IRC17:09
*** pgbridge has quit IRC17:09
*** lyrrad has joined #openstack-swift17:12
*** noark9 has quit IRC17:14
*** rledisez has quit IRC17:16
*** pgbridge has joined #openstack-swift17:17
*** petertr7 is now known as petertr7_away17:18
*** tsg has joined #openstack-swift17:20
openstackgerritPeter Chng proposed openstack/swift: Add round-trip encrypter/decrypter unit tests  https://review.openstack.org/25160617:21
*** jistr|mtg is now known as jistr17:21
openstackgerritPeter Chng proposed openstack/swift: Add round-trip encrypter/decrypter unit tests  https://review.openstack.org/25160617:25
*** dmorita has joined #openstack-swift17:28
portanteany znc users out there encountered slow startup of znc after server reboots?17:33
portantewhat do folks have their server throttle set to?17:34
*** insanidade has joined #openstack-swift17:35
insanidadehi all. is it possible to move data from one container to another?17:35
*** janonymous__ has quit IRC17:36
*** esker has quit IRC17:37
*** garthb_ has quit IRC17:38
*** janonymous_ has joined #openstack-swift17:38
janonymous_kota: hey, on one of my vm instance some ut's were failing, can you have a look if it is useful or it's my env problem only : http://paste.openstack.org/show/481204/17:38
*** manous has joined #openstack-swift17:40
*** ianbrown__ has quit IRC17:52
insanidadeWould anyone please help me with this question? I have a container A and, some time later, I create container B. I synchronize container A and B. What if I decide to delete container A? Does container B still contain the synced files ?17:55
*** ianbrown__ has joined #openstack-swift17:56
*** yarkot has quit IRC17:58
insanidadeanyone ?17:58
pdardeauinsanidade: by synchronize, you mean making a copy of the objects?17:58
insanidadepdardeau: yes. from one container to the other.17:59
insanidadepdardeau: what I'm actually trying to do is to "move" data from one container to the other. I know how "moving" things in object storage is not as easy as in common file systems.18:00
pdardeauinsanidade: once you delete container A, container B will still contain the copies that were made from container A18:00
*** hseipp has quit IRC18:00
insanidadepdardeau: I see. That would help me with that "move" workaround.18:00
pdardeauinsanidade: are you able to give your use case for wanting to move between containers?18:01
insanidadepdardeau: not sure about the details. A customer just asked about it. He tried it with a GUI tool and what the tool did was "download everything and upload to the new cluster"18:02
insanidadepdardeau: that customer wants to know if a "move" feature exists.18:02
pdardeauinsanidade: are you really interested in a container rename?18:03
insanidadeby "move", he clearly means that traditional "move" definition. download TB of data and uploading again is what that customer is trying to avoid.18:03
insanidadepdardeau: that's not just a rename. I believe there's some other data in the destination container and that makes both containers different.18:04
CaioBrentanoinsanidade does this help in anyway? http://docs.openstack.org/developer/swift/overview_container_sync.html18:05
*** jistr has quit IRC18:05
insanidadeCaioBrentano: that url has been in my favorites for a while now :) it sure helps, but I thing I don't have the required access to configure the cluster and enable it.18:06
insanidadesorry for my typos. my keyboard sucks.18:09
ahaletheres also http://developer.openstack.org/api-ref-objectstorage-v1.html#copyObject18:10
*** diazjf has joined #openstack-swift18:11
CaioBrentanoinsanidade in this case I if you have a "simple" solution! In some way, you'll probably have to create all the objects again18:12
insanidadeahale: would that "copy" functionality apply for copies between containers ?18:13
tdasilvayeah, you basically need a tool to issue copy requests to all objects in your source container and then delete them afterwards...18:13
dfgclayg: you there?18:14
insanidadeif that works between containers, yes. I'd have to copy and delete the original objects. that would work.18:14
tdasilvainsanidade: yeah, you can copy between containers and even account18:14
tdasilva*accounts18:14
ahaleyep it works across containers, you just need some scripting round it like tdasilva says18:15
insanidadeahale tdasilva yes, I got it.18:15
notmynamegood morning18:16
CaioBrentanoit's nick to know about this copy method! thanks ahale :)18:17
CaioBrentano*nice18:17
*** alejandrito has quit IRC18:19
*** alejandrito has joined #openstack-swift18:19
dfgclayg: nm18:19
* tdasilva just realized python-swiftclient doesn't have a copy option18:21
notmynametdasilva: watch out. pretty soon you'll be noticing all things swiftclient doesn't have and you'll sound like timburke ;-)18:22
tdasilvalol18:22
tdasilvatimburke: what do you sound like???18:22
timburkei wonder how far you can get with `swift upload -H 'X-Copy-From: ...'`18:22
timburkesomeone who wants some degree of feature parity between client and server?18:23
*** shakamunyi has joined #openstack-swift18:25
tdasilvatimburke: sounds reasonable ;-)18:25
timburkewe still can't even post data to an account (for example, so you could use the bulk ops middleware), though patch 190887 will address that18:25
patchbottimburke: https://review.openstack.org/#/c/190887/ - Use bulk-delete middleware when available18:25
*** jerrygb has quit IRC18:39
insanidadetdasilva ahale what about renaming a container ? is it possible ?18:39
tdasilvainsanidade: not at the moment18:40
*** jerrygb has joined #openstack-swift18:46
*** lucy has joined #openstack-swift18:47
*** lucy is now known as Guest8625318:47
*** Guest86253 has quit IRC18:48
notmynameI talked to someone about a patch that was going to remove the beta tag on the EC docs. but I don't remember the context or who I was talking to18:49
*** david-ly_ has joined #openstack-swift18:50
*** david-ly_ is now known as david-lyle_18:52
insanidadewhat does it mean when I issue a curl command against Swift and receive the following response: "<html><h1>Accepted</h1><p>The request is accepted for processing.</p></html>"  ?18:53
*** david-lyle has quit IRC18:54
notmynameinsanidade: that's fine! that's a 202 status code18:54
insanidadedoes it mean that the requested action will be processed asynchronously ?18:54
notmynamewas it a PUT request?18:54
insanidadenotmyname: yes18:54
notmynameso in that case, you did an overwrite of an existing object. new objects would have given you a 201 Created. an overwrite gives you 202 Accepted.18:54
notmynamethat means that the new data was durably written to multiple disks18:55
notmynameso your data is there18:55
*** manous has quit IRC18:55
notmynamedepending on your cluster (eg hw failures, any business when you were doing the PUT, etc) it *might* be possible that a subsequent GET for that object will return the older version of the file18:56
notmynamehowever, swift will resolve it down to the new version of the file in the background18:56
insanidadenotmyname: I'm just trying to copy a file from one container to the other.18:56
insanidadenotmyname: a file from one container to an empty container.18:56
notmynameinsanidade: the short answer is that a 2xx series response code means that swift has durably stored the data. you're good!18:58
insanidadenotmyname: it's been a while since I issued that command and the destination container remains empty.19:00
notmynameinsanidade: oh, that's weird then. can you pastebin the command you used to do the copy?19:00
notmynamehere's something cool. CaioBrentano shared this with me yesterday. one of the world's biggest soccer stars (neymar jr) has all the static assets on his website hosted in CaioBrentano's swift cluster19:01
*** openstackgerrit has quit IRC19:02
*** openstackgerrit has joined #openstack-swift19:03
insanidadenotmyname: http://pastebin.com/xA7mTapB19:06
notmynameinsanidade: oh!19:06
notmynameok, 2 things I see there19:06
notmynamefirst, you need to PUT to the actual object name you're copying to. in this case, you did a PUT to the container itself, and that's why you got the 202 response19:07
*** tsg has quit IRC19:07
notmynamesecond, the x-copy-from header value needs to be of the form "[/]<src_container>/<src_object>"19:08
insanidadenotmyname: so, I need to explicitly mention the source file name and the destination file name (not only the destination folder) ?19:10
notmynameyes, exactly19:10
insanidadenotmyname: what about wildcards? I'd like to copy all files from source folder to destination folder.19:11
notmynamenope19:11
notmynameinsanidade: http://developer.openstack.org/api-ref-objectstorage-v1.html#copyObject19:12
*** ChubYann has joined #openstack-swift19:12
insanidadenow the server is complaining about Content-Length header not present.19:12
*** hferenc has quit IRC19:12
notmynameyup. PUT requests require a content-length (or chunked transfer encoding). you aren't actually sending a body, so "content-length: 0" is appropriate19:13
*** david-lyle_ is now known as david-lyle19:14
*** janonymous_ has quit IRC19:17
*** esker has joined #openstack-swift19:24
insanidadenotmyname: is there a limit for the file sizes ?19:26
insanidadenotmyname: I can't copy big files (6GB, for instance)19:27
*** esker has quit IRC19:28
*** diazjf has quit IRC19:31
*** diazjf has joined #openstack-swift19:31
notmynameright. COPY will be limited by the per-object limit (default is 5GB)19:32
openstackgerritTim Burke proposed openstack/python-swiftclient: Use application/x-directory content-type for dir markers  https://review.openstack.org/25492119:33
notmynamenow, swift does support larger object using manifest objects, and you can do some stuff to copy the manifest19:33
*** tsg has joined #openstack-swift19:35
notmynamemattoliverau: hurricanerix: I think we need to add subunit to test-requirements for patch 21420619:36
patchbotnotmyname: https://review.openstack.org/#/c/214206/ - Modify functional tests to use testr19:36
notmynamehmm maybe not19:37
mattoliverauThe latest patch is using os-testr.. So it maynot19:37
notmynamemattoliverau: I *love* this time of year for the tz coordination ;-)19:38
notmynamemattoliverau: `SWIFT_TEST_IN_PROCESS=true SWIFT_TEST_IN_PROCESS_CONF_DIR=/tmp tox -efunc` errors with "no module named subunit"19:38
mattoliverauWell its still early, but I happen to be in an airport on my way to the Sydney office.19:39
notmynamebut not doing the in-process works for me. I have subunit installed outside of a venv, so it works if I do the not in process tests19:39
mattoliverauOh really.. Hmm, I thought I nuked my tox env and tried it19:39
mattoliverauSo if you add it to test-req then it works?19:40
notmynamebut I added subunit to test-requirements locally and it stiff broke :-/19:40
notmynameso that was the "hmm maybe not" comment ;-)19:40
mattoliverauHmm19:41
notmynameindeed19:41
notmynameah. maybe....19:42
notmynamegot past that one19:43
notmynamekeep subunit out of t-r19:43
notmynameI had an old .tox dir19:43
notmynamecleared that19:44
notmynamenow whole new errors ;-)19:44
mattoliverau\o/ and :(19:44
notmynameinvalid arguments to pyeclib_c_init. something with liberasure code19:45
mattoliverauIt works on my saio in both in-process and not, and on my desktop.19:45
mattoliverauOK, so its a pyeclib problem :(19:45
notmynamemaybe liberasurecode not being installed?19:47
notmynamenot sure yet19:47
notmynameI did see that tsg joined ;-)19:48
hurricanerixyeah, i saw those errors in the gate jobs, but i don't think i got them locally.19:48
mattoliverauI saw them too, but the last patch finally passed in the gate.. So it might have been a bad nodepool image or something, doesn't help notmyname now tho :p19:49
papercupcan someone here pls point me to documentation for functional tests -- eg:  what's TestFile class ? Is this for testing actual object ?19:49
notmynamepapercup: there's not a lot of docs on the tests themselves, other than inline comments19:49
notmynametest/functional/tests.py is the first place to start. it's a big file19:49
papercupsure thanks. yes .. I'm looking at that exact file for TestFile class19:50
tsgo/ notmyname, sorry don't have enough context in the irc history - is there new gate failure?19:50
papercupI'm trying to figure out what is it actually testign19:50
papercupI thought I'll ask here and get inside with some starter information19:51
eranromacoles around?19:52
tsgnotmyname, mattoliverau: let me know how to repro the "invalid arguments" error and I will take a look right away19:52
notmynamepapercup: IIRC that's for testing objects. it might be in a helper module. I don't remember19:52
papercupnotmyname: some piece of info, thanks for confirming my blind guesses19:54
notmynametsg: I removed my .tox and then ``SWIFT_TEST_IN_PROCESS=true SWIFT_TEST_IN_PROCESS_CONF_DIR=/tmp tox -efunc``19:55
tsgnotmyname, ok - let me try that on the latest swift master19:55
*** dmorita has quit IRC19:56
*** dmorita has joined #openstack-swift19:57
*** petertr7_away is now known as petertr719:58
*** dmorita has quit IRC19:58
*** garthb has joined #openstack-swift20:04
notmynametsg: is there a version attribute of pyeclib somewhere?20:04
tsgnotmyname, there is a PYECLIB_VERSION in ec_iface.py20:05
notmynameahh. the old 'module' object has no attribute 'ec_iface' error20:06
notmynameso liberasurecode isn't installed. right?20:07
notmynameis it possible it picked up a very new version that doesn't have that bundled?20:07
*** mariusv_ has quit IRC20:13
*** mariusv has joined #openstack-swift20:15
*** mariusv has quit IRC20:15
*** mariusv has joined #openstack-swift20:15
*** esker has joined #openstack-swift20:17
* notmyname goes to pick up some lunch20:17
*** yarkot has joined #openstack-swift20:18
*** petertr7 is now known as petertr7_away20:20
*** esker has quit IRC20:21
*** zaitcev has joined #openstack-swift20:23
*** ChanServ sets mode: +v zaitcev20:23
*** silor has quit IRC20:25
*** mariusv has quit IRC20:26
*** chsc has quit IRC20:28
* mattoliverau is boarding his plane so will be afk for a while20:28
*** CaioBrentano has quit IRC20:32
*** CaioBrentano has joined #openstack-swift20:32
*** esker has joined #openstack-swift20:37
*** mariusv_ has joined #openstack-swift20:39
*** gyee has quit IRC20:40
*** dmorita has joined #openstack-swift20:41
*** mariusv_ has quit IRC20:41
gmmahahi, anyone running into issues with doing fresh SAIO install on Ubuntu 14.0420:43
gmmaha?20:43
gmmahaMe and another colleague are running into two different issues.. setup.py is failing with error in setup command: Invalid environment marker: (python_version>='3.0')20:43
gmmahatox fails with ValueError: ("Expected ',' or end-of-list in", "dnspython>=1.12.0;python_version<'3.0'", 'at', ";python_version<'3.0'")20:44
*** yarkot has quit IRC20:49
Zyricgmmaha: Yes, I ran into that too. If I recall correctly it's a pip version issue. I ended up removing the "python_version" parts for the 2.7 line I think. http://stackoverflow.com/questions/33723149/error-during-swift-standalone-install-error-in-setup-command-invalid-environme20:49
*** esker has quit IRC20:49
gmmahaZyric: yeah thats what folks heere also did to get past it.. was wondering if at all we can do something fancy with the requiremtents/tox.ini file to make it work :)20:50
Zyricgmmaha: Good call. Be nice to have a real solution for it especially since it seems to be more common than I thought.20:52
gmmahaZyric: agreed.. let me see what i can fine20:54
notmynameyeah, it's a combination of setuptools, pbr, and maybe pip20:55
notmynamebasically, update everything and it should work20:55
notmyname(yes I know that's a terrible answer, aside from "it works")20:55
gmmahanotmyname: yup, that seems to work. along with installing the dev package for python3 and python2.720:56
gmmahashould we add this to the SAIO setup page for new comers?20:56
*** pdardeau has quit IRC20:56
notmynameprobably ;-)20:57
gmmahadidnt see this until now as i was happy using fedora.. was testing it out for a colleague and ran into this20:57
hrouActually gmmaha, I'd add the steps regarding how to upgrade, that is usually a pain wit pip / setuptools as there's already an OS installed version (that's generally much older, what 14.04 ships with), so I think having those steps is useful20:58
*** pdardeau- has joined #openstack-swift20:58
gmmahahrou: yeah thats definitely useful20:59
*** petertr7_away is now known as petertr721:00
notmynameCaioBrentano: I was just asked (via email) if globo could be featured on the openstack foundation's superuser site. is there some contact I can connect you with?21:00
notmynameor someone there?21:00
hrougmmaha, another FYI is you may find even after upgrading TOX will not work, you generally need to upgrade virtualenv for this, something like pip install --upgrade virtualenv.  Something else to add to that doc :)21:01
gmmahahrou: the list that i upgraded was this sudo pip install -U pip tox pbr virtualenv setuptools21:01
*** jlvillal has joined #openstack-swift21:01
gmmahawith that and the python3-dev package, things are now working just fine21:01
gmmahasorry its libpython3.4-dev21:02
*** thumpba has quit IRC21:02
hrougmmaha, that's great;  Yea I mean generally these don't change, but over the next little while it'll probably be important for any new dev starting with ubunut 14.04 or the like21:02
hroui.e. great idea to document21:02
gmmahahrou: definitely.. given that atleast 3 of us have seen this, its probably fair to put a tiny note21:04
hrougmmaha, oh its way more than 3 :)  Maybe you could add it to this:  http://docs.openstack.org/developer/swift/first_contribution_swift.html21:05
gmmahahrou: should it go into that page or http://docs.openstack.org/developer/swift/development_saio.html?21:06
hrougmmaha, yea not too sure;  Could add it to the "debugging issues" section;  Mind  you that first contribution page does talk about things like rebasing and the like so it is somewhat fitting.21:08
notmynametsg: did you find anything?21:09
tsgnotmyname, sorry had to step out for a meeting - looking again now21:10
tsgnotmyname, this is in a trusty env with no liberasurecode installed?21:10
notmynameI believe there is no liberasurecode installed in the tox environment21:11
tsgnotmyname: not in the tox env, on the system21:13
gmmahahrou: thats true. Let me add it to the first contribution page and see if fits ok21:13
notmynametsg: it's on the system21:14
tsgnotmyname, if it is not installed, versions as of 1.1.2 install liberasurecode prior to installing pyeclib21:14
notmynametsg: this is precise, but I can try trusty too21:14
tsgnotmyname, ok - trying to reproduce now on trusty21:14
tsgnotmyname, can you reproduce it with default "tox -e py27" run?21:14
*** lcurtis has joined #openstack-swift21:15
notmynametsg: yup. repro there too21:16
*** diazjf has quit IRC21:18
*** ianbrown__ has quit IRC21:19
*** diazjf has joined #openstack-swift21:20
tsgnotmyname, I am not able to reproduce with liberasurecode-dev installed at the system level - let me try launching another clean precise instance21:21
notmynameok21:22
*** ianbrown has joined #openstack-swift21:23
*** cdelatte has quit IRC21:23
*** yarkot has joined #openstack-swift21:24
tsgnotmyname, can you please post your log?21:26
*** ianbrown has quit IRC21:27
*** ianbrown has joined #openstack-swift21:28
pdardeau-anyone know Scott Simpson's nick?21:28
notmynamessimpson21:28
pdardeau-notmyname: thx21:29
notmynamepdardeau-: or just bug dfg redbo and glange. they probably all sit within a few feet of him ;-)21:29
pdardeau-notmyname: thanks for tidbit!21:30
glangescott's out of the office for a while, not sure how long21:31
glangepdardeau-: is there anything you need help with?21:32
pdardeau-glange: thx for heads up. i'll post my question in case others know - any background on wanting to increase max number devices in ring?21:32
notmynamepdardeau-: torgomatic has thoughts, I think21:33
notmynamehe's standing at my desk instead of sitting at his own though21:33
*** ianbrown_ has joined #openstack-swift21:33
*** petertr7 is now known as petertr7_away21:33
*** ianbrown has quit IRC21:34
*** petertr7_away is now known as petertr721:34
notmynametsg: I think I have it working21:36
tsgnotmyname, :-) what did you have to do to get it working (I will include in my upcoming update to docs)21:37
notmynameso I hadn't installed liberasurecode[-dev] with apt. I only had the older version that had been installed with pyeclib. I installed it with apt and things seem to work now21:37
torgomaticpdardeau-: if it were me doing things, I'd just be a lazy bum and go from arrays of unsigned 16-bit ints to arrays of unsigned 32-bit ints, but that would more-or-less double the memory usage for Ring objects, and they're already big21:38
torgomaticI mean, that wouldn't stop me from being a lazy bum about it, but it might make me feel guilty21:38
tsgnotmyname, ok - if your older liberasurecode version was 1.0.7 or newer, it should still have worked - perhaps just an ldconfig issue21:40
notmynameyeah, that's what I think. pathing issues rather than code issues21:40
notmynametsg: thanks for looking, though21:40
tsgnotmyname, np - I wasn't able to repro in precise but got notes together for doc update which is good21:41
notmynamegood21:41
*** ianbrown has joined #openstack-swift21:41
tsgnotmyname, hoping Kevin will get to the pyeclib 1.2.0 reviews today and we can release so the doc/requirement updates can go up21:42
notmynamegreat21:42
pdardeau-torgomatic: thx. i'm wondering more about the situation of needing more than 64k devices21:43
notmynamemattoliverau: hurricanerix: the testr patch is now also spewing a lot of logging info (instead of just the test name). have you seen that in your runs? is it something with the os_testr thing?21:43
notmynamepdardeau-: I've been part of some conversations about potential clusters that would need more than 64k devices.21:44
notmynamegranted, I think there will be several other problems to solve well before we hit a limit on the number of individually addressable storage volumes21:44
notmynamebut still, saying "no limit" is better than saying "no limit. ok well maybe there is a limit but it's big and you probably won't hit it and we'll work around it before you get there"21:45
notmynamepdardeau-: I did some back of napkin math on the 64k limit though21:46
notmynamewith 65535 8TB drives and 3 replicas, you're at around 174PB of usable storage21:46
pdardeau-notmyname: good stuff! any ideas on how many object server nodes there would be for that number of drives?21:47
glangeis part of the problem that if you remove a device from a ring, there is a record of it still in the ring?21:47
notmynamedepends how many drive bays per U you have ;-)21:47
glangethere are 90 drive jbods out there21:48
notmynameglange: maybe. even if you had 2**15 drives, that's in the neighborhood of 80-90PB per storage policy21:48
openstackgerritGanesh Maharaj Mahalingam proposed openstack/swift: Modify functional tests to use testr  https://review.openstack.org/21420621:48
openstackgerritGanesh Maharaj Mahalingam proposed openstack/swift: Update SAIO setup for Ubuntu 14.04  https://review.openstack.org/25497021:48
notmynamewhich I can imagine some clusters getting close to these days ;-)21:48
gmmahanotmyname: didnt see the log spew.. let me try and run that again on my setup21:48
ahaleoh the 64k thing, im worried about that21:49
notmynameand of course, that's with 1 drive = 1 volume. you could raid strip a couple of drives too21:49
*** cdelatte has joined #openstack-swift21:49
notmynameahale: getting close on a few clusters?21:49
pdardeau-notmyname: glange: is there a rule of thumb for ratio of devices per server? notwithstanding number of drive bays in chassis? ;-)21:49
ahalemaybe, i was doing some testing but its super slow21:50
openstackgerritGanesh Maharaj Mahalingam proposed openstack/swift: Update SAIO setup for Ubuntu 14.04  https://review.openstack.org/25497021:50
notmynamepdardeau-: no. common deployment options are 12-bay, 24-bay, 32, 45, 60, 8421:52
gmmahanotmyname: for patch 214206 the log spew is the test name and the result? or are you seeing something else?21:52
patchbotgmmaha: https://review.openstack.org/#/c/214206/ - Modify functional tests to use testr21:52
notmynamepdardeau-: well, common ones would be 12, 24, and 60, probably21:53
ahalei wanted to add 40k devices, delete 20k, add 20k, delete 20k. see what happens. if the null dev id's are a problem then im more worried :)21:53
notmynamepdardeau-: but what does the number of server nodes matter? changing the dev_id field is independent of that21:54
notmynameI mean, it matters. just not for this problem21:54
pdardeau-notmyname: just wondering21:55
pdardeau-notmyname: trying to get a mental picture of what such a cluster would look like21:55
pdardeau-my god, it's full of spinning platters!21:55
notmynamelike this http://www.extremetech.com/wp-content/uploads/2013/12/Google-Datacenter.jpg21:56
notmynamenot like this http://farm3.static.flickr.com/2277/2180039413_f54b142ff4_o.jpg21:57
pdardeau-that must be one of those guiness book of world records things, or something21:58
openstackgerritGanesh Maharaj Mahalingam proposed openstack/swift: Update SAIO setup for Ubuntu 14.04  https://review.openstack.org/25497022:04
*** insanidade has quit IRC22:05
notmynamegmmaha: run the testr patch wtih SWIFT_TEST_IN_PROCESS=true SWIFT_TEST_IN_PROCESS_CONF_DIR=/tmp tox -efunc22:06
*** david-ly_ has joined #openstack-swift22:10
* gmmaha goes to try that22:11
*** CaioBrentano has quit IRC22:12
*** david-lyle has quit IRC22:13
*** david-ly_ is now known as david-lyle22:13
gmmahanotmyname: by mistake i rebased patch 214206 and there is now a new version. Is there a recommended way to fix this or is it ok to let it be and Me make sure i dont do it again22:16
patchbotgmmaha: https://review.openstack.org/#/c/214206/ - Modify functional tests to use testr22:16
notmynamegmmaha: are you seeing the log spew too?22:18
gmmahanotmyname: no sir.. just saw the test starting and now its all test_case ... result22:18
jlvillalnotmyname, log spew is a good thing :)  If you mean os-testr prints out lots of output.22:18
notmynamehmm22:18
gmmahado i have to make sure that swift isnt running and move the conf? I believe the vars you gave me took care of that22:18
jlvillalnotmyname, that is a change from testr. Happened to us in Ironic.22:18
jlvillalUseful for when there are errors to debug them.22:19
gmmahanotmyname: was the spew at the start of the test or the end summary?22:19
*** diazjf has quit IRC22:19
notmynameduring22:20
notmynamehere's an example of output for one test(!)22:20
notmynamehttps://gist.github.com/notmyname/432c1a4c072411b0bc6d22:20
gmmahaohh i did not see that22:21
gmmahalet me try and shutdown swift and check22:21
notmynamejlvillal: it's too much to be useful for debugging22:21
notmynamegmmaha: yeah. you don't actually need swift running at all.22:21
notmynamegmmaha: since the in-process tests do it...in-process :-)22:21
jlvillalnotmyname, By spew I thought you meant each test is printed if pass/fail22:21
notmynamejlvillal: I'm fine with that :-)22:22
notmynamebut 100 extra log lines per test? nope22:22
gmmahaOk. yeah its spitting it now22:22
jlvillalnotmyname, mtreinish is the expert on os-testr22:22
jlvillalHe may know why so many extra lines of output22:22
gmmaha:) i thought the IN_PROCESS will make sure that test doesnt check if its running..22:22
mtreinishjlvillal: I know nothing :)22:23
jlvillalmtreinish, Liar! :P22:23
notmynameseems like it's a change with os-testr that is no doing the same log/output capturing22:23
notmynamedoes os-testr capture stderr/stdout?22:23
mtreinishjlvillal: do you have an example for me to look at22:24
notmynamesee the gist above22:24
mtreinishnotmyname: you need to have a fixture collect stdout and attach it to the subunit stream22:24
mtreinishnotmyname: most openstack projects back that into the base test class22:24
jlvillalmtreinish, This patch: https://review.openstack.org/#/c/214206/22:24
jlvillalAnd gist above22:24
mtreinishnotmyname: if you don't do that the stdout will just passthrough22:25
notmynamea monad is just a moniod in the category of endofunctors. what's the problem?22:25
notmynameso what's a subunit stream fixture?22:25
notmynamean earlier patch set was using testr but not os-testr and this wasn't happening.22:26
mtreinishnotmyname: https://github.com/openstack/nova/blob/master/nova/test.py#L177-L187 is an example of how nova does it22:27
*** dustins is now known as dustins|gone22:27
jlvillalmtreinish, notmyname: Something like this? https://github.com/openstack/ironic/blob/master/ironic/tests/base.py#L90-L9722:27
mtreinishjlvillal: yeah that's a good example too22:28
gmmahanotmyname: also if you run swift on the machine and run tox -efunc there is no log spew..22:30
gmmahajust when we use the SWIFT_IN_PROCESS flags22:30
notmynamegmmaha: right! but that's because the servers are set up and doing the normal syslog logging22:30
*** changbl has quit IRC22:31
gmmahaok.. yeah, thats what i thought22:31
notmynameso in this case, they're set up to run in the test process and the logging isn't set up22:31
notmynameunfortunately, that's how the gate job runs. the in-process way.22:31
jlvillalmtreinish, As a note this is 'functional' testing if that makes any difference.22:31
gmmahaaah22:31
notmynameso I don't know if this affects that or not22:31
gmmahalet me try and add the log capturing into it22:31
notmynameok, thanks22:31
mtreinishjlvillal: I don't think that makes a difference :)22:32
*** jordanP has quit IRC22:35
*** yarkot has quit IRC22:39
*** petertr7 is now known as petertr7_away22:40
*** ianbrown_ has quit IRC22:55
*** klrmn has quit IRC22:56
*** ianbrown_ has joined #openstack-swift22:57
*** klrmn1 has quit IRC22:57
*** klrmn has joined #openstack-swift22:58
*** ianbrown has quit IRC23:02
openstackgerritCatherine Northcott proposed openstack/swift: Add support for storage policies to have more than one name  https://review.openstack.org/24197823:06
*** km has joined #openstack-swift23:06
*** km is now known as Guest5281423:06
*** chlong has quit IRC23:10
*** badari_ has joined #openstack-swift23:11
*** badari has quit IRC23:13
*** ho has joined #openstack-swift23:17
hogood morning!23:20
ZyricGood morning ho23:20
hoZyric: thanks!23:21
*** blmartin has quit IRC23:21
notmynamegmmaha: any luck with supressing all the logging?23:29
*** kei_yama has joined #openstack-swift23:32
gmmahanotmyname: somehow i managed to completely destroy my swift setup..23:32
gmmahajust revived it and workign on the patch now23:32
notmynamelol23:32
gmmahathe small change i did isnt working23:32
gmmahawill keep cehcking23:32
notmynameok23:33
*** eranrom has quit IRC23:33
*** eranrom has joined #openstack-swift23:33
notmynamegmmaha: just left a comment there with what I saw23:37
gmmahanotmyname: cool. my tests still arent working.. Will keep trying and drop an update here if i am completely stuck23:38
notmynameok23:38
notmynamethanks23:38
mattoliverauho: morning23:43
*** sgundur has joined #openstack-swift23:43
homattoliverau: morning!23:44
*** jerrygb has quit IRC23:55
*** jerrygb has joined #openstack-swift23:56
*** dewsday has joined #openstack-swift23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!