Tuesday, 2015-04-28

mattoliverauho: morning!00:00
homattoliverau: morning!00:00
*** openstack has joined #openstack-swift00:08
*** zhill has joined #openstack-swift00:24
*** minwoob has quit IRC00:27
openstackgerritKota Tsuyuzaki proposed openstack/swift: Don't show the query string for inner usage  https://review.openstack.org/17713200:30
*** kota_ has joined #openstack-swift00:31
kota_morning00:32
*** bkopilov has quit IRC00:32
hokota_: morning!00:36
*** annegentle has joined #openstack-swift00:36
mattoliveraukota_: morning!00:38
kota_mattoliverau: you, too!00:39
*** annegentle has quit IRC00:42
*** dmorita has joined #openstack-swift00:43
*** bkopilov has joined #openstack-swift00:50
openstackgerritKota Tsuyuzaki proposed openstack/swift: Don't show the query string for inner usage  https://review.openstack.org/17713200:53
*** fifieldt has joined #openstack-swift01:32
openstackgerritMatthew Oliver proposed openstack/swift: Add concurrent reads option to proxy  https://review.openstack.org/11771001:33
*** zhill has quit IRC01:35
*** asettle has quit IRC01:36
*** annegentle has joined #openstack-swift01:37
*** vinsh has quit IRC01:41
*** esker has joined #openstack-swift01:48
openstackgerritMatthew Oliver proposed openstack/swift: Add missing docstring in direct_client  https://review.openstack.org/17766401:49
openstackgerritMerged openstack/swift-bench: Fix bug when using --log-level option  https://review.openstack.org/17528502:00
mattoliverauLol, I like it when I go to make object-updater in my sharding POC shard aware.. only to find it looks like I've already done it!02:11
*** bkopilov has quit IRC02:11
mattoliverauI don't remember doing it, so will just assume it works and wont spend the brain cycles on it :P02:11
*** annegentle has quit IRC02:12
*** vinsh has joined #openstack-swift02:38
*** pberis has quit IRC02:49
*** NM has joined #openstack-swift02:58
*** silor has joined #openstack-swift03:02
*** NM has quit IRC03:05
claygmattoliverau: does that like happen to you a lot?  "go to make object-updater in my sharding POC shard aware.."03:19
*** NM has joined #openstack-swift03:28
mattoliverauclayg: all the time.. I just wish it would be more generic cause you tend to only need to do that once :p03:34
claygheh03:34
notmynamemattoliverau: sorry for waking you up so early :-)03:34
mattoliveraunotmyname: lol nps, its your work day.. I happened to be awake and then my phone buzzed :)03:35
mattoliverauSlowly working on an updated spec now. Between other tasks that is :)03:36
*** NM has quit IRC03:41
*** NM has joined #openstack-swift03:43
*** silor has quit IRC03:55
*** NM has quit IRC03:56
*** bkopilov has joined #openstack-swift04:07
*** kei_yama has quit IRC04:55
*** kei_yama has joined #openstack-swift04:58
*** silor has joined #openstack-swift05:20
*** silor has quit IRC05:25
*** silor has joined #openstack-swift05:25
*** vinsh has quit IRC06:01
*** vinsh has joined #openstack-swift06:01
*** SkyRocknRoll has joined #openstack-swift06:07
*** sandywalsh has quit IRC06:14
*** sandywalsh_ has joined #openstack-swift06:15
*** sandywalsh has joined #openstack-swift06:21
*** sandywalsh_ has quit IRC06:23
*** zul has joined #openstack-swift06:31
*** zul has quit IRC06:36
*** sandywalsh has quit IRC06:51
*** sandywalsh_ has joined #openstack-swift06:51
*** sandywalsh has joined #openstack-swift06:59
*** kota_ has quit IRC06:59
*** sandywalsh_ has quit IRC06:59
*** zul has joined #openstack-swift06:59
*** ppai has joined #openstack-swift07:03
*** mmcardle has joined #openstack-swift07:04
*** asettle has joined #openstack-swift07:08
*** mmcardle has left #openstack-swift07:12
*** geaaru has joined #openstack-swift07:31
*** pcaruana has joined #openstack-swift07:33
*** jistr has joined #openstack-swift07:55
*** openstackgerrit has quit IRC08:20
*** openstackgerrit has joined #openstack-swift08:21
*** esker has quit IRC08:21
*** resker has joined #openstack-swift08:21
*** acoles_away is now known as acoles08:37
*** jordanP has joined #openstack-swift08:39
openstackgerritAlistair Coles proposed openstack/swift: Make SSYNC receiver return a response when initial checks fail  https://review.openstack.org/17783608:47
openstackgerritHisashi Osanai proposed openstack/swift: Enable Object Replicator's failure count in recon  https://review.openstack.org/13834208:50
*** aix has joined #openstack-swift09:07
*** knl has joined #openstack-swift09:43
*** fifieldt has quit IRC09:58
openstackgerritAlistair Coles proposed openstack/swift: Fix intermittent container replicator test failure  https://review.openstack.org/17811609:58
*** knl has quit IRC10:01
*** xianghui has quit IRC10:04
*** xianghui has joined #openstack-swift10:04
*** xianghui has quit IRC10:07
*** zigo has quit IRC10:29
*** silor has quit IRC10:33
*** zigo_ has joined #openstack-swift10:33
*** zul has quit IRC10:45
*** aix has quit IRC10:47
*** jamielennox is now known as jamielennox|away10:57
*** knl has joined #openstack-swift10:58
*** aix has joined #openstack-swift10:58
*** early has quit IRC11:15
*** early has joined #openstack-swift11:18
*** cdelatte has quit IRC11:20
*** delattec has quit IRC11:20
*** knl has quit IRC11:20
*** silor has joined #openstack-swift11:27
*** silor1 has joined #openstack-swift11:29
*** silor has quit IRC11:29
*** jistr is now known as jistr|class11:34
*** aix has quit IRC11:34
*** silor1 has quit IRC11:34
*** zul has joined #openstack-swift11:35
*** silor has joined #openstack-swift11:36
*** spudly has quit IRC11:40
*** knl has joined #openstack-swift11:40
*** silor has quit IRC11:43
*** cdelatte has joined #openstack-swift11:45
*** delattec has joined #openstack-swift11:45
*** dmorita has quit IRC11:49
*** ho has quit IRC11:49
*** aix has joined #openstack-swift11:50
*** fanyaohong has joined #openstack-swift11:59
*** knl has quit IRC12:20
*** resker has quit IRC12:21
*** ppai has quit IRC12:25
*** mordred has quit IRC12:35
*** mordred has joined #openstack-swift12:35
*** proteusguy has quit IRC12:39
*** proteusguy has joined #openstack-swift12:42
*** bkopilov has quit IRC12:42
-openstackstatus- NOTICE: Gate is experiencing epic failures due to issues with mirrors, work is underway to mitigate and return to normal levels of sanity12:43
*** ChanServ changes topic to "Gate is experiencing epic failures due to issues with mirrors, work is underway to mitigate and return to normal levels of sanity"12:43
*** spudly has joined #openstack-swift12:46
*** jistr|class is now known as jistr13:01
*** zul has quit IRC13:02
*** swat30 has quit IRC13:02
*** haomaiwang has quit IRC13:02
*** NM has joined #openstack-swift13:04
*** swat30 has joined #openstack-swift13:07
*** raginbajin has quit IRC13:08
*** raginbajin has joined #openstack-swift13:09
*** annegentle has joined #openstack-swift13:13
*** zul has joined #openstack-swift13:13
*** minwoob has joined #openstack-swift13:15
*** km has quit IRC13:17
*** kei_yama has quit IRC13:20
*** krykowski has joined #openstack-swift13:26
*** esker has joined #openstack-swift13:35
*** esker has quit IRC13:40
minwoobacoles: Hi. Are there any near term plans for this?13:45
minwoobhttps://trello.com/c/Am97H6uj/152-ecdiskfilewriter-is-calling-hash-cleanup-listdir-twice13:45
acolesminwoob: yes i have a refactoring in mind that will clean that up - its not a bug, just unnecessary13:47
*** esker has joined #openstack-swift13:48
minwoobacoles: I see. I did some initial testing on it, but wanted to touch base with you before digging into it further.13:50
minwoobSome number of tests are failing, but it seems like an interesting problem that will help one learn a lot of the EC code base.13:54
acolesminwoob: which tests?13:54
acolesminwoob: do you mean tests failing after you made changes?13:55
minwoobacoles: Yes. After some changes were made.13:56
acolesoic13:56
minwoobDo you want me to post them here, or in a PM?13:56
acolesminwoob: pastebin would be best13:58
*** SkyRocknRoll has quit IRC14:02
minwoobThis is the latest snapshot of those tests: http://paste.ubuntu.com/10925729/14:04
minwoobThere are other snapshots as well (with different results), but that one is the result of what I see as the closest fix.14:06
*** knl has joined #openstack-swift14:07
acolesminwoob: i'd need to see what changes you made to be able to comment - maybe you want to push a review to gerrit, even though tests fail?14:09
*** fanyaohong has quit IRC14:09
minwoobacoles: Sure. I'd be happy to do that.14:14
acolesminwoob: ok. add me as a reviewer on gerrit when you do, or ping me here14:14
minwoobacoles: Do you mind if I work with it a bit further, before pushing it up for review?14:14
acolesminwoob: sure.14:15
minwoobacoles: Will do.14:15
acolesminwoob: so what I had in mind was to make both ECDiskFilewriter and DiskFileWriter call hash_cleanup_listdir during their commit() methods and remove the call from _finalize_put, so that its only called once for either class.14:17
* acoles is waving hands :)14:17
acolesminwoob: don't worry if you have some tests failing we can probably figure those out from a review.14:17
*** silor has joined #openstack-swift14:18
*** bkopilov has joined #openstack-swift14:19
minwoobacoles: I see. My solution was to have it in the _finalize_put method instead, but I can explore using commit() instead.14:22
*** wbhuber has joined #openstack-swift14:22
acolesminwoob: for ECDiskFileWriter is must be called after the .durable is written in commit()14:23
acolesminwoob: hmmm, but when we write a .meta file it must be called in finalize_put for both DiskFileWriter types, so I am revising my thoughts ;)14:23
acolesminwoob: maybe pass a flag to finalize_put to make the call to hash_cleanup_listdir conditional???14:24
acolesminwoob: ok lets see what you come up with in review !14:24
*** jrichli has joined #openstack-swift14:25
minwoobacoles: Okay. Will keep you updated!14:25
acolesminwoob: hmmm again, but of course commit is called for a .meta so my first thought might well work after all - sorry!14:25
minwoobacoles: Yes, that sounds like the way to go.14:26
acolesminwoob: ok let me know how it goes14:27
acolesminwoob: thank!14:27
minwoobacoles: Okay. Sounds good.14:29
*** notmyname has quit IRC14:31
*** notmyname has joined #openstack-swift14:34
*** ChanServ sets mode: +v notmyname14:34
*** annegentle has quit IRC14:35
acolesdfg_: ok, yes i see it used there in bulk.py too. thx14:36
*** annegentle has joined #openstack-swift14:46
*** annegentle has quit IRC14:54
*** breitz has quit IRC15:05
*** breitz has joined #openstack-swift15:06
*** zul has quit IRC15:06
*** panbalag has joined #openstack-swift15:26
*** panbalag has left #openstack-swift15:26
hurricanerixgoodmorning15:26
*** annc has joined #openstack-swift15:27
*** NM has quit IRC15:27
*** NM has joined #openstack-swift15:27
notmynamegood morning15:31
-openstackstatus- NOTICE: gerrit has been restarted to clear an issue with its event stream. any change events between 14:43-15:30 utc should be rechecked or have their approval votes reapplied to trigger jobs15:37
*** petertr7 has joined #openstack-swift15:43
pelusemorning15:43
*** bkopilov has quit IRC15:44
*** clds has quit IRC15:44
*** clds has joined #openstack-swift15:44
*** bkopilov has joined #openstack-swift15:45
acolespeluse: you're back! good break?15:45
*** annegentle has joined #openstack-swift15:46
peluseacoles, hi!  yeah, it was very relaxing - mostly :)15:49
*** ChanServ changes topic to "Review Dashboard: http://goo.gl/8IUcKl | Summary Dashboard: http://goo.gl/qHus5v | Summit scheduling: https://etherpad.openstack.org/p/liberty-swift-summit-topics | Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/"15:53
openstackgerritAlistair Coles proposed openstack/python-swiftclient: Add test for timeout being passed to keystone client  https://review.openstack.org/17825815:59
*** jistr has quit IRC16:02
*** annegentle has quit IRC16:03
*** mahatic has joined #openstack-swift16:12
*** pcaruana has quit IRC16:14
*** vinsh has quit IRC16:17
*** haomaiwa_ has joined #openstack-swift16:23
*** krykowski has quit IRC16:25
*** ajiang has joined #openstack-swift16:28
*** vinsh has joined #openstack-swift16:31
*** gyee has joined #openstack-swift16:51
*** ir2ivps8_ has quit IRC17:11
*** thumpba has joined #openstack-swift17:13
*** vinsh has quit IRC17:15
*** thumpba_ has joined #openstack-swift17:15
*** knl has quit IRC17:17
*** thumpba has quit IRC17:18
*** jordanP has quit IRC17:22
*** sandywalsh has quit IRC17:23
*** vinsh has joined #openstack-swift17:23
*** ir2ivps8_ has joined #openstack-swift17:34
*** aix has quit IRC17:36
claygmornint!17:53
*** annegentle has joined #openstack-swift17:54
*** itlinux has joined #openstack-swift17:55
*** ir2ivps8_ has quit IRC17:58
*** geaaru has quit IRC18:01
openstackgerritSamuel Merritt proposed openstack/swift-specs: Symlinks in Swift  https://review.openstack.org/17360918:02
claygtorgomatic: man you're all over that symlink spec18:03
claygtorgomatic: I wish we were somehow better at merging sooner18:04
*** jamielennox|away is now known as jamielennox18:04
*** annegentle has quit IRC18:09
*** annegentle has joined #openstack-swift18:10
*** thumpba_ has quit IRC18:23
*** ir2ivps8_ has joined #openstack-swift18:23
*** thumpba has joined #openstack-swift18:25
openstackgerritMinwoo Bae proposed openstack/swift: For both the ECDiskFileWriter and the DiskFileWriter, the hash_cleanup_listdir function should now be called in the commit() methods, instead of in _finalize_put() and _finalize_durable().  https://review.openstack.org/17831718:29
torgomaticclayg: yeah, we still have to figure out the versioning things18:33
* torgomatic has tried nothing and is all out of ideas18:33
*** minwoob_ has joined #openstack-swift18:44
*** minwoob has quit IRC18:45
*** welldannit has joined #openstack-swift18:48
*** minwoob_ has quit IRC18:57
*** minwoob has joined #openstack-swift18:58
*** jlk has quit IRC19:04
*** proteusguy has quit IRC19:04
*** jlk has joined #openstack-swift19:06
*** annc has quit IRC19:06
*** mahatic has quit IRC19:06
*** jlk has quit IRC19:07
*** jlk has joined #openstack-swift19:07
*** erlon has joined #openstack-swift19:07
*** delattec has quit IRC19:09
*** cdelatte has quit IRC19:10
*** itlinux has quit IRC19:17
*** proteusguy has joined #openstack-swift19:17
*** haomaiwa_ has quit IRC19:17
*** haomaiwa_ has joined #openstack-swift19:18
*** doxavore has joined #openstack-swift19:18
*** 20WAA5XQW has joined #openstack-swift19:20
*** 6JTAAZ692 has joined #openstack-swift19:20
*** 6JTAAZ692 has quit IRC19:41
*** 20WAA5XQW has quit IRC19:41
*** openstackstatus has quit IRC19:52
*** openstackstatus has joined #openstack-swift19:54
*** ChanServ sets mode: +v openstackstatus19:54
*** gyee has quit IRC20:00
*** cdelatte has joined #openstack-swift20:06
*** delattec has joined #openstack-swift20:06
*** acoles is now known as acoles_away20:09
*** silor has quit IRC20:26
*** knl has joined #openstack-swift20:27
*** itlinux has joined #openstack-swift20:28
*** knl has quit IRC20:29
*** delattec has quit IRC20:30
*** cdelatte has quit IRC20:31
*** rmcall has joined #openstack-swift20:33
*** wer has quit IRC20:35
*** wer has joined #openstack-swift20:35
*** zigo_ is now known as zigo20:50
*** NM has quit IRC20:52
*** knl has joined #openstack-swift20:55
*** knl has quit IRC20:56
*** wer has quit IRC20:58
*** gyee has joined #openstack-swift21:02
*** gyee has quit IRC21:11
*** wer has joined #openstack-swift21:11
*** gyee has joined #openstack-swift21:14
*** wer has quit IRC21:21
*** wer has joined #openstack-swift21:21
*** zhill has joined #openstack-swift21:24
*** zhill has quit IRC21:33
*** annegentle has quit IRC21:44
*** zhill has joined #openstack-swift21:47
*** annegentle has joined #openstack-swift21:48
*** mwheckmann has joined #openstack-swift21:49
*** ajiang has quit IRC21:52
mwheckmannhi all. question about storage policies and geo replication: Is it possible to have a different swift.conf in some regions of a single Swift cluster?21:52
mwheckmannwhat I mean by that is the following: Can different regions in the *same* cluster use a different storage policy as the default?21:53
mwheckmannfor example, region-1 has policy:0 as the default and region-2 has policy:1 as the default.21:54
mwheckmannLet's say I had 2 regions: EMEA and NCSA and 3 policies21:54
notmynamemwheckmann: interesting question21:55
mwheckmannpolicy:0 would only include disks in NCSA21:55
mwheckmannpolicy:1 would only include disks in EMEA21:55
mwheckmannand policy:2 would include disks in both EMEA & NCSA21:55
notmynamemwheckmann: assuming you had the same hash prefix/suffix in the 2 different swift.conf files, yes it would probably work21:55
notmynameI think it opens you up to some issues though21:56
mwheckmannI would want the default for nodes and proxies in NCSA to be policy:021:56
mwheckmannand in EMEA I would want the default to be policy:121:56
notmynameyou'd be much more likely to get containers that are created in two different policies and have to be reconciled21:56
mwheckmannnotmyname: That would be the only issue?21:57
notmynameI don't know. still thinking :-)21:57
mwheckmannok :)21:57
notmynamethat's never been the design of the system, so it's something new to think about21:58
mwheckmannotherwise, would there be an alternative for my use case? Or a best practice for what I want to achieve?21:58
mwheckmannWe don't see the need have the default policy span two regions. That could get expensive and cause bottlenecks quickly21:59
mwheckmannobviously having the default policy span devices in both regions combined w/ read/write affinities would solve the problem, but it's not what we want.22:00
mwheckmannWe want geo-replication to be an "opt-in" option for the users22:00
mattoliverauMorning22:01
notmynamehello mattoliverau22:03
notmynamemwheckmann: I think it would work. but you're going into uncharted waters :-)22:04
mwheckmannnomyname: Ok. To be continued then... Any recommended testing scenarios?22:05
mwheckmannany recommendation for a better way to do this (if any)?22:05
mwheckmannWas the system designed with the assumption that geo replication between regions should be the default policy?22:06
notmynameno, more like the default would be the default everywhere22:06
notmynameand that any proxy would handle any request the same way22:06
mwheckmannright.22:06
*** esker has quit IRC22:07
notmynameso you' d have to take extra care that a client talking to a proxy in region A wouldn't suddenly start talking to a proxy in region B22:07
mwheckmannok. That shouldn't be a problem.22:08
mwheckmannOK. I'll do additional research and I'll make sure that I get good test coverage and get back to you all with the results22:09
notmynamecool22:09
mwheckmannnotmyname: thanks for the help. It's much appreciated.22:10
notmynameI'd think you'd want to account for what happens when a proxy goes down. and how long it takes to reconcile if you create the same container in different regions with different policies22:10
mwheckmannok.22:10
mwheckmannI'll have to look at the doc about the container-reconciler again. I thought it was only an issue within the *same* policy if a split-brain situation occurs22:12
*** doxavore has quit IRC22:13
mwheckmannnotmyname: thanks again. I'm out for the rest of the day.22:13
notmynamea policy can only be set on a container when the container is created. but if you have a split brain, you could set different polices on the same container. in which case the reconciler will fix it (move the data to the one created first)22:13
mwheckmannright. It's the other way around. got it!22:14
mwheckmannIn my use case, I'm assuming that most accounts will write exclusively in one region, so it should be less of a problem right?22:15
notmynamemwheckmann: the worrisome words there are "assuming" and "most" and "less" ;-)22:16
mwheckmannindeed. It only has to happen once for it to be a problem.22:17
mwheckmannI'll give some more thought to it and run some tests.22:17
notmynamelike I said, you're in uncharted waters, so that's cool. the big difference is that you have proxies that do not behave the same way (violating "every proxy does the same thing")22:18
notmynameand It hink it just might work. but definitely should be tested :-)22:18
*** wer has quit IRC22:27
*** wbhuber has quit IRC22:40
*** jrichli has quit IRC22:41
*** annegentle has quit IRC22:46
*** thumpba has quit IRC22:47
*** fanyaohong has joined #openstack-swift22:55
zaitcev"The tempurl should tract the scope of the signed url" -- go go gadget dict.org23:00
*** vinsh_ has joined #openstack-swift23:14
*** vinsh has quit IRC23:14
claygnotmyname: do you still have the link to the design summit schedule for swift stuff?23:20
notmynamethe spreadsheet?23:20
claygnotmyname: yeah i thought it was just a google doc - showed which rooms swift had and on what days?23:20
notmynameyup here23:20
notmynamehttps://docs.google.com/spreadsheets/d/1VsFdRYGbX5eCde81XDV7TrPBfEC7cgtOFikruYmqbPY/edit#gid=56996312823:20
*** petertr7 has quit IRC23:21
zaitcevGate threw up with this - /opt/stack/new/devstack-gate/devstack-vm-gate.sh: line 411: cd: /opt/stack /old/devstack: No such file or directory23:24
zaitcev2 paths for cd?23:24
zaitcevI'm scared to consider what that means.23:24
zaitcevoh, wait. It's /opt/stack/old/devstack. Never mind, I'm an idiot.23:25
notmynamethat, I doubt :-)23:26
*** vinsh_ has quit IRC23:27
*** vinsh has joined #openstack-swift23:27
*** erlon has quit IRC23:31
*** vinsh has quit IRC23:32
*** vinsh has joined #openstack-swift23:52
*** esker has joined #openstack-swift23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!