| *** lyrrad has quit IRC | 00:02 | |
| *** delatte has quit IRC | 00:08 | |
| *** pgbridge has quit IRC | 00:11 | |
| openstackgerrit | Samuel Merritt proposed openstack/swift: Skip SLO-reconciling probe test when SLO is off https://review.openstack.org/312698 | 00:15 |
|---|---|---|
| *** chlong_ has joined #openstack-swift | 00:22 | |
| *** chlong_ has quit IRC | 00:22 | |
| kota_ | good morning | 00:23 |
| *** chlong_ has joined #openstack-swift | 00:27 | |
| *** chlong_ has quit IRC | 00:28 | |
| mattoliverau | kota_: morning | 00:29 |
| kota_ | mattoliverau: morning, I'm reading the irc log looks like a bunch of conversation yesterday. | 00:32 |
| kota_ | tdasilva: great work for pyeclib/liberasurecode migration, I will look at, thanks! | 00:40 |
| *** thumpba has joined #openstack-swift | 00:48 | |
| hosanai | good morning | 00:51 |
| kota_ | hosanai: morning | 00:53 |
| hosanai | kota_: morning! | 00:53 |
| hosanai | clayg: I fixed vsaio (with keystone) in my repo. The content of the fix is removed last two commits and used stable/liberty branch of keystone. | 00:53 |
| mattoliverau | hosanai: morning | 00:56 |
| hosanai | mattoliverau: morning! | 00:56 |
| *** ozialien10 has quit IRC | 00:59 | |
| tdasilva | kota_: thanks, I have to push a some fixes | 01:01 |
| *** sgundur- has left #openstack-swift | 01:01 | |
| kota_ | k | 01:02 |
| tdasilva | can anybody figure out why patch 311219 is not merging? | 01:03 |
| patchbot | tdasilva: https://review.openstack.org/#/c/311219/ - swift - Remove ThreadPool class | 01:03 |
| kota_ | tdasilva: looks like patch 311219 is waiting the previous depended patch landed. | 01:10 |
| patchbot | kota_: https://review.openstack.org/#/c/311219/ - swift - Remove ThreadPool class | 01:10 |
| kota_ | but patch 310864 (the previous patch) looks stalling at zuul verification. | 01:11 |
| patchbot | kota_: https://review.openstack.org/#/c/310864/ - swift - Remove threads_per_disk setting | 01:11 |
| kota_ | not sure | 01:11 |
| *** tqtran has quit IRC | 01:12 | |
| tdasilva | kota_: duh, I'm still not used to the new gerrit gui when it comes to dependencies, older gui was much easier to see | 01:13 |
| tdasilva | kota_: thanks | 01:13 |
| kota_ | tdasilva: agree, current gui is too complicated so I just downloaded the patch set via git review -d <n> and look at the git log history ;) | 01:14 |
| openstackgerrit | Merged openstack/swift: Remove threads_per_disk setting https://review.openstack.org/310864 | 01:36 |
| *** klamath has quit IRC | 01:37 | |
| openstackgerrit | Merged openstack/swift: Remove ThreadPool class https://review.openstack.org/311219 | 01:37 |
| kota_ | tdasilva: congrats :) | 01:37 |
| tdasilva | lol | 01:38 |
| *** klamath has joined #openstack-swift | 01:38 | |
| *** suyash has quit IRC | 01:43 | |
| *** dmorita has joined #openstack-swift | 01:54 | |
| *** natarej has quit IRC | 01:58 | |
| *** dmorita has quit IRC | 01:58 | |
| *** natarej has joined #openstack-swift | 01:59 | |
| *** NM has quit IRC | 02:13 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Fix EC ring validation at ring reload https://review.openstack.org/268993 | 02:15 |
| *** sgundur- has joined #openstack-swift | 02:16 | |
| *** sgundur- has left #openstack-swift | 02:16 | |
| openstackgerrit | Samuel Merritt proposed openstack/swift: Make eventlet.tpool's thread count configurable in object server https://review.openstack.org/289664 | 02:19 |
| *** gyee has quit IRC | 02:20 | |
| notmyname | #success threads per disk has been removed | 02:23 |
| openstackstatus | notmyname: Added success to Success page | 02:23 |
| *** klamath has quit IRC | 02:41 | |
| *** sheel has joined #openstack-swift | 02:53 | |
| *** dmorita has joined #openstack-swift | 02:54 | |
| *** sgundur- has joined #openstack-swift | 02:58 | |
| *** sgundur- has left #openstack-swift | 02:58 | |
| *** thumpba has quit IRC | 03:04 | |
| *** links has joined #openstack-swift | 03:21 | |
| *** ankur has joined #openstack-swift | 03:27 | |
| *** Jeffrey4l has joined #openstack-swift | 03:35 | |
| openstackgerrit | Merged openstack/swift: resurrect gholt blog posts on building consistent hashing ring https://review.openstack.org/314270 | 03:39 |
| *** dmorita has quit IRC | 03:44 | |
| *** dmorita has joined #openstack-swift | 03:56 | |
| *** dmorita has quit IRC | 04:01 | |
| *** pcaruana has joined #openstack-swift | 04:02 | |
| openstackgerrit | Tim Burke proposed openstack/swift: Allow concurrent bulk deletes https://review.openstack.org/311817 | 04:02 |
| jrichli | clayg sgundur- acoles eranrom : I was wrong that my config was the issue. I never have gotten a successful run on those 3 container-sync probetests using anything later than patch 270961 | 04:13 |
| patchbot | jrichli: https://review.openstack.org/#/c/270961/ - swift - Container-Sync to perform HEAD before PUT object o... (MERGED) | 04:13 |
| jrichli | acoles is not seeing the error, and I have run with the same contents as he has in my container-sync-realms.conf | 04:14 |
| jrichli | eranrom: the error I am seeing happens every time - not just intermittently | 04:15 |
| *** arch-nemesis has joined #openstack-swift | 04:16 | |
| openstackgerrit | Mahati Chamarthy proposed openstack/swift: crypto: Remove BYOK support https://review.openstack.org/314856 | 04:19 |
| *** esker has joined #openstack-swift | 04:24 | |
| *** dmorita has joined #openstack-swift | 04:27 | |
| *** esker has quit IRC | 04:28 | |
| *** dmorita has quit IRC | 04:30 | |
| *** arch-nemesis has quit IRC | 04:35 | |
| *** psachin has joined #openstack-swift | 04:42 | |
| *** takashi has joined #openstack-swift | 04:47 | |
| takashi | good afternoon :-) | 04:47 |
| kota_ | takashi: good afternoon | 04:49 |
| *** takashi has quit IRC | 04:52 | |
| *** takashi has joined #openstack-swift | 04:55 | |
| *** rcernin has joined #openstack-swift | 04:58 | |
| *** thumpba has joined #openstack-swift | 05:05 | |
| *** thumpba has quit IRC | 05:10 | |
| openstackgerrit | Mahati Chamarthy proposed openstack/swift: crypto: Override flag to fix probe tests https://review.openstack.org/314865 | 05:19 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Fix EC ring validation at ring reload https://review.openstack.org/268993 | 05:20 |
| *** ChubYann has quit IRC | 05:26 | |
| janonymous | pdardeau: Nice work on ring overview doc. | 05:30 |
| *** silor has joined #openstack-swift | 05:33 | |
| *** silor1 has joined #openstack-swift | 05:38 | |
| *** silor has quit IRC | 05:39 | |
| *** silor1 is now known as silor | 05:39 | |
| *** ankur has quit IRC | 06:07 | |
| mahatic | jrichli: could you post the 3 probe test errors you see with container-sync? Want to verify | 06:08 |
| *** itlinux has joined #openstack-swift | 06:18 | |
| eranrom | mahati | 06:22 |
| *** david-lyle has quit IRC | 06:23 | |
| *** david-lyle has joined #openstack-swift | 06:27 | |
| eranrom | mahatic, jrichli, acoles_: The tests that fail are in test_container_sync.py. Amongst them are: test_sync_with_post_as_copy, test_sync_with_stale_container_rows | 06:27 |
| eranrom | Basically what the test does is (1) create a source and target containers with the appropriate metadata for unidirectional replication | 06:28 |
| eranrom | (2) add some stuff to the source | 06:28 |
| eranrom | (3) have the sync daemon to work | 06:28 |
| eranrom | (4) validate that objects are replicated | 06:28 |
| mahatic | eranrom: thanks! are they 2 errors or 3? I also see this one: ERROR: test_sync_with_fast_post (test.probe.test_container_sync.TestContainerSync) | 06:29 |
| eranrom | mahatic: there are 3 but I need to run once more to find it. | 06:29 |
| *** jmccarthy has quit IRC | 06:29 | |
| *** jmccarthy has joined #openstack-swift | 06:30 | |
| eranrom | What happens is that replication fails, because on the target side, the container_sync middleware fails to retrieve the sync key (when using get_container_info), since getting the info always seem to fail on the first time. | 06:30 |
| *** dmorita has joined #openstack-swift | 06:31 | |
| *** mmcardle has joined #openstack-swift | 06:31 | |
| eranrom | The get_info code seem to populate the cache but fail to actually return the info. The second time get_info is called the cache is populated and everything works fine. | 06:32 |
| eranrom | Thus, a workaround would be e.g. to do head on the target container as soon as it is created with the sync key metadata. | 06:32 |
| eranrom | But changing the test accordingly would be cheating :0) | 06:32 |
| eranrom | This is why I posted yesterday a question on the get_info code. | 06:33 |
| eranrom | I will try to dig deeper into this. | 06:33 |
| *** ppai has joined #openstack-swift | 06:34 | |
| mahatic | eranrom: thanks for the details, that helps | 06:34 |
| eranrom | mahatic: thank you for looking. | 06:34 |
| *** dmorita has quit IRC | 06:35 | |
| hosanai | eranrom: interesting. it | 06:47 |
| hosanai | eranrom: sorry. wait a minute | 06:48 |
| *** zaitcev has quit IRC | 06:50 | |
| *** thumpba has joined #openstack-swift | 07:06 | |
| -openstackstatus- NOTICE: pip 8.1.2 broke our local python mirror, some jobs will fail with "No matching distribution found". We're investigating. Do not "recheck" until the issue is solved | 07:08 | |
| *** ChanServ changes topic to "pip 8.1.2 broke our local python mirror, some jobs will fail with "No matching distribution found". We're investigating. Do not "recheck" until the issue is solved" | 07:08 | |
| *** rcernin has quit IRC | 07:09 | |
| *** rcernin has joined #openstack-swift | 07:10 | |
| hosanai | eranrom: i try to understand the root of code. Is it passed L575 at versioned_writes.py? here make a req with env.copy() at L551 in versioned_writes.py then uses it at L575 | 07:11 |
| *** thumpba has quit IRC | 07:11 | |
| hosanai | eranrom: if above root is correct, i would like to try env instead of env.copy() at L551 to narrow down the problem. | 07:11 |
| hosanai | eranrom: btw i have an idea for workaround; when getting container info fails, tries to get it again in base.py | 07:12 |
| *** SkyRocknRoll has joined #openstack-swift | 07:12 | |
| eranrom | hosanai: regarding the workaround, I tried it (in get_info I called the get from cache once more) but then got some strange unit tests failures | 07:14 |
| eranrom | hosanai: The lines above (e.g. L551) are in basy.py or swob.py? | 07:14 |
| hosanai | eranrom: L551 in versioned_writes.py | 07:15 |
| hosanai | eranrom: https://github.com/openstack/swift/blob/master/swift/common/middleware/versioned_writes.py#L551 | 07:15 |
| *** rledisez has joined #openstack-swift | 07:16 | |
| eranrom | hosanai: gotcha, you are saying that this env.copy breaks the assumption made by the get_info code (regarding GETorHEAD_base updating the env)? | 07:17 |
| hosanai | eranrom: yep, i don't have container-sync env now (i couldn't reproduce it) so i guess the code is related to this issue. | 07:21 |
| eranrom | hosanai,: wow good catch!!! Can I just try to remove it from the pipeline? | 07:22 |
| eranrom | would it break anything? | 07:22 |
| hosanai | eranrom: my idea is use "req = Request(env)" instead of "req = Request(env.copy())" for investigation... | 07:25 |
| *** daemontool_ has joined #openstack-swift | 07:26 | |
| eranrom | hosanai: right, but I thought that there might be a ggod reason why it has a copy so I asked about removing it from the pipeline. Anyway, the probtests work when the versioned_writes are removed from the pipeline!!! | 07:27 |
| eranrom | Let me try with your suggestion now. | 07:28 |
| *** klrmn has quit IRC | 07:31 | |
| eranrom | hosanai,: your suggestion also works! That was a superfast root cause analysis!!! | 07:31 |
| *** jmccarthy1 has joined #openstack-swift | 07:46 | |
| eranrom | acoles_,:Any chance that the container sync probe tests work for you due to not having versioned_writes in the pipeline? | 07:47 |
| *** chlong has quit IRC | 07:49 | |
| kota_ | eranrom: just small note, env is a dictionary for HTTP environment variable and if it's updated, the update will be remaining anywhere. | 07:50 |
| kota_ | eronrom: that's reason why env.copy() is needed to save the original environment dictionary. | 07:51 |
| openstackgerrit | Merged openstack/python-swiftclient: Check responses when retrying bodies https://review.openstack.org/269252 | 07:52 |
| kota_ | eranrom: for safety, trying to use make_subrequest might be a solution to keep the env safe | 07:52 |
| hosanai | eranrom: thanks for the trial :-) it's not reach to the root cause. | 07:59 |
| *** ntt has quit IRC | 08:01 | |
| *** jordanP has joined #openstack-swift | 08:01 | |
| *** SkyRocknRoll has quit IRC | 08:03 | |
| hosanai | eranrom: the approach works, the env.copy is not necessary for L575 so move "req = Request(env.copy())" to L580 . | 08:03 |
| hosanai | eranrom: only for COPY req | 08:04 |
| eranrom | gotcha. So why are you saying this is not the root cause :-) ? | 08:05 |
| hosanai | eranrom: we just know the root of problem but don't know the reason | 08:06 |
| *** mingdang1 has joined #openstack-swift | 08:06 | |
| eranrom | you mean the reason why the copy middleware does an env.copy? | 08:06 |
| eranrom | agree | 08:06 |
| kota_ | hmm...probably I am missing some key points, quit out from the conversation. | 08:06 |
| eranrom | kota_,no worries, here is the point: from some reason the versioned_writes middleware creates a new request with an enc.copy for each request that passes through it. Thus, any modifications done on the env after versioned_writes is not seen by any middleware before it. | 08:07 |
| *** ntt has joined #openstack-swift | 08:08 | |
| *** mingdang1 has quit IRC | 08:08 | |
| hosanai | eranrom: i'm in meeting now so i will check the conversation later, sorry... | 08:08 |
| *** mingdang1 has joined #openstack-swift | 08:08 | |
| *** dmk0202 has joined #openstack-swift | 08:08 | |
| eranrom | hosanai,: sure, thatks for the help! | 08:08 |
| eranrom | kota_,does this make sense? Do you think that make_subrequest bypasses this? | 08:09 |
| km | はい。残業なしの方向で進めたい、その方向でまずは見積もる | 08:10 |
| kota_ | not sure but make_subrequest is for keeping some important variables. | 08:10 |
| kota_ | km: wooa? | 08:10 |
| kota_ | well, wait, I don't have clear memory for it. | 08:11 |
| km | sorry miss | 08:12 |
| *** ntt has quit IRC | 08:12 | |
| kota_ | eranrom: maybe, i'm still in fog. do you mean the env updated in the subrequest will be missing at the middleware located in left of versioned_writes, right? | 08:13 |
| *** km has left #openstack-swift | 08:14 | |
| eranrom | kota_,right. more precisely any updates made to the right of versioned writes will not bee seen | 08:14 |
| *** asettle has joined #openstack-swift | 08:14 | |
| eranrom | kota_,: that is any updates to the env. | 08:15 |
| kota_ | interesting, and sounds reasnable. that is because it will prevent any information leak to left side. | 08:15 |
| kota_ | and versioned_writes should be at the most right side in the pipeline. | 08:16 |
| *** SkyRocknRoll has joined #openstack-swift | 08:16 | |
| kota_ | so the proxy-app is only a man can update the env. | 08:16 |
| kota_ | ah, except proxy-logging and proxy-app | 08:17 |
| kota_ | eranrom: so what's env value do you expect to pass to left side? | 08:17 |
| *** daemontool_ has quit IRC | 08:18 | |
| eranrom | kota_,: so in general many 'middlewares' do a get_info_(account/container). the get_info code looks at the cache, and if the info is not in cache, it generates a head request that reaches the proxy app. Now, the proxy/controller/base.py GETorHEAD_base populates the cache but also puts the required info on the env | 08:19 |
| eranrom | kota_,: and so the env is where the get_info code (called by the middleware) is looking for the info. | 08:20 |
| *** daemontool_ has joined #openstack-swift | 08:20 | |
| eranrom | kota_,: the result is that any first call to get_info would fail. On the seconde call it will find it in cache. | 08:20 |
| kota_ | yes | 08:21 |
| kota_ | (the first call try env info, and then try memcache and then GETorHEAD and cache it into memcache and env) | 08:22 |
| eranrom | kota_,: I think so yes | 08:25 |
| kota_ | and the env will be passed through from left to right. | 08:26 |
| *** Shashikant86 has joined #openstack-swift | 08:26 | |
| *** cbartz has joined #openstack-swift | 08:26 | |
| kota_ | so I'm wondering why there is a problem at the left middleware if a middleware in right update the account/container info that will be added only when the left didn't do get_info. | 08:28 |
| kota_ | the left middlware does something (refer the info) when the response comming back from right middleware? | 08:29 |
| eranrom | kota_, exactly. | 08:29 |
| kota_ | ah, the anlysis is still WIP? | 08:30 |
| kota_ | perhaps | 08:31 |
| eranrom | get_info generates a head request with some env. this is done to the left of versioned_writes. then versioned writes copies the env, then GETorHEAD updates the copied env, and so the code in get info that is after the head does not see that change | 08:31 |
| eranrom | well, this is something that hosanai has found. He also made the point that versioned_writes does not need to do the copy for every request that passes it, but rather only for request whose method is COPY (and perhaps also delete and put) | 08:32 |
| *** Shashikant86 has quit IRC | 08:33 | |
| eranrom | two initial test where I (1) removed the versioned_writes from the pipeline and (2) changes the env.copy to env seem to soleve the problem | 08:33 |
| eranrom | kota_,: so this is where we stand | 08:33 |
| kota_ | gocha | 08:34 |
| kota_ | but no answer right now, but sounds like it may appear when merging "decouple versioned writes from COPY" | 08:36 |
| kota_ | that patch changes the handling COPY to GET/PUT and it touches env dictionary. | 08:37 |
| eranrom | need to look at the patch | 08:38 |
| *** takashi has quit IRC | 08:38 | |
| *** jistr has joined #openstack-swift | 08:38 | |
| *** Shashikant86 has joined #openstack-swift | 08:44 | |
| *** Shashikant86 has quit IRC | 08:45 | |
| hosanai | eranrom, kota_: i'm back. i have been searching patches which are related to this. could you double check/try patch 280977? | 08:50 |
| patchbot | hosanai: https://review.openstack.org/#/c/280977/ - swift - Make info caching work across subrequests | 08:50 |
| *** kei_yama has quit IRC | 08:53 | |
| *** ndk has quit IRC | 08:54 | |
| *** ndk_ has joined #openstack-swift | 08:55 | |
| hosanai | eranrom, kota_: could you also double-check patch 280978? | 08:55 |
| patchbot | hosanai: https://review.openstack.org/#/c/280978/ - swift - Fix up get_account_info and get_container_info | 08:55 |
| eranrom | hosanai: will review thanks! | 08:56 |
| kota_ | hosanai: looks like what we might need (not sure), thanks. | 08:59 |
| kota_ | hosanai: I need more time to look at the problem and solution. | 09:00 |
| *** thumpba has joined #openstack-swift | 09:10 | |
| *** acoles_ is now known as acoles | 09:14 | |
| *** thumpba has quit IRC | 09:15 | |
| *** tesseract has joined #openstack-swift | 09:20 | |
| *** daemontool_ has quit IRC | 09:21 | |
| acoles | eranrom: i have versioned_writes in pipeline, but even if it is not explicitly there it *should* be auto-inserted - take a look at your proxy log at startup and it will have a log of the pipeline (grep "Pipeline is" ) | 09:22 |
| *** daemontool_ has joined #openstack-swift | 09:22 | |
| kota_ | eranrom: I'm not sure but I'm realizing if we can doubt the memcache setting? | 09:27 |
| kota_ | scrolling back to the log, you described as the first call failed, and second succeeded. | 09:27 |
| acoles | eranrom: but...I had allow_versioned_writes=False, when True I see the tests fail :/ which makes sense because when you remove versioned_writes from the pipeline, it will be auto-inserted but with its default settings which is allow_versioned_writes=false | 09:28 |
| *** takashi has joined #openstack-swift | 09:28 | |
| acoles | eranrom: and the copied request environ is not used when allow_versioned_writes is false | 09:28 |
| kota_ | if memcached server halted with some reasons, memocache.get in get_info failed but the second will succeed because the HEAD result will be in env. | 09:29 |
| kota_ | not sure though. | 09:29 |
| kota_ | need to look at/and test. | 09:30 |
| *** takashi has quit IRC | 09:30 | |
| kota_ | eranrom: sorry I'm running out of time, keep it in my heads, hope to talk later. | 09:32 |
| kota_ | see you. | 09:32 |
| *** daemontool_ has quit IRC | 09:40 | |
| *** daemontool_ has joined #openstack-swift | 09:40 | |
| *** daemontool_ has quit IRC | 09:40 | |
| *** daemontool_ has joined #openstack-swift | 09:40 | |
| eranrom | acoles, kota_: alright, so it seems like hosanai nailed the problem. I need to go offline now. talk to you later. Thanks! | 09:42 |
| *** sanchitmalhotra has joined #openstack-swift | 09:43 | |
| *** sanchitmalhotra1 has quit IRC | 09:43 | |
| acoles | eranrom: kota_ i just made a small patch that fixes it, will push to gerrit fyi | 09:43 |
| mahatic | eranrom: glad to know the root cause. hosanai: nice catch! | 09:49 |
| *** sanchitmalhotra1 has joined #openstack-swift | 09:54 | |
| *** sanchitmalhotra has quit IRC | 09:56 | |
| hosanai | eranrom: how to re-produce the problem? just execute probe test? i still couldn't re-produce it... | 10:04 |
| hosanai | acoles: oh, you already made a patch for it. great! | 10:04 |
| acoles | hosanai: kota_ eranrom jrichli i didn't see a bug report so made one https://bugs.launchpad.net/swift/+bug/1580528 | 10:05 |
| openstack | Launchpad bug 1580528 in OpenStack Object Storage (swift) "container sync probe tests fail when allow_versioned_writes is true" [Undecided,New] | 10:05 |
| acoles | you may like to add your diagnosis of the cause ^^ | 10:07 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Use original environ in versioned_writes container handling (WIP) https://review.openstack.org/314947 | 10:07 |
| *** daemontool_ has quit IRC | 10:08 | |
| *** daemontool_ has joined #openstack-swift | 10:08 | |
| *** mingdang1 has quit IRC | 10:10 | |
| acoles | hosanai: kota_ eranrom jrichli that is a quick fix ^^, I don't have time yet to know if it is a complete fix, or the best fix, and the patch could do with a unit test, so it's WIP and I am more than happy for someone else to take it on as co-author or squash or copy | 10:10 |
| *** mingdang1 has joined #openstack-swift | 10:10 | |
| *** stevemar has quit IRC | 10:13 | |
| *** stevemar has joined #openstack-swift | 10:15 | |
| hosanai | with this bug staticweb with versioned_write might not working... i have to leave now see you later. | 10:15 |
| *** hosanai has quit IRC | 10:15 | |
| *** natarej has quit IRC | 10:16 | |
| *** natarej has joined #openstack-swift | 10:17 | |
| *** daemontool_ has quit IRC | 10:17 | |
| *** daemontool has joined #openstack-swift | 10:18 | |
| *** naseer036 has joined #openstack-swift | 10:32 | |
| *** daemontool has quit IRC | 10:48 | |
| *** naseer036 has quit IRC | 10:49 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - don't break conditional GETs on unencrypted objects https://review.openstack.org/314740 | 10:49 |
| *** jistr is now known as jistr|mtg | 11:00 | |
| openstackgerrit | Merged openstack/swift: crypto - add functional test to verify container listing detail https://review.openstack.org/307121 | 11:34 |
| *** jamie_h has quit IRC | 11:46 | |
| *** ppai has quit IRC | 11:50 | |
| *** ChanServ changes topic to "Let's talk! We're nice | Review Dashboard: https://goo.gl/mtEv1C | Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/ | Meetings: https://wiki.openstack.org/wiki/Meetings/Swift | Community Dashboard: http://not.mn/swift/swift_community_dashboard.html" | 11:59 | |
| -openstackstatus- NOTICE: We have a workaround for our mirrors to attempt to translate package names if a match isn't immediately obvious. A more complete fix is yet to come. It is now safe to 'recheck' any jobs that failed due to "No matching distribution found". Please join #openstack-infra if you discover more problems. | 11:59 | |
| *** ppai has joined #openstack-swift | 12:01 | |
| *** jistr|mtg is now known as jistr | 12:11 | |
| *** mmcardle has quit IRC | 12:16 | |
| *** asettle has quit IRC | 12:20 | |
| *** ppai has quit IRC | 12:23 | |
| *** marcusvrn_ has joined #openstack-swift | 12:24 | |
| *** mmcardle has joined #openstack-swift | 12:27 | |
| *** openstackgerrit has quit IRC | 12:33 | |
| *** openstackgerrit has joined #openstack-swift | 12:33 | |
| *** ppai has joined #openstack-swift | 12:36 | |
| *** jamielennox is now known as jamielennox|away | 12:45 | |
| *** ppai has quit IRC | 12:46 | |
| *** asettle has joined #openstack-swift | 12:47 | |
| *** daemontool has joined #openstack-swift | 12:51 | |
| *** links has quit IRC | 12:52 | |
| *** daemontool has quit IRC | 12:53 | |
| *** jamielennox|away is now known as jamielennox | 12:53 | |
| *** daemontool has joined #openstack-swift | 12:53 | |
| *** jamie_h has joined #openstack-swift | 12:56 | |
| *** takashi has joined #openstack-swift | 12:57 | |
| *** pauloewerton has joined #openstack-swift | 12:58 | |
| *** rickyrem has joined #openstack-swift | 13:01 | |
| kota_ | hmm.... looks like the reason why versioned_writes does env.copy() is that versioned_writes can mangle the env['PATH_INFO'] when the method is COPY? | 13:16 |
| kota_ | if it's true, req instance in __call__ method should use the original (not copy) and then do make_subrequest if another path (e.g. for get_source or for put to versions container) | 13:18 |
| kota_ | just dumping what I thought. | 13:18 |
| openstackgerrit | Monty Taylor proposed openstack/swift: Update paths to point to git.openstack.org https://review.openstack.org/315028 | 13:21 |
| openstackgerrit | Monty Taylor proposed openstack/swift: Add godep manifest file https://review.openstack.org/315029 | 13:21 |
| *** BAKfr has quit IRC | 13:26 | |
| *** BAKfr has joined #openstack-swift | 13:28 | |
| *** dmorita has joined #openstack-swift | 13:32 | |
| kota_ | ah, the reason why object_request is not affected the env.copy() problem is it will call self.app(env, start_response) directly w/o env copy. | 13:33 |
| *** cdelatte has joined #openstack-swift | 13:33 | |
| kota_ | but it is still complicated with the code. hm.... | 13:33 |
| *** daemontool has quit IRC | 13:34 | |
| *** dmorita has quit IRC | 13:36 | |
| *** klamath has joined #openstack-swift | 13:38 | |
| *** cdelatte has quit IRC | 13:38 | |
| *** klamath has quit IRC | 13:38 | |
| *** chlong has joined #openstack-swift | 13:38 | |
| *** klamath has joined #openstack-swift | 13:38 | |
| openstackgerrit | Nandini Tata proposed openstack/swift: add Tempest Plugin for functional tests https://review.openstack.org/313676 | 13:52 |
| *** SkyRocknRoll has quit IRC | 13:53 | |
| *** ametts has joined #openstack-swift | 13:56 | |
| *** thumpba has joined #openstack-swift | 13:57 | |
| *** daemontool has joined #openstack-swift | 13:58 | |
| *** links has joined #openstack-swift | 14:01 | |
| *** _JZ_ has joined #openstack-swift | 14:01 | |
| pdardeau | janonymous: thx | 14:12 |
| *** sgundur- has joined #openstack-swift | 14:13 | |
| *** daemontool has quit IRC | 14:13 | |
| *** mingdang1 has quit IRC | 14:20 | |
| pdardeau | good morning | 14:22 |
| kota_ | pdardeau: good morning | 14:23 |
| pdardeau | kota_: o/ | 14:23 |
| pdardeau | kota_: surprised to see you online at this hour! | 14:23 |
| kota_ | pdardeau: :) usually I'm here at wednesday night. | 14:24 |
| *** daemontool has joined #openstack-swift | 14:31 | |
| *** hezhiqiang has joined #openstack-swift | 14:36 | |
| *** lowecase has joined #openstack-swift | 14:37 | |
| acoles | kota_: yeah i don't think that env.copy is required for *all* requests, patch 314947 | 14:38 |
| patchbot | acoles: https://review.openstack.org/#/c/314947/ - swift - Use original environ in versioned_writes container... | 14:38 |
| acoles | kota_: maybe that patch ^^ should use the original env for some obj requests too?? | 14:39 |
| acoles | jrichli: ^^ | 14:39 |
| *** takashi has quit IRC | 14:39 | |
| kota_ | acoles: probably the reason why object request is not affected is the object_request can take the original env like self.object_request(*snip*)(env, start_response) | 14:41 |
| kota_ | i.e. object_request will return self.app and it will take the original env (not copied) | 14:42 |
| kota_ | and then the self.app call is probably for the original request (e.g. PUT object, in my looking). | 14:43 |
| acoles | kota_: you mean line 583? | 14:44 |
| kota_ | yes | 14:44 |
| kota_ | it seems all requests in object_request method are subrequests | 14:45 |
| *** daemontool has quit IRC | 14:45 | |
| kota_ | i mean just only for PUT object case though. | 14:45 |
| kota_ | (COPY too but I didn't check deeply for DELETE object case) | 14:46 |
| acoles | kota_: yes, the object put path always returns self.app, then self.app(env, start_response) is called like you say | 14:49 |
| acoles | the delete path is less obvious. | 14:49 |
| *** pgbridge has joined #openstack-swift | 14:49 | |
| acoles | I think the delete path may end up with the copied environ being used in the response | 14:50 |
| kota_ | acoles: maybe true, in my memory, the delete case's response should be from DELETE versioned_container/previous_version_object | 14:51 |
| *** lowecase has quit IRC | 14:52 | |
| *** lowecase has joined #openstack-swift | 14:52 | |
| kota_ | that HTTP request path info should be different from the original env, so I think it would be handled as subrequest with copied env. | 14:53 |
| kota_ | not yet look around there. | 14:53 |
| acoles | kota_: anyway I agree that the env should not be copied so early in the __call__ | 14:53 |
| *** links has quit IRC | 14:54 | |
| kota_ | acoles: exactly, imo, we should copy (i like to use make_env or make_subrequest) only when it's obviously subrequset different from the original request path. | 14:55 |
| *** daemontool has joined #openstack-swift | 14:55 | |
| kota_ | ah...today is becomming tommorow, pend this as tommorow's task | 14:57 |
| *** lowecase has quit IRC | 14:57 | |
| kota_ | acoles: sorry, will look at your patch tommorow. thanks. | 14:57 |
| *** lowecase has joined #openstack-swift | 14:58 | |
| * tdasilva is trying to come up to speed on this discussion | 14:58 | |
| *** pgbridge has quit IRC | 15:01 | |
| acoles | tdasilva: summary - versioned_writes makes a copy of the request environ, which means that get_info updates the info in the copied environ, which seems to cause container_sync probe tests to fail because a get_info doesn't return container info, although I'm not sure why the info isn't found in memcache. | 15:02 |
| tdasilva | acoles: mmm..interesting...while looking at the bug, it seems that the problem only shows up if user explicitly sets allow_versioned_writes to True ??? | 15:03 |
| tdasilva | weird | 15:03 |
| *** lowecase has quit IRC | 15:03 | |
| acoles | tdasilva: yes because otherwise versioned_writes does not actually use the copied env for container requests | 15:04 |
| tdasilva | btw...if I remember correctly, with copy middleware, we get rid of the env.copy all together | 15:04 |
| tdasilva | because copy requests never reach versioned_writes middleware | 15:04 |
| acoles | of course! | 15:05 |
| tdasilva | https://review.openstack.org/#/c/156923/40/swift/common/middleware/versioned_writes.py@532 | 15:05 |
| patchbot | tdasilva: patch 156923 - swift - Refactor server side copy as middleware | 15:05 |
| tdasilva | acoles: would that help? | 15:06 |
| *** arch-nemesis has joined #openstack-swift | 15:12 | |
| *** jistr has quit IRC | 15:13 | |
| *** diogogmt has joined #openstack-swift | 15:14 | |
| *** esker has joined #openstack-swift | 15:17 | |
| *** pgbridge has joined #openstack-swift | 15:20 | |
| acoles | tdasilva: i think it would! kota_ eranrom jrichli ^^ | 15:20 |
| acoles | copy middleware will save us :) | 15:20 |
| tdasilva | acoles: re copy middleware, there's already a +2 from timburke but mattoliverau left a -1 that I think is worth making the changes he suggested, especially the one around renaming the middleware to just copy | 15:21 |
| tdasilva | it would be a small enough change, maybe we can still land this patch this week :| ??? | 15:22 |
| admin6 | Hi there. Sorry to disturb you with some stupid questions but since I’ve upgraded my swift servers to v2.7, I have a lot of messages "STDERR: UnicodeDecodeError" in the object-replicator logs that I didnt’ saw before with v2.5. These logs appears when rebalancing a replica3 object ring (more logs in http://pastebin.com/nUprVRrk), have you suggestions regarding that encondind/decoding problem/warning ? | 15:23 |
| jrichli | tdasilva: let's hope timburke is ok with the name change :-) | 15:25 |
| tdasilva | jrichli: yeah, I'd like to hear from timburke first to make sure he is ok with it | 15:26 |
| *** esker has quit IRC | 15:26 | |
| *** esker has joined #openstack-swift | 15:27 | |
| *** klrmn has joined #openstack-swift | 15:27 | |
| acoles | tdasilva: yes, I agree about the name, I just never had the courage to say so - +1 for mattoliverau | 15:28 |
| acoles | also agree, check with timburke | 15:28 |
| acoles | tdasilva: just leaving a couple of replies to mattoliverau's other comments | 15:28 |
| tdasilva | acoles: cool! thanks | 15:29 |
| acoles | tdasilva: how about just 'cp' | 15:29 |
| acoles | JK | 15:29 |
| tdasilva | hehe | 15:29 |
| jrichli | :-) | 15:30 |
| *** jistr has joined #openstack-swift | 15:31 | |
| *** diogogmt has quit IRC | 15:33 | |
| *** jistr has quit IRC | 15:33 | |
| *** jistr has joined #openstack-swift | 15:33 | |
| acoles | tdasilva: the doc dup could be cleaned up in a follow up but imho if the name is changing it should change before merging | 15:34 |
| tdasilva | acoles: agreed | 15:35 |
| *** dmorita has joined #openstack-swift | 15:42 | |
| *** dmorita has quit IRC | 15:43 | |
| sgundur- | jrichli: hi, sorry had missed the conv yesterday, now went through the discussion in the logs! | 15:44 |
| jrichli | sgundur- : np. looks like there is something to fix after all :-) | 15:45 |
| sgundur- | jrichli: so now my probetests should pass on master + 314947 ? | 15:47 |
| *** rcernin has quit IRC | 15:47 | |
| jrichli | sgundur- : yes. I haven't tested that out yet, but will be looking at that patch later today. | 15:48 |
| sgundur- | jrichli: great, I ll try it out now | 15:48 |
| openstackgerrit | David Goetz proposed openstack/swift: go: shuffle suffix dirs when replicating a partition https://review.openstack.org/315138 | 15:51 |
| acoles | jrichli: sgundur- note conversation above from tdasilva that this should also fix the probe test errors https://review.openstack.org/#/c/156923/40/swift/common/middleware/versioned_writes.py@532 | 15:51 |
| patchbot | acoles: patch 156923 - swift - Refactor server side copy as middleware | 15:51 |
| acoles | be interested if someone has time to verify that | 15:51 |
| *** dmorita has joined #openstack-swift | 15:51 | |
| *** dmorita has quit IRC | 15:51 | |
| *** dmorita_ has joined #openstack-swift | 15:51 | |
| tdasilva | disclaimer: all I said is that in copy middleware env.copy is removed :D | 15:52 |
| jrichli | acoles: i will try that first | 15:53 |
| * jrichli can't right now, but will soon | 15:54 | |
| *** lakshmiS has joined #openstack-swift | 15:55 | |
| acoles | admin6: can you file that as a bug report? https://bugs.launchpad.net/swift/+filebug and if you are able to also include any device names you might have in your ring with non-ascii characters? | 15:56 |
| acoles | admin6: or I can file it if you don't have/don't want a launchpad account | 15:57 |
| *** joeljwright has joined #openstack-swift | 15:57 | |
| *** ChanServ sets mode: +v joeljwright | 15:57 | |
| *** diogogmt has joined #openstack-swift | 15:59 | |
| acoles | admin6: btw that's not a stupid question | 16:00 |
| admin6 | I don’t have a launchpasd account and I’m not familiar with that. Whet do you mean by device names with non-ascii chars? | 16:00 |
| admin6 | acoles: thanks :-) | 16:00 |
| *** nadeem has joined #openstack-swift | 16:01 | |
| *** nadeem has quit IRC | 16:01 | |
| *** nadeem has joined #openstack-swift | 16:02 | |
| admin6 | acoles: here is my swift-ring-builder result for that ring http://pastebin.com/tfC5pawr | 16:03 |
| janonymous | acloes: is it double encoding case? | 16:03 |
| *** dmk0202 has quit IRC | 16:07 | |
| *** pgbridge has quit IRC | 16:07 | |
| *** esker has quit IRC | 16:08 | |
| *** lyrrad has joined #openstack-swift | 16:11 | |
| *** gyee has joined #openstack-swift | 16:13 | |
| admin6 | acoles: I juste remember already having an ubuntu one account so I’ve create a bug report : https://bugs.launchpad.net/swift/+bug/1580678 | 16:20 |
| openstack | Launchpad bug 1580678 in OpenStack Object Storage (swift) "UnicodeDecodeError when rebalancing a ring" [Undecided,New] | 16:20 |
| admin6 | acoles: I must leave now, I’ll be available tomorrow. thanks. | 16:21 |
| acoles | admin6: thanks | 16:21 |
| *** cbartz has quit IRC | 16:23 | |
| *** klrmn has quit IRC | 16:25 | |
| *** rledisez has quit IRC | 16:34 | |
| timburke | good morning | 16:37 |
| clayg | jrichli: heh, after "fixing" my config (servers per port) the container sync probe tests passed for me | 16:42 |
| clayg | jrichli: but I am still getting 3 error tests - something about KeyError - is there *really* a patch for that which hasn't landed yet (!?) | 16:42 |
| *** Jeffrey4l has quit IRC | 16:43 | |
| *** nadeem has quit IRC | 16:44 | |
| notmyname | I just +A'd https://review.openstack.org/#/c/311899/ to get some of the probe tests fixed | 16:46 |
| patchbot | notmyname: patch 311899 - swift - Fix probe tests from commit cf48e75 | 16:46 |
| AJaeger | notmyname: IMHO https://review.openstack.org/#/c/298313 is ready for merge now - the other-requirements.txt introduction. | 16:47 |
| AJaeger | You can always add additional files later on... | 16:48 |
| clayg | what is *this* "Suppression partition" !? | 16:48 |
| notmyname | AJaeger: ack | 16:48 |
| clayg | notmyname: so patch 311899 reverts the default port changes - that's cool - but did the saio instructions update remakerings? | 16:51 |
| patchbot | clayg: https://review.openstack.org/#/c/311899/ - swift - Fix probe tests from commit cf48e75 | 16:51 |
| *** nadeem has joined #openstack-swift | 16:51 | |
| clayg | notmyname: I'm more worried that if someone did manage to get through the saio instructions they'd end up with a ring config that won't pass probetests? | 16:51 |
| notmyname | clayg: no, I don't think so. that was the bug | 16:51 |
| clayg | i suppose I could have just looked :P | 16:52 |
| notmyname | ok, stupid question | 16:53 |
| notmyname | I can pip install -r reqs just fine | 16:53 |
| clayg | ^ not a question | 16:53 |
| notmyname | but `tox -e bindep` gives me the classic "error in setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Expected ',' or end-of-list in dnspython>=1.12.0;python_version<'3.0' at ;python_version<'3.0'" error | 16:54 |
| notmyname | why is the tox env not getting the right pip pbr setuptools whatever? | 16:54 |
| clarkb | because virtualenv bundles all of that stuff | 16:54 |
| clarkb | so you need to upgrade virtualenv | 16:54 |
| clayg | <snicker> | 16:54 |
| clarkb | (this is one reason that the distros consider bundling to be terribleness) | 16:55 |
| *** asettle has quit IRC | 16:55 | |
| notmyname | clayg: told you it was a stupid question | 16:56 |
| *** nadeem has quit IRC | 16:56 | |
| notmyname | clayg: sorry. I didn't build my whole box from latest-nightly sources | 16:56 |
| notmyname | AJaeger: "ERROR: InvocationError: '/home/swift/swift/.tox/bindep/bin/bindep test'" | 16:57 |
| clayg | <snicker> | 16:57 |
| *** psachin has quit IRC | 16:58 | |
| notmyname | AJaeger: https://gist.github.com/notmyname/a8bcd0e2a74f6e6c61d087467e13ceb6 | 16:58 |
| AJaeger | notmyname: strange ;( | 16:58 |
| notmyname | AJaeger: is that just telling me that the commands didnt' work? or that there's an error somewhere? | 16:58 |
| AJaeger | notmyname: thanks for the gist. It tells you - see line 8 - that you do not have libsqlite3-dev pkg-config installed | 16:59 |
| notmyname | AJaeger: also, does this mean bindep should be added to test-requirements.txt? | 16:59 |
| AJaeger | But should have... | 16:59 |
| notmyname | AJaeger: "should". I mean, I kinda know that this box does in fact work with swift ;-) | 16:59 |
| AJaeger | notmyname: no, the idea here is to not add it since none of the other jobs needs it. | 16:59 |
| acoles | clayg: you're a genius! | 17:00 |
| acoles | clayg: https://github.com/openstack/swift/blob/master/swift/locale/fr/LC_MESSAGES/swift.po#L871 | 17:00 |
| notmyname | or maybe my dev box is working because I'm not linking anythign against sqlite? | 17:00 |
| AJaeger | notmyname: that might be the case here... | 17:00 |
| acoles | and so Successful rsync becomes https://github.com/openstack/swift/blob/master/swift/locale/fr/LC_MESSAGES/swift.po#L968 | 17:00 |
| klamath | can someone help explain the splice function in the object-server conf? | 17:01 |
| acoles | clayg: note the è in Succès de Rsync | 17:01 |
| AJaeger | notmyname: bindep environment includes a dependency on bindep itself - a developer should be able to run "tox -e bindep" and it works, so it should have minimal dependencies. | 17:01 |
| klamath | does that help with the o_temp issues with xfs? | 17:01 |
| *** jistr has quit IRC | 17:02 | |
| AJaeger | You might have in test-requirements packages that need devel libraries, and therefore don't want to install them to avoid a chicken/egg problem. | 17:02 |
| sgundur- | jrichli: acoles - i tried current master + patch156923 ; probetests pass | 17:02 |
| acoles | sgundur-: thanks | 17:02 |
| sgundur- | jrichli: acoles: but at the end of unittests - I get this log -http://paste.openstack.org/show/496777/ | 17:02 |
| sgundur- | probably from here - https://github.com/openstack/swift/blob/5d150166692da4edb70aae7eb4a209ae1ae2d57f/swift/proxy/controllers/obj.py#L1450 | 17:03 |
| AJaeger | notmyname: does that answer your questions? | 17:03 |
| clayg | acoles: i sorta want *my* logs to say "Succès de Rsync" now :'( | 17:03 |
| AJaeger | speaking about Infra: There's a change to run keystone V3 jobs on swift for project-config: https://review.openstack.org/#/c/313659/ - I'd like some swift folks to review that, please | 17:04 |
| patchbot | AJaeger: patch 313659 - openstack-infra/project-config - Run Swift functional tests in Identity v3-only | 17:04 |
| sgundur- | acoles: the unittests pass, but just that log at the end | 17:04 |
| notmyname | AJaeger: well, it's an answer :-) | 17:05 |
| *** mragupat has joined #openstack-swift | 17:06 | |
| notmyname | AJaeger: I don't see anything that I'd expect to require sqlite dev headers | 17:06 |
| *** jmccarthy1 has quit IRC | 17:07 | |
| AJaeger | notmyname: should I remove those two entries from my change? | 17:07 |
| *** BAKfr has quit IRC | 17:08 | |
| AJaeger | I can respin easily... Just like to finish this soon if possible before I disappear for two weeks of vacation... | 17:08 |
| notmyname | unless someone else has reason to believe that we actually require libsqlite3-dev or pkg-config | 17:08 |
| acoles | clayg: how about rotating locales so you get a new language every day ;) | 17:08 |
| AJaeger | let's remove them - and you can always add back ;) | 17:08 |
| openstackgerrit | Andreas Jaeger proposed openstack/swift: List system dependencies for running common tests https://review.openstack.org/298313 | 17:09 |
| AJaeger | So, let's see whether Jenkins agrees with testing ;) | 17:09 |
| *** BAKfr has joined #openstack-swift | 17:10 | |
| clayg | acoles: I only like the ones where rsync comes of sounding smug - "Do you *like* the way I push le bytes?" | 17:10 |
| acoles | sgundur-: ah, the ChunkWriteTimeout returns :( | 17:10 |
| acoles | clayg: you mean urrrrsync ;) | 17:11 |
| clayg | it's like every time we write a new unittest based on those in-process guys | 17:11 |
| *** hezhiqiang has quit IRC | 17:12 | |
| *** nadeem has joined #openstack-swift | 17:12 | |
| *** ChubYann has joined #openstack-swift | 17:16 | |
| *** klrmn has joined #openstack-swift | 17:17 | |
| *** tqtran has joined #openstack-swift | 17:18 | |
| acoles | AJaeger: ack re patch 313659, I'll try to take a look | 17:28 |
| patchbot | acoles: https://review.openstack.org/#/c/313659/ - openstack-infra/project-config - Run Swift functional tests in Identity v3-only | 17:28 |
| AJaeger | thanks, acoles | 17:30 |
| *** dmorita has joined #openstack-swift | 17:31 | |
| *** openstackgerrit has quit IRC | 17:33 | |
| *** openstackgerrit has joined #openstack-swift | 17:33 | |
| *** dmorita_ has quit IRC | 17:34 | |
| *** sileht has quit IRC | 17:37 | |
| *** silor has quit IRC | 17:43 | |
| *** sileht has joined #openstack-swift | 17:44 | |
| *** acoles is now known as acoles_ | 17:45 | |
| jrichli | clayg: not sure what the KeyError is for. There were 2 issues I knew of: the ports - which is fixed by a patch about to be merged - and the 3 container-sync issues related to versioning that you wont see if you have versioned writes off. | 17:53 |
| *** CaioBrentano has joined #openstack-swift | 17:53 | |
| *** dmorita has quit IRC | 17:54 | |
| *** thumpba has quit IRC | 17:54 | |
| *** dmorita has joined #openstack-swift | 17:54 | |
| *** thumpba has joined #openstack-swift | 18:05 | |
| *** ChubYann has quit IRC | 18:11 | |
| *** dmorita has quit IRC | 18:12 | |
| *** dmorita has joined #openstack-swift | 18:13 | |
| *** dmorita has quit IRC | 18:13 | |
| *** raildo has joined #openstack-swift | 18:17 | |
| raildo | hey guys, I'm working to create a new job that runs swift functional tests in a identity v3-only devstack, and adds this job to the swift pipeline but I need someone on swift core team to take a look on this patch: https://review.openstack.org/#/c/313659/ | 18:18 |
| patchbot | raildo: patch 313659 - openstack-infra/project-config - Run Swift functional tests in Identity v3-only | 18:18 |
| openstackgerrit | Monty Taylor proposed openstack/swift: Use govendor to help manage requirements https://review.openstack.org/315196 | 18:18 |
| notmyname | raildo: earlier acoles_ said he'd look at it. (but it's already evening for him so probably not until tomorrow at the earliest) | 18:20 |
| raildo | notmyname: ok, np. thanks :) | 18:20 |
| notmyname | dfg_: so you know where the monty patches are coming from.... | 18:21 |
| notmyname | dfg_: for looking at the best way to do dependency management, there's godep, glide, and govendor. he's proposing patches for each of those so we can compare and see what they're like | 18:21 |
| notmyname | dfg_: probably we'll use either govendor or glide over godep | 18:22 |
| notmyname | dfg_: and nobody's expecting these specifically to land. it's more of "ok, that's what it looks like, can we live with it and does it work" sort of thing | 18:22 |
| openstackgerrit | Monty Taylor proposed openstack/swift: Use glide for dependency management https://review.openstack.org/315200 | 18:23 |
| *** ChubYann has joined #openstack-swift | 18:23 | |
| sgundur- | jrichli: I tried master+311899+314947 ; probetests fail with - http://paste.openstack.org/show/496792/ ; is it my local settings ? | 18:25 |
| jrichli | sgundur- : interesting. I haven't seen that one before. BUT, I haven't tried running with the fix today yet. Will do soon and let you know. thx! | 18:26 |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - cleanup decrypter exception handling https://review.openstack.org/304806 | 18:27 |
| jrichli | has anyone run with a compressed container DB? | 18:28 |
| sgundur- | jrichli: ok, thanks :) | 18:28 |
| *** natarej has quit IRC | 18:33 | |
| *** natarej has joined #openstack-swift | 18:34 | |
| clayg | weeee | 18:38 |
| clayg | jrichli: pretty sure the KeyError things were the port changes in the tests? | 18:38 |
| jrichli | clayg: ah, ok. i am surprised you only got 3 of those. i never did test that version :-) | 18:39 |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - cleanup decrypter exception handling https://review.openstack.org/304806 | 18:39 |
| tdasilva | timburke: around? | 18:40 |
| timburke | tdasilva: yep | 18:40 |
| tdasilva | timburke: hey! not sure if you had a chance to read up on the convo around copy middleware, but mattoliverau made some suggestions for changes | 18:41 |
| *** rvasilets__ has joined #openstack-swift | 18:41 | |
| timburke | yeah, i'd been following along a bit. took a long weekend, but was catching up last night | 18:41 |
| tdasilva | timburke: was wondering if you could take a look and think that they are ok to be made and get a quick +2 from you again :) | 18:41 |
| timburke | will do | 18:41 |
| tdasilva | ok, I won't push the changes until you give your opinion as I'd prefer to not lose your +2 at this point. mattoliverau said he would be willing to +2 also and leave changes for a follow-up, but I think the name change is worth doing now | 18:42 |
| openstackgerrit | Alistair Coles proposed openstack/swift: crypto - cleanup encrypter error handling https://review.openstack.org/312662 | 18:43 |
| *** acoles_ is now known as acoles | 18:43 | |
| *** mragupat has quit IRC | 18:44 | |
| timburke | tdasilva: yeah, if we're doing the name change, better to do it now. may as well push; i'll be sure to find time today | 18:47 |
| tdasilva | timburke: thanks! | 18:47 |
| acoles | raildo: will those new v3 only func tests be non-voting | 18:49 |
| raildo | acoles: yes | 18:49 |
| raildo | acoles: https://review.openstack.org/#/c/313659/4/jenkins/jobs/projects.yaml this job-suffix 'nv' represent that the job will be non-voting | 18:50 |
| patchbot | raildo: patch 313659 - openstack-infra/project-config - Run Swift functional tests in Identity v3-only | 18:50 |
| acoles | raildo: got it, i suspected that might be the case. like notmyname said I won't be able to look properly til later this week, just finishing today | 18:51 |
| *** tesseract has quit IRC | 18:51 | |
| raildo | acoles: np :) | 18:52 |
| acoles | clayg: might you be able to approve this patch 290391 ? I fear another merge conflict... | 18:52 |
| patchbot | acoles: https://review.openstack.org/#/c/290391/ - swift - Rename hash_cleanup_listdir tests | 18:52 |
| *** joeljwright has quit IRC | 18:52 | |
| acoles | clayg: or perhaps I should say "approuvez s'il-vous-plaît" ;) | 18:54 |
| pdardeau | acoles: clayg: pretty soon we'll see the nick 'grandfromage' show up here :) | 18:56 |
| openstackgerrit | Thiago da Silva proposed openstack/swift: Refactor server side copy as middleware https://review.openstack.org/156923 | 18:57 |
| acoles | pdardeau: lol | 18:58 |
| *** mmcardle has quit IRC | 19:03 | |
| * notmyname afk for a bit | 19:09 | |
| *** NM has joined #openstack-swift | 19:13 | |
| *** acoles is now known as acoles_ | 19:20 | |
| *** vint_bra has joined #openstack-swift | 19:21 | |
| dfg_ | notmyname: ok cool. we will check them out on our end | 19:30 |
| openstackgerrit | Merged openstack/swift: Fix probe tests from commit cf48e75 https://review.openstack.org/311899 | 19:31 |
| clayg | yay! | 19:31 |
| *** lakshmiS_ has joined #openstack-swift | 19:38 | |
| *** thumpba has quit IRC | 19:38 | |
| notmyname | dfg_: great, thanks | 19:38 |
| *** dmk0202 has joined #openstack-swift | 19:38 | |
| *** lakshmiS has quit IRC | 19:39 | |
| notmyname | torgomatic: thanks for another great ML post :-) | 19:40 |
| *** thumpba has joined #openstack-swift | 19:42 | |
| *** thumpba has quit IRC | 19:43 | |
| *** thumpba has joined #openstack-swift | 19:43 | |
| *** thumpba has quit IRC | 19:45 | |
| *** thumpba has joined #openstack-swift | 19:45 | |
| *** nadeem has quit IRC | 19:50 | |
| *** nadeem has joined #openstack-swift | 19:51 | |
| notmyname | AJaeger: patch 298313 looks good in the gate :-) | 19:52 |
| patchbot | notmyname: https://review.openstack.org/#/c/298313/ - swift - List system dependencies for running common tests | 19:52 |
| notmyname | the golang ML thread has 124 messages from the last 8 days. that's only 15.5 per day! come on! we can do better. with a little extra effort, I think we can get it up to 20 messages per day! | 19:57 |
| *** mmcardle has joined #openstack-swift | 19:59 | |
| *** daemontool has quit IRC | 20:09 | |
| *** daemontool has joined #openstack-swift | 20:14 | |
| NM | Hey guys. Just to be sure: the maximum number of objects per container, depends on a lot of things, right? Like memory, disk i/o, network… | 20:14 |
| glange | NM_ yes, also if you have big containers you might have troubles with disks holding those big containers filling up before other container holding disks | 20:16 |
| glange | and you'll also just see a long decline in performance as the number of objects increases | 20:17 |
| notmyname | NM: where the performance is in object PUTs per second in that container | 20:17 |
| glange | and maybe deletes | 20:17 |
| glange | but not gets | 20:18 |
| notmyname | glange: people shouldn't be deleting data in swift anyway ;-) | 20:18 |
| glange | hah | 20:18 |
| glange | the filling up disks to early thing can be a big problem depending on your cluser/usage patterns | 20:18 |
| glange | it like doesn't really have a good fix, I think? | 20:19 |
| NM | Well, right now we have 5M objects in a single container. Not sure if it's a big number for swift. | 20:19 |
| notmyname | NM: depends mostly on what kind of media your containers are on. if on flash (SSD) and you're ok with <200 PUTs/s, then you're probably fine until you start running out of space on the SSDs | 20:20 |
| glange | yeah, SSDs fixes a lot of pain there | 20:20 |
| notmyname | NM: and the other thing to check is the container replication time. large containers will get to a point where it's extremely difficult to keep them in sync. | 20:21 |
| glange | NM_ the work around is sharding your objects across many containers | 20:21 |
| NM | What would define a container as large? Number of objects? | 20:22 |
| notmyname | NM: at 5M objects in a single container (and I assume, growing), it's probably a good idea to think about how you could shard the data across containers | 20:22 |
| glange | just mod_ on the name | 20:22 |
| ahale | or you make a ring change and two huge dbs end up on the same device and suddenly fill up completely | 20:22 |
| NM | I'm almost sure we will reach 10M if we do nothing. | 20:22 |
| glange | write your application to shard | 20:22 |
| glange | 10 containers, 1 million objects per container | 20:22 |
| NM | glange: I'll talk to our devs. | 20:24 |
| glange | dude, devs are the worst :) | 20:24 |
| glange | good luck | 20:24 |
| tdasilva | lol | 20:24 |
| NM | hahaha! We go in group of 3, to talk to devs. Usually it's kind persuasive | 20:26 |
| notmyname | dfg_: is https://github.com/rackerlabs/swift/tree/hummingbird_production up to datE? | 20:26 |
| dfg_ | notmyname: no | 20:27 |
| notmyname | dfg_: there's been a few questions about how to do builds for the golang stuff, so I was poking around at what you had done | 20:27 |
| dfg_ | notmyname: oh. well the debian stuff in there is prob not changed but we haven't updated the code there for a while because we changed where we release from to something internal | 20:30 |
| NM | Thanks guy. | 20:30 |
| NM | notmyname: I think it's time to have ssd :D | 20:31 |
| notmyname | NM: you don't?! yeah, always put accounts and containers on flash drives :-) | 20:31 |
| notmyname | dfg_: ok. I'm not particularly concerned with the .deb building (although yes that will be a good starting point for other stuff). maybe a better thing to ask about is how you do tests on your dev VM | 20:32 |
| notmyname | dfg_: so maybe I can imagine what that would look like on a CI/gate image | 20:32 |
| NM | notmyname: I'll quote you to our backoffice team. | 20:32 |
| notmyname | NM: "some guy on the internet totally said that we need to buy a whole bunch of SSDs" | 20:33 |
| NM | notmyname: Seenm legit. | 20:33 |
| *** pcaruana has quit IRC | 20:42 | |
| dfg_ | notmyname: redbo made a little CI server: http://104.239.166.47/job/Swift%20Golang%20Checks/163/ it posts to the gerrit but the link needs to be updated to just run off of :80 | 20:43 |
| *** rcernin has joined #openstack-swift | 20:44 | |
| notmyname | dfg_: that looks like it's running the upstream functests against a hummingbird deployment. which is good, but doesn't do the hummingburd unittests. right? | 20:44 |
| dfg_ | ya i think it does | 20:46 |
| dfg_ | ya- at the top of the console input | 20:46 |
| dfg_ | bbiab | 20:46 |
| *** suyash has joined #openstack-swift | 20:47 | |
| notmyname | dfg_: ah, ok. I see | 20:47 |
| *** nadeem has quit IRC | 20:49 | |
| *** joeljwright has joined #openstack-swift | 20:50 | |
| *** ChanServ sets mode: +v joeljwright | 20:50 | |
| notmyname | dfg_: having a "set -x" on your test script and/or seeing the job definition on that jenkins server would be really really helpful. | 20:53 |
| *** Harin has joined #openstack-swift | 20:55 | |
| *** Harin has quit IRC | 20:56 | |
| *** thumpba has quit IRC | 20:56 | |
| jrichli | sgundur- : my probetests run cleanly now with those patches - I am not seeing the same error that you have | 20:57 |
| *** takashi has joined #openstack-swift | 20:57 | |
| bkeller` | mine seem fine too | 20:57 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 20:58 |
| *** acoles_ is now known as acoles | 20:59 | |
| kota_ | mornin | 20:59 |
| sgundur- | jrichli: thanks, ll recheck if I have messed up my settings! | 20:59 |
| *** raildo is now known as raildo-afk | 21:00 | |
| notmyname | swift team meeting time in #openstack-meeting | 21:00 |
| *** m_kazuhiro has joined #openstack-swift | 21:03 | |
| *** m_kazuhi_ has joined #openstack-swift | 21:07 | |
| *** m_kazuhiro has quit IRC | 21:10 | |
| *** ekarlso has quit IRC | 21:11 | |
| *** ekarlso has joined #openstack-swift | 21:14 | |
| *** pauloewerton has quit IRC | 21:15 | |
| openstackgerrit | Merged openstack/swift: Rename hash_cleanup_listdir tests https://review.openstack.org/290391 | 21:20 |
| acoles | clayg: thanks ^^ | 21:21 |
| clayg | acoles: i just click whatever buttons you tell me to | 21:21 |
| *** jordanP has quit IRC | 21:23 | |
| *** lakshmiS_ has quit IRC | 21:26 | |
| *** nadeem has joined #openstack-swift | 21:28 | |
| *** nadeem has quit IRC | 21:29 | |
| *** nadeem has joined #openstack-swift | 21:30 | |
| *** mmcardle has quit IRC | 21:34 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 21:43 |
| timburke | oh, tdasilva: i think the copy patch may close https://bugs.launchpad.net/swift/+bug/1260446 as a drive-by... thoughts? | 21:48 |
| openstack | Launchpad bug 1260446 in OpenStack Object Storage (swift) "when you copy a slo the destination doesn't get its content-length and etag set correctly in the container listings" [Undecided,In progress] - Assigned to Alistair Coles (alistair-coles) | 21:48 |
| *** NM has quit IRC | 22:00 | |
| *** CaioBrentano has quit IRC | 22:01 | |
| mattoliverau | timburke, tdasilva: I'll checkout and hopefully +2/+A copy patch post meeting I'm going into. | 22:02 |
| mattoliverau | unless someone does it first of course | 22:03 |
| *** takashi has quit IRC | 22:06 | |
| *** m_kazuhi_ has quit IRC | 22:06 | |
| *** nadeem has quit IRC | 22:07 | |
| *** kmARC has joined #openstack-swift | 22:07 | |
| acoles | tdasilva: there is a functional test here https://review.openstack.org/#/c/248219/2/test/functional/tests.py which you could use to verify part of fixing bug1260446 (not suggesting you necessarily add it to the patch at this point) | 22:08 |
| patchbot | acoles: patch 248219 - swift - Fix listing of SLO manifest bytes after POST or COPY | 22:08 |
| acoles | clayg: so IIRC you're client driven policy migration use case has the client make any POSTs to the target, and the symlink 'rejects' POSTs with a redirect, but the client shouldn't really be POSTing to a symlink? I think I convinced myself that was workable. | 22:10 |
| acoles | s/you're/your/ :? | 22:10 |
| *** jordanP has joined #openstack-swift | 22:13 | |
| *** jordanP has quit IRC | 22:14 | |
| *** acoles is now known as acoles_ | 22:19 | |
| *** ozialien10 has joined #openstack-swift | 22:21 | |
| *** acoles_ is now known as acoles | 22:21 | |
| acoles | tdasilva: also sgundur- verified that the copy patch fixes the container sync probe test bug https://launchpad.net/bugs/1580528 | 22:22 |
| openstack | Launchpad bug 1580528 in OpenStack Object Storage (swift) "container sync probe tests fail when allow_versioned_writes is true" [Undecided,New] | 22:22 |
| *** acoles is now known as acoles_ | 22:27 | |
| *** dmk0202 has quit IRC | 22:47 | |
| *** vint_bra has quit IRC | 22:53 | |
| *** joeljwright has quit IRC | 22:56 | |
| *** bill_az has joined #openstack-swift | 22:58 | |
| *** Jeffrey4l has joined #openstack-swift | 22:59 | |
| *** jamielennox is now known as jamielennox|away | 23:02 | |
| *** mingdang1 has joined #openstack-swift | 23:03 | |
| *** mingdang1 has quit IRC | 23:13 | |
| *** zigo has quit IRC | 23:14 | |
| *** mariusv has quit IRC | 23:14 | |
| *** mingdang1 has joined #openstack-swift | 23:14 | |
| *** zigo has joined #openstack-swift | 23:15 | |
| *** arch-nemesis has quit IRC | 23:16 | |
| *** NM has joined #openstack-swift | 23:16 | |
| *** mariusv has joined #openstack-swift | 23:16 | |
| *** mariusv has quit IRC | 23:16 | |
| *** mariusv has joined #openstack-swift | 23:16 | |
| *** ozialien10 has quit IRC | 23:27 | |
| *** ozialien10 has joined #openstack-swift | 23:28 | |
| *** kei_yama has joined #openstack-swift | 23:30 | |
| *** diogogmt_ has joined #openstack-swift | 23:34 | |
| *** diogogmt_ has quit IRC | 23:34 | |
| *** tamizh_geek has quit IRC | 23:34 | |
| *** tamizh_geek has joined #openstack-swift | 23:36 | |
| *** diogogmt has quit IRC | 23:37 | |
| *** ozialien10 has quit IRC | 23:37 | |
| *** ozialien10 has joined #openstack-swift | 23:38 | |
| *** hosanai has joined #openstack-swift | 23:40 | |
| *** ChanServ sets mode: +v hosanai | 23:40 | |
| hosanai | good morning! | 23:43 |
| timburke | good morning hosanai! | 23:44 |
| notmyname | hello hosanai | 23:44 |
| hosanai | timburke, notmyname: hello | 23:46 |
| notmyname | so that's interesting. xenial doesn't let me create an XFS partition with a 256 byte inode. must be 512 | 23:50 |
| notmyname | 23:50 | |
| notmyname | "Minimum inode size for CRCs is 512 bytes" | 23:50 |
| *** jamielennox|away is now known as jamielennox | 23:51 | |
| notmyname | and 512 is now the default | 23:51 |
| notmyname | my older saio had 256 byte inodes | 23:54 |
| notmyname | that was precise. anyone want to check on trusty? | 23:54 |
| *** rvasilets__ has quit IRC | 23:56 | |
| clarkb | notmyname: http://manpages.ubuntu.com/manpages/xenial/man8/mkfs.xfs.8.html that man page would appear to be buggy in that case (still says minimum and default is 256 bytes) | 23:56 |
| notmyname | clarkb: yeah, it definitely was the old default :-) | 23:57 |
| timburke | notmyname: assuming i'm reading the output of xfs_info correctly, trusty seems to work with (and default to) 256 | 23:57 |
| notmyname | timburke: thanks | 23:57 |
| notmyname | clarkb: weird. I have xfsprogs 4.3.0 installed (came with xenial server) | 23:58 |
| notmyname | definitely an error though | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!