| kota_ | good morning | 01:23 |
|---|---|---|
| mattoliverau | kota_: morning | 01:25 |
| mattoliverau | clayg: playing with your vsaio (cause i'm working remotely and want to have something to play with locally on my laptop, especially as I fly to barca on the weekend) and thankyou so much for having both the proxy and the noauth proxy.. makes my life so much easier.. and it means I don't have to set up a noauth one.. you rock! | 01:26 |
| kota_ | mattoliverau: morning | 01:28 |
| *** zhengyin has joined #openstack-swift | 01:32 | |
| kota_ | clayg: will look at your bug report | 02:40 |
| kota_ | ah, proposed patch for stable branch doesn't appear here? | 02:41 |
| kota_ | https://review.openstack.org/#/c/387123/ | 02:41 |
| patchbot | patch 387123 - swift (stable/newton) - Prevent ssync writing bad fragment data to diskfile | 02:41 |
| kota_ | that's a back port for newton | 02:41 |
| kota_ | will try for mitaka acoles said, it doesn't conflict but an unit test failed. | 02:42 |
| charz | kota_: mattoliverau morning | 02:49 |
| mattoliverau | charz: o/ | 02:50 |
| *** sgundur has joined #openstack-swift | 03:09 | |
| *** sgundur has quit IRC | 03:15 | |
| *** chlong has joined #openstack-swift | 03:30 | |
| *** psachin has joined #openstack-swift | 03:49 | |
| *** winggundamth has joined #openstack-swift | 03:55 | |
| *** cshastri has joined #openstack-swift | 04:12 | |
| mahatic | good morning | 04:18 |
| mahatic | tarantool: hello, around? | 04:18 |
| *** cshastri_ has joined #openstack-swift | 04:20 | |
| *** psachin has quit IRC | 04:20 | |
| *** cshastri has quit IRC | 04:23 | |
| *** cshastri_ has quit IRC | 04:25 | |
| *** cshastri_ has joined #openstack-swift | 04:25 | |
| mahatic | notmyname: I did happen to chat with tarantool earlier on opw channel. So although the deadline to put in the application is today, she can still continue to make first contribution (and later edit the application), for the next couple of weeks. Then the decision will be based on her contribution and mentor's willingness to accept. | 04:26 |
| winggundamth | hi everybody. anyone know how to delete account under keystoneauth? I have 150 million objects under the project and I want to remove it all but I don't want to remove instances under that project. | 04:28 |
| *** sures has joined #openstack-swift | 04:30 | |
| sures | hii all, I deployed openstack swift with one proxy and one storage node. I able to upload and download objects. But while uploading 20 objects of size 5.1GB parallely i am getting an error 503 service unavialable. | 04:32 |
| sures | please someone help? | 04:32 |
| *** StraubTW has quit IRC | 04:32 | |
| *** StraubTW has joined #openstack-swift | 04:33 | |
| *** psachin_ has joined #openstack-swift | 04:37 | |
| *** sheel has joined #openstack-swift | 05:00 | |
| *** cshastri_ is now known as cshastri | 05:08 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:24 | |
| *** linoespinoza has joined #openstack-swift | 05:27 | |
| *** qwertyco has joined #openstack-swift | 05:29 | |
| *** ChubYann has quit IRC | 05:38 | |
| *** ppai has joined #openstack-swift | 05:47 | |
| *** linoespinoza has quit IRC | 05:48 | |
| *** dmorita has joined #openstack-swift | 05:49 | |
| kota_ | sures: sounds like they are over the object size limit (5GB in default) but it's little odd because that should be not 503 but sort of 4xx. | 05:55 |
| *** klrmn has quit IRC | 05:56 | |
| kota_ | acoles: i finally reach out the reason why we hit the assertion failure in the backporting for stable/mitaka | 05:56 |
| kota_ | acoles: the first good (?) commit we can cherry-pick without change is https://github.com/openstack/swift/commit/4c11833a9cbff499725365e535e217f3eae3c442 | 05:56 |
| *** psachin_ has quit IRC | 05:57 | |
| kota_ | I'm telling as mitaka(bad) -> https://github.com/openstack/swift/commit/4c11833a9cbff499725365e535e217f3eae3c442 (good) -> master (good) | 05:57 |
| kota_ | that patch is removing the evnetlet coros for writing down the entire chunk. i.e. before the commit 4c11833a9cbff499725365e535e217f3eae3c442, we need one more extra tranpoline to ensure both writing chunk and writing a log | 05:58 |
| sures | kota i am giving command like this "swift upload c1 file_5.1 --segment-size 536870912" | 06:00 |
| sures | and i am running 20 uploading processes with the same size objects | 06:00 |
| kota_ | sures: ok, you're using static large objects | 06:00 |
| sures | kota: yes | 06:01 |
| sures | kota: Is swift proxyserver is unable to handle those many requests? | 06:01 |
| kota_ | sures: would you be able to see swift logs (proxy or objects)? it might be helpful to know what happened in your swift. | 06:02 |
| kota_ | sures: might be but i cannot judge that yet. | 06:02 |
| sures | kota: here is my object log http://paste.openstack.org/show/585891/ | 06:05 |
| kota_ | acoles: https://gist.github.com/bloodeagle40234/2098382d7157cf92cbca9a39907566fb for mitaka/stable -> cherry-pick 3218f8b064e462d901466b04a4813e15ec96da85 worked in my environment | 06:05 |
| *** rcernin has joined #openstack-swift | 06:06 | |
| kota_ | sures: looking | 06:07 |
| kota_ | sures: it looks like swift client log? not a severside ones? | 06:08 |
| *** dmorita_ has joined #openstack-swift | 06:09 | |
| *** dmorita has quit IRC | 06:09 | |
| sures | kota: ok i will post server side log also | 06:11 |
| * kota_ wants gerrit bot works on stable branch too | 06:15 | |
| kota_ | acoles, clayg: https://review.openstack.org/#/c/387172/ , here is for stable/mitaka | 06:16 |
| patchbot | patch 387172 - swift (stable/mitaka) - Prevent ssync writing bad fragment data to diskfile | 06:16 |
| sures | kota: here is my server side log http://paste.openstack.org/show/585894/ | 06:17 |
| kota_ | sures: k, it looks like some of the PUTs got a network timeout which could happen while transferring the data from proxy to object server | 06:20 |
| sures | kota: for single request it is working properly | 06:22 |
| kota_ | sures: i'm wondering if you could make sure your network (or disk) speed have been enought to write. | 06:22 |
| kota_ | (or, if something buggy things happens, it might appear in the objct-sever logs) | 06:23 |
| sures | kota: i have two servers with high configurations | 06:25 |
| sures | each has 96 GB RAM, 2 octacore processors and 10 Gige back to back connectivity | 06:27 |
| kota_ | sures: sounds great environment | 06:31 |
| notmyname | winggundamth: (drive by comment, since I'm getting offline now) use a reseller admin auth token to DELETE the account in swift. that will mark it as deleted, and swift will reap (ie delete/clean uo) all the data in the background. this is how you delete all the data in swift for an account without affecting the rest of the openstack services | 06:32 |
| kota_ | sures: hmm, how your object-sever log said? | 06:32 |
| kota_ | notmyname:!!! | 06:32 |
| notmyname | winggundamth: but be careful if you have eg glance or cinder images stored in that user's account in swift. if you delete everything in swift for that user, you'll lose those too | 06:32 |
| notmyname | kota_: hello :-) | 06:32 |
| notmyname | kota_: just reading a little scrollback and email before going to bed :-) | 06:32 |
| notmyname | kota_: thanks for doing the backport | 06:33 |
| notmyname | kota_: I'll get to that late tomorrow or on tuesday | 06:33 |
| *** dmorita_ has quit IRC | 06:33 | |
| kota_ | notmyname: no problem, acoles and clayg are great. | 06:33 |
| *** dmorita has joined #openstack-swift | 06:33 | |
| kota_ | k | 06:33 |
| notmyname | FTI for the summit, https://www.openstack.org/summit/barcelona-2016/summit-schedule/events/17353/ops-swift-feedback is in the ops track, not the swift track | 06:33 |
| notmyname | and add any questions or thoughts on the linked etherpad | 06:34 |
| notmyname | I'll be setting the swift schedule tomorrow, I hope | 06:34 |
| kota_ | notmyname: sounds good, thanks for giving the info | 06:35 |
| notmyname | yeah, I'll keep bugging people about it too :-) | 06:35 |
| notmyname | ok, good night | 06:35 |
| kota_ | notmyname: good night | 06:36 |
| sures | kota: one minute | 06:37 |
| sures | please | 06:37 |
| sures | hi all, while doing swift stat i am getting error like this in logs | 06:38 |
| sures | http://paste.openstack.org/show/585895/ | 06:38 |
| sures | please someone help on this | 06:38 |
| * kota_ added the ops-swifft-feedback session to my schedule | 06:39 | |
| *** Jeffrey4l has quit IRC | 06:45 | |
| winggundamth | notmyname: thank you so much for the answer. I think the same but found this error when I'm using curl to delete account via ResellerAdmin account. http://paste.openstack.org/show/585897/ | 06:48 |
| winggundamth | notmyname: btw good night! | 06:48 |
| winggundamth | notmyname: if you wake up and have a time. please advice why it show 405 Method Not Allowed when I'm trying to DELETE account | 06:50 |
| *** dmorita has quit IRC | 06:53 | |
| *** dmorita has joined #openstack-swift | 06:53 | |
| *** StevenK has quit IRC | 06:55 | |
| winggundamth | notmyname: I found that I need to put allow_account_management = true to proxy-server. I'm going to try it right now and let you know the result | 06:58 |
| *** Jeffrey4l has joined #openstack-swift | 07:00 | |
| *** tesseract has joined #openstack-swift | 07:03 | |
| *** tesseract is now known as Guest21077 | 07:04 | |
| *** hseipp has joined #openstack-swift | 07:05 | |
| *** dmorita has quit IRC | 07:14 | |
| *** dmorita has joined #openstack-swift | 07:15 | |
| *** amoralej|off is now known as amoralej | 07:16 | |
| openstackgerrit | Tin Lam proposed openstack/swift: Added conditional headers to HEAD request documentation https://review.openstack.org/387204 | 07:17 |
| *** geaaru has joined #openstack-swift | 07:24 | |
| *** dmorita has quit IRC | 07:24 | |
| *** dmorita has joined #openstack-swift | 07:24 | |
| *** openstackgerrit has quit IRC | 07:34 | |
| *** openstackgerrit has joined #openstack-swift | 07:34 | |
| *** rledisez has joined #openstack-swift | 07:40 | |
| *** dmorita_ has joined #openstack-swift | 07:47 | |
| *** dmorita has quit IRC | 07:48 | |
| *** StevenK has joined #openstack-swift | 07:52 | |
| *** jordanP has joined #openstack-swift | 08:03 | |
| *** dmorita_ has quit IRC | 08:03 | |
| *** qwertyco has quit IRC | 08:10 | |
| winggundamth | notmyname: I successful delete the account. but account reaper seems like not delete the objects. how could I check on this? | 08:13 |
| *** sures has quit IRC | 08:24 | |
| *** acoles_ is now known as acoles | 08:32 | |
| acoles | kota_: thanks for doing those backports | 08:33 |
| *** geaaru has quit IRC | 08:34 | |
| *** dmorita has joined #openstack-swift | 08:40 | |
| acoles | kota_: you are a genius for finding the extra coro in mitaka! | 08:44 |
| acoles | notmyname: re bug 1633647 clayg basically wrote exactly what I was going to first thing today, and I would ideally have that fixed and in next release, as well as backported. | 08:46 |
| openstack | bug 1633647 in OpenStack Object Storage (swift) "bad fragment data not detected in audit" [High,Confirmed] https://launchpad.net/bugs/1633647 | 08:46 |
| *** geaaru has joined #openstack-swift | 08:47 | |
| winggundamth | notmyname: So I changed delay_reaping from 7 days to 60 seconds. still waiting for the result | 08:50 |
| *** janonymous has quit IRC | 08:53 | |
| openstackgerrit | Alistair Coles proposed openstack/python-swiftclient: Use ConfigParser instead of SafeConfigParser https://review.openstack.org/369819 | 08:57 |
| *** pcaruana has joined #openstack-swift | 09:08 | |
| acoles | kota_: looks like here https://review.openstack.org/#/c/311219/1/swift/obj/diskfile.py@1412 we lost a sleep() to trampoline after writing the chunk?? | 09:14 |
| patchbot | patch 311219 - swift - Remove ThreadPool class (MERGED) | 09:14 |
| acoles | kota_: (that makes no difference to your backport, I just happened to notice it) | 09:14 |
| *** chlong has quit IRC | 09:26 | |
| *** ppai has quit IRC | 09:41 | |
| *** dmorita has quit IRC | 09:42 | |
| *** dmorita has joined #openstack-swift | 09:42 | |
| kota_ | acoles: might be? not sure if we need to tranpoline for each chunk writing or not | 09:51 |
| charz | Guys, just a simeple question, can I issue a bulk-delete with a single container name to delete a container that have amount of objects? | 09:52 |
| charz | Or I need to list all objects from the container and add them to list and put the container name in the end of line. | 09:52 |
| *** ppai has joined #openstack-swift | 09:56 | |
| acoles | charz: swiftclient CLI will do that for you - 'swift delete <container name>' will delete all objects in the container | 10:03 |
| acoles | charz: if you are making your own bulk-delete request to the swift API then you do need to list all the objects in the request body https://github.com/openstack/swift/blob/e09c4ee7800e82aa09ca2f6ae375420b766182a4/swift/common/middleware/bulk.py#L148-L190 | 10:07 |
| charz | acoles: thx, yeah! I did some tests on it and I got 409 conflict when I only put a container name on the list. When I tried all objects and put the container name in last part. And I'll able to remove all objects and the container in single curl request. | 10:10 |
| acoles | charz: right. note that there is limit of 10k objects per bulk delete, if your container has more you'll need a series of bulk deletes with the container name at the end of the final one. | 10:12 |
| charz | acoles: thx, just make sure the behavior is matched to our expected. | 10:12 |
| charz | acoles: one more question, do we able to see the report I mean https://github.com/openstack/swift/blob/e09c4ee7800e82aa09ca2f6ae375420b766182a4/swift/common/middleware/bulk.py#L170-L174 in swift logs? | 10:15 |
| charz | acoles: or that's only send to client even I enabled the debug mode in proxy-server.conf. | 10:18 |
| acoles | charz: I can't see any logging for the individual delete's in the proxy bulk middleware code so I think you only have the content of the response body that is returned to the client | 10:24 |
| *** janonymous has joined #openstack-swift | 10:44 | |
| *** ppai has quit IRC | 10:46 | |
| openstackgerrit | Merged openstack/python-swiftclient: Use ConfigParser instead of SafeConfigParser https://review.openstack.org/369819 | 10:48 |
| *** cdelatte has joined #openstack-swift | 10:50 | |
| *** delattec has joined #openstack-swift | 10:52 | |
| *** cdelatte has quit IRC | 10:54 | |
| *** ppai has joined #openstack-swift | 10:58 | |
| *** hseipp has quit IRC | 11:01 | |
| openstackgerrit | Karen Chan proposed openstack/swift: Mirror X-Trans-Id to X-OpenStack-Request-Id https://review.openstack.org/387354 | 11:02 |
| *** hseipp has joined #openstack-swift | 11:10 | |
| *** derekjhyang has quit IRC | 11:10 | |
| *** dmorita has quit IRC | 11:53 | |
| *** dmorita has joined #openstack-swift | 11:54 | |
| *** hseipp has quit IRC | 11:54 | |
| *** hseipp has joined #openstack-swift | 11:54 | |
| *** amoralej is now known as amoralej|lunch | 12:11 | |
| *** cshastri has quit IRC | 12:17 | |
| *** dmorita has quit IRC | 12:19 | |
| *** sai has quit IRC | 12:19 | |
| *** Guest23990 is now known as zigo | 12:21 | |
| *** sai has joined #openstack-swift | 12:27 | |
| *** x1fhh9zh has joined #openstack-swift | 12:41 | |
| *** x1fhh9zh has quit IRC | 12:42 | |
| *** sileht has quit IRC | 12:42 | |
| *** sileht has joined #openstack-swift | 12:49 | |
| *** psachin` has joined #openstack-swift | 12:50 | |
| tarantool | mahatic: hello mahatic, I'm in UTC +2 time zone, just seen your message :-) | 12:51 |
| tarantool | mahatic: I'll see what I can do | 12:52 |
| tarantool | mahatic: and thank you! | 12:52 |
| *** ppai has quit IRC | 12:53 | |
| mahatic | tarantool: sure, good luck! If you pick something from here https://bugs.launchpad.net/swift/+bugs?field.tag=low-hanging-fruit and there's already an assignee but no activity in a while, you can check with the assignee if they're working on it or if you could pick it up | 12:54 |
| *** StraubTW has quit IRC | 12:57 | |
| *** x1fhh9zh has joined #openstack-swift | 12:59 | |
| *** klamath has joined #openstack-swift | 13:00 | |
| tarantool | mahatic: great idea! Hopefully the assignees will be ok with that :-) | 13:01 |
| *** kei_yama has quit IRC | 13:05 | |
| *** pcaruana has quit IRC | 13:18 | |
| *** dmorita has joined #openstack-swift | 13:18 | |
| *** SkyRocknRoll has quit IRC | 13:26 | |
| *** StraubTW has joined #openstack-swift | 13:30 | |
| *** x1fhh9zh has quit IRC | 13:44 | |
| *** x1fhh9zh has joined #openstack-swift | 13:45 | |
| *** admin6 has joined #openstack-swift | 13:52 | |
| *** admin6 has quit IRC | 13:56 | |
| *** sgundur has joined #openstack-swift | 14:02 | |
| *** amoralej|lunch is now known as amoralej | 14:03 | |
| *** hogepodge has quit IRC | 14:04 | |
| *** dmorita has quit IRC | 14:09 | |
| *** sheel has quit IRC | 14:10 | |
| *** admin6 has joined #openstack-swift | 14:11 | |
| *** x1fhh9zh has quit IRC | 14:21 | |
| *** x1fhh9zh has joined #openstack-swift | 14:23 | |
| *** dmorita has joined #openstack-swift | 14:25 | |
| *** x1fhh9zh has quit IRC | 14:28 | |
| *** klrmn has joined #openstack-swift | 14:28 | |
| *** dmorita has quit IRC | 14:30 | |
| *** silor has joined #openstack-swift | 14:35 | |
| *** silor has quit IRC | 14:40 | |
| *** silor has joined #openstack-swift | 14:41 | |
| *** silor has quit IRC | 14:45 | |
| *** silor has joined #openstack-swift | 14:45 | |
| *** dmorita has joined #openstack-swift | 14:56 | |
| *** sgundur has quit IRC | 14:57 | |
| *** dmorita has quit IRC | 15:01 | |
| *** pcaruana has joined #openstack-swift | 15:03 | |
| *** jlwhite has quit IRC | 15:03 | |
| *** mvk has quit IRC | 15:09 | |
| *** jlwhite has joined #openstack-swift | 15:12 | |
| *** Trixboxer has joined #openstack-swift | 15:18 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: EC: reconstruct using non-durable fragments https://review.openstack.org/376630 | 15:21 |
| openstackgerrit | Alistair Coles proposed openstack/swift: EC: reconstruct using non-durable fragments https://review.openstack.org/376630 | 15:24 |
| jrichli | Has anyone configured swift to work as a request processor of the Apache2 server as described in http://docs.openstack.org/developer/swift/apache_deployment_guide.html ? | 15:24 |
| *** Trixboxer has quit IRC | 15:26 | |
| acoles | tdasilva: I refactored the probe tests in patch 376630. Needed to rebase due to .durables going away. | 15:26 |
| patchbot | https://review.openstack.org/#/c/376630/ - swift - EC: reconstruct using non-durable fragments | 15:26 |
| *** rcernin has quit IRC | 15:26 | |
| tdasilva | acoles: cool, i will check it out! thanks! | 15:27 |
| tdasilva | notmyname: I saw a comment over the weekend about cutting a new release, I think patch 376630 should be part of it... | 15:27 |
| patchbot | https://review.openstack.org/#/c/376630/ - swift - EC: reconstruct using non-durable fragments | 15:27 |
| *** klrmn has quit IRC | 15:31 | |
| *** sgundur has joined #openstack-swift | 15:37 | |
| *** dmorita has joined #openstack-swift | 15:43 | |
| *** jistr is now known as jistr|biab | 15:46 | |
| *** dmorita has quit IRC | 15:47 | |
| *** david-lyle_ has joined #openstack-swift | 15:52 | |
| *** bill_az has joined #openstack-swift | 15:53 | |
| *** david-lyle has quit IRC | 15:53 | |
| *** hogepodge has joined #openstack-swift | 15:53 | |
| *** gaurangt has joined #openstack-swift | 15:54 | |
| *** gaurangt has left #openstack-swift | 15:54 | |
| *** rcernin has joined #openstack-swift | 16:10 | |
| *** ChubYann has joined #openstack-swift | 16:11 | |
| *** Vinsh_ has joined #openstack-swift | 16:12 | |
| *** Vinsh has quit IRC | 16:12 | |
| *** hseipp has quit IRC | 16:17 | |
| *** Vinsh has joined #openstack-swift | 16:21 | |
| *** Vinsh_ has quit IRC | 16:21 | |
| *** jistr|biab is now known as jistr | 16:22 | |
| *** Vinsh_ has joined #openstack-swift | 16:23 | |
| *** Vinsh has quit IRC | 16:23 | |
| *** Vinsh_ has quit IRC | 16:23 | |
| *** david-lyle_ is now known as david-lyle | 16:24 | |
| *** Vinsh has joined #openstack-swift | 16:24 | |
| *** Vinsh has quit IRC | 16:26 | |
| *** Vinsh has joined #openstack-swift | 16:26 | |
| *** diogogmt has joined #openstack-swift | 16:28 | |
| *** Vinsh has quit IRC | 16:29 | |
| *** dmorita has joined #openstack-swift | 16:30 | |
| *** dmorita has quit IRC | 16:34 | |
| *** diogogmt has quit IRC | 16:37 | |
| *** diogogmt has joined #openstack-swift | 16:38 | |
| *** sgundur has quit IRC | 16:39 | |
| *** sgundur has joined #openstack-swift | 16:46 | |
| *** rledisez has quit IRC | 16:49 | |
| *** sgundur has quit IRC | 16:51 | |
| *** amoralej is now known as amoralej|off | 16:57 | |
| clayg | mattoliverau: i'm open to any sweet configuration ninja dev tricks like that - having a noauth proxy was something I always did - and since I coudln't think of it getting in anyones way I made it the default? | 16:59 |
| clayg | kota_: acoles: any thing you need to handoff with the EC reconstructor? I saw we have the ssync fix up against backports? | 17:00 |
| clayg | dose that include the just released newton? | 17:00 |
| acoles | clayg: there are backports proposed for newton and mitaka | 17:01 |
| clayg | notmyname: procedure wise is there anything we can do if we want to cut a single backport release with the ssync & auditor fixes in the same tag? | 17:01 |
| clayg | is liberty dead? | 17:01 |
| acoles | patch 387123 | 17:01 |
| patchbot | https://review.openstack.org/#/c/387123/ - swift (stable/newton) - Prevent ssync writing bad fragment data to diskfile | 17:01 |
| acoles | and patch 387172 | 17:02 |
| patchbot | https://review.openstack.org/#/c/387172/ - swift (stable/mitaka) - Prevent ssync writing bad fragment data to diskfile | 17:02 |
| acoles | clayg: did you statr work on an auditor fix? I just started working on it... | 17:02 |
| *** sgundur has joined #openstack-swift | 17:03 | |
| *** jordanP has quit IRC | 17:06 | |
| clayg | acoles: i haven't started - and I need to bounce off this am for another thing - be happy to take over what you have? | 17:07 |
| clayg | ... this afternoon/evening | 17:07 |
| acoles | clayg: ok I'll plough on, so far I just wrote a test. my thinking is that the diskfile should do the check rather than in auditor, seems to sit with the other etag checking that DiskFileReader does | 17:08 |
| *** klrmn has joined #openstack-swift | 17:11 | |
| *** zaitcev has joined #openstack-swift | 17:13 | |
| *** ChanServ sets mode: +v zaitcev | 17:13 | |
| *** dmorita has joined #openstack-swift | 17:16 | |
| *** stradling has joined #openstack-swift | 17:18 | |
| clayg | acoles: I was also leaning that way - let us catch it on proxy read as well | 17:18 |
| acoles | clayg: good point | 17:19 |
| clayg | good thing we have the ECDiskFile for ec specific behaiors at the object layer! You remember that one time we thoguth "wouldn't it be neat if the object server didn't know/care it was storying EC data!?" | 17:19 |
| clayg | past me was so naive | 17:19 |
| clayg | acoles: cool yeah push up a WIP i'll look for it in a bit | 17:20 |
| *** tqtran has joined #openstack-swift | 17:21 | |
| *** dmorita has quit IRC | 17:21 | |
| acoles | clayg: k | 17:22 |
| openstackgerrit | Hanxi Liu proposed openstack/swift: Add links to more detailed overview in overview_architecture https://review.openstack.org/381446 | 17:23 |
| *** psachin` has quit IRC | 17:24 | |
| *** vinsh has joined #openstack-swift | 17:29 | |
| *** sheel has joined #openstack-swift | 17:36 | |
| *** stradling has quit IRC | 17:46 | |
| *** dmorita has joined #openstack-swift | 17:48 | |
| *** dmorita has quit IRC | 17:52 | |
| *** geaaru has quit IRC | 17:54 | |
| openstackgerrit | Ondřej Nový proposed openstack/swift: Use less restrictive permission for recon cache https://review.openstack.org/387591 | 17:57 |
| onovy | ^^ anyone have working drive-audit in production with recon middleware? because i don't know how it could work without this change :) | 17:58 |
| *** sgundur has quit IRC | 17:59 | |
| *** stradling has joined #openstack-swift | 18:03 | |
| *** sgundur has joined #openstack-swift | 18:06 | |
| *** cschwede|mtg is now known as cschwede | 18:23 | |
| *** vint_bra has joined #openstack-swift | 18:23 | |
| *** Guest21077 has quit IRC | 18:29 | |
| *** dmorita has joined #openstack-swift | 18:34 | |
| *** dmorita has quit IRC | 18:39 | |
| *** stradling has quit IRC | 18:43 | |
| openstackgerrit | Ondřej Nový proposed openstack/swift: Use less restrictive permission for recon cache https://review.openstack.org/387591 | 18:43 |
| *** sgundur has quit IRC | 18:56 | |
| *** admin6 has quit IRC | 19:01 | |
| *** sgundur has joined #openstack-swift | 19:09 | |
| *** StraubTW has quit IRC | 19:10 | |
| *** StraubTW has joined #openstack-swift | 19:10 | |
| clayg | onovy: I don't really love that solution - but I also don't use swift-drive-audit - perhaps we could track down donough acoles or locan brown and get some context on how the thing worked originally | 19:14 |
| onovy | clayg: hi. my pov is: we have world readable swift configs, rings and pid files. anyone can set permissions to /var/cache/swift directory to correctly handle data security | 19:15 |
| onovy | i know there is better solution, for example oslo.rootwrap for swift-drive-audit | 19:16 |
| onovy | but to be honest, it think this is simple working no-one-harms solution | 19:16 |
| onovy | only reason why it's not 0x644 by default is using temp file (ugly) hack | 19:17 |
| *** dmorita has joined #openstack-swift | 19:21 | |
| *** dmorita has quit IRC | 19:26 | |
| jrichli | notmyname, timburke, onovy, anyone else: I'd like you vote on how to proceed with patch 218490 | 19:27 |
| patchbot | https://review.openstack.org/#/c/218490/ - swift - Automatic refresh of memcache config settings | 19:27 |
| jrichli | There are differing opinions on whether or not to default to current behavior or the added behavior of automatically refreshing memcache config settings. | 19:27 |
| onovy | ah, memcache :) | 19:27 |
| jrichli | mattoliverau voted to default to auto-refresh | 19:27 |
| * onovy votes to disable auto-refresh, which is current behaviour | 19:28 | |
| *** silor has quit IRC | 19:28 | |
| jrichli | great, thank you. so we have 1 to 1 now. need more votes! | 19:28 |
| onovy | do i have more than one vote? :) | 19:28 |
| * onovy votes again to not enable auto-refresh by default | 19:29 | |
| jrichli | nice :-) | 19:29 |
| onovy | ah, you are not vote bot! damn. | 19:29 |
| jrichli | noted as a strong vote for current behavior default! | 19:29 |
| onovy | :)) | 19:29 |
| onovy | thanks | 19:29 |
| clayg | jrichli: I'm not sure I'm against the feature - but i like the consistency we have currently - somewhat shy on why memcache should have a special-case beahvior? | 19:34 |
| clayg | if you're dropping a node out of the memache ring - what's wrong with sending HUP? | 19:34 |
| clayg | is this part of a larger effort to try and apply non-deterministic config reloading to all of swift? | 19:35 |
| onovy | clayg: see my comment in review. i had almost same argument | 19:35 |
| onovy | use unix-standardized-way (tm) for config reloading => SIGHUP | 19:35 |
| clayg | sorry, long review history and it's currently clear | 19:35 |
| clayg | ... er there's no -1 up at the top (fresh patch?) | 19:36 |
| onovy | fresh patch and i give up :) | 19:36 |
| onovy | but it's my first comment | 19:37 |
| jrichli | clayg: it was something people asked for in the Vancouver summit ops session. | 19:37 |
| jrichli | clayg: from notmyname on the history: "No, reloading on a SIGHUP isn't needed (at least at this point). This feature is mirroring the way that rings are reloaded automatically: periodically based on mtime instead of through explicit operator action. And, in fact, SIGHUB is already handled by swift's processes to mean "graceful shutdown"." | 19:38 |
| clayg | hrmm... interesting | 19:38 |
| onovy | btw: ring file is "config" file too and i'm completly fine to reload it only on SIGHUP | 19:38 |
| clayg | jrichli: that's from an epad somewhere? | 19:38 |
| clayg | onovy: same here! | 19:39 |
| onovy | so i don't like any config-auto-reloading at all, that's it :) | 19:39 |
| onovy | but i'm fine if it's optional feature, disabled by default | 19:39 |
| clayg | jrichli: ^ my vote is like that too | 19:39 |
| onovy | clayg: ok, can we go back to chmod discussion pls? :) | 19:40 |
| clayg | sorry - i'm stilling bring up some stuff to look at that more closely | 19:40 |
| clayg | onovy: ^ RE: permissions of recon drops & other config artifacts | 19:41 |
| onovy | ok, np | 19:41 |
| jrichli | clayg: search for 'memcache' in https://etherpad.openstack.org/p/liberty-swift-ops-feedback | 19:42 |
| jrichli | and sorry to interrupt the conversation | 19:43 |
| clayg | jrichli: onovy: maybe we need to double check that HUP does reload the memcache config? I almost don't see how it could *not* - maybe something about the way paste is dealing with configs or specifically the external memcache config? | 19:44 |
| onovy | SIGHUP ends swift process now | 19:44 |
| onovy | so yes, it "reload" memcache.conf :) | 19:44 |
| clayg | onovy: yeah, certainly the wsgi parent is died and new one is spawned - so it *must* - I don't think any daemons respond to HUP | 19:45 |
| onovy | in unix world, SIGTERM = stop daemons cleanly, SIGHUP = reloads config. in swift, SIGTERM = stop daemons, kill connections, SIGHUP = stop daemon cleanly | 19:45 |
| clayg | the comment in the etherpad is strange "yeah that a bug" what's a bug? missing autoreload is obvioulsy a feature. someone must have suggested that reload doesn't work... but I don't see how that can be true | 19:46 |
| clayg | onovy: yeah that sounds about right! :) | 19:46 |
| onovy | clayg: this is bug ^ i think :) | 19:47 |
| onovy | not missing memcache autoreloader | 19:47 |
| *** sheel has quit IRC | 19:50 | |
| jrichli | if you want consistency, then you'd have to stick with re-start to re-load. | 19:50 |
| jrichli | otherwise, if you want this to be an optional feature but invoked by a signal, is there a different signal you'd want to use? | 19:50 |
| onovy | i should be optional feature invoked automagically | 19:51 |
| openstackgerrit | Alistair Coles proposed openstack/swift: WIP: Make ECDiskFileReader check fragment metadata https://review.openstack.org/387655 | 19:51 |
| onovy | not by a signal | 19:51 |
| jrichli | ok | 19:51 |
| acoles | clayg: it passes pep8 checks :) ^^ | 19:51 |
| acoles | clayg: patch 387655 | 19:51 |
| patchbot | https://review.openstack.org/#/c/387655/ - swift - WIP: Make ECDiskFileReader check fragment metadata | 19:51 |
| clayg | acoles: go to bed! | 19:52 |
| clayg | acoles: thanks | 19:52 |
| acoles | clayg: there's a bunch of unit tests failing I guess because they don't have valid EC frags in their diskfiles, ho hum! and yeah I wanna go to bed/beer | 19:52 |
| acoles | clayg: TODO notes in commit message, your already co-authored so i expect to see it merged by tomorrow am ;) | 19:53 |
| clayg | acoles: ah of course - ho hum indeed | 19:53 |
| clayg | acoles: har har | 19:53 |
| acoles | clayg: we could make the audit change configurable and default to off so the current tests all pass | 19:53 |
| acoles | clayg: JK | 19:53 |
| acoles | kota_: FYI patch 387655 ^^ | 19:54 |
| patchbot | https://review.openstack.org/#/c/387655/ - swift - WIP: Make ECDiskFileReader check fragment metadata | 19:54 |
| acoles | good night | 19:54 |
| clayg | onovy: RE permissions I'm ok with 644 - but i'm less good with enforcing it in code and leaving the recon dumps owned by root (as I said in all of my setups' they are owned by swift user) | 19:55 |
| onovy | but it will be still owned by swift user in your setup | 19:55 |
| clayg | onovy: you said use of some such temp file forces them to 0600? I'm concerned that perhaps umask on system is 0600 and set to 0644 in code with no way to change - I would perfer to respect configuration of the system if possible - and not do anything that makes it exceedingly hard for operator to get default system behavior | 19:56 |
| onovy | so maybe we should read umask value and set permission to it? | 19:57 |
| clayg | onovy: I think swift-drive-audit requireing elevated permissions for mucking with fstab and unmount disks is reasonable - but otherwise would expect it to create files in /var/cache/swift as configured user - I think that is the bug that has existed since the feature was introduced | 19:57 |
| onovy | so 1. we should run swift-drive-audit by root | 19:58 |
| onovy | 2. change owner of /var/cache/swift to user from config? | 19:58 |
| onovy | and keep 0600? | 19:58 |
| patchbot | https://review.openstack.org/#/c/600/ - nova - live migration/boot-from-volume test completed (ABANDONED) | 19:58 |
| onovy | lol. | 19:58 |
| clayg | onovy: that is perhaps a solution - but i'm not sure that fixes the issue when root creates the file - or allows non root to overwrite? | 19:58 |
| onovy | if drive-audit will run by root, it can overwrite anything | 19:59 |
| clayg | onovy: but don't all things update same recon file with their stats? | 19:59 |
| onovy | yep and all files will be owned by swift | 19:59 |
| onovy | so it's fine | 19:59 |
| onovy | 1. create temp file, write content, owner will be process owner (root in drive-audit) | 20:00 |
| onovy | 2. chown it to swift | 20:00 |
| onovy | 3. rename | 20:00 |
| *** acoles is now known as acoles_ | 20:01 | |
| clayg | onovy: yeah that plus maybe use a .recon-file.update.tmp or something instead of /tmp | 20:01 |
| onovy | but that's another story/patch | 20:01 |
| onovy | (imho) | 20:02 |
| clayg | onovy: yeah so I think if root creates a file as root even with 644 - the swift user can still overwrite it I guess because he has permissions in the directory | 20:02 |
| *** tqtran has quit IRC | 20:02 | |
| onovy | yep, but that's not case. this will never happen | 20:02 |
| clayg | the swift-drive-audit never wins the race to create the dump cache file? | 20:03 |
| onovy | it will create file in /tmp | 20:03 |
| onovy | than change owner to swift | 20:03 |
| onovy | and than rename | 20:03 |
| onovy | so it will never exists as root in cache dir | 20:03 |
| clayg | oh - but I thought you were suggesting that should be a follow up? | 20:03 |
| clayg | in your existing change - it still works because swift user can read (644) and overwrite/rename (owns the dir) | 20:04 |
| onovy | ah, no. i can change corrent os.chmod to chown | 20:04 |
| clayg | onovy: ah, yeah I like that quite a bit then | 20:04 |
| onovy | i want another patch for 'maybe use a .recon-file.update.tmp or something instead of /tmp' | 20:04 |
| onovy | that's all | 20:04 |
| clayg | so you were saying the use a not /tmp file being an follow up | 20:04 |
| clayg | #weareinagreemnt | 20:04 |
| onovy | yep | 20:05 |
| onovy | cool, so i will write code now :) | 20:05 |
| clayg | cool - good chat - always a pleasure | 20:05 |
| clayg | thanks for making swift better! | 20:05 |
| onovy | thanks too | 20:05 |
| *** dmorita has joined #openstack-swift | 20:08 | |
| openstackgerrit | Shashirekha Gundur proposed openstack/swift: Invalidate cached tokens api https://review.openstack.org/370319 | 20:08 |
| *** bill_az has quit IRC | 20:09 | |
| *** bill_az has joined #openstack-swift | 20:11 | |
| *** dmorita has quit IRC | 20:12 | |
| *** sgundur has quit IRC | 20:16 | |
| *** sgundur has joined #openstack-swift | 20:17 | |
| *** pcaruana has quit IRC | 20:25 | |
| *** tqtran has joined #openstack-swift | 20:33 | |
| *** admin6 has joined #openstack-swift | 20:45 | |
| mattoliverau | Morning | 20:51 |
| *** dmorita has joined #openstack-swift | 20:55 | |
| *** dmorita has quit IRC | 21:00 | |
| *** sgundur has quit IRC | 21:02 | |
| *** mvk has joined #openstack-swift | 21:05 | |
| *** sgundur has joined #openstack-swift | 21:09 | |
| *** dmorita has joined #openstack-swift | 21:26 | |
| *** slunkad has joined #openstack-swift | 21:30 | |
| *** slunkad has quit IRC | 21:30 | |
| *** dmorita has quit IRC | 21:31 | |
| *** sgundur has quit IRC | 21:43 | |
| *** vint_bra has quit IRC | 21:45 | |
| openstackgerrit | Ondřej Nový proposed openstack/swift: Set owner of drive-audit recon cache to swift user https://review.openstack.org/387591 | 21:49 |
| onovy | clayg: ^ better? :) | 21:49 |
| onovy | gn | 21:54 |
| *** bill_az has quit IRC | 22:03 | |
| *** bill_az has joined #openstack-swift | 22:05 | |
| *** rcernin has quit IRC | 22:06 | |
| clayg | onovy: it's not bad | 22:08 |
| *** bill_az has quit IRC | 22:12 | |
| *** dmorita has joined #openstack-swift | 22:13 | |
| zaitcev | clayg: ewww, this is exactly what I was afraid of | 22:15 |
| *** dmorita has quit IRC | 22:18 | |
| clayg | zaitcev: what what'd i do? | 22:19 |
| clayg | zaitcev: you and onovy do whatever you think is best for that script - i shouldn't have stepped in it | 22:20 |
| clayg | ;P | 22:20 |
| zaitcev | I'm unsure too. It just looks like an overcomplication now. | 22:20 |
| clayg | zaitcev: you liked patch #2 better with the 0644 thing - I don't really like forcing that umask on those files - I think the whole thing is janked' up | 22:20 |
| zaitcev | You think this is bad, but we have a similar issue with ceilometer, only worse. | 22:21 |
| zaitcev | Worse, because Swift does not know what files the ceilometer middleware is creating behind our backs. | 22:22 |
| clayg | zaitcev: maybe the rootwrap shenanigans is worth it? you want over complicated - I sorta like an early fork with drop priv then use a multiprocessing queue to send over the recon updates | 22:22 |
| zaitcev | ouch | 22:22 |
| zaitcev | But yes, I thought about an IPC. Just didn't want to actually develop a patch. | 22:23 |
| clayg | ... same | 22:23 |
| clayg | :) | 22:23 |
| *** StraubTW has quit IRC | 22:47 | |
| *** bill_az has joined #openstack-swift | 22:52 | |
| *** admin6 has quit IRC | 22:53 | |
| *** dmorita has joined #openstack-swift | 22:53 | |
| *** admin6 has joined #openstack-swift | 22:54 | |
| zaitcev | A good thing though - nobody suggested stat.S_IREAD|stat.S_IWRITE|stat.S_IRGRP|stat.S_IROTH instead of 0o644 | 23:06 |
| clayg | rofl | 23:06 |
| clayg | zaitcev: now you have to look up the thread where linus rails on those? | 23:07 |
| zaitcev | clayg: I didn't know, I'll look | 23:07 |
| *** klamath has quit IRC | 23:29 | |
| *** kei_yama has joined #openstack-swift | 23:35 | |
| *** chlong has joined #openstack-swift | 23:40 | |
| *** _JZ_ has joined #openstack-swift | 23:42 | |
| *** vint_bra has joined #openstack-swift | 23:44 | |
| *** admin6 has quit IRC | 23:51 | |
| *** dmorita has quit IRC | 23:52 | |
| *** vint_bra has quit IRC | 23:53 | |
| jamielennox | hey all, https://review.openstack.org/#/c/298968/ is the patch that adds basic keystoneauth support to swiftclient - any chance we can progress this? | 23:56 |
| patchbot | patch 298968 - python-swiftclient - Adding keystoneauth sessions support | 23:56 |
| *** vint_bra has joined #openstack-swift | 23:56 | |
| jamielennox | it would really simplify the glance_store swift backend | 23:56 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!