| *** tone_zrt has joined #openstack-swift | 00:28 | |
| *** vint_bra1 has quit IRC | 00:35 | |
| *** tovin07_ has joined #openstack-swift | 01:27 | |
| *** mat128 has joined #openstack-swift | 01:30 | |
| *** d0ugal has quit IRC | 01:36 | |
| *** psachin has joined #openstack-swift | 01:39 | |
| *** m_kazuhiro has joined #openstack-swift | 01:44 | |
| *** d0ugal has joined #openstack-swift | 02:00 | |
| *** mat128 has quit IRC | 02:23 | |
| *** mat128 has joined #openstack-swift | 02:24 | |
| *** dja has quit IRC | 02:29 | |
| *** mat128 has quit IRC | 02:29 | |
| *** dja has joined #openstack-swift | 02:29 | |
| *** dja has quit IRC | 03:03 | |
| *** dja has joined #openstack-swift | 03:04 | |
| *** klrmn has quit IRC | 03:25 | |
| *** mat128 has joined #openstack-swift | 03:27 | |
| *** zaitcev has quit IRC | 03:32 | |
| *** wes_dillingham has quit IRC | 03:34 | |
| *** gkadam has joined #openstack-swift | 03:35 | |
| *** gkadam_ has joined #openstack-swift | 03:40 | |
| *** gkadam has quit IRC | 03:41 | |
| *** dja has quit IRC | 03:41 | |
| *** kei_yama has quit IRC | 03:42 | |
| *** dja has joined #openstack-swift | 03:42 | |
| kota_ | hello world | 03:50 |
|---|---|---|
| *** kei_yama has joined #openstack-swift | 04:02 | |
| mattoliverau | kota_: o/ | 04:16 |
| *** dja has quit IRC | 04:16 | |
| *** dja has joined #openstack-swift | 04:16 | |
| kota_ | mattoliverau: | 04:18 |
| kota_ | mattoliverau: o/ | 04:18 |
| *** Dinesh_Bhor has joined #openstack-swift | 04:19 | |
| *** thurloat has joined #openstack-swift | 04:26 | |
| *** m_kazuhiro_ has joined #openstack-swift | 04:32 | |
| *** m_kazuhiro has quit IRC | 04:35 | |
| *** mabrams has joined #openstack-swift | 04:37 | |
| *** SkyRocknRoll has joined #openstack-swift | 04:45 | |
| *** dja has quit IRC | 04:51 | |
| *** dja has joined #openstack-swift | 04:51 | |
| *** mabrams has quit IRC | 05:19 | |
| *** mabrams has joined #openstack-swift | 05:22 | |
| *** dja has quit IRC | 05:26 | |
| *** dja has joined #openstack-swift | 05:27 | |
| *** cshastri has joined #openstack-swift | 05:32 | |
| *** ChubYann has quit IRC | 05:38 | |
| *** spectr has joined #openstack-swift | 05:52 | |
| *** dja has quit IRC | 05:55 | |
| *** dja has joined #openstack-swift | 05:58 | |
| *** geaaru__ has quit IRC | 06:02 | |
| *** thurloat has quit IRC | 06:27 | |
| *** dja has quit IRC | 06:33 | |
| *** dja has joined #openstack-swift | 06:33 | |
| *** kallenp has joined #openstack-swift | 06:34 | |
| *** kallenp has left #openstack-swift | 06:35 | |
| *** hseipp has joined #openstack-swift | 06:58 | |
| *** thurloat has joined #openstack-swift | 06:58 | |
| *** pcaruana has joined #openstack-swift | 07:00 | |
| *** dja has quit IRC | 07:17 | |
| *** dja has joined #openstack-swift | 07:18 | |
| *** thurloat has quit IRC | 07:18 | |
| *** jistr is now known as jistr|mtgs | 07:26 | |
| *** tesseract has joined #openstack-swift | 07:27 | |
| *** m_kazuhiro_ has quit IRC | 07:36 | |
| *** oshritf has joined #openstack-swift | 07:44 | |
| *** thurloat has joined #openstack-swift | 07:47 | |
| acoles | good morning | 07:48 |
| *** geaaru has joined #openstack-swift | 07:48 | |
| *** dja has quit IRC | 07:53 | |
| *** dja has joined #openstack-swift | 07:54 | |
| *** cbartz has joined #openstack-swift | 08:02 | |
| *** cbartz has quit IRC | 08:15 | |
| *** cbartz has joined #openstack-swift | 08:28 | |
| *** dja has quit IRC | 08:28 | |
| *** dja has joined #openstack-swift | 08:29 | |
| *** spectr has quit IRC | 08:35 | |
| *** spectr has joined #openstack-swift | 08:49 | |
| *** dja has quit IRC | 09:03 | |
| *** dja has joined #openstack-swift | 09:03 | |
| *** mvk has quit IRC | 09:27 | |
| *** spectr has quit IRC | 09:31 | |
| *** spectr has joined #openstack-swift | 09:31 | |
| *** dja has quit IRC | 09:42 | |
| *** dja has joined #openstack-swift | 09:43 | |
| *** kei_yama has quit IRC | 09:55 | |
| *** mvk has joined #openstack-swift | 10:02 | |
| *** spectr has quit IRC | 10:04 | |
| *** tovin07_ has quit IRC | 10:12 | |
| *** dja has quit IRC | 10:17 | |
| *** dja has joined #openstack-swift | 10:18 | |
| *** spectr has joined #openstack-swift | 10:20 | |
| openstackgerrit | HCLTech-SSW proposed openstack/swift master: Add ability to undelete an account. https://review.openstack.org/507808 | 10:36 |
| *** spectr has quit IRC | 10:45 | |
| *** spectr has joined #openstack-swift | 10:47 | |
| openstackgerrit | Merged openstack/swift feature/deep: sharder needs rsync module conf option https://review.openstack.org/513788 | 10:51 |
| openstackgerrit | Merged openstack/swift master: Clean up a couple hand-rolled mocks. https://review.openstack.org/513870 | 10:51 |
| openstackgerrit | Merged openstack/swift master: Clean up memcache tests https://review.openstack.org/513858 | 10:51 |
| *** dja has quit IRC | 10:52 | |
| *** dja has joined #openstack-swift | 10:52 | |
| *** oshritf_ has joined #openstack-swift | 11:04 | |
| *** oshritf has quit IRC | 11:06 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-swiftclient master: Updated from global requirements https://review.openstack.org/89250 | 11:19 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements https://review.openstack.org/88736 | 11:21 |
| *** dja has quit IRC | 11:27 | |
| *** dja has joined #openstack-swift | 11:27 | |
| *** silor has joined #openstack-swift | 11:33 | |
| *** kallenp has joined #openstack-swift | 11:44 | |
| *** oshritf_ has quit IRC | 11:51 | |
| *** kallenp has left #openstack-swift | 11:54 | |
| *** NM has joined #openstack-swift | 12:01 | |
| *** wes_dillingham has joined #openstack-swift | 12:03 | |
| *** dja has quit IRC | 12:08 | |
| *** dja has joined #openstack-swift | 12:09 | |
| *** silor has quit IRC | 12:12 | |
| *** silor has joined #openstack-swift | 12:12 | |
| *** dhellmann has quit IRC | 12:15 | |
| *** dhellmann has joined #openstack-swift | 12:16 | |
| *** oshritf_ has joined #openstack-swift | 12:23 | |
| *** mariusv has quit IRC | 12:25 | |
| *** mariusv has joined #openstack-swift | 12:26 | |
| *** pcaruana has quit IRC | 12:41 | |
| *** nikivi has joined #openstack-swift | 12:43 | |
| *** dja has quit IRC | 12:43 | |
| *** dja has joined #openstack-swift | 12:44 | |
| *** gkadam_ has quit IRC | 12:44 | |
| *** kallenp has joined #openstack-swift | 12:45 | |
| *** kallenp has quit IRC | 12:50 | |
| *** mwheckmann has joined #openstack-swift | 12:50 | |
| *** pcaruana has joined #openstack-swift | 12:57 | |
| *** pcaruana has quit IRC | 13:01 | |
| *** mat128 has quit IRC | 13:03 | |
| *** dja has quit IRC | 13:19 | |
| *** dja has joined #openstack-swift | 13:19 | |
| *** SkyRocknRoll has quit IRC | 13:20 | |
| *** MeltedLux has quit IRC | 13:30 | |
| *** MeltedLux has joined #openstack-swift | 13:33 | |
| *** mat128 has joined #openstack-swift | 13:45 | |
| *** mabrams has quit IRC | 13:54 | |
| *** dja has quit IRC | 13:54 | |
| *** dja has joined #openstack-swift | 13:54 | |
| *** MeltedLux has quit IRC | 14:03 | |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep: simple tool to display shard relationships https://review.openstack.org/514329 | 14:10 |
| *** zaitcev has joined #openstack-swift | 14:19 | |
| *** ChanServ sets mode: +v zaitcev | 14:19 | |
| *** dja has quit IRC | 14:29 | |
| *** dja has joined #openstack-swift | 14:29 | |
| *** wes_dillingham has quit IRC | 14:31 | |
| *** wes_dillingham has joined #openstack-swift | 14:33 | |
| *** cshastri has quit IRC | 14:35 | |
| *** MeltedLux has joined #openstack-swift | 14:38 | |
| *** psachin has quit IRC | 14:42 | |
| *** kallenp has joined #openstack-swift | 14:51 | |
| *** MeltedLux has quit IRC | 14:52 | |
| *** dja has quit IRC | 15:04 | |
| *** dja has joined #openstack-swift | 15:05 | |
| *** mjturek has joined #openstack-swift | 15:06 | |
| *** klrmn has joined #openstack-swift | 15:08 | |
| cschwede | notmyname: Hi! Can you have a look at patch 505092, please? It's a backport that only you can approve (afaict). Would be great to get this into Pike | 15:20 |
| patchbot | https://review.openstack.org/#/c/505092/ - python-swiftclient (stable/pike) - Add support for versionless endpoints | 15:20 |
| *** MeltedLux has joined #openstack-swift | 15:23 | |
| *** mjturek1 has joined #openstack-swift | 15:29 | |
| *** mjturek has quit IRC | 15:30 | |
| *** gyee has joined #openstack-swift | 15:37 | |
| *** dja has quit IRC | 15:39 | |
| *** dja has joined #openstack-swift | 15:40 | |
| *** wes_dillingham has quit IRC | 15:42 | |
| *** wes_dillingham has joined #openstack-swift | 15:43 | |
| *** MeltedLux has quit IRC | 15:45 | |
| *** SkyRocknRoll has joined #openstack-swift | 15:50 | |
| *** MeltedLux has joined #openstack-swift | 15:57 | |
| notmyname | good morning | 15:57 |
| notmyname | cschwede: ack | 15:57 |
| *** vint_bra has joined #openstack-swift | 15:59 | |
| clayg | cschwede: ok, thanks for the ping | 16:09 |
| clayg | cschwede: if I had extra review bandwidth today I'd probably burn it on patch 507808 or patch 509306 (and it's pre-req) | 16:10 |
| patchbot | https://review.openstack.org/#/c/507808/ - swift - Add ability to undelete an account. | 16:10 |
| patchbot | https://review.openstack.org/#/c/509306/ - swift - Let clients request heartbeats during SLO PUTs | 16:10 |
| *** klrmn has quit IRC | 16:14 | |
| *** dja has quit IRC | 16:16 | |
| *** HCLTech-SSW has joined #openstack-swift | 16:16 | |
| *** dja has joined #openstack-swift | 16:17 | |
| timburke | good morning | 16:17 |
| *** itlinux has joined #openstack-swift | 16:18 | |
| *** HCLTech-SSW_ has joined #openstack-swift | 16:19 | |
| *** HCLTech-SSW has quit IRC | 16:20 | |
| *** hseipp has quit IRC | 16:22 | |
| *** SkyRocknRoll_ has joined #openstack-swift | 16:23 | |
| clayg | HCLTech-SSW_: I saw you updated patch 507808 - I'd definitely like to give it another go this week - last time I looked there was a lack of tests - it seems to be getting a LOT closer and good docs too!!! I'm excited to take another look - I really really hope it's polished up to fine shine and we can get that bad boy on master! You're working hard and doing amazing work! | 16:23 |
| patchbot | https://review.openstack.org/#/c/507808/ - swift - Add ability to undelete an account. | 16:23 |
| *** drewn3ss has quit IRC | 16:23 | |
| *** mjturek1 has quit IRC | 16:33 | |
| *** HCLTech-SSW_ has quit IRC | 16:34 | |
| clayg | umm... https://github.com/google/grumpy | 16:36 |
| clayg | idk, unladen swallow was also a google project as well AFAIK... | 16:37 |
| *** cbartz has quit IRC | 16:43 | |
| *** mjturek has joined #openstack-swift | 16:45 | |
| *** mvk has quit IRC | 16:49 | |
| *** dja has quit IRC | 16:51 | |
| *** dja has joined #openstack-swift | 16:51 | |
| *** NM has quit IRC | 16:58 | |
| *** NM has joined #openstack-swift | 17:10 | |
| *** SkyRocknRoll_ has quit IRC | 17:10 | |
| *** klrmn has joined #openstack-swift | 17:11 | |
| *** tesseract has quit IRC | 17:18 | |
| *** SkyRocknRoll has quit IRC | 17:23 | |
| *** hseipp has joined #openstack-swift | 17:28 | |
| *** dja has quit IRC | 17:28 | |
| *** dja has joined #openstack-swift | 17:29 | |
| *** NM has quit IRC | 17:31 | |
| *** mvk has joined #openstack-swift | 17:32 | |
| *** hseipp has quit IRC | 17:32 | |
| *** NM has joined #openstack-swift | 17:35 | |
| *** SkyRocknRoll has joined #openstack-swift | 17:37 | |
| *** NM has quit IRC | 17:40 | |
| *** SkyRocknRoll has quit IRC | 17:49 | |
| *** NM has joined #openstack-swift | 17:55 | |
| *** kallenp has quit IRC | 17:56 | |
| *** kallenp has joined #openstack-swift | 17:56 | |
| *** kallenp has quit IRC | 17:57 | |
| *** kallenp has joined #openstack-swift | 17:57 | |
| *** NM has quit IRC | 17:57 | |
| openstackgerrit | Tim Burke proposed openstack/swift feature/deep: Shard a shard https://review.openstack.org/514406 | 17:58 |
| *** NM has joined #openstack-swift | 18:00 | |
| *** dja has quit IRC | 18:04 | |
| *** dja has joined #openstack-swift | 18:05 | |
| acoles | timburke: nice! | 18:05 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep: Delete orphaned shard containers https://review.openstack.org/513468 | 18:06 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep: probe test undelete https://review.openstack.org/514411 | 18:06 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep: WIP undelete of shard containers plus probe test https://review.openstack.org/514412 | 18:06 |
| acoles | oops | 18:06 |
| *** mjturek1 has joined #openstack-swift | 18:07 | |
| *** mjturek has quit IRC | 18:07 | |
| notmyname | please pay attention to https://wiki.openstack.org/wiki/Swift/PriorityReviews so we can have a release ASAP | 18:07 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep: Delete orphaned shard containers https://review.openstack.org/513468 | 18:08 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep: WIP undelete of shard containers plus probe test https://review.openstack.org/514412 | 18:08 |
| *** SkyRocknRoll has joined #openstack-swift | 18:08 | |
| acoles | mattoliverau: timburke torgomatic I've put some thoughts on the etherpad about deleting/undeleting sharded containers and their shards. I think it requires a change to how sharding metadata is managed. Comments welcome. | 18:10 |
| acoles | https://etherpad.openstack.org/p/deep-containers | 18:10 |
| acoles | at the tail ^^ | 18:11 |
| acoles | the probe test here illustrates the problem https://review.openstack.org/514412 - that patch has a temproray hack to let the metadata live | 18:12 |
| patchbot | patch 514412 - swift (feature/deep) - WIP undelete of shard containers plus probe test | 18:12 |
| *** acoles has left #openstack-swift | 18:14 | |
| *** esnyder has joined #openstack-swift | 18:25 | |
| esnyder | Can anyone point me to the code that would be generating log messages on proxy node like: | 18:26 |
| esnyder | proxy-server: STDERR: ERROR:root:Timeout talking to memcached: 10.166.160.93:11211 | 18:26 |
| clayg | cschwede: I'm not going to review patch 505092 today! sorry - try and remind me again on Friday? | 18:26 |
| patchbot | https://review.openstack.org/#/c/505092/ - python-swiftclient (stable/pike) - Add support for versionless endpoints | 18:26 |
| esnyder | git grep is failing me | 18:26 |
| esnyder | https://www.irccloud.com/pastebin/vftYM2C4/ | 18:28 |
| timburke | esnyder: it's coming out of https://github.com/openstack/swift/blob/2.15.1/swift/common/memcached.py#L166-L167 | 18:28 |
| esnyder | timburke: thanks | 18:28 |
| timburke | yeah, that's the one :-) | 18:28 |
| notmyname | esnyder: either proxy base controller, proxy server, or memcache client | 18:32 |
| notmyname | oh, timburke beat me to it :-) | 18:32 |
| esnyder | thanks to you both. Any pointers regarding where memcache fits into things architecturally? I see thousands to 10s of thousands of those messages in our daily proxy logs, and am trying to understand what it means, what sort of problems it might indicate, and what (if anything) we should do about it? | 18:34 |
| *** silor has quit IRC | 18:34 | |
| notmyname | the proxy servers talk to memcache | 18:35 |
| *** kallenp1 has joined #openstack-swift | 18:36 | |
| notmyname | and they talk to any memcache server defined in the proxy confit (or the memcache config) in /etc/swift/* | 18:36 |
| *** geaaru has quit IRC | 18:36 | |
| notmyname | the proxy stores tokens and account/container info in memcache | 18:36 |
| notmyname | (there's a couple of other things too like rate limiting info) | 18:36 |
| notmyname | mostly tokens and account/container info | 18:36 |
| *** kallenp2 has joined #openstack-swift | 18:37 | |
| notmyname | for eg container info, it's so wedont' have to go to a container disk for every request | 18:37 |
| notmyname | same for accounts | 18:37 |
| notmyname | for tokens, it's so we dont' have to go back to the auth system for every request (if the token is still valid). normally the swift cluster is *much* more scalable than the auth system | 18:37 |
| notmyname | a couple of important things you've probably already realized... | 18:38 |
| notmyname | first, do *NOT* give public access to the memcache servers. restrict them to requests from the proxy servers (or the swift cluster). firewall rules are reasonable here | 18:38 |
| notmyname | second, if you have a geographically disperse cluster (eg part in LA and part in NYC or London), use separate memcache pools for each region. it will really help performance and reliability | 18:39 |
| *** kallenp has quit IRC | 18:40 | |
| *** kallenp1 has quit IRC | 18:40 | |
| esnyder | notmyname: awesome, thanks! my takeaway from this is: these errors likely indicate that some requests are slower than need by (because it's going all the way down for whatever info it expected to check memcache for). | 18:41 |
| esnyder | but i'm still not 100% sure if this indicates a general problem w/ memcache pool, or how to troubleshoot? | 18:42 |
| notmyname | you could check the memcache config for the number of allowed connections to see if it reasonably matches the request rate your cluster is seeing | 18:43 |
| esnyder | cool, thanks | 18:43 |
| notmyname | eg if you've got 10 proxy servers, each with a memcache server, and you're doing 1000 req/sec, each memcache server should expect to see about 100 req/sec (1000/10) | 18:43 |
| notmyname | IIRC the default max memcache connections is 10 or something like that | 18:44 |
| notmyname | otherwise, normal ops monitoring to find out where the bottleneck is (network, cpu, etc) and then adjusting | 18:44 |
| esnyder | ok, thanks again | 18:45 |
| *** NM has quit IRC | 18:47 | |
| *** SkyRocknRoll has quit IRC | 18:50 | |
| *** dja has quit IRC | 18:52 | |
| *** itlinux has quit IRC | 18:52 | |
| *** dja has joined #openstack-swift | 18:52 | |
| *** kallenp2 has quit IRC | 18:53 | |
| *** SkyRocknRoll has joined #openstack-swift | 18:53 | |
| *** itlinux has joined #openstack-swift | 18:56 | |
| *** NM has joined #openstack-swift | 18:59 | |
| *** ChubYann has joined #openstack-swift | 19:10 | |
| *** dja has quit IRC | 19:27 | |
| *** dja has joined #openstack-swift | 19:27 | |
| *** SkyRocknRoll has quit IRC | 19:30 | |
| *** chsc has joined #openstack-swift | 19:30 | |
| *** chsc has joined #openstack-swift | 19:30 | |
| *** newmember has quit IRC | 19:38 | |
| *** itlinux has quit IRC | 19:39 | |
| *** wes_dillingham has quit IRC | 19:56 | |
| *** dja has quit IRC | 20:04 | |
| *** dja has joined #openstack-swift | 20:04 | |
| *** NM has quit IRC | 20:09 | |
| *** mat128 has quit IRC | 20:24 | |
| *** dja has quit IRC | 20:40 | |
| *** dja has joined #openstack-swift | 20:42 | |
| *** dja has quit IRC | 21:16 | |
| *** dja has joined #openstack-swift | 21:17 | |
| *** mwheckmann has quit IRC | 21:38 | |
| *** wes_dillingham has joined #openstack-swift | 21:41 | |
| *** dja has quit IRC | 21:52 | |
| *** dja has joined #openstack-swift | 21:54 | |
| mattoliverau | morning | 21:59 |
| *** dja has quit IRC | 22:28 | |
| *** dja has joined #openstack-swift | 22:28 | |
| *** dja has quit IRC | 23:03 | |
| *** dja has joined #openstack-swift | 23:03 | |
| *** vint_bra has quit IRC | 23:21 | |
| *** kei_yama has joined #openstack-swift | 23:24 | |
| *** mjturek1 has quit IRC | 23:30 | |
| *** chsc has quit IRC | 23:35 | |
| *** mat128 has joined #openstack-swift | 23:47 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!