| *** mikecmpbll has quit IRC | 00:45 | |
| *** gyee has quit IRC | 00:54 | |
| *** takamatsu_ has joined #openstack-swift | 01:26 | |
| *** takamatsu has quit IRC | 01:26 | |
| openstackgerrit | Merged openstack/python-swiftclient master: Update hacking version https://review.openstack.org/627724 | 02:45 |
|---|---|---|
| *** ccamacho has quit IRC | 02:48 | |
| *** two_tired has joined #openstack-swift | 03:01 | |
| *** psachin has joined #openstack-swift | 03:04 | |
| *** two_tired has quit IRC | 05:02 | |
| *** gmann has quit IRC | 05:28 | |
| openstackgerrit | Merged openstack/swift master: py3: port some more middleware tests https://review.openstack.org/638287 | 05:42 |
| zaitcev | So, this went in, but versioned writes didn't. | 05:56 |
| mattoliverau | version writes needed another recheck :( | 06:30 |
| *** e0ne has joined #openstack-swift | 07:29 | |
| *** e0ne has quit IRC | 07:43 | |
| *** e0ne has joined #openstack-swift | 07:44 | |
| *** e0ne has quit IRC | 07:44 | |
| *** e0ne has joined #openstack-swift | 07:50 | |
| *** e0ne has quit IRC | 08:10 | |
| *** tkajinam has quit IRC | 08:13 | |
| *** hseipp has joined #openstack-swift | 08:18 | |
| *** mikecmpbll has joined #openstack-swift | 09:06 | |
| *** hseipp has quit IRC | 09:31 | |
| *** mikecmpbll has quit IRC | 09:38 | |
| *** mikecmpbll has joined #openstack-swift | 09:39 | |
| *** takamatsu_ has quit IRC | 10:04 | |
| *** takamatsu has joined #openstack-swift | 10:05 | |
| *** early has quit IRC | 10:15 | |
| *** e0ne has joined #openstack-swift | 10:19 | |
| *** early has joined #openstack-swift | 10:21 | |
| *** takamatsu_ has joined #openstack-swift | 10:23 | |
| *** takamatsu has quit IRC | 10:24 | |
| *** rcernin has quit IRC | 10:31 | |
| *** hseipp has joined #openstack-swift | 10:42 | |
| *** takamatsu_ has quit IRC | 10:48 | |
| *** takamatsu has joined #openstack-swift | 10:49 | |
| *** ccamacho has joined #openstack-swift | 10:54 | |
| *** ccamacho has quit IRC | 10:54 | |
| *** takamatsu has quit IRC | 10:54 | |
| *** takamatsu has joined #openstack-swift | 10:58 | |
| kota_ | alecuyer: sorry I couldn't make my time about LOSF today but you seems to find the newer grpcio would work with eventlet. | 11:13 |
| kota_ | if the newer (and lower from 1.15.0) is fine, we may change the patch to set just lower requirement of grpcio in requirements.txt rather than mangle the constarints settings. | 11:14 |
| kota_ | i.e. `grpcio >= <enough version to able to run>` is easier. | 11:15 |
| *** gmann has joined #openstack-swift | 11:49 | |
| alecuyer | kota_: no worries. It does seem to work with 1.15.0 and i'm running git bisect to find where it started working, but that takes some time | 12:09 |
| *** guimaluf has joined #openstack-swift | 12:26 | |
| guimaluf | hi folks, I've a production cluster running swift and keystone at kilo version. My ring is splited in two regions with 4 nodes each. One of our regions is completely off and we found that one file is lost cause all the three copies are in the same region, the one is out... why is this happen? swift should not put a file within the same region, I always trust in that premise... my storage nodes are far away from been full... I | 12:39 |
| guimaluf | woudl appreciate if someone could shed light in the subject | 12:39 |
| *** ccamacho has joined #openstack-swift | 12:39 | |
| guimaluf | checking dispersion report I realize that 189 objects are lost | 12:51 |
| alecuyer | guimaluf: I am not familiar with multi region, but are you using write affinity ? (I understand this will make copies to other regions asynchronously) | 12:57 |
| DHE | unfortunately it's not a guarantee, just a best effort. maybe the ring needed periodic rebalancing, or the regions were not well balanced in terms of [sum of] device weights | 13:11 |
| DHE | there is a way to make a guarantee in the current version, but it's an app marked as "experimental" and you'll always have exactly the same number of copies of an object in each region | 13:11 |
| guimaluf | thank you DHE and alecuyer. now, how can I investigate which objects or partitions were affected? | 13:23 |
| guimaluf | can I namely the partitions that are within the same region? | 13:23 |
| guimaluf | I imagine that is impossible to figure out which or how many files were there, right? | 13:24 |
| DHE | how many zones per region? | 13:25 |
| guimaluf | r1z{1..4} and r2z1 | 13:26 |
| guimaluf | zones were servers, that's why when I add r2 I set up only z1 | 13:26 |
| guimaluf | added* | 13:26 |
| DHE | and r1 is down? | 13:26 |
| guimaluf | r2 was deleted | 13:27 |
| guimaluf | not down :/ removed... blown up | 13:27 |
| DHE | oh... that's not expected... | 13:27 |
| guimaluf | yes... :/ that's why I'm a little bit in dispair | 13:28 |
| guimaluf | I've never realize that 0.58 dispersion could do this kind of thing... | 13:29 |
| DHE | well, the report command you want is probably: swift-ring-builder [filename.builder] dispersion --verbose | 13:29 |
| guimaluf | now I need to know how many objects were affected.. | 13:29 |
| guimaluf | I did this already | 13:29 |
| guimaluf | I know that 189 partitions were at r2 | 13:29 |
| guimaluf | swift-ring-builder /etc/swift/object.builder list_parts r2 | grep '3$' | 13:32 |
| *** psachin has quit IRC | 13:35 | |
| guimaluf | how can I find more info about partition? which server/device/zone its store? | 13:35 |
| *** ccamacho has quit IRC | 13:50 | |
| *** ccamacho has joined #openstack-swift | 13:51 | |
| *** ccamacho has quit IRC | 15:01 | |
| *** e0ne has quit IRC | 16:11 | |
| *** e0ne has joined #openstack-swift | 16:20 | |
| *** hseipp has quit IRC | 16:21 | |
| notmyname | good morning | 16:28 |
| *** e0ne has quit IRC | 16:31 | |
| *** gyee has joined #openstack-swift | 16:40 | |
| *** itlinux has joined #openstack-swift | 16:47 | |
| *** mikecmpbll has quit IRC | 17:04 | |
| alecuyer | kota_: It seems that the introduction of a background python poller thread broke grpc client with gevent (8f7bc54ff2256179ede6d4a6d1a8792b7820fc28). Per the comments, this was a "temporary workaround". This was removed in a75712a71f01c06d7e5868a75cb62b26d775e5df , as the thread is now in C core. With that patch, it does not hang anymore. Im not done yet but I wanted to tell you what I found for now. Ill put more info on the g | 17:08 |
| alecuyer | oogle drive doc | 17:08 |
| alecuyer | s/gevent/eventlet/ | 17:08 |
| *** hseipp has joined #openstack-swift | 17:09 | |
| notmyname | FYI http://lists.openstack.org/pipermail/openstack-discuss/2019-February/003073.html | 17:34 |
| timburke | mattoliverau, huh. well there goes my theory -- https://github.com/python/cpython/commit/dc1650c landed in 3.5.3 ... | 17:41 |
| *** takamatsu_ has joined #openstack-swift | 17:48 | |
| *** takamatsu has quit IRC | 17:48 | |
| *** mikecmpbll has joined #openstack-swift | 17:59 | |
| *** takamatsu_ has quit IRC | 18:03 | |
| *** takamatsu_ has joined #openstack-swift | 18:06 | |
| *** e0ne has joined #openstack-swift | 19:18 | |
| *** hseipp has quit IRC | 19:24 | |
| *** itlinux has quit IRC | 19:41 | |
| * zaitcev is cheering for versioned writes | 19:50 | |
| zaitcev | I'm asking myself if we want to make probe tests more tolerant in some ways. | 19:50 |
| zaitcev | If I become the PTL, I'll need to sit down with the infra guys and study how all that machinery works, what are problems. | 19:51 |
| zaitcev | Maybe it's not just the sponsor money that contributes to this. | 19:51 |
| zaitcev | er, the lack of | 19:51 |
| *** e0ne has quit IRC | 20:08 | |
| *** itlinux has joined #openstack-swift | 20:24 | |
| *** itlinux has quit IRC | 20:51 | |
| *** itlinux has joined #openstack-swift | 21:01 | |
| *** itlinux has quit IRC | 21:02 | |
| *** itlinux has joined #openstack-swift | 21:13 | |
| *** itlinux has quit IRC | 21:20 | |
| *** itlinux has joined #openstack-swift | 21:39 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: Make test_multiphase_put_drains_extra_commit_junk_disconnect less flakey https://review.openstack.org/638778 | 22:23 |
| openstackgerrit | Merged openstack/swift master: py3: port the container https://review.openstack.org/570320 | 23:07 |
| openstackgerrit | Merged openstack/swift master: py3: port versioned_writes https://review.openstack.org/638010 | 23:21 |
| *** thurloat has joined #openstack-swift | 23:25 | |
| *** itlinux has quit IRC | 23:35 | |
| *** itlinux has joined #openstack-swift | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!