| *** gyee has quit IRC | 00:17 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: Support long-running multipart uploads https://review.openstack.org/575818 | 00:25 |
|---|---|---|
| *** Belgar81 has joined #openstack-swift | 00:55 | |
| timburke | clayg: tdasilva: https://review.openstack.org/#/c/617046/ | 00:56 |
| patchbot | patch 617046 - swift - s3api: Change default location to us-east-1 - 3 patch sets | 00:56 |
| openstackgerrit | Tim Burke proposed openstack/swift master: s3api: Allow some forms of server-side-encryption https://review.openstack.org/619127 | 01:19 |
| timburke | good call clayg -- that's way better ^^^ | 01:19 |
| notmyname | oh cool. is that the first time we're using admin swift info? | 01:22 |
| timburke | i think maybe | 01:22 |
| timburke | i love the idea of that being more than a client-facing tool though | 01:23 |
| notmyname | timburke: ugh. the check_drive patch failed after 6 hours with a probetest failure | 01:48 |
| notmyname | do you remember if I can recheck now, or do I have to wait until after zuul reports back? | 01:48 |
| *** itlinux has joined #openstack-swift | 01:50 | |
| notmyname | I'll check it later tonight. hopefully we can have it landed by our morning tomorrow | 01:53 |
| *** mahatic has quit IRC | 02:01 | |
| mattoliverau | notmyname: I'll keep an eye on it and recheck it at the end of my day if it hasn't landed by then. | 02:02 |
| *** psachin has joined #openstack-swift | 03:32 | |
| openstackgerrit | Merged openstack/python-swiftclient master: Change openstack-dev to openstack-discuss https://review.openstack.org/621900 | 03:43 |
| openstackgerrit | Merged openstack/swift master: fixed _check_node() in the container sharder https://review.openstack.org/621727 | 03:58 |
| openstackgerrit | Merged openstack/python-swiftclient master: Add delimiter to get_account(). https://review.openstack.org/619857 | 04:27 |
| *** e0ne has joined #openstack-swift | 07:38 | |
| *** rcernin has quit IRC | 07:56 | |
| *** pcaruana has joined #openstack-swift | 07:58 | |
| *** pcaruana is now known as muttley | 07:58 | |
| *** rcernin has joined #openstack-swift | 08:03 | |
| *** gkadam has joined #openstack-swift | 08:16 | |
| *** rcernin has quit IRC | 08:33 | |
| *** admin6_ has joined #openstack-swift | 08:34 | |
| *** admin6 has quit IRC | 08:37 | |
| *** admin6_ is now known as admin6 | 08:37 | |
| zigo | Is there an Android app to do a phone backup using swift natively? | 08:48 |
| *** mikecmpbll has joined #openstack-swift | 08:51 | |
| *** mahatic has joined #openstack-swift | 08:59 | |
| *** ChanServ sets mode: +v mahatic | 08:59 | |
| *** admin6_ has joined #openstack-swift | 09:28 | |
| *** admin6 has quit IRC | 09:31 | |
| *** admin6_ is now known as admin6 | 09:31 | |
| *** admin6_ has joined #openstack-swift | 09:36 | |
| *** admin6 has quit IRC | 09:39 | |
| *** admin6_ is now known as admin6 | 09:39 | |
| *** e0ne has quit IRC | 10:24 | |
| *** e0ne has joined #openstack-swift | 10:27 | |
| *** mvkr has joined #openstack-swift | 10:36 | |
| *** Belgar81 has quit IRC | 10:54 | |
| *** admin6_ has joined #openstack-swift | 11:07 | |
| *** admin6 has quit IRC | 11:10 | |
| *** admin6_ is now known as admin6 | 11:10 | |
| *** admin6_ has joined #openstack-swift | 11:34 | |
| *** admin6 has quit IRC | 11:37 | |
| *** admin6_ is now known as admin6 | 11:37 | |
| *** NM has joined #openstack-swift | 11:55 | |
| *** gkadam_ has joined #openstack-swift | 12:00 | |
| *** gkadam has quit IRC | 12:03 | |
| *** gkadam_ has quit IRC | 12:03 | |
| *** gkadam has joined #openstack-swift | 12:05 | |
| *** psachin has quit IRC | 12:08 | |
| *** gkadam_ has joined #openstack-swift | 12:55 | |
| *** gkadam has quit IRC | 12:58 | |
| *** admin6_ has joined #openstack-swift | 13:00 | |
| *** admin6 has quit IRC | 13:03 | |
| *** admin6_ is now known as admin6 | 13:03 | |
| *** muttley has quit IRC | 13:08 | |
| *** rcernin has joined #openstack-swift | 13:17 | |
| *** muttley has joined #openstack-swift | 13:21 | |
| *** muttley has quit IRC | 13:25 | |
| *** muttley has joined #openstack-swift | 13:26 | |
| *** rcernin has quit IRC | 13:29 | |
| *** muttley has quit IRC | 13:29 | |
| *** pcaruana has joined #openstack-swift | 13:34 | |
| *** kota_ has quit IRC | 13:36 | |
| *** kota_ has joined #openstack-swift | 13:38 | |
| *** ChanServ sets mode: +v kota_ | 13:38 | |
| *** pcaruana has quit IRC | 13:39 | |
| *** pcaruana has joined #openstack-swift | 13:43 | |
| *** pcaruana has quit IRC | 13:47 | |
| *** hseipp has joined #openstack-swift | 13:50 | |
| *** dr_gogeta86 has joined #openstack-swift | 13:55 | |
| openstackgerrit | Merged openstack/swift master: api-ref: Clean up container and object response code formatting https://review.openstack.org/622607 | 14:16 |
| *** psachin has joined #openstack-swift | 14:32 | |
| *** admin6_ has joined #openstack-swift | 15:16 | |
| *** admin6 has quit IRC | 15:18 | |
| *** admin6_ is now known as admin6 | 15:18 | |
| *** gkadam_ has quit IRC | 15:19 | |
| *** admin6_ has joined #openstack-swift | 16:04 | |
| *** admin6 has quit IRC | 16:07 | |
| *** admin6_ is now known as admin6 | 16:07 | |
| *** takamatsu has quit IRC | 16:10 | |
| *** SkyRocknRoll has joined #openstack-swift | 16:23 | |
| *** e0ne has quit IRC | 16:30 | |
| *** admin6 has quit IRC | 16:32 | |
| *** psachin has quit IRC | 16:41 | |
| *** NM has quit IRC | 17:01 | |
| *** NM has joined #openstack-swift | 17:04 | |
| *** hseipp has quit IRC | 17:21 | |
| notmyname | good morning | 17:41 |
| notmyname | patch 592231 | 17:41 |
| patchbot | https://review.openstack.org/#/c/592231/ - swift - s3api: Include '-' in S3 ETags of normal SLOs - 6 patch sets | 17:41 |
| *** NM has quit IRC | 17:43 | |
| *** mikecmpbll has quit IRC | 17:47 | |
| clayg | does s3api really just hide any container that happens to end in `_segments`? | 17:49 |
| *** NM has joined #openstack-swift | 17:52 | |
| *** gyee has joined #openstack-swift | 17:55 | |
| clayg | timburke: p 618301 has no tests 🙄 | 17:58 |
| patchbot | https://review.openstack.org/#/c/618301/ - swift - s3api: Allow concurrent multi-deletes - 3 patch sets | 17:58 |
| timburke | clayg: but it didn't *break* any tests, either! :P | 18:07 |
| *** e0ne has joined #openstack-swift | 18:08 | |
| timburke | clayg: and no, it's not _segments that's special (though maybe it'll hide that, too?) -- it uses +segments for MPUs specifically because + isn't valid under either the old or new bucket naming rules | 18:08 |
| timburke | if a bucket name isn't valid, why return it? | 18:08 |
| *** e0ne has quit IRC | 18:09 | |
| clayg | TIL "The bucket name cannot contain underscores" | 18:11 |
| timburke | bu it *used* to be able to, iirc. set dns_compliant_bucket_names=false and it should show up | 18:16 |
| timburke | "The legacy rules for bucket names in the US East (N. Virginia) Region allowed bucket names to be as long as 255 characters, and bucket names could contain any combination of uppercase letters, lowercase letters, numbers, periods (.), hyphens (-), and underscores (_)." | 18:16 |
| timburke | so there are some ceph s3 tests that still test 250 chars, 251, 252, 253, 254, 255, 256, 280, and 3000 :P | 18:18 |
| *** SkyRocknRoll has quit IRC | 18:29 | |
| NM | Hi everyone! Is anyone using any feature/tool/middleware to undelete objects? I found swift_undelete on Github but it has been while since the project was updated. | 18:34 |
| timburke | NM: i'd go with something like the history mode for versioned writes, personally -- are you looking to have it apply cluster-wide, or just for a container or two? | 18:38 |
| timburke | if we ever get the defaulter merged (see p 342857) we could even have versioned_writes look for and apply a default x-delete-after header as part of the copy... | 18:40 |
| patchbot | https://review.openstack.org/#/c/342857/ - swift - Add defaulter middleware - 7 patch sets | 18:40 |
| notmyname | zigo: I have no idea if there's any native android->swift backup tool. but weirdly I did just see this on twitter: https://www.infomaniak.com/en/professional-tools/swiss-backup | 18:48 |
| notmyname | zigo: seems like maybe they do some sort of backup to swift? | 18:48 |
| NM | timburke: right now, just for a container. Someone made a implementation of x-delete-at and after some bad results people are asking if we could offer an undelete feature. I think versioning won't protect the objects for this case… | 18:48 |
| openstackgerrit | Tim Burke proposed openstack/swift master: s3api: Allow some forms of server-side-encryption https://review.openstack.org/619127 | 18:55 |
| timburke | NM: yeah, versioning and object expiration don't exactly mesh well... | 18:56 |
| openstackgerrit | Merged openstack/swift master: s3api: Include '-' in S3 ETags of normal SLOs https://review.openstack.org/592231 | 19:17 |
| NM | timburke: thank you anyway. 👍🏼 | 19:21 |
| * clayg I'm trying to boto and seeing: | 19:26 | |
| clayg | DEBUG:boto:encountered gaierror exception, reconnecting | 19:26 |
| clayg | gaierror is like a get hostname call or something right? I tried `host='127.0.0.1'` and saio... i don't know what the actual error is - but my syslog isn't showing any requests | 19:27 |
| notmyname | fungi: I'm seeing some odd things in zuul. specifically, I've seen two patches with tests reported failed in the dashboard but they've both merged with no rechecks | 19:29 |
| notmyname | https://review.openstack.org/#/c/592231/ merged just find with no reported errors, but this morning clarkb pointed me at http://logs.openstack.org/31/592231/6/gate/swift-probetests-centos-7/7bde795/job-output.txt.gz#_2018-12-06_17_32_48_836444 which is a failed job | 19:29 |
| patchbot | patch 592231 - swift - s3api: Include '-' in S3 ETags of normal SLOs (MERGED) - 6 patch sets | 19:29 |
| notmyname | and last night https://review.openstack.org/#/c/621727/ landed but I saw while it was in the gate queue with a big red x next to it | 19:29 |
| patchbot | patch 621727 - swift - fixed _check_node() in the container sharder (MERGED) - 5 patch sets | 19:30 |
| *** mikecmpbll has joined #openstack-swift | 19:30 | |
| timburke | clayg: probably trying to connect to <bucket>.saio or something? | 19:30 |
| notmyname | sorry.. saw http://logs.openstack.org/27/621727/5/gate/swift-probetests-centos-7/7ac3dba/job-output.txt.gz | 19:30 |
| clayg | aparently that's the failure mode if you try and skip the `calling_format=boto.s3.connection.OrdinaryCallingFormat()` bit from examples!? | 19:30 |
| clayg | @timburke brilliant! DEBUG:boto:Host: lots-of-files.127.0.0.1:8080 | 19:31 |
| notmyname | fungi: the hash after the test name is different for the ones reporting errors (comparing the failing log with the reported log that passed), but I don't know what that means | 19:31 |
| timburke | notmyname: most likely another ahead of it in the queue failed (most likely for another project), so *all* the jobs for the patch that merged got re-enqueued without the other project's patch applied, and *then* the jobs all passed | 19:31 |
| notmyname | timburke: ah, maybe so. | 19:32 |
| notmyname | I think I've noticed it because it's happened twice in the last 12 hours or so | 19:32 |
| timburke | sometimes i kinda wish we didn't get all the auto-retries, just to force us to fix our flaky tests... not real often though ;-) | 19:32 |
| fungi | notmyname: looks like 592231 failed build 7bde795 but then another change ahead of it failed and so all the jobs for that change were rerun again without the failing change ahead of it. if you look at the items list for http://logs.openstack.org/31/592231/6/gate/swift-probetests-centos-7/7bde795/zuul-info/inventory.yaml you'll see what other changes were included with that run | 19:32 |
| notmyname | fungi: ah ok. so what timburke said about the job being re-enqueued | 19:33 |
| fungi | right. zuul won't consider job results from a dependent pipeline valid if there's a change with failing jobs "ahead" of that one | 19:34 |
| timburke | clayg: i wonder if i could convince boto to take a patch to check whether a host is actually an IP address and if so, skip straight to the bucket-in-path request... | 19:34 |
| timburke | was it boto or boto3? | 19:34 |
| fungi | so it moves the earliest failing change aside and then reparents all the subsequent changes to the nearest non-failing change | 19:34 |
| notmyname | ...which gives a definitive answer to something I've been wondering. always wait for zuul to report back with failure *before* sending a recheck | 19:35 |
| fungi | well, yes that too. unless zuul completely dequeues a change first it's going to ignore recheck comments | 19:35 |
| notmyname | because if it's a transient failure, failing stuff ahead of it will reenqueue it (and earlier than a human recheck would place it) | 19:36 |
| fungi | right, for example you don't want a bad nova patch in a change ahead of yours to cause a devstack failure in your swift change and have it be ejected as a bad patch | 19:37 |
| fungi | so we try the switch change again without the failing nova change to make sure that wasn't the reason the swift change was also failing | 19:37 |
| fungi | s/switch/swift/ | 19:37 |
| notmyname | fungi: so what you're saying is that as long as our patches have the most stable tests, then the system automatically works most optimally for us :-) | 19:37 |
| fungi | basically | 19:38 |
| notmyname | stable tests + few dependencies = best possible openstack gate queues | 19:38 |
| timburke | notmyname: but few dependencies won't necessarily help us -- as long as we're in the integrated gate queue, we'll have ready-to-merge code getting re-enqueued because of someone else's failure | 19:40 |
| notmyname | timburke: I just mean that if we don't have a dependency on nova, then it's impossible for a nova patch to break our tests. meaning that given the reality that nova and swift patches are tested together, we only need to "handle" failures based on our own code | 19:42 |
| timburke | mmm -- true | 19:42 |
| *** mikecmpbll has quit IRC | 19:54 | |
| *** rcernin has joined #openstack-swift | 20:12 | |
| openstackgerrit | Merged openstack/swift master: Change openstack-dev to openstack-discuss https://review.openstack.org/622834 | 20:19 |
| *** NM has quit IRC | 20:45 | |
| *** NM has joined #openstack-swift | 20:47 | |
| *** NM has quit IRC | 20:53 | |
| sorrison | Hi all, we upgraded our swift cluster to 2.15 about a week ago and now our accounts don't seem to be updating their object count or size, container count works fine though. Any ideas where to look? | 21:14 |
| timburke | sorrison: have you looked at the container-updater logs? | 21:16 |
| timburke | also, sorry -- i've been meaning to look at p 603529 -- i think i've even got a decent setup to test it ATM... | 21:17 |
| patchbot | https://review.openstack.org/#/c/603529/ - swift - s3 secret caching - 10 patch sets | 21:17 |
| sorrison | timburke: nothing in logs for container updater. ran it manually on one host and got: | 21:21 |
| sorrison | Container update single threaded sweep completed: 8.64s, 20 successes, 0 failures, 414 with no changes | 21:21 |
| sorrison | [updater_last_sweep] low: 2, high: 34, avg: 21.3, total: 361, Failed: 0.0%, no_result: 0, reported: | 21:22 |
| sorrison | timeburke: So the container-updater is the process that updates the account with the number of objects and size of objects? | 21:41 |
| sorrison | timburke: ^ | 21:41 |
| timburke | sorrison: k, pretty sure you're getting bit by bug 1710328 which was fixed in https://github.com/openstack/swift/commit/6d16079 -- can you upgrade to 2.15.1 instead of 2.15.0? alternatively, i know we did some backports for that guy -- tip of stable/ocata should have https://github.com/openstack/swift/commit/69c715c | 21:42 |
| openstack | bug 1710328 in OpenStack Object Storage (swift) "object server deadlocks when a worker thread logs something" [High,Fix released] https://launchpad.net/bugs/1710328 - Assigned to Samuel Merritt (torgomatic) | 21:42 |
| timburke | yeah, container-updater takes the stats that got rolled up at the container level and sends them on to the account | 21:43 |
| sorrison | We are running 2.15.1 from ubuntu cloud archive package | 21:43 |
| sorrison | 2.15.1-0ubuntu3~cloud0 to be precise | 21:43 |
| timburke | oh | 21:43 |
| timburke | good | 21:43 |
| timburke | hmm.... | 21:43 |
| sorrison | (which is pike) | 21:43 |
| sorrison | Just checked and the code for https://github.com/openstack/swift/commit/6d16079 is in the version we running | 21:45 |
| timburke | ah! i was thinking of https://bugs.launchpad.net/swift/+bug/1722951 :-) | 21:45 |
| openstack | Launchpad bug 1722951 in OpenStack Object Storage (swift) "Container updater may be stuck and not make progress" [High,Fix released] - Assigned to Samuel Merritt (torgomatic) | 21:45 |
| sorrison | ok I see, fixed in 2.16.0 we are going to upgrade to the queens version next week so should all be good then | 21:46 |
| timburke | notmyname: should we maybe tag some stable releases so downstream knows that we've got some fixes for ocata and pike? 'cause this was fixed for pike, too, in https://github.com/openstack/swift/commit/bf5b49c | 21:48 |
| notmyname | timburke: yeah, when we do the next release, we should just do a whole round of tagging for all the stable branches | 22:06 |
| timburke | notmyname: in that case, i ought to take another look at what's landed since the various stable branches were cut and see if there's anything *else* we ought to backport | 22:07 |
| notmyname | timburke: there's *always* something else to land | 22:14 |
| mattoliverau | morning | 22:40 |
| openstackgerrit | Tim Burke proposed openstack/swift master: s3api: Allow concurrent multi-deletes https://review.openstack.org/618301 | 23:21 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!