Wednesday, 2018-05-30

openstackgerritSamuel Merritt proposed openstack/swift master: Remove unhelpful log message in copy middleware  https://review.openstack.org/57104700:00
openstackgerritTim Burke proposed openstack/swift master: Make account ACLs work with unicode in user/account names  https://review.openstack.org/57104800:01
openstackgerritTim Burke proposed openstack/swift master: Make COPY work with unicode account names  https://review.openstack.org/57104900:01
openstackgerritTim Burke proposed openstack/swift master: functests: Clean up objects better  https://review.openstack.org/57105000:01
openstackgerritTim Burke proposed openstack/swift master: Improve user logging  https://review.openstack.org/57105100:01
timburke^^^ a whole bunch of crap to shake out of looking harder at https://review.openstack.org/#/c/558974/00:01
patchbotpatch 558974 - swift - Support unicode characters in tempauth user names. (MERGED)00:01
timburkethe good news is, test/functional/tests.py test/functional/test_object.py now pass with tempauth users like tést:tester00:02
timburkethe bad news is, a whole bunch of other stuff still doesn't00:02
*** rledisez has quit IRC00:04
*** rledisez has joined #openstack-swift00:05
*** alecuyer has quit IRC00:06
*** alecuyer has joined #openstack-swift00:06
*** DHE has quit IRC00:06
*** zigo has quit IRC00:06
*** dhellmann has quit IRC00:06
*** afazekas has quit IRC00:06
*** zigo_ has joined #openstack-swift00:06
*** dhellmann has joined #openstack-swift00:11
*** afazekas has joined #openstack-swift00:11
*** ChanServ has quit IRC00:12
*** DHE has joined #openstack-swift00:15
*** ChanServ has joined #openstack-swift00:33
*** barjavel.freenode.net sets mode: +o ChanServ00:33
*** amandap has joined #openstack-swift00:36
*** Renich has quit IRC00:37
openstackgerritSamuel Merritt proposed openstack/swift master: container-updater: log LockTimeout exceptions at INFO, not ERROR  https://review.openstack.org/57104200:38
*** Renich has joined #openstack-swift01:05
*** gyankum has joined #openstack-swift01:29
*** DHE has quit IRC01:31
*** gyee has quit IRC01:31
*** DHE has joined #openstack-swift01:44
*** dr_gogeta86 has quit IRC01:45
*** armaan has quit IRC01:48
*** armaan has joined #openstack-swift01:49
openstackgerritMerged openstack/swift master: Add s3api headers to allowed_headers by default  https://review.openstack.org/57101302:03
*** bkopilov_ has quit IRC02:08
*** psachin has joined #openstack-swift02:42
openstackgerritMerged openstack/swift master: authors/changelog update for 2.18.0 release  https://review.openstack.org/57061902:43
notmynamehttps://review.openstack.org/#/c/571067/  <-- 2.18.0 tag request02:48
patchbotpatch 571067 - releases - Swift 2.18.0 release02:48
notmynamebiggest swift release ever02:48
mattoliverauand best!02:57
*** bkopilov_ has joined #openstack-swift03:24
openstackgerritMatthew Oliver proposed openstack/swift master: Adds read_only middleware  https://review.openstack.org/23374903:26
mattoliverautimburke: I'm taking a look at the read_only middleware. if you want to merge your follow up fixes to the patch to the existing, add a co-author (if you want) and then add your +2 again, I'll go test it out and put mine on. :)04:04
mattoliverauor we can just land the 2, that fine to. I updated the commit message so it'll need you +2 again anyway04:05
*** links has joined #openstack-swift04:11
*** amandap has quit IRC04:17
*** amandap has joined #openstack-swift04:30
*** psachin has quit IRC04:46
*** mvk has joined #openstack-swift04:58
openstackgerritMerged openstack/swift master: Catch and log errors from process_container, then keep updating  https://review.openstack.org/57062205:00
*** eandersson has joined #openstack-swift05:10
openstackgerritMerged openstack/swift master: Remove unhelpful log message in copy middleware  https://review.openstack.org/57104705:19
*** lifeless_ has quit IRC05:19
*** lifeless has joined #openstack-swift05:19
kota_hello world.05:42
kota_wow nice. a lot of s3api staffs in the gerrit.05:43
*** eandersson has quit IRC05:48
*** eandersson has joined #openstack-swift05:49
kota_hmm... vda1 looks like  size: 80.00 GB... ceph-s3tests eat the volume?05:55
kota_ah... no. it seems like the loopback devices of the disks too small. Let me try to increase.06:00
*** cshastri has joined #openstack-swift06:02
openstackgerritKota Tsuyuzaki proposed openstack/swift master: WIP: Add ceph-s3 test non-voting job  https://review.openstack.org/56470006:02
*** armaan has quit IRC06:09
*** armaan has joined #openstack-swift06:09
f0oHi I'm having a bit of an issue, I just updated one of our swift nodes and now the container-server and replication-server wont start anymore. they all fail with the same error, related to liberasurecode and pyeclib. logs: http://paste.openstack.org/show/722322/ - Any pointers?06:12
f0oif it's of any use, I got pyeclib version 1.5.0 and liberasurecode 1.1.106:14
mattoliverauf0o: I think the dynamic linking warnings there happened in an older version of liberasurecode. Lastest version of liberasure is 1.5.0 (I think, according to the source). Any chance you can try a later version?06:21
f0omattoliverau: yeah I saw that too now, I'm filing a bug with upstream gentoo to bump the ebuild06:22
kota_IIRC, _liberasurecode_version() has been supported around 1.2.0...06:22
*** lifeless has quit IRC06:23
*** lifeless has joined #openstack-swift06:25
mattoliveraunice thanks kota_06:26
mattoliverauI'll go ping prometheanfire, he's a gentoo dev06:27
kota_that looks newer, https://github.com/openstack/liberasurecode/commit/09d8bbf3 since 1.4.006:27
kota_then, using https://github.com/openstack/pyeclib/blob/master/src/c/pyeclib_c/pyeclib_c.c#L122606:29
kota_I totally forget how older liberasurecode/pyeclib can be supported though.06:29
mattoliveraucool, I've bugged prometheanfire.. always fun to have an excuse to do that :P06:32
f0oanyone knows what license liberasurecode is under?06:33
*** pcaruana has joined #openstack-swift06:33
kota_IIRC, 3-clause BSD06:34
kota_https://github.com/openstack/pyeclib/blob/master/src/c/pyeclib_c/pyeclib_c.c#L122606:35
kota_no invalid link06:35
kota_https://github.com/openstack/liberasurecode/blob/master/COPYING06:35
f0oty06:35
kota_oh, it might be 2-clauses?06:36
f0ofor a package maintainer's eyes it's BSD :D06:36
* kota_ is checking the sentences06:36
kota_hmm... it looks like 2-clause06:39
kota_it doesn't have the 3rd clause.06:39
f0oupdate liberasurecode to 1.5.0 worked flawless06:41
f0othanks :)06:41
f0osubmitting my ebuild to upstream now, so they can update their repos06:41
f0ohttps://bugs.gentoo.org/65688206:45
f0ofor the record06:45
*** hseipp has joined #openstack-swift06:47
*** m_kazuhiro has joined #openstack-swift06:54
*** cshastri has quit IRC06:57
openstackgerritKota Tsuyuzaki proposed openstack/swift master: WIP: Add ceph-s3 test non-voting job  https://review.openstack.org/56470006:58
*** wangyuxin has quit IRC07:07
*** yuxin_ has joined #openstack-swift07:07
mattoliverauf0o: nice work and thanks for letting us know07:09
*** ccamacho has joined #openstack-swift07:11
*** tesseract has joined #openstack-swift07:20
*** cshastri has joined #openstack-swift07:22
*** rcernin has quit IRC07:27
*** geaaru has joined #openstack-swift07:28
mattoliverauf0o: gentoo has fixed it. And your bug is closed :)07:30
*** cbartz has joined #openstack-swift07:32
*** armaan has quit IRC07:48
*** armaan has joined #openstack-swift07:49
*** amandap has quit IRC07:53
*** mikecmpbll has joined #openstack-swift07:58
*** m_kazuhiro has quit IRC08:01
*** armaan has quit IRC08:04
*** armaan has joined #openstack-swift08:04
f0oyep just saw the ebuild pushed to my gentoo mirrors08:30
f0onow I can continue and update all the test of the nodes ;)08:31
f0oOT how can I restore an object that was mistakenly created with an expiry tag?08:38
f0oa customer just poked me about that, I can see the file on disk so it's not scrubbed yet - can I just bump the timestamp tag?08:39
f0oor will that create a world of hurt in the backend?08:39
*** lifeless_ has joined #openstack-swift08:44
*** lifeless has quit IRC08:44
*** silor has joined #openstack-swift08:55
*** kei_yama has quit IRC08:56
acolestimburke: notmyname: re https://review.openstack.org/#/c/570989/ ...looks like the default sharding size/threshold was 10M since first commit on feature/deep, but documented as 1M. The change makes sense.08:58
patchbotpatch 570989 - swift - Change default sharding threshold to 1,000,000 obj... (MERGED)08:58
*** armaan has quit IRC09:05
*** armaan has joined #openstack-swift09:05
*** armaan has quit IRC09:09
*** armaan has joined #openstack-swift09:13
*** dr_gogeta86 has joined #openstack-swift09:25
*** dr_gogeta86 has quit IRC09:25
*** dr_gogeta86 has joined #openstack-swift09:25
*** silor has quit IRC09:45
acolesf0o: you'd need to delete the x-delete-at metadata in *all replicas* of the  object on disk file. The best I came up with last time this was asked is to delete all the metadata for the object using something like this https://gist.github.com/alistairncoles/722e3e8a6d7107c9fd93c1db7ff7d73c which requires your user to not care about any user metadata, and requires swift 2.15.009:53
kota_timburke: we finally get the s3-tests compatibility summary at the gate! -> http://logs.openstack.org/00/564700/19/check/swift-tox-func-s3api-ceph-s3tests-tempauth/5816ec8/output/ceph-s3-summary.log09:54
acolesf0o: The simplest fix might be to ship the content of the .data file you can see on disk back to your user and suggest they upload it again without an x-delete-at/after header09:55
kota_idk how it affects to the probetests as TIMEOUT tho :/09:56
*** zigo_ is now known as zigo09:56
openstackgerritAlexandre Lécuyer proposed openstack/swift master: Configure diskfile per storage policy  https://review.openstack.org/44712910:08
*** bkopilov_ has quit IRC10:18
*** psachin has joined #openstack-swift10:21
*** lifeless_ has quit IRC10:32
*** gyankum has quit IRC10:41
*** amandap has joined #openstack-swift10:44
*** lifeless has joined #openstack-swift11:01
*** armaan has quit IRC11:16
*** armaan has joined #openstack-swift11:17
f0oacoles: thanks (also sorry for the late answer, lunch ;)) - I ended up shipping the files on disk to him and asked him to reupload it either versioned or without the expiry11:28
f0oI'll still keep the gist you linked, it will come in handy later on I think :)11:29
*** armaan has quit IRC11:33
*** armaan_ has joined #openstack-swift11:33
*** lifeless_ has joined #openstack-swift11:37
*** lifeless has quit IRC11:38
tdasilvaacoles: wondering if we should come up with a way to remove the expiration on a object via API11:47
tdasilvaacoles: would that cause all kinds of 'post-as-copy' like issues?11:48
*** armaan_ has quit IRC11:49
*** DHE has quit IRC11:54
*** gyankum has joined #openstack-swift11:54
*** DHE has joined #openstack-swift11:57
acolestdasilva: it's easy enough to remove before the expiration time is reached. after that it is a race with the expirer. it requires a POST to be accepted by an expired object, which we allow internally with an X-Backend- header.12:09
tdasilvaacoles: got it!12:10
acolestdasilva: I don't immediately see a consistency obstacle, but the race with expirer would be a problem12:12
*** armaan has joined #openstack-swift12:32
*** mvenesio has joined #openstack-swift12:37
*** lifeless_ has quit IRC12:41
*** armaan has quit IRC12:44
*** sajjadg has joined #openstack-swift12:59
*** mvenesio has quit IRC13:02
*** armaan has joined #openstack-swift13:08
*** cshastri has quit IRC13:23
*** SkyRocknRoll has joined #openstack-swift13:25
*** links has quit IRC13:29
*** links has joined #openstack-swift13:30
*** cshastri has joined #openstack-swift13:35
*** links has quit IRC13:44
*** bkopilov_ has joined #openstack-swift13:44
*** cshastri has quit IRC13:45
*** klamath_ has joined #openstack-swift13:57
*** SkyRocknRoll has quit IRC14:01
*** klamath has quit IRC14:01
openstackgerritAlistair Coles proposed openstack/swift master: Resolve TODO's in test/probe/test_sharder.py  https://review.openstack.org/57120914:08
*** SkyRocknRoll has joined #openstack-swift14:14
*** armaan has quit IRC14:18
*** armaan has joined #openstack-swift14:19
notmynamethe 2.18.0 tag landed! congrats, everyone14:24
openstackgerritJohn Dickinson proposed openstack/swift master: Added PyPI validation check for the readme.  https://review.openstack.org/57121514:41
*** armaan has quit IRC14:49
openstackgerritJohn Dickinson proposed openstack/swift master: Added PyPI validation check for the readme.  https://review.openstack.org/57121514:55
*** amandap has quit IRC14:58
*** klamath_ is now known as klamath14:59
*** armaan has joined #openstack-swift15:01
*** gyee has joined #openstack-swift15:05
*** zaitcev_ has joined #openstack-swift15:17
*** ChanServ sets mode: +v zaitcev_15:17
*** zaitcev has quit IRC15:20
*** sajjadg has quit IRC15:30
*** pcaruana has quit IRC15:33
*** SkyRocknRoll has quit IRC15:34
*** amandap has joined #openstack-swift15:34
*** SkyRocknRoll has joined #openstack-swift15:36
*** cbartz has quit IRC15:45
*** amandap has quit IRC15:48
*** amandap has joined #openstack-swift15:48
DHEsweet... been looking forward to sharding...15:54
openstackgerritGilles Biannic proposed openstack/swift master: Add template in proxy to create custom and anonymous access-logs  https://review.openstack.org/54894816:21
*** pcaruana has joined #openstack-swift16:23
*** hseipp has quit IRC16:24
*** amandap has quit IRC16:33
*** mikecmpbll has quit IRC16:35
timburketdasilva: acoles: fwiw, i think it should be an operator-only feature. if i set an object to expire and then my creds get compromised a day later, i don't want there to be any *extra* security risk16:45
timburkeyou might be able to do something clever with dropping a .meta with a bumped offset to keep timestamps roughly the same...16:45
timburkemaybe it could still be exposed through the proxy, but limited to reseller admins?16:46
*** klamath has quit IRC16:47
*** klamath has joined #openstack-swift16:48
tdasilvatimburke: i'm not sure...i guess it really depends on how the "feature" is exposed to end-users?? if it's on a UI and it is "easy" to set an expiration for example, then users might make the mistake and try to undo16:49
*** armaan has quit IRC16:50
*** klamath has quit IRC17:01
timburkeif it isn't *already* expired, sure. but i'd be nervous about having a location 404 on GET, 404 on whatever flavor of POST you'd use to implement the feature, and then turn around and start 200ing on subsequent GETs...17:02
*** klamath has joined #openstack-swift17:07
*** tesseract has quit IRC17:10
*** gyankum has quit IRC17:23
*** klamath has quit IRC17:42
*** klamath_ has joined #openstack-swift17:42
tdasilvatimburke: oh yeah, if it's already expired, then agree it needs a different mechanism...17:43
*** psachin has quit IRC17:57
*** geaaru has quit IRC18:02
*** SkyRocknRoll has quit IRC18:04
*** mvenesio has joined #openstack-swift18:05
openstackgerritTim Burke proposed openstack/swift master: WIP: Add ceph-s3 test non-voting job  https://review.openstack.org/56470018:07
timburkekota_: looks like the timeouts started coming in patchset 13... only thing that should affect probe tests is https://review.openstack.org/#/c/564700/12..13/playbooks/saio_single_node_setup/install_dependencies.yaml i think? i'll try taking system packages one a a time until it works :-/18:09
patchbotpatch 564700 - swift - WIP: Add ceph-s3 test non-voting job18:09
*** pcaruana has quit IRC18:09
*** mikecmpbll has joined #openstack-swift18:09
*** ccamacho has quit IRC18:11
openstackgerritMerged openstack/swift master: container-updater: log LockTimeout exceptions at INFO, not ERROR  https://review.openstack.org/57104218:14
*** klamath_ is now known as klamath18:39
claygyay sharding!19:13
*** lifeless has joined #openstack-swift19:31
claygso, there was something with cmd and python2?  My vsaio didn't build yesterday.  Is there a well known fix?19:32
*** nicodemus_ has joined #openstack-swift19:34
nicodemus_Hello19:34
claygnicodemus_: HI!19:36
nicodemus_I have a question: what would be a good strategy for migrating an account from one swift cluster to another? The scenario is: there's an account that has about 50TB of data spread across 10 containers19:36
nicodemus_Since I cannot download everything to an intermediate host and then upload, I was thinking about doing a 'swift list' for each container, saving the list of objects, and then for each object issuing a 'swift download' from cluster 1 and then a 'swift upload' to cluster 219:38
nicodemus_These two clusters are independent one from the other, and the keystone account exists on both keystone servers19:38
claygnicodemus_: maybe https://rclone.org/ ???19:40
nicodemus_clayg: I didn't know about rclone, will look into it19:46
nicodemus_Do you know if rclone accepts swift as both the source and the destination? Or it expects different types of remotes?19:48
claygshould allow swift -> swift19:49
nicodemus_thanks clayg !19:53
claygtimburke: notmyname: how do y'all do that thing where you can tell what's in the merge pipeline?  I was thinking about building ss-release-2.18.0.1 off 56aeb20f9893ca2c9bfc6ea8456b02895c1082f519:59
*** mvenesio has quit IRC20:08
*** mvenesio has joined #openstack-swift20:08
*** mvenesio has quit IRC20:13
*** armaan has joined #openstack-swift20:15
notmynameclayg: look at http://zuul.openstack.org20:28
claygnotmyname: you honestly i look at that a lot... then basically conclude... I have no idea what any of this means20:29
claygit's BEAUTIFUL tho!20:29
notmynameclayg: type in swift at the top, see that the rest of it is empty20:29
timburketry searching for swift20:29
timburkeall clear20:29
notmynameclayg: did you find the gate answer? I can't find it quickly in my history. yes, there's a well-known fix20:30
notmynamesorry, not gate. the cmd and python220:30
claygwell, it worked when I tried again today - so maybe the internet fixed it?20:31
claygok, well i'm still not sure i understand the zuul thing - but I'll look at it again another day and maybe some spot the difference insight will help me understand the meaning of all those categories better?20:32
timburkeclayg: notmyname: fwiw, i've got a script like http://paste.openstack.org/show/722378/ that i use sometimes20:32
timburke(which has some obvious debugging stuff toward the top...)20:33
claygtimburke: that script looks pretty good!20:34
claygthe api is noauth/read-only?20:34
timburkethat's how you get it in the browser20:37
openstackgerritTim Burke proposed openstack/swift master: WIP: Add ceph-s3 test non-voting job  https://review.openstack.org/56470020:38
*** itlinux has joined #openstack-swift20:44
notmynamemeeting time in 15 minutes20:45
*** m_kazuhiro has joined #openstack-swift20:51
kota_good morning20:52
notmynamekota_: hello. did you get home ok?20:54
*** zaitcev_ is now known as zaitcev20:55
notmynamemeeting time in #openstack-meeting /cc kota_ m_kazuhiro mattoliverau clayg cschwede tdasilva zaitcev torgomatic timburke acoles rledisez20:59
kota_notmyname: yeah, it was good flight to be back home.21:00
notmynamegood :-)21:00
*** nicodemus_ has quit IRC21:21
*** lifeless has quit IRC21:26
*** lifeless_ has joined #openstack-swift21:26
*** lifeless has joined #openstack-swift21:38
*** lifeless_ has quit IRC21:38
*** armaan has quit IRC21:39
openstackgerritSamuel Merritt proposed openstack/swift master: Log updater stats once per pass, not per policy  https://review.openstack.org/57131821:41
*** itlinux has quit IRC21:43
*** klamath has quit IRC21:54
*** klamath has joined #openstack-swift21:56
*** m_kazuhiro has quit IRC21:56
*** klamath_ has joined #openstack-swift21:58
claygi've been thinking about big/old code bases (like swift) and how new contributors engage with them and why...21:58
*** klamath has quit IRC21:58
claygmost of my open source contributions (outside of swift) have been bug fixes... some docs... lots of bug reports and WOMM "patches"21:59
claygdoes anyone in here maintain any other projects?  timur you basically maintain bouncestorage/docker-swift and fog/fog-openstack (storage)22:02
claygbut both of those are like ... "hey this is useful" "yeah it's abandoned, you want it" "umm.... i guess?"22:02
*** klamath_ has quit IRC22:02
*** klamath has joined #openstack-swift22:03
*** lifeless has quit IRC22:03
claygthat's happened to me before... but I can't think of a time where I came to a project that a half-dozen folks were working on to the tune of 10s of commits a week and was just like "hey can help merge stuff?"  why would I even want to do that - either as an individual, or as an employee?22:04
*** lifeless has joined #openstack-swift22:04
remix_tjclayg: i contributed only with bugfixes for years, now i started pushing some features, but are few lines contributions22:06
remix_tjand more or less because i'm a sysadmin and my company thinks that fixing things is not what i should do22:06
remix_tjbut if on ansible 2.5 my colleagues can deploy without issues a vm on ovirt, is because of my fix :-)22:07
remix_tji'd like to start contributing with more things, also for work, but is not feasible for me, i've low skills and low time22:08
claygyes, it's hard to imagine someone working on a significant feature "in their spare time" - and equally hard to imagine someone dumping a "small feature" over the wall to be maintained by whoever is still around... most likely is 1) drive by fixes 2) corporate subsidized long-term/full-time maintenance (including redesign feature/architecture/technical investment)22:13
claygremix_tj: congrats on the ansible fix!22:14
remix_tji'm trying to develop a middleware for implementing retention features on swift. At the moment my middleware is of 150 rows. About 100 of them are copied from "template". And is blocked there since 2 weeks ago. I'm not sure i will ever end also because there's no specific interest on my company to have a swift setup and for using that feature22:16
remix_tjbut there are tons of useless things developed by other departments (like an SSO/IAM portal with a custom protocol instead of using a classic SAML opensource product)22:17
remix_tjand i don't understand why i can't invest time on things that may be useful and have to allow people to loose time on mantaining a thing that should be replaced with a marked-standard product22:18
*** rcernin has joined #openstack-swift22:22
*** amandap has joined #openstack-swift22:23
*** amandap has quit IRC22:28
*** klamath_ has joined #openstack-swift22:29
*** klamath has quit IRC22:29
*** lifeless has quit IRC22:33
*** klamath_ has quit IRC22:35
*** lifeless has joined #openstack-swift22:35
*** klamath has joined #openstack-swift22:36
*** mikecmpbll has quit IRC22:53
*** klamath has quit IRC22:54
notmynameremix_tj: that's cool22:55
remix_tj?22:55
notmynamejust catching up on the conversation22:55
remix_tjwhat part of the conversation is cool? :-)22:56
notmynameheh22:56
*** klamath has joined #openstack-swift22:56
remix_tji'm going to bed now, see you tomorrow22:56
notmynameyou've got a middleware that's doing new, interesting things. that's cool. of course, your frustration isn't cool. same with confusion about "why that and not this" of corporate priorities22:56
notmynameremix_tj: when you say you're blocked, is that by upstream reviews? or something else?22:57
remix_tjdevelopment22:57
notmynameah22:57
remix_tji have to go on with code, i can't find time at the moment to continue, but i'll do when possible22:57
*** klamath has quit IRC23:07
*** kei_yama has joined #openstack-swift23:09
*** lifeless has quit IRC23:11
*** klamath has joined #openstack-swift23:12
openstackgerritJohn Dickinson proposed openstack/swift master: removed specific reference to two core reviewers  https://review.openstack.org/57132823:14
*** mikecmpbll has joined #openstack-swift23:34
*** klamath has quit IRC23:35
*** mikecmpbll has quit IRC23:35
*** mikecmpbll has joined #openstack-swift23:38
*** mikecmpbll has quit IRC23:40
*** lifeless has joined #openstack-swift23:41
*** amandap has joined #openstack-swift23:41
*** gerardgine has joined #openstack-swift23:47
*** klamath has joined #openstack-swift23:50
*** lifeless has quit IRC23:58
*** klamath has quit IRC23:58
*** lifeless has joined #openstack-swift23:59
*** klamath has joined #openstack-swift23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!