Thursday, 2017-08-31

*** alenavolk has quit IRC00:10
*** alenavolk has joined #openstack-swift00:11
*** alenavolk has joined #openstack-swift00:12
*** thegreenhundred has joined #openstack-swift00:14
*** tovin07_ has joined #openstack-swift00:30
*** alenavolk has quit IRC00:31
*** alenavolk has joined #openstack-swift00:32
*** alenavolk has quit IRC00:32
*** alenavolk has joined #openstack-swift00:32
*** alenavolk has quit IRC00:33
*** catintheroof has quit IRC00:35
*** SkyRocknRoll has joined #openstack-swift00:43
*** kei_yama has quit IRC00:44
*** spotz has quit IRC01:00
*** sawinetehleet has joined #openstack-swift01:06
*** mat128 has joined #openstack-swift01:06
sawinetehleetHello, is it possible to strip the path from multiple objects that are uploaded like so: swift upload --skip-identical --verbose --header "X-Delete-After: 300" test /root/uptest/*/*01:07
sawinetehleetI would only like to store them with the filename, without the full path01:07
*** klrmn has quit IRC01:14
*** alenavolk has joined #openstack-swift01:29
*** spotz has joined #openstack-swift01:35
*** mat128 has quit IRC01:41
*** alenavolk has quit IRC02:00
*** alenavolk has joined #openstack-swift02:01
*** alenavolk has quit IRC02:01
*** alenavolk has joined #openstack-swift02:02
*** alenavolk has quit IRC02:02
*** catintheroof has joined #openstack-swift02:03
*** alenavolk has joined #openstack-swift02:16
*** kiennt has joined #openstack-swift02:20
*** two_tired has joined #openstack-swift02:23
*** __david_ has quit IRC02:25
*** SkyRocknRoll has quit IRC02:31
*** mat128 has joined #openstack-swift02:40
*** klrmn has joined #openstack-swift03:06
*** catintheroof has quit IRC03:07
sawinetehleetCan you apply header to the segments ?03:09
*** mat128 has quit IRC03:12
*** itlinux has quit IRC03:33
sawinetehleetIs there a workaround this bug? https://bugs.launchpad.net/python-swiftclient/+bug/115995103:36
openstackLaunchpad bug 1159951 in python-swiftclient "swift client header is not used for uploading large object" [Undecided,In progress] - Assigned to Tong Li (litong01)03:36
*** links has joined #openstack-swift03:39
*** gkadam_ has joined #openstack-swift03:40
*** alenavolk has quit IRC03:42
*** psachin has joined #openstack-swift03:43
*** vito_ has joined #openstack-swift03:53
*** vito_ has quit IRC03:54
*** mat128 has joined #openstack-swift04:09
*** klrmn has quit IRC04:23
*** gyee has quit IRC04:23
*** two_tired has quit IRC04:29
*** itlinux has joined #openstack-swift04:35
*** itlinux has quit IRC04:52
*** vint_bra has joined #openstack-swift04:53
*** vint_bra has quit IRC04:58
*** ianychoi__ has joined #openstack-swift05:12
*** ianychoi_ has quit IRC05:16
*** psachin has quit IRC06:10
*** kiennt has left #openstack-swift06:13
*** psachin has joined #openstack-swift06:14
*** links has quit IRC06:15
*** psachin has quit IRC06:16
*** psachin has joined #openstack-swift06:16
*** pcaruana has joined #openstack-swift06:22
*** silor has joined #openstack-swift06:39
*** hseipp has joined #openstack-swift06:40
*** rcernin has joined #openstack-swift06:40
*** vint_bra has joined #openstack-swift06:41
*** vint_bra has quit IRC06:46
*** cbartz has joined #openstack-swift07:03
*** hoonetorg has quit IRC07:04
*** hoonetorg has joined #openstack-swift07:17
*** silor has quit IRC07:25
*** links has joined #openstack-swift07:28
*** tesseract has joined #openstack-swift07:32
*** geaaru has joined #openstack-swift07:43
*** cschwede has joined #openstack-swift08:01
*** ChanServ sets mode: +v cschwede08:01
acolesgoo dmorning08:06
*** openstackgerrit has quit IRC08:17
*** vint_bra has joined #openstack-swift08:30
*** vint_bra has quit IRC08:34
*** kei_yama has joined #openstack-swift08:40
*** openstackgerrit has joined #openstack-swift08:40
openstackgerritAlistair Coles proposed openstack/swift master: sharding: Test and refactor for _misplaced_objects  https://review.openstack.org/45189608:40
*** openstackgerrit has quit IRC09:03
mattoliverauacoles: morning09:07
acolesmattoliverau: o/09:07
mattoliverauYeah we should squish it in :)09:07
acolesmattoliverau: ok. you want me to or can you?09:07
acolesI'm at a desk with coffee, you may be in an armchair with beer :)09:08
*** hoonetorg has quit IRC09:36
mattoliveraulol, your more then welcome too.09:41
mattoliverauBec had her gallbladder removed on Tuesday, and is sore and can't bend or lift anything. She's doing well, but means I have _alot_ of errands especially post work hours :)09:43
acolesmattoliverau: ouch. you take car of her, I'll take care of the patch :)09:44
mattoliverau:)09:44
acolescare*09:44
*** psachin has quit IRC09:45
mattoliverauI'm playing with the sharder some, had some thoughts on where to improve it, concurrancy etc, to make a tiny step in the right direction of potentially getting something like the auditor to feed it rather then it also having to walk through the container directories. The latter being a 2.0 or 3.0 of course. I've got a purdy digram and hope we can discuss at PTG.09:48
acolesinteresting09:49
acolesgetting 1.0 done with a roadmap for improvements sounds like a plan09:49
mattoliverau+109:49
*** baojg_ has joined #openstack-swift09:51
*** baojg has quit IRC09:52
mattoliverauthe auditor currently just opens a db file and calls a get_info to see if it throws an exception. So getting it to actually test by getting a peice of useful metadata should almost be a noop :)09:52
*** hoonetorg has joined #openstack-swift09:53
acolescontainer sync has a pattern whereby the replicator finds sync'd containers in its post_replicate_hook09:56
acolessame place also feeds the reconciler - IDK if that is similar scenario to sharder09:57
*** psachin has joined #openstack-swift09:58
*** kei_yama has quit IRC10:06
*** tovin07_ has quit IRC10:11
*** vint_bra has joined #openstack-swift10:18
*** vint_bra has quit IRC10:22
*** szaher has quit IRC10:41
*** szaher has joined #openstack-swift10:46
*** abhitechie has joined #openstack-swift10:58
*** psachin has quit IRC11:08
*** abhitechie has quit IRC11:11
*** silor has joined #openstack-swift11:12
*** jistr is now known as jistr|biab11:12
*** psachin has joined #openstack-swift11:19
*** mat128 has joined #openstack-swift11:37
*** hseipp has quit IRC11:48
*** thegreenhundred has quit IRC11:55
*** hseipp has joined #openstack-swift11:59
*** mvk has joined #openstack-swift12:03
*** hseipp has quit IRC12:06
*** vint_bra has joined #openstack-swift12:06
*** vint_bra has quit IRC12:11
*** jistr|biab is now known as jistr12:20
*** mat128 has quit IRC12:27
*** catintheroof has joined #openstack-swift12:30
*** mat128 has joined #openstack-swift12:31
*** jaosorior has quit IRC12:37
*** thegreenhundred has joined #openstack-swift12:38
*** jaosorior has joined #openstack-swift12:38
*** gkadam_ has quit IRC12:40
*** jaosorior has quit IRC12:44
*** jaosorior has joined #openstack-swift12:45
*** jaosorior has quit IRC12:45
*** jaosorior has joined #openstack-swift12:46
*** abhitechie has joined #openstack-swift13:11
*** mat128 has quit IRC13:17
*** EmreSoku_ has joined #openstack-swift13:19
*** ianychoi__ is now known as ianychoi13:20
EmreSoku_hi guys, I have a stack with one proxy and three data servers, the proxy just died, i have no backup13:21
EmreSoku_do you think there's a way to recover still?13:21
EmreSoku_data servers are alive.13:23
EmreSoku_but I don't know if without a key or anything they're recoverable.13:23
*** gyee has joined #openstack-swift13:41
*** ChubYann has quit IRC13:56
*** jaosorior has quit IRC13:57
*** psachin has quit IRC14:04
*** ChubYann has joined #openstack-swift14:10
*** alenavolk has joined #openstack-swift14:15
*** itlinux has joined #openstack-swift14:15
*** gyee has quit IRC14:17
*** gyee has joined #openstack-swift14:17
*** hseipp has joined #openstack-swift14:20
*** vint_bra has joined #openstack-swift14:31
*** alenavolk has quit IRC14:31
*** alenavolk has joined #openstack-swift14:37
*** alenavolk has quit IRC14:55
*** openstackgerrit has joined #openstack-swift14:59
openstackgerritViktor Varga proposed openstack/swift master: Use more specific asserts in test/unit/common/middleware  https://review.openstack.org/34277014:59
openstackgerritAlistair Coles proposed openstack/swift master: Respect co-builder partition moves when min_part_hours is zero  https://review.openstack.org/49963415:10
acolesEmreSoku_: you will need a copy of you swift.conf file and ring files which you should have on the storage nodes, to setup a replacement proxy15:23
acolesEmreSoku_: if you were using encryption middleware then you will need the encryption key15:24
acoleswhich should not be on the storage nodes !15:24
EmreSoku_thanks acoles, i do have swift.conf and it only has one variable; swift_hash_path_suffix15:26
openstackgerritGábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/common  https://review.openstack.org/34278115:26
openstackgerritGábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/container  https://review.openstack.org/34280815:26
openstackgerritGábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/obj tests  https://review.openstack.org/34283015:26
openstackgerritGábor Antal proposed openstack/swift master: Update hacking and enable off-by-default checks  https://review.openstack.org/48662715:26
EmreSoku_ring files, you mean account.ring.gz container.ring.gz I guess?15:27
EmreSoku_and object.ring.gz15:27
EmreSoku_of course15:27
*** SkyRocknRoll has joined #openstack-swift15:28
*** alenavolk has joined #openstack-swift15:30
*** EmreSoku_ has quit IRC15:32
*** pcaruana has quit IRC15:32
notmynamegood morning15:34
timburkegood morning15:42
*** itlinux has quit IRC15:46
*** klrmn has joined #openstack-swift15:50
*** alenavolk has quit IRC15:51
*** EmreSokullu has joined #openstack-swift15:53
*** alenavolk has joined #openstack-swift15:54
*** mvk has quit IRC15:59
*** oshritf has quit IRC16:07
*** mat128 has joined #openstack-swift16:15
*** hoonetorg has quit IRC16:18
*** EmreSokullu has quit IRC16:20
*** hseipp has quit IRC16:24
*** geaaru has quit IRC16:28
*** jistr is now known as jistr|afk16:31
*** chsc has joined #openstack-swift16:36
*** chsc has joined #openstack-swift16:36
openstackgerritAlistair Coles proposed openstack/swift master: Add container sharding to Swift containers  https://review.openstack.org/42390616:38
acolesmattoliverau: squashed ^^16:38
*** mat128 has quit IRC16:44
*** itlinux has joined #openstack-swift16:46
*** silor has quit IRC16:46
*** joeljwright has joined #openstack-swift16:50
*** ChanServ sets mode: +v joeljwright16:50
*** alenavolk has quit IRC17:00
*** alenavolk has joined #openstack-swift17:04
*** psachin has joined #openstack-swift17:11
*** rcernin has quit IRC17:12
*** tesseract has quit IRC17:14
*** mat128 has joined #openstack-swift17:21
*** joeljwright has quit IRC17:31
*** se-tester_ has joined #openstack-swift17:39
*** alenavolk has quit IRC17:42
*** jistr|afk is now known as jistr17:50
*** psachin has quit IRC17:52
*** links has quit IRC17:52
notmynamedid we do the thing where slo uploads in the client sets the expiry time of the segments to the same value as the manifest?17:56
notmynametimburke: ^ ?17:56
notmynameor was that a Bad Idea (tm) because segments could be in multiple SLOs? (but they're named uniquely via `swift`)17:56
notmynamesawinetehleet: (moving from pm) another thing you could do with a manifest of the segments is create a temporary-access URL to the manifest object and then share it with someone else18:01
notmynamethen they could get the archives and not need credentials to the storage service18:02
notmyname...but I don't know if you need that functionality or not for your archives18:02
*** cschwede has quit IRC18:09
*** alenavolk has joined #openstack-swift18:11
timburkenotmyname: we did not. the trouble always came back around to "how do we deal with the POST that changes or removes the expiry?" and the only way we could see forward for that was "well, HEAD before every POST"18:11
notmynametimburke: which results in "...ewww"18:11
timburkeyup18:11
claygdoes crazy swiftclient already do a HEAD before every POST?  or PUT anyway...18:13
timburkeclayg: before every PUT and DELETE, not POST (yet)18:13
timburke--leave-segments disables it18:14
claygtimburke: i'm so glad you know these things18:14
notmyname+118:14
notmynamesawinetehleet: note what timburke for improved speed in your script (the --leave-segments)18:14
*** se-tester_ has quit IRC18:16
*** jistr is now known as jistr|off18:21
*** mat128_ has joined #openstack-swift18:34
*** mat128 has quit IRC18:37
*** mat128_ has quit IRC18:43
*** mat128 has joined #openstack-swift18:43
*** rcernin has joined #openstack-swift19:01
*** cbartz has quit IRC19:03
*** hoonetorg has joined #openstack-swift19:14
notmynametdasilva: on https://bugs.launchpad.net/swift/+bug/1609231 where are you seeing it checked on reads? I can't find that in the code (check_object_creation() is only in the PUT methods of the proxy obj controller and the obj server)19:30
openstackLaunchpad bug 1609231 in OpenStack Object Storage (swift) "check MAX_ACCOUNT_NAME_LENGTH in all requests" [Low,New] - Assigned to zheng yin (yin-zheng)19:30
notmynametdasilva: ah, hmmm... constraints.MAX_ACCOUNT_NAME_LENGTH is explicitly used (not the helper)19:31
tdasilvanotmyname: maybe i'm looking at the wrong place, but I saw this: https://github.com/openstack/swift/blob/master/swift/proxy/controllers/account.py#L5619:32
notmynameadded in https://review.openstack.org/#/c/27724/ ?19:33
patchbotpatch 27724 - swift - Autocreate cleanups (MERGED)19:33
notmynameoh, no. that just moved19:33
*** SkyRocknRoll has quit IRC19:33
notmynametd https://github.com/openstack/swift/commit/1a586323825ca681a7c6739bc6f6fd9131932a2c19:36
notmynametdasilva: https://github.com/openstack/swift/commit/1a586323825ca681a7c6739bc6f6fd9131932a2c19:36
notmynamewhich makes sense, right? reads and autocreate an account. and when an account is created, check the lenght limits19:37
notmynametdasilva: looks like at some point a refactoring may have lost the "and if autocreate" part of the check19:38
notmynamemaybe?19:38
tdasilvanotmyname: p 27724 calls out that the autocreate part was removed on purpose19:39
patchbothttps://review.openstack.org/#/c/27724/ - swift - Autocreate cleanups (MERGED)19:39
notmynameah, in part because we lazy autocreate now. we only autocreate the account on first container create, so we fake it. which means the account read is pretending to act like it's creating the account, but it's not, so it has to do the len check?19:39
tdasilvayep, that was the conclusion i was reaching too19:40
tdasilvaso even thou we are not really creating the account, we are 'fake' creating it, or lazily as you put it19:40
notmynamewhich seems like it has the gap of a read on a previously created account that was created before the account lenght max was reduced19:41
notmynamegap == spurious error19:41
tdasilvayes19:41
notmynameso, I *want* to say, "that's unfortunate. don't change cluster constraints when there's data in the cluster"19:41
notmynamehowever... there *is* a gap19:41
tdasilvamaybe we could just move the check?19:42
notmynameand I'm guessing you've found some cluster that is hitting this?19:42
tdasilvaoh no...sorry...that was just me hitting the bug triage etherpad19:42
notmynameheh, ok19:42
notmynameyeah, I think moving the check under line 76 (the elif autocreate) would probably be better)19:42
tdasilvawhat's weird about that is that now a GET request might or might not return a MAX_ACCOUNT_NAME_LENGTH error19:43
tdasilvaand it would be sort of hidden from the user why that is happening19:44
tdasilvais that true?19:44
notmynametrying to enumerate the possibilities: 1. it would return length error if it couldn't find the account and the requested name is too long19:44
notmyname2. it wouldn't if the account were found19:45
notmyname3. it wouldn't if the account weren't found and autocreate is off (generally rare, Ithink)19:45
notmynamewhat about deleted account with too long of a name that is being fetched as a precursor to being autocreated?19:46
notmynamedeleted but not yet reaped19:46
*** silor has joined #openstack-swift19:47
notmynamedo you see other possiblities?19:47
tdasilvanotmyname: i think you got it, my point is more like, let's say max is changed from 10 to 3. It is possible that a GET on account1 might return OK, but a GET on account2 might return error MAX_ACCOUNT_NAME_LENGTH19:48
notmynameah, right. and the user doesn't know the difference. but internally account1 was created before the constraint change. and account2 has never been created19:49
tdasilvait might not be obvious to the user why one get returns success and the other returns saying the account name is too long, while they have the same lenght19:49
notmynameyeah19:49
tdasilvaright!19:49
notmynameyes, I agree that is a confusing situation for end users19:50
notmynameagain, by first reaction is to include the operator's pager number in the error message and say "call this guy and ask him why he made your life more difficult"19:50
notmyname;-)19:50
tdasilvalol19:51
tdasilvawe should put operators phone number on the /info request19:51
notmynameyes! (as long as it's not mine)19:51
notmynameI don't think it sounds good to include the constraint at time of creation with an account response19:52
notmynameand I don't think we can include "current limit is X but it used to be Y and Z" with the error19:52
notmynamehave you ever heard of someone chaging the lenth limit away from the default? why is it configurable at all?19:53
notmynameisn't 256 characters enough for anyone?19:53
notmyname(only half joking)19:53
tdasilvai have not, but maybe the bug submitter ran into that issue? I can only assume19:55
notmynameyeah, same here19:55
notmynameso I'm tempted to say we should move the check, but I don't see a clear path forward to reduce the end-user confusion in your 10->3 example19:56
tdasilvaprobably should be marked as wishlist and see if more people complain about it19:58
notmynamemeh... low priority bug for moving the check19:59
notmynameIMO19:59
tdasilvaworks20:00
tdasilvanotmyname: need to drop off, will be back later20:00
notmynametdasilva: thanks for the bug triage! :-)20:00
timburkewhy not just 404 on GET if the name's too long, and wait on the constraint error until they actually try to PUT something?20:08
*** hseipp has joined #openstack-swift20:11
notmynametimburke: after moving the check? you'd get a 404 when doing an account GET/HEAD, then you'd never do a container PUT? the way it is now, you'd get a 200 on the account GET/HEAD20:11
notmynameand it's very unlikely the user has the ability to actually PUT the account (if that's even enabled in the cluster!)20:12
timburkeright... but it was too long anyway, right?20:13
timburkei suppose i oughta actually look at the code20:14
notmynametimburke: no? I think tdasilva got the right triage. we may get a small move of some code out of it, but low priority :-)20:14
*** hseipp has quit IRC20:18
rcerninping swift3 guys, do we need to also include the s3token entry point?20:27
rcerninswift3 is failing with  LookupError: Entry point 's3token' not found in egg 'swift3' (dir: /usr/lib/python2.7/site-packages; protocols: paste.filter_factory, paste.filter_app_factory; entry_points:)20:28
rcerninseems there is still some piece missing.20:28
timburkercernin: what version of swift3 are you using?20:28
rcerninbut egg is there20:28
rcerninhi timburke20:29
rcerninuno momento20:29
rcernintimburke: 1.11.0-620:30
timburkehrm... should be there... https://github.com/openstack/swift3/blob/v1.11/setup.cfg#L3220:30
timburkelemme poke around a bit...20:30
rcernintimburke: thats what I was thinking!20:31
rcerninstrange20:31
timburkewhat's the use line look like?20:32
timburkeand i'm trying to remember what a proper egg looks like -- i know my dev install has a .egg-info dir with a entry_points.txt in it...20:34
rcerninuse = egg:swift3#s3token20:34
timburkelooks right20:35
rcerninoh my bad!:( timburke++ sorry.. pebkac20:35
rcerninthe other config misses use line!20:36
timburkercernin: glad you solved it :-)20:36
rcernintimburke: http://paste.openstack.org/show/620136/ this is the current s3token20:36
rcernintimburke: may i ask about the paste.filter_factory?20:37
timburkeah... i wonder if paste is getting confused by having both use *and* paste.filter_factory listed...20:37
rcerninyeah could cause issue20:39
*** silor has quit IRC20:54
*** mat128 has quit IRC20:55
*** catintheroof has quit IRC21:31
*** alenavolk has quit IRC21:37
*** alenavolk has joined #openstack-swift21:37
*** alenavolk has quit IRC21:37
*** alenavolk has joined #openstack-swift21:38
*** alenavolk has quit IRC21:38
*** alenavolk has joined #openstack-swift21:39
*** alenavolk has quit IRC21:39
*** alenavolk has joined #openstack-swift21:40
*** alenavolk has quit IRC21:40
*** abhinavtechie has joined #openstack-swift22:29
*** abhitechie has quit IRC22:30
*** rcernin has quit IRC22:30
*** itlinux has quit IRC22:34
*** thegreenhundred has quit IRC22:38
*** abhinavtechie has quit IRC22:54
*** chsc has quit IRC22:57
openstackgerritTim Burke proposed openstack/swift master: api-ref: update docs links  https://review.openstack.org/49929322:57
*** kei_yama has joined #openstack-swift23:31
*** abhitechie has joined #openstack-swift23:36
*** kei_yama has quit IRC23:47
*** chsc has joined #openstack-swift23:53
*** chsc has joined #openstack-swift23:53
*** catintheroof has joined #openstack-swift23:55
openstackgerritTim Burke proposed openstack/swift master: Stop clearing req.acl in DLO  https://review.openstack.org/49985623:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!