Thursday, 2017-10-19

torgomaticman, adding a new xattr sounds easy until you realize that there's diskfile :|00:07
*** caiobrentano has joined #openstack-swift00:30
*** tovin07_ has joined #openstack-swift00:46
*** wes_dillingham has joined #openstack-swift00:46
*** klrmn has quit IRC01:15
*** two_tired has joined #openstack-swift01:17
*** vint_bra has joined #openstack-swift01:45
*** bkopilov has quit IRC02:07
*** m_kazuhiro has joined #openstack-swift02:16
*** gyee has quit IRC02:21
*** klrmn has joined #openstack-swift02:47
*** caiobrentano has quit IRC02:57
*** vint_bra has quit IRC03:07
*** gkadam has joined #openstack-swift03:23
*** tovin07 has quit IRC03:25
*** tovin07 has joined #openstack-swift03:26
*** bkopilov has joined #openstack-swift03:29
openstackgerritMatthew Oliver proposed openstack/swift master: Make max_ranges configurable  https://review.openstack.org/49341803:29
openstackgerritMerged openstack/swift master: tighten up drop_privileges unit tests  https://review.openstack.org/51299203:36
openstackgerritMerged openstack/swift master: Remove swift-temp-url man page.  https://review.openstack.org/51319403:39
*** tovin07 has quit IRC04:21
*** tovin07 has joined #openstack-swift04:21
*** psachin has joined #openstack-swift04:22
*** two_tired has quit IRC04:28
*** wes_dillingham has quit IRC04:40
*** m_kazuhiro_ has joined #openstack-swift04:42
*** m_kazuhiro has quit IRC04:45
*** mabrams has joined #openstack-swift04:50
*** newmember has quit IRC05:11
*** newmember has joined #openstack-swift05:11
*** ChubYann has quit IRC05:15
kota_timburke: hmm... i may be mis-read for the usage on multipart_byteranges_to_document_iters that doesn't work as I expected at http://logs.openstack.org/43/512543/5/check/legacy-swift3-tox-tempauth/10e604c/job-output.txt.gz#_2017-10-19_03_27_07_28066705:16
kota_:/05:16
kota_I'd prefer to use patch set 4 (use email.parser) rather than digging the reason more for now. And then fix it in s3api feature branch if needed.05:21
kota_timburk: I'll add the suggestion to use multipart_byteranges_to_document_iters around there05:22
kota_lq05:22
kota_oops, i mis-type, thinking of using console, not irc client :/05:23
openstackgerritKota Tsuyuzaki proposed openstack/swift feature/s3api: Import all swift3 code as s3api  https://review.openstack.org/51228405:28
*** spectr has joined #openstack-swift05:48
*** klrmn has quit IRC06:01
*** parab0la has joined #openstack-swift06:02
*** hoonetorg has quit IRC06:09
*** parab0la has left #openstack-swift06:19
*** hoonetorg has joined #openstack-swift06:25
*** hseipp has joined #openstack-swift06:37
*** pcaruana has joined #openstack-swift06:55
*** newmember has quit IRC07:18
*** newmember has joined #openstack-swift07:19
*** tesseract has joined #openstack-swift07:21
*** geaaru has joined #openstack-swift07:32
*** m_kazuhiro_ has quit IRC07:44
openstackgerritMerged openstack/swift master: Be more tolerant of exception messages from sqlite  https://review.openstack.org/51320608:17
*** cbartz has joined #openstack-swift08:20
acolesgood morning08:21
kota_mattoliverau: thx for the comment on freedom feature branch!08:25
kota_acoles: morning08:25
acoleskota_: o/08:25
kota_notmyname: did you create trello board for swift3->s3api migration?08:27
kota_that would be helpful to get the status among me, tdasilva, notmyname, timburke and all swift contributors.08:28
acoleskota_: *freedom* feature branch ? I like the sound of that!08:28
*** vint_bra has joined #openstack-swift08:34
*** vint_bra has quit IRC08:41
kota_acoles: i just re-call the matt's comments on https://review.openstack.org/#/c/512284/ :P08:43
patchbotpatch 512284 - swift (feature/s3api) - Import all swift3 code as s3api08:43
acoleskota_: OIC. I was thinking we had a new feature branch, like feature/freedom-from-backwards-compatibility :P08:45
kota_acoles: it's much fun!08:45
acoleswe should only work on feature branches, it's so liberating08:46
acoles;)08:46
*** spectr has quit IRC08:47
openstackgerritKota Tsuyuzaki proposed openstack/swift feature/s3api: Continue merge swift3 middleware  https://review.openstack.org/51317008:47
kota_tdasilva: that's a version w/ no unit test failures (at least, in my local)08:47
* kota_ likes more code cleanup (in particular naming somewhere) tho.08:48
*** spectr has joined #openstack-swift08:48
*** spectr has quit IRC08:48
*** spectr has joined #openstack-swift08:50
acoleskota_: I wonder if we should try to get patch https://review.openstack.org/#/c/499634/ merged before the next release? I know you are very busy, so I am just mentioning it :)08:51
patchbotpatch 499634 - swift - Respect co-builder partition moves when min_part_h...08:51
acolesmattoliverau: thanks for your comments on patch 51275608:53
patchbothttps://review.openstack.org/#/c/512756/ - swift (feature/deep) - WIP only make single request to get container list...08:53
acolesI've been wondering how to best handle bad response from one shard08:53
acolesalso wondering overnight if this whole method could be deferred for initial reduce scope - i.e. while in sharding state just continue to get the root container listing and accept that any updates to the shards are not yet included in listing08:55
*** tovin07 has quit IRC09:16
*** tovin07 has joined #openstack-swift09:17
*** oshritf has quit IRC09:29
*** SkyRocknRoll has joined #openstack-swift09:34
*** SkyRocknRoll has quit IRC09:34
*** SkyRocknRoll has joined #openstack-swift09:34
kota_acoles: good idea to get it into the release.09:37
kota_I added the patch into PriorityReviews09:39
kota_acoles: and i found my draft comments on patch set 2 :,(  It seems like I was doing reviewing but tons of things intercepted me :/09:40
acoleskota_: thanks!09:41
kota_still needs to get my head up to the patch w/ doing other works (in particular, s3api and symlink for m_kazuhiro)09:41
acoleskota_: yeah I understand you have a lot going on. I hope that maybe clayg will also be able to review it.09:42
*** oshritf has joined #openstack-swift09:45
kota_acoles: thx, I think I will be able to take a time to look at it in the next week and expect it can be ok for the next week according to the today's meeting info...09:48
*** oshritf has quit IRC09:48
acoleskota_: great, thank you09:48
*** SkyRocknRoll has quit IRC09:48
*** _ix has joined #openstack-swift09:50
_ixGood mornig, folks. If anyone is about, I'm wondering if anyone has some ideas as to what the problem might be in my swift nodes today.09:51
kota_hmmm.... why dsvm gate failed with....?09:51
_ixWe've been battling high disk utilization for the past few months in debating whether we expand storage or begin storing offsite.09:51
_ixYesterday, we started removing a number of objects, and they were deleted across the nodes... but, one swift node's disks trended upwards, and now while the majority of the cluster is at around 60% utilization, the one in question is sitting at 95-99 across all of its disks.09:53
*** oshritf has joined #openstack-swift10:01
*** tovin07_ has quit IRC10:01
*** oshritf has quit IRC10:05
*** oshritf has joined #openstack-swift10:08
*** oshritf has quit IRC10:09
*** oshritf has joined #openstack-swift10:18
*** mvk has quit IRC10:19
*** oshritf has quit IRC10:26
*** oshritf has joined #openstack-swift10:27
*** vint_bra has joined #openstack-swift10:27
*** _ix has quit IRC10:37
*** oshritf has quit IRC10:39
*** bkopilov has quit IRC10:47
*** mvk has joined #openstack-swift10:48
*** wes_dillingham has joined #openstack-swift10:48
*** oshritf has joined #openstack-swift10:52
*** wes_dillingham has quit IRC10:56
tdasilvakota_: ack on both patches11:23
tdasilvakota_, notmyname: https://trello.com/b/ZloaZ23t/s3api11:24
tdasilvai'm going to start with creating lists somewhat similar to what container sharding is doing, then we can modify/add as we go....11:25
tdasilvaacoles, rledisez: in case you are interested: https://lists.fosdem.org/pipermail/fosdem/2017-October/002616.html11:28
acolestdasilva: thanks11:28
*** itlinux has joined #openstack-swift11:34
*** caiobrentano has joined #openstack-swift11:39
*** kallenp has joined #openstack-swift11:44
kallenpPlase, Can I ask here for contact for Pavel Kvasnicka "OpenStack Swift *master*" ? Thanks, Petr Kallen, Czech Republic11:46
rlediseztdasilva: thx for the link11:46
tdasilvakallenp: onovy might be able to help with contacting Pavel11:48
kallenptdasilva: thanks11:48
tdasilvakallenp: https://github.com/openstack/swift/blob/master/AUTHORS#L27911:49
acoleskallenp: https://review.openstack.org/#/q/owner:pavel.kvasnicka%2540firma.seznam.cz+status:open11:49
tdasilvaI need to step out for a bit, be back later11:52
kallenpacoles: thanks11:52
*** vint_bra has joined #openstack-swift11:53
*** silor has joined #openstack-swift11:54
kallenpMy frined ask me: Howto connect NextCloud to Swift object storage ? I don't know nothing about it. I am need to create new user with password and give him read only permissions for all data. I found this documentation on NextCloud https://docs.nextcloud.com/server/11/admin_manual/configuration_files/primary_storage.html But I really dont'know something about Swift. Can I ask here for a help to do it ?11:54
openstackgerritAlistair Coles proposed openstack/swift feature/deep: only make single request to get container listings  https://review.openstack.org/51275611:54
*** mvk has quit IRC11:54
*** rcernin has joined #openstack-swift11:58
acoleskallenp: you are welcome to ask questions here about swift. You will find docs here https://docs.openstack.org/swift/latest/11:59
kallenpacoles: ok and thanks for your time, can I ask you howto to do it step-by-step ?12:00
acoleskallenp: we have docs to help you setup swift. for a dev setup start here https://docs.openstack.org/swift/latest/development_saio.html12:01
acolesfull install guides for swift and other openstack projects are here https://docs.openstack.org/pike/#install-guides12:02
acolesfrom the nextcloud page you linked to, it looks like it assumes a keystone identity service, you'll find an install guide for that under that link too ^^12:04
*** mvk has joined #openstack-swift12:07
*** psachin has quit IRC12:13
*** NM has joined #openstack-swift12:15
kallenpOK, but really, I am need this only for one-time.12:15
*** itlinux has quit IRC12:15
*** bkopilov has joined #openstack-swift12:18
*** catintheroof has joined #openstack-swift12:27
kallenpThis command does not work for me "swift list" . I have these output "Auth version 2.0 requires OS_AUTH_URL, OS_USERNAME, OS_PASSWORD, and12:39
kallenpOS_TENANT_NAME OS_TENANT_ID to be set or overridden with --os-auth-url,12:39
kallenp--os-username, --os-password, --os-tenant-name or os-tenant-id. Note:12:39
kallenpadding "-V 2" is necessary for this."12:39
kallenpReally I know nothing about swift.12:39
kallenpCan I ask here for a help ?12:40
kallenpI really need it only for one time12:40
kallenpI found this basics video https://www.youtube.com/watch?v=N76-HAxRymM&t=861s but it really not work.12:40
*** catintheroof has quit IRC12:42
*** gkadam has quit IRC12:45
*** SkyRocknRoll has joined #openstack-swift12:47
acoleskallenp: the swift CLI needs those options to authenticate with an identity service - you either need to set them in you env or pass them on command line - examples here https://docs.openstack.org/python-swiftclient/latest/cli/index.html#authentication12:48
*** catintheroof has joined #openstack-swift12:51
*** SkyRocknRoll has quit IRC12:52
*** catintheroof has quit IRC12:53
*** wes_dillingham has joined #openstack-swift12:58
*** StevenK has quit IRC12:59
*** mjturek has joined #openstack-swift13:14
kota_tdasilva: thx for the trello board. I'm get in now.13:14
*** StevenK has joined #openstack-swift13:16
*** StevenK is now known as Guest2579213:16
*** Guest25792 is now known as StevenK13:18
*** rcernin has quit IRC13:21
*** itlinux has joined #openstack-swift13:27
*** rcernin has joined #openstack-swift13:35
*** oshritf has quit IRC13:38
*** rcernin has quit IRC13:40
*** oshritf has joined #openstack-swift14:00
*** itlinux has quit IRC14:07
*** kallenp1 has joined #openstack-swift14:08
tdasilvaacoles: how often do you merge master into feature branch?14:08
acolestdasilva: I think timburke is aiming to do it once per week for feature/deep14:09
acolesof if something lands on master that you need, then sooner14:09
acolesI guess it depends how much you anticipate changes on master causing merge conflicts, and keeping on top of that14:09
*** kallenp has quit IRC14:10
tdasilvaacoles: cool, thanks! can you share that paste you have again on steps to merge master to feature branch?14:10
acoleswith crypto I think I may have aimed for once per two weeks14:10
tdasilvaacoles: oh i see, so it doesn't have to be that often14:10
tdasilvait really depends as you said14:11
acolestdasilva: http://paste.openstack.org/show/624084/14:13
tdasilvaacoles: thanks14:14
*** kallenp1 has quit IRC14:14
*** jrichli- is now known as jrichli14:16
*** ChanServ sets mode: +v jrichli14:16
*** itlinux has joined #openstack-swift14:17
*** mat128 has joined #openstack-swift14:28
*** kallenp has joined #openstack-swift14:31
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Allow sharded container to be deleted (when empty)  https://review.openstack.org/51340414:32
*** itlinux has quit IRC14:34
*** spectr has quit IRC14:38
*** gkadam has joined #openstack-swift14:38
*** gyee has joined #openstack-swift14:42
*** mwheckmann has joined #openstack-swift14:49
*** gkadam is now known as gkadam-away15:05
*** klrmn has joined #openstack-swift15:11
*** newmember has quit IRC15:15
*** MVenesio has joined #openstack-swift15:21
*** mvk has quit IRC15:30
*** chsc has joined #openstack-swift15:31
*** chsc has joined #openstack-swift15:31
*** mjturek has quit IRC15:34
*** mabrams has quit IRC15:38
notmynamegood mornign15:39
notmynametdasilva: thanks for the trello board (/me marks it off of today's todo list)15:39
tdasilvanotmyname: sorry to jump the gun, we (kota_ and I) just wanted to start adding some tasks there...15:41
notmynameno worries at all :-)15:41
*** chsc has quit IRC15:47
*** kallenp has quit IRC15:48
*** cbartz has quit IRC15:56
*** klrmn has quit IRC16:05
*** ianychoi has quit IRC16:06
*** ianychoi has joined #openstack-swift16:07
*** silor has quit IRC16:10
*** klrmn has joined #openstack-swift16:11
*** geaaru has quit IRC16:12
*** pcaruana has quit IRC16:18
notmynamekota_: stay safe with https://pmm.nasa.gov/extreme-weather/gpm-sees-intensifying-typhoon-lan-heading-toward-japan16:19
*** silor has joined #openstack-swift16:21
*** MVenesio has quit IRC16:21
notmynamenote that keystone is dropping v2.0 support in the queens release http://lists.openstack.org/pipermail/openstack-dev/2017-October/123783.html16:21
timburkegood morning16:38
*** kallenp has joined #openstack-swift16:39
*** kallenp has quit IRC16:39
timburkenotmyname: i wonder at what point we'll need to drag v2 auth into swiftclient... and whether that will need to include the keystoneauth plugin(s?) (which people may legit want for the SDK following https://github.com/openstack/python-swiftclient/commit/73e4296), or just the simple password-based stuff we prioritize for the CLI...16:41
*** chsc has joined #openstack-swift16:41
notmynametimburke: yeah, curious about (or worried about)16:42
timburketdasilva: acoles: yeah, once/week is my ideal, but i'm definitely willing to let it slip longer16:42
timburkenotmyname: so far they're *only* talking about the server, yeah? has there been *any* mention of dropping client support?16:42
notmynamedoesn't client support rely on the server being able to validate it?16:43
notmynameor are you saying the client can translate it to a server-acceptable v3 format?16:43
*** silor has quit IRC16:44
timburkeno, i'm saying that clients released today still need to be able to work with servers that were deployed last year. whether there are func tests to validate it or not16:44
timburke(or, y'know, for swift, servers that were deployed five years ago, ideally)16:45
notmynameah, right16:45
notmynamethat's a *great* question16:45
*** NM has quit IRC16:47
timburkehmm... i wonder how egg entry points deal with having multiple sources for a given name...16:47
*** gkadam-away has quit IRC16:48
notmynametimburke: I shouted into the void^UI asked in #openstack-dev16:49
*** hseipp has quit IRC16:50
*** NM has joined #openstack-swift16:51
*** tesseract has quit IRC17:04
notmynamekeystone v2 removed in queens for the server. client support likely dropped in rocky (ie 6 months later)17:05
*** oshritf has quit IRC17:10
timburkei saw. which means we've got 6-9 months to figure out what scope we want to continue supporting, and drag that code into swiftclient17:13
*** NM has quit IRC17:28
*** chinkara has joined #openstack-swift17:32
*** csmart has quit IRC17:43
*** NM has joined #openstack-swift17:45
*** oshritf has joined #openstack-swift17:54
*** MVenesio has joined #openstack-swift17:58
openstackgerritTim Burke proposed openstack/swift master: Add support for data segments to SLO and SegmentedIterable  https://review.openstack.org/36537117:59
*** NM has quit IRC18:08
*** NM has joined #openstack-swift18:15
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Do Not Merge - deleted root could delete shards  https://review.openstack.org/51346418:15
*** ChubYann has joined #openstack-swift18:19
notmynamehttps://wiki.openstack.org/wiki/Swift/PriorityReviews has been updated with 2.16.0 things18:21
notmynameshould be anything on that list that surprises anyone (other than "that hasn't landed yet?")18:21
tdasilvayay, p 371150 made the list18:22
patchbothttps://review.openstack.org/#/c/371150/ - swift - Return 404 on a GET if tombstone is newer18:22
notmynametdasilva: that's one of the "what?! that hasn't landed yet? :sad:" ones ;-)18:23
tdasilvaheh18:23
tdasilvathere's also a pretty long list on the 'Needs final approval' section of the dashboard, maybe some of those could land too18:24
notmynameindeed18:24
timburkegood reminder that we ought to address the ci failures for https://review.openstack.org/#/c/478611/ -- are we aiming for a swiftclient release anytime soon, too?18:28
patchbotpatch 478611 - python-swiftclient - Allow for object uploads > 5GB from stdin.18:28
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Delete orphaned shard containers  https://review.openstack.org/51346818:28
timburkeif we *do*, i'd kinda like to see https://review.openstack.org/#/c/498069/ and https://review.openstack.org/#/c/481202/ too18:29
patchbotpatch 498069 - python-swiftclient - Stop mutating header dicts18:29
patchbotpatch 481202 - python-swiftclient - Allow --meta on upload18:29
notmynameseems all the logs for p 478611 have been removed (?) so I'm rechecking it18:31
patchbothttps://review.openstack.org/#/c/478611/ - python-swiftclient - Allow for object uploads > 5GB from stdin.18:31
*** vint_bra has quit IRC18:31
acolesgood night18:31
timburkenotmyname: zaitcev said the failures looked legit... i don't actually remember what they were, though18:32
notmynameyeah.. that's why I was looking too :-)18:32
*** oshritf has quit IRC18:32
tdasilvaacoles: good night18:35
*** vint_bra has joined #openstack-swift18:45
*** vint_bra has quit IRC19:02
*** NM has quit IRC19:03
*** MVenesio has quit IRC19:07
openstackgerritTim Burke proposed openstack/swift master: Let clients request heartbeats during SLO PUTs  https://review.openstack.org/50930619:14
openstackgerritTim Burke proposed openstack/swift master: Make xml responses less insane  https://review.openstack.org/51348019:14
timburkeclayg: ^^^ you see what you make me do???19:19
claygDrive-by: stop being so quadratic.  /me snickers19:20
clayg*so* quadratic19:20
clayg</valley-girl>19:20
timburkei'm sure we're still accidentally quadratic *somewhere else*19:20
*** NM has joined #openstack-swift19:22
timburkeoh, also, there was a driveby to properly fucking *escape strings coming from god-knows-where before we jam them into something claiming to be XML*. forgot to include that one19:22
claygoic, we have to make it not insane *first*19:23
timburkegd right19:23
*** kallenp has joined #openstack-swift19:23
*** newmember has joined #openstack-swift19:23
clayglol at lower bound - dood - YOU'RE ON FIRE!19:23
timburkei'm fairly certain i'll never be a professional java developer. i'm OK with that19:23
claygi'll definately take a look by my EoD - thanks for being awesome timburke19:24
timburkethanks19:24
*** kallenp has joined #openstack-swift19:24
*** kallenp has joined #openstack-swift19:26
* clayg crosses fingers tempest doesn't notice the api change :'(19:26
*** kallenp has quit IRC19:26
timburkeif it does, i'm just rolling back the heartbeat patch. everything's a <delete>! why not! words mean nothing!19:29
*** kallenp has joined #openstack-swift19:31
claygright... words mean nothing.  I'd also be fine with only the new api being correct (or not supporting xml).  Or maybe we can fix this one - you know how paranoid I get :'(19:31
claygidk, i guess tempest gets mad when we want to make a backwards incompatible change... maybe they won't "let" us... idk19:32
claygwe fixed that status code thingy tho right!?  and that was fine19:32
claygi think...19:32
*** kallenp has joined #openstack-swift19:33
*** NM has quit IRC19:35
*** NM has joined #openstack-swift19:39
*** vint_bra has joined #openstack-swift19:40
*** kallenp has quit IRC19:42
*** kallenp has joined #openstack-swift19:47
*** wes_dillingham has quit IRC20:04
*** kallenp has quit IRC20:07
openstackgerritMerged openstack/swift master: Document keystone role element in container ACL  https://review.openstack.org/49384220:08
*** ukaynar has joined #openstack-swift20:38
*** mat128 has quit IRC20:48
*** ukaynar has quit IRC21:00
*** ukaynar has joined #openstack-swift21:00
*** ukaynar has quit IRC21:05
*** ukaynar has joined #openstack-swift21:06
*** ukaynar has quit IRC21:10
openstackgerritTim Burke proposed openstack/swift master: Let clients request heartbeats during SLO PUTs  https://review.openstack.org/50930621:24
*** caiobrentano has quit IRC21:29
*** mvk has joined #openstack-swift21:48
*** NM has quit IRC21:51
*** wes_dillingham has joined #openstack-swift21:56
*** csmart has joined #openstack-swift22:01
openstackgerritMerged openstack/swift feature/s3api: Import all swift3 code as s3api  https://review.openstack.org/51228422:09
openstackgerritMerged openstack/swift feature/deep: Allow sharded container to be deleted (when empty)  https://review.openstack.org/51340422:10
*** vint_bra has quit IRC22:16
mattoliveraumorning22:23
notmynamehello mattoliverau22:23
*** ukaynar has joined #openstack-swift22:27
*** mwheckmann has quit IRC22:30
mattoliveraunotmyname: o/22:44
timburketorgomatic: there's the one i was thinking of! https://review.openstack.org/#/c/502529/ -- i had it backwards; it was to *make sure* that the auditor ran the migration. and you're already +2, so thanks :-)22:56
patchbotpatch 502529 - swift - Create policy_stat table in auditor if missing22:56
timburkefixes bug 1435506! who wants to +A? ;-)22:58
openstackbug 1435506 in OpenStack Object Storage (swift) "Account DB policy_stat Table Needs Some More Creating" [Low,Confirmed] https://launchpad.net/bugs/143550622:58
*** wes_dillingham has quit IRC23:15
*** wes_dillingham has joined #openstack-swift23:26
*** newmember has quit IRC23:41
torgomatictimburke: notmyname: thoughts on adding metadata checksums? do you care if it's in the first patch or is a second, dependent patch better?23:54
notmynameyou mean first patch checks them and adds them to new data, second patch adds them to old data?23:59
torgomaticyep23:59
notmynamesorry, I missed the part where you were splitting it into multiple patches23:59
torgomaticwell, I haven't submitted it yet23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!