Thursday, 2016-07-28

tdasilvaclayg: I was ready to +2 it, but I held back since acoles_ added me as a co-author00:00
*** asettle has quit IRC00:05
claygtdasilva: screw that!  jrichli's already got a +1 on there - that's gotta cover the difference ;)00:08
tdasilvalol00:08
notmynameheh00:08
*** Suyi has quit IRC00:08
*** hosanai has joined #openstack-swift00:20
*** ChanServ sets mode: +v hosanai00:20
*** m_kazuhiro has joined #openstack-swift00:54
*** cdelatte has quit IRC00:55
*** adu has quit IRC00:59
openstackgerritDaisuke Morita proposed openstack/swift: WIP: Changing Policies  https://review.openstack.org/20932900:59
*** tqtran has quit IRC00:59
*** kei_yama has quit IRC01:02
*** kei_yama has joined #openstack-swift01:03
*** gyee has quit IRC01:03
*** chlong has joined #openstack-swift01:06
*** trananhkma has joined #openstack-swift01:09
*** adu has joined #openstack-swift01:09
pdardeaunotmyname: clayg: on patch 347546 it *may* be speaking about upcoming change. i can't say which dimension it's in :-)01:32
patchbotpdardeau: https://review.openstack.org/#/c/347546/ - swift - Moved ipv4 & ipv6 validations to the common utils01:32
pdardeaunotmyname: clayg: iirc, there's an existing patch from torgomatic (sorry, don't have number handy) that would make use of the refactoring01:32
*** NM has joined #openstack-swift01:40
*** NM has quit IRC01:47
claygpdardeau: it was a back to the future e qutoe :)01:47
claygpdardeau: it's acctually sorta fitting -> https://www.youtube.com/watch?v=CUcNM7OsdsY01:49
claygthe bridge *will* exists in 1985!01:49
*** dmorita_ has quit IRC01:56
*** uCHobby has quit IRC02:17
openstackgerritMerged openstack/swift: Add region in ring structure & deployment guide  https://review.openstack.org/32014602:20
*** kei_yama has quit IRC02:22
*** kei_yama has joined #openstack-swift02:23
*** adu has quit IRC02:26
*** tqtran has joined #openstack-swift02:57
*** mingyu has joined #openstack-swift02:58
*** mingyu has quit IRC03:00
*** adu has joined #openstack-swift03:01
*** tqtran has quit IRC03:01
*** abhitechie has joined #openstack-swift03:35
jrichliclayg: :-)03:41
*** bill_az has quit IRC03:49
*** NM has joined #openstack-swift03:50
*** NM has quit IRC03:52
*** dmorita has joined #openstack-swift03:57
*** bill_az has joined #openstack-swift04:01
*** dmorita has quit IRC04:02
*** links has joined #openstack-swift04:03
*** abhitechie has quit IRC04:03
*** ppai has joined #openstack-swift04:27
*** psachin has joined #openstack-swift04:30
*** adu has quit IRC04:32
*** klrmn has quit IRC04:40
*** bill_az has quit IRC04:44
*** tqtran has joined #openstack-swift04:59
*** tqtran has quit IRC05:04
*** flwang1 has quit IRC05:09
*** asettle has joined #openstack-swift05:10
*** asettle has quit IRC05:14
*** ChubYann has quit IRC05:15
*** KriskK has joined #openstack-swift05:18
*** SkyRocknRoll has joined #openstack-swift05:43
*** zaitcev has quit IRC05:46
*** asettle has joined #openstack-swift06:03
*** asettle has quit IRC06:08
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873606:08
*** _JZ_ has quit IRC06:10
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/swift: [WIP] Testing latest u-c  https://review.openstack.org/31844106:10
*** pcaruana has joined #openstack-swift06:17
*** silor has joined #openstack-swift06:18
*** silor1 has joined #openstack-swift06:22
*** silor has quit IRC06:24
*** silor1 is now known as silor06:24
*** oshritf has joined #openstack-swift06:42
*** tesseract- has joined #openstack-swift06:42
*** geaaru has joined #openstack-swift06:44
*** kei_yama has quit IRC06:53
*** asettle has joined #openstack-swift06:57
openstackgerritzheng yin proposed openstack/swift: Simple code and improve the readability of the code  https://review.openstack.org/34814906:57
*** kei_yama has joined #openstack-swift07:00
*** asettle has quit IRC07:01
*** hseipp has joined #openstack-swift07:04
*** ppai has quit IRC07:06
openstackgerritzheng yin proposed openstack/swift: Simple code and improve the readability of the code  https://review.openstack.org/34814907:07
*** permalac has joined #openstack-swift07:13
*** flwang1 has joined #openstack-swift07:29
*** ppai has joined #openstack-swift07:30
*** asettle has joined #openstack-swift07:33
*** asettle has quit IRC07:38
*** rledisez has joined #openstack-swift07:40
openstackgerritPeter Lisák proposed openstack/swift: Add a jitter for check of time synchronization.  https://review.openstack.org/34025007:42
openstackgerritPeter Lisák proposed openstack/swift: Add a jitter for check of time synchronization.  https://review.openstack.org/34025007:44
*** chlong has quit IRC07:47
*** KriskK has quit IRC07:52
*** KriskK has joined #openstack-swift07:54
*** asettle has joined #openstack-swift07:56
*** asettle has quit IRC08:01
cschwedemahatic: just stumbled upon your Swift talk at Linuxcon Berlin. Congrats :)08:17
mahaticChanServ: thanks :) that will be my first linux conf (if I get to go). Will you be attending?08:18
mahaticcschwede: ^08:18
mahaticlol, I posted to chanserv08:18
cschwedemahatic: i hope so; it’s quite close from my home (2 hours by train), so there is a chance to learn more about Swift for me ;)08:19
mahaticcschwede: lol, sure08:20
*** permalac has quit IRC08:26
*** permalac has joined #openstack-swift08:26
*** geaaru has quit IRC08:29
openstackgerritlokesh s proposed openstack/swift: Py3:Fixes TypeError:NamedTemporaryFile() got an unexpected keyword  https://review.openstack.org/34786408:36
*** ellen has quit IRC08:36
openstackgerritChristian Schwede proposed openstack/swift: Allow full storage URL as input to swift-temp-url  https://review.openstack.org/34817308:39
*** geaaru has joined #openstack-swift08:41
*** asettle has joined #openstack-swift08:44
*** asettle has quit IRC08:44
*** ppai has quit IRC08:44
*** asettle has joined #openstack-swift08:44
*** admin6__ has joined #openstack-swift08:49
*** admin6 has quit IRC08:52
*** admin6__ is now known as admin608:52
*** acoles_ is now known as acoles08:52
*** dmorita has joined #openstack-swift08:57
*** ppai has joined #openstack-swift08:58
*** dmorita has quit IRC09:02
*** ouchkernel has quit IRC09:03
*** daemontool has joined #openstack-swift09:03
*** admin6 has quit IRC09:04
*** asettle has quit IRC09:05
*** asettle has joined #openstack-swift09:06
*** ouchkernel has joined #openstack-swift09:09
*** jordanP has joined #openstack-swift09:10
*** permalac has quit IRC09:25
*** hosanai has quit IRC09:31
*** ppai has quit IRC09:43
*** ma91 has joined #openstack-swift09:45
ma91HI, I have a set of files that I want to load on Swift with SwiftOnFile…. once they are represented as objects, I want to allow one user to create temp-urls to these files so that they could be downloaded by anybody having the URL. What kind of ACL shall I set in order to grant temp-url creation to this user?09:47
ma91is read access enough? does this user need to belong to operator_roles?09:49
*** ppai has joined #openstack-swift09:55
*** jordanP_ has joined #openstack-swift09:56
*** jordanP has quit IRC10:00
openstackgerritlokesh s proposed openstack/swift: Py3:Fixes TypeError:NamedTemporaryFile() got an unexpected keyword  https://review.openstack.org/34786410:08
openstackgerritAlistair Coles proposed openstack/swift: Fix broken link in associated projects doc  https://review.openstack.org/34821310:10
*** zhengyin has joined #openstack-swift10:28
openstackgerritGleb Samsonov proposed openstack/swift: go: proxyserver's version with mongodb backend This is our implemetation for some swift proxy functions. Not product-ready yet. But you can take some middlewares or ideas from here  https://review.openstack.org/28715710:32
openstackgerritGleb Samsonov proposed openstack/swift: go: proxyserver's version with mongodb backend This is our implemetation for some swift proxy functions. Not product-ready yet. But you can take some middlewares or ideas from here  https://review.openstack.org/28715710:46
*** jordan__ has joined #openstack-swift11:20
*** m_kazuhiro has quit IRC11:20
*** abhitechie has joined #openstack-swift11:22
*** jordanP_ has quit IRC11:24
*** acoles is now known as acoles_11:25
*** acoles_ is now known as acoles11:25
*** asettle has quit IRC11:27
*** asettle has joined #openstack-swift11:41
*** ma91 has left #openstack-swift11:47
*** chlong has joined #openstack-swift11:54
*** hoonetorg has quit IRC11:56
openstackgerritChristian Schwede proposed openstack/swift: Allow full storage URL as input to swift-temp-url  https://review.openstack.org/34817312:06
openstackgerritlokesh s proposed openstack/swift: Py3: Fixes eventlet ImportError: No module named urllib2  https://review.openstack.org/34827212:11
*** trananhkma has quit IRC12:12
openstackgerritlokesh s proposed openstack/swift: Py3: Fixes eventlet ImportError: No module named urllib2  https://review.openstack.org/34827212:13
*** NM has joined #openstack-swift12:19
*** _JZ_ has joined #openstack-swift12:21
*** KriskK has quit IRC12:26
*** hoonetorg has joined #openstack-swift12:27
*** kei_yama has quit IRC12:28
openstackgerritMerged openstack/swift: Fix broken link in associated projects doc  https://review.openstack.org/34821312:36
*** links has quit IRC12:40
*** klamath has joined #openstack-swift12:46
*** klamath has joined #openstack-swift12:46
*** dmorita has joined #openstack-swift12:58
*** dmorita has quit IRC13:02
*** tqtran has joined #openstack-swift13:02
*** tqtran has quit IRC13:07
*** NM has quit IRC13:08
*** NM has joined #openstack-swift13:09
*** daemontool has quit IRC13:21
*** ppai has quit IRC13:21
*** bill_az has joined #openstack-swift13:21
tdasilvaacoles: hello, I've been adding support for conditional gets for patch 232162, but I had to change a bit from the plan you had laid out https://etherpad.openstack.org/p/swift_symlinks13:32
patchbottdasilva: https://review.openstack.org/#/c/232162/ - swift - Symlink implementation.13:32
tdasilvainstead of using * for X-Object-Sysmeta-Symlink-Etag, i'm using a fake etag value13:32
tdasilvabecause if we use * then any bogus etag the user sends will return Ok13:33
*** ppai has joined #openstack-swift13:35
tdasilvathat is, if I add * to the list of values in the If-Match header send by the user, then it doesn't matter what value the user actually passed in, because the * always matches13:35
openstackgerritlokesh s proposed openstack/swift: Py3: Fixes AssertionError  https://review.openstack.org/34832213:37
acolestdasilva: I think I was meaning that the symlinker would add * to If-Match only for the request to the symlink.13:39
acolestdasilva: ah, but if its not a symlink then that doesn't work?13:39
acolestdasilva: or rather it does work when it shouldn't ;)13:40
acolestdasilva: is that the case you're referring to - when the client supplied url is not actually a symlink?13:40
tdasilvaacoles: yeah, the problem also is that we never know if a request is for a symlink or not13:40
tdasilvaonly after the fact13:40
tdasilvaso the user sends a get for a/c/o -H 'If-Match: some_etag'13:41
tdasilvawhen the sym middleware receives that req, it doesn't know if a/c/o is a symlink or not, so I would need to append * to it every time13:42
tdasilvaon the PUT is different, because I only add X-Object-Sysmeta-Symlink-Etag if user is PUTting a symlink13:43
ppaiHi guys, is swift packaged in pypi and available for over pip install ? Couldn't locate it on https://pypi.python.org/13:43
tdasilvappai: only found the swift client so far13:46
tdasilvappai: looks like it is not, I don't see a pypi-jobs for it on the project-config repo13:49
acolestdasilva: yeah you are right, you need to use something other than *.13:49
*** abhitechie has quit IRC13:49
*** abhitechie has joined #openstack-swift13:50
tdasilvaacoles: and I think it is only needed for If-Match and not for If-None-Match13:50
tdasilvaunless the user sends empty etag13:50
tdasilvaempty string etag13:50
acolestdasilva: in fact, looking a the photo I have of the san antonio whiteboard, we had x-object-sysmeta-symlink-etag="symlink". I just wrote different in the etherpad on the plane!13:50
tdasilvaoh, hehe13:51
tdasilvaacoles: now the challenge is "If-Modified-Since"13:52
ppaiI'm trying to locate openstack packages for Fedora 24 and seems like nobody has them. Fedora 23 shipped kilo and then they stopped shipping openstack packages since liberty. RDO moved all their repos to CentOS Could SIG repo which only hosts packages for centos 713:53
tdasilvappai: have you asked in the #rdo channel?13:54
acolestdasilva: https://goo.gl/photos/sXkusVLTbxSRZ1Z8913:54
ppaiI'll go do that.13:54
acolestdasilva: hmmm, we didn't discuss that in SAT :o13:54
*** SkyRocknRoll has quit IRC13:55
tdasilvaacoles: my job was easy so far, all the hard work was done in SAT ;) now just need to complete these missings bits13:57
tdasilvaacoles: altough I did change somethings from what you wrote down on the etherpad, specifically line 9813:58
tdasilvaon GET with symlink=true, i'm just returning the symlink, similar to what we do with slo manifest13:59
acolestdasilva: for if-modified-since, won't the symlink timestamp be the same as the target? for auto-tiering at least14:01
tdasilvaacoles: for middleware that creates the symlink (e.g., auto-tiering, versioning) most likely, but for manual policy migration probably not, right?14:02
acolestdasilva: right, can't have the client setting the timestamp14:02
*** abhitechie has quit IRC14:03
pdardeauclayg: thx for movie reference. sadly, i've never seen that movie (hanging head in shame). just got added to my must see list.14:03
tdasilvapdardeau: back to the future?14:04
pdardeautdasilva: yep14:04
tdasilvapdardeau: now I don't feel so bad about "The Blues Brothers" ;)14:04
tdasilvaacoles: ^^^14:05
*** ukaynar has joined #openstack-swift14:05
pdardeautdasilva: more shame for me. i haven't seen that one either. :-(14:05
acoleslol14:06
tdasilvapdardeau: another one for the list then, acoles said it is a must watch14:06
pdardeautdasilva: but (in case it helps my cause at all) i've seen holy grail and life of brian many times (and have a set of handy quotes from those)14:07
acolesBlues Brother has to trump BTTF by a long way14:07
*** admin6 has joined #openstack-swift14:10
*** ppai has quit IRC14:10
*** diogogmt has joined #openstack-swift14:23
jrichliacoles: thanks for patch 348292 and for reviewing.  I will work on your latest in_process feedback this morning14:28
patchbotjrichli: https://review.openstack.org/#/c/348292/ - openstack-infra/project-config - Add swift job tox-func-in-process-encryption14:28
acolesjrichli: np. its nearly there :)14:28
*** ukaynar has quit IRC14:31
*** diogogmt has quit IRC14:32
*** ukaynar has joined #openstack-swift14:34
*** jistr is now known as jistr|call14:35
mmotianiGood Morning14:36
ukaynarHello all, I am trying to integrate swiftfs with hadoop. It seems like the Swiftfs is doing authentication by using a "POST method" is it possible to use "Get method" instead of post? Does the swiftfs support this?14:41
*** asettle has quit IRC14:43
*** jistr|call is now known as jistr14:49
*** asettle has joined #openstack-swift14:51
*** vinsh has joined #openstack-swift15:01
notmynamegood morning15:16
*** openstackgerrit has quit IRC15:18
*** openstackgerrit has joined #openstack-swift15:18
*** pcaruana has quit IRC15:22
*** diogogmt has joined #openstack-swift15:24
jrichliupdated patch 34710115:30
patchbotjrichli: https://review.openstack.org/#/c/347101/ - swift - Enable in-process func tests to optionally use enc...15:30
jrichligerrit has been ignoring me lately :-(15:31
*** jistr is now known as jistr|afk15:42
notmynamejrichli: what does that mean?15:54
jrichlinotmyname: oh, just that gerrit has not posted a couple notifications of my patchsets.  so i have done it manually a couple times.15:55
notmynameah15:56
notmynamecome one openstackgerrit! get yourself sorted!15:56
jrichli:-)15:56
*** silor has quit IRC15:59
notmynameI know this is probably wading into a religious war (that I'd never win)... I've been upgrading an old server this week. As a not-even-part-time sysadmin, I like some of the changes in the last 4 years (the last time I upgraded a server). configuring a firewall and services that automatically run seems less arcane now15:59
claygawwww we missed our chance to answer ma91 about no such thing as tempurl acls?16:00
claygukaynar: swiftfs?  there is an openstack project that does hadoop provisioning -> https://wiki.openstack.org/wiki/Sahara16:04
claygit has some swift support - that is esentially "pull" using GET http method.16:04
*** psachin has quit IRC16:04
acolesjrichli: its not just you, gerrit also shuns me some days16:15
*** nadeem has joined #openstack-swift16:16
jrichliacoles: no rush, but I am curious about your opinions on the recent comments in bug 148779116:20
openstackbug 1487791 in OpenStack Object Storage (swift) "No tests exist for POST to DLO manifest file" [Undecided,In progress] https://launchpad.net/bugs/1487791 - Assigned to Janie Richling (jrichli)16:20
*** adu has joined #openstack-swift16:21
*** dmorita has joined #openstack-swift16:22
*** nadeem has quit IRC16:25
*** sanchitmalhotra1 has joined #openstack-swift16:26
*** nadeem has joined #openstack-swift16:26
*** sanchitmalhotra has quit IRC16:27
*** sanchitmalhotra1 is now known as sanchitmalhotra16:27
*** jistr|afk is now known as jistr16:30
*** tesseract- has quit IRC16:32
*** rledisez has quit IRC16:34
acolesjrichli: ack but will be next week, I'm OOO tomorrow16:34
*** dmorita has quit IRC16:36
*** dmorita has joined #openstack-swift16:38
*** dmorita has quit IRC16:38
*** dmorita has joined #openstack-swift16:38
timburkegood morning16:41
*** hseipp has quit IRC16:41
*** gyee has joined #openstack-swift16:43
*** jordan__ has quit IRC16:44
*** gyee has quit IRC16:48
*** KriskK has joined #openstack-swift16:52
*** asettle has quit IRC16:54
*** KriskK has quit IRC16:54
*** oshritf has quit IRC16:56
*** chsc has joined #openstack-swift16:58
*** McMurlock1 has joined #openstack-swift17:07
*** klrmn has joined #openstack-swift17:08
*** tqtran has joined #openstack-swift17:33
*** tqtran has quit IRC17:38
*** adu has quit IRC17:38
*** ppai has joined #openstack-swift17:40
*** ppai has quit IRC17:42
*** asettle has joined #openstack-swift17:51
jrichliacoles: ok, np.  have a nice Friday off!17:55
*** bsdkurt has joined #openstack-swift17:59
tdasilvaacoles: so, I kinda solved the issue with a different approach, now I check for the symlink sysmeta header on errors18:02
*** Suyi_ has joined #openstack-swift18:02
openstackgerritAlistair Coles proposed openstack/swift: Make container sync copy SLO manifests  https://review.openstack.org/34849518:04
acolestdasilva: neat. I guess that works for If-Modified-Since too?18:04
tdasilvaacoles: it is nice because it works for conditional gets and range requests18:04
bsdkurtI did a massive delete on my R&D lab while two storage nodes were down (didn't notice). Now that the nodes are up they are slowing deleting their copies (3-copy setup). What process does that detecting and deleting?18:05
bsdkurtI'd like to speed that up if possible by giving more workers18:05
*** zaitcev has joined #openstack-swift18:06
*** ChanServ sets mode: +v zaitcev18:06
timburkebsdkurt: object replication18:07
bsdkurttimburke: thank you18:07
acolestdasilva: yep, sounds good.18:07
jrichliacoles: I see that you have proposed that the encryption gate test be non-voting.  Is there a reason for that?18:08
openstackgerritThiago da Silva proposed openstack/swift: Symlink implementation.  https://review.openstack.org/23216218:08
*** asettle has quit IRC18:08
acolesjrichli: its usual for a new jenkins job to be non-voting when it is first added, in case for some reason the job is bad and produces false-negatives. We'd make it voting after some periof of "probation".18:09
jrichliacoles: ok.  i wondered if it was something like that.18:09
jrichlitdasilva:  nice work!  I plan on taking a look at the latest symlinks patch soon.18:10
tdasilvaacoles: one issue that we currently have with symlinks is that we get different behavior for when post_as_copy=true or false. timburke also noticed that issues, but we haven't thought of a solution yet, if you have a chance, it would be great to get your opinion18:10
tdasilvaclayg, jrichli: ^^^ yours too18:11
jrichlipost is always a problem child :-)18:12
notmynameacoles: thanks for proposing that new test18:13
timburkei think there's a reason AWS doesn't support it :P18:14
bsdkurtTo speed up my deletes due to node down, does the following sound reasonable? [object-replicator] run_pause = 1 concurrency = 6018:15
notmynamethere's been several things in swift over the years that we've done differently than s3. and then normally at some point I have a "oh that's why they did it that way" moment where I realize they sidestepped a bunch of problems ;-)18:15
*** siva_krish has joined #openstack-swift18:16
*** siva_krish has left #openstack-swift18:16
notmynamebsdkurt: increasing the concurrency is the right thing. lowering run_pause (isn't it called "interval" now?) will reduce the amount of time between cycles, but eg 30 or 60 seconds there is not really going to matter in the context of a replication run that takes hours18:16
bsdkurtah.. ok thank you John18:17
*** KriskK has joined #openstack-swift18:18
*** tqtran has joined #openstack-swift18:19
claygtorgomatic: ntata: I'm going to rebase patch 347546 onto patch 291959 and update the commits - thanks for all the context!18:22
patchbotclayg: https://review.openstack.org/#/c/347546/ - swift - Moved ipv4 & ipv6 validations to the common utils18:22
patchbotclayg: https://review.openstack.org/#/c/291959/ - swift - Handle IPv6 addresses in swift-get-nodes.18:22
*** geaaru has quit IRC18:33
*** catintheroof has joined #openstack-swift18:34
*** KriskK has quit IRC18:37
*** ChubYann has joined #openstack-swift18:41
acolestdasilva: is is this patch 232162 ?18:42
patchbotacoles: https://review.openstack.org/#/c/232162/ - swift - Symlink implementation.18:42
tdasilvaacoles: correct18:42
acolestdasilva: and is the different post behaviour manifested in the functional test errors?18:44
acolestdasilva: just wondering where to start looking18:44
timburketdasilva: acoles: just dropped an idea for how we could always return the 307 on POST18:45
claygcschwede: do you have any idea if the bad/annoying behavior you're fixing in patch 348173 also exists in the client command `swift tempulr`?18:46
patchbotclayg: https://review.openstack.org/#/c/348173/ - swift - Allow full storage URL as input to swift-temp-url18:46
claygtimburke: always is s scary word18:46
timburkeclayg: yeah! let's 307 all the POSTs! that'll resolve *all* of our fast-post vs. post-as-copy troubles!18:48
claygoic18:48
claygi...18:48
claygdon't know if you're joking18:48
tdasilvaacoles, timburke: for now I changed the functional test for post request to check if post_as_copy is true or false, so that the tests is skipped and it won't fail. you could remove that skip to see it fail18:49
claygfixes *2* bugs -> patch 27272418:50
patchbotclayg: https://review.openstack.org/#/c/272724/ - swift - Copy headers correctly when copying object18:50
clayg... i'm going to end up looking at it aren't I :'(18:50
tdasilvaclayg, i'm on it right now actually18:50
tdasilvawanted to take another look at it before I +A18:51
* clayg hugs tdasilva 18:51
claygtdasilva: well *yeah* - you're the man18:51
tdasilvalol18:51
timburkeclayg: i keep meaning to do it...but then i get distracted at the possibility of *writing* code instead of reviewing it18:52
tdasilvatimburke: heh18:52
claygtimburke: ain't not body got time for that18:54
tdasilvaacoles: so yeah, basically the issue is that with post_as_copy, symlink middleware never sees a post request, so it doesn't 30718:55
tdasilvaacoles: If I remember correctly, the copy middleware does add a 'post_as_copy' flag to the env, so maybe I could add a check of that when handling a PUT18:56
zaitcevguys, I have a quick question if anyone remembers: is there a standard header for the an HTTP proxy to pass towards back end, which contains original Host: and scheme? Otherwise back-end can't form Location: properly for 3xx redirects.18:56
claygzaitcev: does the backend currently ever do 3xx?18:56
zaitcevclayg: staticweb does18:56
claygzaitcev: but that's int he proxy?18:57
zaitcevclayg: That's why I said "HTTP Proxy" not Swift proxy.18:57
claygzaitcev: I think the proxy has some support to get something about the original request from the load balanacer18:57
claygzaitcev: oh I think I confused "towards the back end" - I should have read it as "HTTP proxy to pass towards the *swift proxy*"18:58
claygand yeah, I think there's something - but I don't remember where :\18:58
claygzaitcev: X-Forwarded-For is in proxy logging - something like that sounds right?18:59
zaitcevclayg: To put is simply, Tempest fails two tests, test_web_index and test_web_listing_css, if it runs against Swift that has SSL-terminating LB in front, like HAproxy.18:59
notmynamesounds like it might be a bug in swift? doesn't the format "///:some/url/path" mean "use the current scheme"?19:02
zaitcevhttps://bugs.launchpad.net/mos/+bug/153707119:02
openstackLaunchpad bug 1537071 in Mirantis OpenStack 9.x "[Tempest] [SSL] 2 tests for object storage failed with ssl configuration" [High,Confirmed] - Assigned to Kostiantyn Danylov (kdanylov)19:02
notmynameor like with tempauth we let the config set the actual url that comes back19:02
*** dmorita has quit IRC19:02
notmynameso you can set https even if the ssl is being terminated externally19:02
*** fifieldt has quit IRC19:02
claygok, seems like a reasonable thing to fix - but lots of people run swift behind a lb - so it's probably something in the haproxy config that tells it to forward the right values in the right headers where swift will look for them IIRC19:02
zaitcevnotmyname: swob returns Location: with netloc. I verified with tcpdump19:02
notmynameisnt' the problem that the proxy isn't configured for https so it only returns the http netloc?19:03
notmynameotherwise it would Just Work, right?19:03
zaitcevI'm hoping for something like X-Forwarded-For, here. Seems like a cleaner solution.19:03
claygzaitcev: sometimes you can configure http proxies to do location re-writes19:03
acolestdasilva: yes there's a swift.post-as-copy flag in the environ if that help19:04
claygzaitcev: well dig around proxy-logging and see what it's doing with the header - if you can't make that work either fix the proxy to do location rewrite or file a bug and fix it?19:04
openstackgerritRebecca Finn proposed openstack/swift: Check object metadata constraints after authorizing  https://review.openstack.org/34229519:05
zaitcevclayg: oh yeah, that's another thing. proxy-logging does not log anything in this case, although it's properly inserted twice in the pipeline. I ended debugging it by inserting q and running tcpdump. I need to look into this too, everyone tends to trust proxy-logging, but it lies.19:05
zaitcevI only ever noticed it bacause I captured tx-id, and then noticed that a HEAD to Swift container-server is loggerd with this tx-id, but nothing from proxy.19:06
notmynamezaitcev: IMO I think the bug report comment has the right answer: staticweb should be configured to send back the right scheme (TLS or not)19:06
acolestdasilva: timburke I'll try to catch with you on the POST to symlink thing next week, out of time now, sorry.19:07
*** KriskK has joined #openstack-swift19:07
* acoles is ooo til Monday19:07
zaitcevnotmyname: well yea, that's one way to do it, like tempauth.19:07
tdasilvaacoles: no worries! enjoy your long weekend! :-)19:07
timburkeno worries. have a good weekend acoles!19:07
acolesthanks guys! hoping the surf is up :)19:07
notmynamezaitcev: yeah. I'm just voting for that option19:07
notmynamealso, I'm going to lunch now19:08
notmynameacoles: have fun!19:08
*** acoles is now known as acoles_19:08
timburkezaitcev: looks like mod_proxy uses X-Forwarded-Host19:08
openstackgerritClay Gerrard proposed openstack/swift: Moved ipv4 & ipv6 validations to the common utils  https://review.openstack.org/34754619:09
openstackgerritClay Gerrard proposed openstack/swift: Handle IPv6 addresses in swift-get-nodes.  https://review.openstack.org/29195919:09
claygdamnit gerrit19:10
clayglb tend to be configurable in this regard if they're expecting the underlying app to do the right then they need some flexibility because there's a lot of apps that might be down there...19:10
*** KriskK has quit IRC19:11
clayg... but good ones also have ability to do location rewrite -> https://www.haproxy.com/doc/aloha/7.0/haproxy/http_rewriting.html#rewrite-anywhere-in-the-response-using-regexes19:11
claygi think it'd be *fine* to improve swift too - if it makes it easier to configure for different popular lb solutions...19:11
zaitcevIn our case HAproxy is deployed by Director, so the rewrite clause needs to get added by Puppet. Not the fun Thursday I hoped for.19:13
*** fifieldt has joined #openstack-swift19:18
openstackgerritJanie Richling proposed openstack/swift: Enable in-process func tests to optionally use encryption  https://review.openstack.org/34710119:19
*** fifieldt has quit IRC19:27
*** dmorita has joined #openstack-swift19:27
*** dmorita has quit IRC19:32
*** cdelatte has joined #openstack-swift19:34
*** fifieldt has joined #openstack-swift19:37
*** mvk has quit IRC19:40
*** vinsh has quit IRC19:51
*** asettle has joined #openstack-swift19:58
*** asettle has quit IRC19:58
*** adu has joined #openstack-swift20:14
*** flwang1 has quit IRC20:18
openstackgerritClay Gerrard proposed openstack/swift: Allow full storage URL as input to swift-temp-url  https://review.openstack.org/34817320:18
*** gyee has joined #openstack-swift20:21
*** ukaynar has quit IRC20:25
*** greghaynes has quit IRC20:25
*** cdelatte has quit IRC20:26
openstackgerritClay Gerrard proposed openstack/swift: Always reset epoch when reseting last_part_moves  https://review.openstack.org/34697920:33
*** greghaynes has joined #openstack-swift20:37
*** oshritf has joined #openstack-swift20:42
openstackgerritTim Burke proposed openstack/swift: Store SLO Etag and swift_bytes in sysmeta  https://review.openstack.org/34753820:45
*** jamielennox|away is now known as jamielennox20:57
claygwith dispersion is bigger gooder or badder?21:02
claygis mohit not in chan or did I forget his handle?21:10
claygredbo: torgomatic: have either you you guys thought much about patch 34686521:11
patchbotclayg: https://review.openstack.org/#/c/346865/ - swift - Delete old tombstones21:11
claygzaitcev: I know you've tought about change some21:11
clayg... who else is a replication guru?21:11
zaitcevnot me but I was going to look at this21:11
zaitcevSince I'm on core I cannot shrink from reviewing too much, so no matter what...21:12
zaitcevKota probably knows that area.21:12
claygtorgomatic: also for your list interesting ring change -> patch 346475 (cc swifterdarrell RE: fixing min_part_hours 0)21:12
patchbotclayg: https://review.openstack.org/#/c/346475/ - swift - For any part, only one replica can move in a rebal...21:12
claygnotmyname: how's the review assignment tool going?21:13
claygisn't there a way to assign reviewers to bits?21:13
clayghrmm... i see the "Add..." button next to Reviewers - would it be passive agressive to just assign changes I like to people :)21:15
*** ukaynar has joined #openstack-swift21:28
*** NM has quit IRC21:35
claygzaitcev: i don't follow your comment on patch 291959 - you're just questioning my example script that uses the old add syntax for rings to demonstrate the fix for swift-get-nodes???21:46
patchbotclayg: https://review.openstack.org/#/c/291959/ - swift - Handle IPv6 addresses in swift-get-nodes.21:46
*** adu has quit IRC21:46
claygFWIW if the swift-ring-builder add parser doesn't respect []'s around ipv6 ips that's probably a seperate bug - ntata is fixing the swift-ring-builder output formatting in patch 31938721:48
patchbotclayg: https://review.openstack.org/#/c/319387/ - swift - swift-ring-builder output corrected for ipv621:48
clayg... but I think that's orthogonal to torgomatic's old swift-get-nodes fix - you added 2nd +2 but not +A maybe?21:49
zaitcevclayg: I'm not questioning as much as pointing out that it's probably unnecessary. There should be something like --ip that takes v4 and v6 addresses21:50
claygzaitcev: to *curl* !?  are we talking about patch 291959 or not?21:50
patchbotclayg: https://review.openstack.org/#/c/291959/ - swift - Handle IPv6 addresses in swift-get-nodes.21:50
zaitcevclayg: you have example swift-ring-builder ipv6test.builder add r1z3-1:1:1:1:1:1:1:1:6030/sdb3 121:51
zaitcevwhich is not curl is it now21:51
*** flwang1 has joined #openstack-swift21:51
zaitcevAnyway, the code looked good to me, so +2. I should've not left any confusing comments.21:52
claygah, yeah my test script - i mean i was just making a ring with ipv6 addresses to show the output of swift-get-nodes with ipv6 - specifically the curl commands which get generated21:53
clayg... they're wrong for ipv6 currently, sam's patch makes them the fixed, my example script to make a ring (however poorly executed) demonstrates the sam's fix to swift-get-nodes is correct IMHO21:53
claygno your comment was probably founded - just it was to *me* not to sam's patch - that could have been more clear21:54
claygsorry, IMHO ;)21:54
*** dmorita has joined #openstack-swift21:57
notmynameclayg: workflow question for you. with vSAIO, how do you deal with moving something non-swift to your SAIO to test? eg 3rd party middleware or just your own code/scripts22:03
*** oshritf has quit IRC22:04
timburkezaitcev: looks like mod_proxy uses X-Forwarded-Host22:04
timburkebah, sorry zaitcev -- wrong window22:04
pdardeauclayg: mohit nick is mmotiani22:04
pdardeauor mmotiani_22:05
claygnotmyname: I've made an entry in my global git ignore for /.scratch/ - then I just drop that folder in the root of most crap I hack on - with vboxsf /vagrant/.scratch is where it all goes down when I need it on the node22:08
*** vinsh has joined #openstack-swift22:08
notmynamepdardeau: basically, start typing "m"s and then hit <Tab> when you get tired22:08
notmynameclayg: ah, ok. so cp to .scratch on your host, then *magic* and it's on your vSAIO22:08
claygmmotiani: hey I wanted to warn you that cschwede is already working on fixin swift-temp-url (the script i now hate with the passion of a thousand burning suns!)22:08
claygnotmyname: yup22:09
notmynameclayg: that turns into a subdir of the swift code then. right?22:09
claygnotmyname: you'll need to have *something* in your global gitignore to make it so you can have stuff in your git dir that isn't messed with tho22:09
claygnotmyname: nah, it's *just* in /vagrant/.scratch - totally outside of /vagrant/swift or ~/swift - so for a middleware you may still have to do something to drop it somewhere importable - same as you might do for any other sort of middleware kinda thing22:10
*** asettle has joined #openstack-swift22:10
claygnotmyname: it's possible I don't understand the full scope of what you're trying to do22:10
notmynamenah, I think you've got it22:10
openstackgerritMerged openstack/swift: Copy headers correctly when copying object  https://review.openstack.org/27272422:11
notmynamebut I thought /vagrant/ on the guest is the actual swift code. ie .../vsaio/swift on the host22:11
notmynametherefore /vagrant/.scratch would have to be .../vsaio/swift/.scratch22:11
notmynameclayg: you filed a bug earlier today. aren't we going to take swift-temp-url out behind the woodshed and end our misery?22:11
claygI think my .scratch folder (which again is a convention i use on *all* the projects I hack on) for my vsaio acctually has a few other projects checked out in it dfg's slowdown, and ricks trace thing - I think just go to /vagrant/.scratch/<project> dir and `pip install -e .` them so I can refernce them by egg in my `proxy-server.conf.d/base.conf-template` and the pipelines...22:12
*** dmorita has quit IRC22:12
claygnotmyname: HELL FUCKING YES THAT'S THE TICKET22:12
claygnotmyname: no /vagrant on the guest is the root of the project /vagrant/swift is swift code (?)22:12
claygpretty sure...22:12
claygyes def22:12
*** mmcardle has joined #openstack-swift22:13
*** dmorita has joined #openstack-swift22:13
timburkenotmyname: fwiw, i made a branch to include swift3, might be a useful reference if you wanted to have your stuff automatically added to pipelines, etc. -- relevant commit is https://github.com/swiftstack/vagrant-swift-all-in-one/commit/c047b4f22:13
*** dmorita has quit IRC22:14
*** dmorita_ has joined #openstack-swift22:14
claygtimburke: i love me some swift3!!!! let's just merge that down onto the master!?22:14
timburkeoh, hey, that must predate the stackforge rename!22:16
claygheh :)22:16
notmynametimburke: what?! you givn' me bad configs?22:16
claygwell then let's *not* merge it down to master!22:16
*** asettle has quit IRC22:17
timburkenah, it redirects fine. just mildly interesting22:17
claygoic22:17
*** mmcardle has quit IRC22:17
mmotianiclayg, Got it. I will unassign that bug to me.22:26
*** McMurlock1 has quit IRC22:27
claygmmotiani: probably a good call - it's cschwede's mess now ;)22:29
* clayg hugs cschwede 22:29
mmotianiclayg :-)22:29
*** chsc has quit IRC22:36
*** MadOtis has joined #openstack-swift22:39
MadOtisHello all... I'm back...  I had the issue, yesterday, with the swift authentication not coming back correctly.  I rebuilt the entire environment with Ubuntu, and I'm seeing the exact same issue... but, I have a little more detail this time...22:40
notmynameyay details22:41
*** nadeem has quit IRC22:41
MadOtisSorry, I was waiting for paste.openstack.org to open; but its not responding, so I'm reverting to pastebin...22:42
MadOtisHere is what is getting dumped out to syslog on the controller: http://pastebin.com/s4rSFfU722:43
MadOtisI wasn't seeing this detail in Centos, but it's plainly in the syslog in Ubuntu.22:43
claygMadOtis: you're loosing some important bits from syslog...22:43
clayg$MaxMessageSize 20k22:44
claygor something like that maybe?22:44
claygwhat's a "DiscoveryFailure"22:45
claygkeystone - you're werid22:45
MadOtisclayg: in the paste, or I missed some details from syslog?22:45
claygMadOtis: I think your syslog configuration is truncating messages22:46
claygthose tracebacks can be longish22:46
MadOtisOh, that's possible... it was a bare-bones 14.0422:46
claygyeah so.... $MaxMessageSize 20k or something like that22:46
*** ukaynar has quit IRC22:47
MadOtisok, let me bump it up and re-test22:47
clayg... but there was a few gems in those keystone tracebacks - "DiscoveryFailure" and the like - i just don't know what any of them mean22:47
claygi feel like on my vsaio I get properly formatted tracebacks in the logs - I like the #012 behavior for grepping production logs - but I think the message size limit is a thing to consider...22:48
clayg... but I don't know what triggers the behavior?  is it a toggle?  is it when loglevel > DEBUG or something?22:48
claygdoes anyone know what I'm talking about with the #012?  I know ahale and briancline get it22:49
clayghey i'm out - talk to you cats later22:51
mattoliverauMorning22:51
clarkbkeystone does version discovery by default now I think. Basically the client server negotiate whether or not to use v2 or v322:53
MadOtisIs this one better?  http://pastebin.com/1PRD7Gv222:53
clarkblike you need to set a specific version or update your client if that is failing22:53
notmynameMadOtis: try adding "--auth-version 3" to the `swift` command you're running (based on what clarkb said)22:57
MadOtisok, trying...22:57
MadOtisSame results22:58
*** klrmn has quit IRC23:00
notmynamehmm..IIRC from yesterday, you had an envvar already setting that23:00
*** klrmn has joined #openstack-swift23:01
MadOtiscorrect, the openrc files (one for admin, and one for demo)23:04
*** mvk has joined #openstack-swift23:07
*** dmorita_ has quit IRC23:10
*** dmorita has joined #openstack-swift23:11
notmynameMadOtis: ok, yeah. you've got more info today. let's see what we can figure otu23:16
MadOtiscool23:17
notmynameclayg is right. the interesting issue is the DiscoveryFailure23:17
MadOtisIn the interim, I'm installing ceilometer while waiting for direction.23:17
notmynameoh.23:17
notmynameum23:18
notmynameyeah, sure. I'm sure you'll probably be fine23:18
notmynamegood luck23:18
notmynameMadOtis: can you give me your pipeline again? I don't have the link23:20
MadOtisSure, one sec...23:20
MadOtispipeline = catch_errors gatekeeper healthcheck proxy-logging cache container_sync bulk tempurl ratelimit authtoken keystoneauth container-quotas account-quotas slo dlo versioned_writes proxy-logging proxy-server23:20
notmynameok23:21
notmynamewhat version of keystonemiddleware do you have installed?23:22
openstackgerritMatthew Oliver proposed openstack/swift: Add end_marker and reverse options to direct_client  https://review.openstack.org/32622623:24
notmynameMadOtis: while being technically accurate yet completely unhelpful, so far what I can tell is that keystone is trying to do some sort of discovery, getting an error, and then raising an exception that is not caught23:26
notmynameI don't have any idea why keystone is doing the version discovery or how to make it work23:26
MadOtisI hate it when that happens!23:26
notmynamedid you find the keystonemiddleware veruson?23:27
MadOtisOh, sorry, I missed that request... let me check23:27
notmynameMadOtis: new plan. is it possible to switch to auth version 2?23:29
MadOtisI can try23:30
MadOtisOS_IDENTITY_API_VERSION?23:31
notmynamethat seems likely23:32
MadOtisAnd... python-keystonemiddleware is v:4.4.0-3~cloud023:33
MadOtisroot@controller:~# export OS_IDENTITY_API_VERSION=2 root@controller:~# swift stat Authorization Failure. Authorization Failed: The resource could not be found. (HTTP 404) (Request-ID: req-91c66bbc-6293-4c3a-811e-2a72a054336a)23:33
*** kei_yama has joined #openstack-swift23:36
notmynamewell that's better23:37
notmynamemight need to comment out (or unset) the domain setting23:37
notmynamewhere better == not a traceback23:38
*** _JZ_ has quit IRC23:42
MadOtiswhere?  In the proxy-server.conf for keystone authtoken?23:43
notmynameno, it's one of the envvars keystone has for v323:43
MadOtisAh!  OS_PROJECT_DOMAIN_NAME23:44
notmynameyeah23:45
notmynamealso maybe OS_USER_DOMAIN_NAME if it's there23:45
MadOtisroot@controller:~# swift stat Unauthorized. Check username/id, password, tenant name/id and user/tenant domain name/id.23:45
notmynamehmmm23:45
MadOtisroot@controller:~# unset OS_USER_DOMAIN_NAME root@controller:~# swift stat Unauthorized. Check username/id, password, tenant name/id and user/tenant domain name/id.23:45
MadOtiseating dinner... brb23:52
*** itlinux has joined #openstack-swift23:55
*** klamath has quit IRC23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!