| *** rpedde_ has joined #openstack-swift | 00:02 | |
| *** Alex_Gaynor_ has joined #openstack-swift | 00:02 | |
| *** matt____ has joined #openstack-swift | 00:02 | |
| *** StevenK_ has joined #openstack-swift | 00:03 | |
| *** pandemicsyn2 has joined #openstack-swift | 00:04 | |
| *** mhu1 has joined #openstack-swift | 00:06 | |
| *** jdag_____ has joined #openstack-swift | 00:07 | |
| *** matt____ has quit IRC | 00:08 | |
| *** vu has joined #openstack-swift | 00:08 | |
| *** bgmccollum_ has joined #openstack-swift | 00:09 | |
| *** zacksh_ has joined #openstack-swift | 00:09 | |
| *** charz has joined #openstack-swift | 00:09 | |
| *** jroll|dupe has joined #openstack-swift | 00:09 | |
| *** otherjon_ has joined #openstack-swift | 00:12 | |
| *** EmilienM_ has joined #openstack-swift | 00:13 | |
| *** matt____ has joined #openstack-swift | 00:13 | |
| *** charz_ has quit IRC | 00:13 | |
| *** otherjon has quit IRC | 00:13 | |
| *** jdag____ has quit IRC | 00:13 | |
| *** Alex_Gaynor has quit IRC | 00:13 | |
| *** sileht has quit IRC | 00:13 | |
| *** rpedde has quit IRC | 00:13 | |
| *** jroll has quit IRC | 00:13 | |
| *** pandemicsyn has quit IRC | 00:13 | |
| *** mhu has quit IRC | 00:13 | |
| *** bgmccollum has quit IRC | 00:13 | |
| *** EmilienM has quit IRC | 00:13 | |
| *** StevenK has quit IRC | 00:13 | |
| *** zacksh has quit IRC | 00:13 | |
| *** mattoliverau has quit IRC | 00:13 | |
| *** mhu1 is now known as mhu | 00:13 | |
| *** rpedde_ is now known as rpedde | 00:13 | |
| *** jroll|dupe is now known as jroll | 00:13 | |
| *** EmilienM_ is now known as EmilienM | 00:13 | |
| *** matt____ is now known as mattoliverau | 00:13 | |
| *** otherjon_ is now known as otherjon | 00:13 | |
| *** sileht has joined #openstack-swift | 00:14 | |
| *** Alex_Gaynor_ is now known as Alex_Gaynor | 00:14 | |
| *** jdag_____ is now known as jdag____ | 00:14 | |
| *** dmsimard is now known as dmsimard_away | 00:19 | |
| *** bnelson has quit IRC | 00:32 | |
| *** bnelson has joined #openstack-swift | 00:37 | |
| *** HenryG has quit IRC | 00:37 | |
| *** chuck_ has joined #openstack-swift | 00:39 | |
| *** addnull has joined #openstack-swift | 01:04 | |
| *** mitz_ has joined #openstack-swift | 01:05 | |
| *** vu has quit IRC | 01:19 | |
| *** mrsnivvel has joined #openstack-swift | 01:53 | |
| *** haomaiwang has joined #openstack-swift | 02:10 | |
| *** IgnacioCorderi has joined #openstack-swift | 02:33 | |
| *** IgnacioCorderi has quit IRC | 02:38 | |
| hugokuo | Morning | 03:15 |
|---|---|---|
| mattoliverau | Morning hugokuo | 03:20 |
| hugokuo | how are you today ? | 03:20 |
| *** haomaiwang has quit IRC | 03:31 | |
| *** haomaiwang has joined #openstack-swift | 03:32 | |
| *** haomaiw__ has joined #openstack-swift | 03:38 | |
| *** haomaiwang has quit IRC | 03:42 | |
| mattoliverau | hugokuo: I'm good, time to another week :) You? (Sorry for late reply I was at lunch) | 03:45 |
| *** evanjfraser has joined #openstack-swift | 04:12 | |
| *** addnull has quit IRC | 04:31 | |
| *** bkopilov has quit IRC | 04:54 | |
| *** bkopilov has joined #openstack-swift | 04:55 | |
| *** kopparam has joined #openstack-swift | 04:56 | |
| *** ppai has joined #openstack-swift | 04:59 | |
| *** kopparam has quit IRC | 04:59 | |
| *** kopparam has joined #openstack-swift | 05:02 | |
| *** aix has quit IRC | 05:02 | |
| *** addnull has joined #openstack-swift | 05:14 | |
| *** byeagerz has quit IRC | 05:17 | |
| *** addnull has quit IRC | 05:18 | |
| *** byeager_away has joined #openstack-swift | 05:19 | |
| *** bkopilov has quit IRC | 05:21 | |
| *** addnull has joined #openstack-swift | 05:23 | |
| *** IgnacioCorderi has joined #openstack-swift | 05:43 | |
| *** occupant has quit IRC | 05:55 | |
| *** occupant has joined #openstack-swift | 05:55 | |
| *** k4n0 has joined #openstack-swift | 05:57 | |
| *** bkopilov has joined #openstack-swift | 06:20 | |
| *** bkopilov has quit IRC | 06:29 | |
| *** bkopilov has joined #openstack-swift | 06:37 | |
| openstackgerrit | Christian Schwede proposed a change to openstack/swift: Limit partition movement when rebalancing https://review.openstack.org/121422 | 06:46 |
| *** kevinbenton has joined #openstack-swift | 06:50 | |
| *** kopparam has quit IRC | 07:06 | |
| kevinbenton | hi, is someone online that can help me understand one of the swift functional tests? | 07:11 |
| *** haomaiw__ has quit IRC | 07:13 | |
| *** haomaiwang has joined #openstack-swift | 07:13 | |
| *** IgnacioCorderi has quit IRC | 07:15 | |
| *** bnelson has quit IRC | 07:27 | |
| *** bnelson has joined #openstack-swift | 07:28 | |
| *** haomaiw__ has joined #openstack-swift | 07:29 | |
| *** haomaiwang has quit IRC | 07:32 | |
| *** homegrown1 has left #openstack-swift | 07:38 | |
| *** kopparam has joined #openstack-swift | 07:38 | |
| openstackgerrit | Christian Schwede proposed a change to openstack/swift: Limit partition movement when rebalancing https://review.openstack.org/121422 | 07:52 |
| *** nshaikh has joined #openstack-swift | 07:59 | |
| *** jdag____ has quit IRC | 08:04 | |
| *** jdag____ has joined #openstack-swift | 08:04 | |
| *** sileht has quit IRC | 08:05 | |
| *** ppai has quit IRC | 08:05 | |
| *** ppai has joined #openstack-swift | 08:05 | |
| *** k4n0 has quit IRC | 08:05 | |
| *** k4n0 has joined #openstack-swift | 08:05 | |
| *** sileht has joined #openstack-swift | 08:18 | |
| *** nshaikh has quit IRC | 08:47 | |
| *** k4n0 has quit IRC | 08:48 | |
| *** ChanServ sets mode: +v chmouel_ | 08:58 | |
| *** chmouel_ is now known as chmouel | 08:59 | |
| *** mkollaro has joined #openstack-swift | 09:02 | |
| *** k4n0 has joined #openstack-swift | 09:07 | |
| *** cschwede has joined #openstack-swift | 09:10 | |
| *** sileht has quit IRC | 09:19 | |
| *** aix has joined #openstack-swift | 09:19 | |
| *** kopparam has quit IRC | 09:20 | |
| *** kopparam has joined #openstack-swift | 09:21 | |
| *** kopparam has quit IRC | 09:26 | |
| *** kopparam has joined #openstack-swift | 09:26 | |
| openstackgerrit | Alistair Coles proposed a change to openstack/swift: Update documentation for using keystone auth https://review.openstack.org/121481 | 09:27 |
| *** kopparam has quit IRC | 09:31 | |
| *** sileht has joined #openstack-swift | 09:33 | |
| *** kopparam has joined #openstack-swift | 09:39 | |
| *** mkollaro has quit IRC | 09:59 | |
| *** kopparam has quit IRC | 10:06 | |
| *** Trixboxer has joined #openstack-swift | 10:06 | |
| Trixboxer | Hi, how can we check the running swift version on Ubuntu ? | 10:08 |
| *** kopparam has joined #openstack-swift | 10:12 | |
| cschwede | Trixboxer: recent Swift versions expose the release when requesting /info. for example: curl http://swift_proxy_ip/info returns "version": "2.1.0…..". if you used the regular ubuntu packages, you could also check with dpkg -l "swift-*" | grep "^ii" | 10:14 |
| Trixboxer | yeah I found the dpkg -l and the version is quite old 1.4.8-0ubuntu2 | 10:15 |
| Trixboxer | part of swift-core I guess | 10:15 |
| Trixboxer | thanks cschwede :) | 10:16 |
| Trixboxer | are you the same guy after swiftbrowser ? | 10:16 |
| cschwede | Trixboxer: yes :) | 10:17 |
| *** kopparam_ has joined #openstack-swift | 10:17 | |
| cschwede | Trixboxer: if you have questions on swiftbrowser, let me know | 10:18 |
| *** koppara__ has joined #openstack-swift | 10:18 | |
| *** kopparam has quit IRC | 10:21 | |
| *** kopparam_ has quit IRC | 10:22 | |
| smart_developer | cschwede : Running the curl command you suggested (curl http://swift_proxy_ip/info) with the Swift proxy's ip seems to return "curl: (7) couldn't connect to host" | 10:27 |
| *** cschwede has quit IRC | 10:27 | |
| smart_developer | Does anyone know how to obtain /info ? | 10:27 |
| smart_developer | I'm using Swift version 1.13. | 10:27 |
| openstackgerrit | Alistair Coles proposed a change to openstack/swift: Fix internal link to keystoneauth in documentation https://review.openstack.org/121423 | 10:29 |
| openstackgerrit | Alistair Coles proposed a change to openstack/swift: Update documentation for using keystone auth https://review.openstack.org/121481 | 10:29 |
| acoles | cschwede: ^^ https://review.openstack.org/121423 depends on your fix to the link in logs.rst but I needed to rebase it | 10:31 |
| *** Krast has joined #openstack-swift | 10:34 | |
| *** Krast has quit IRC | 10:35 | |
| *** HenryG has joined #openstack-swift | 10:37 | |
| *** k4n0 has quit IRC | 10:42 | |
| *** mrsnivvel has quit IRC | 10:47 | |
| acoles | smart_developer: did you include port number e.g. curl http://swift_proxy_ip:8080/info | 10:50 |
| *** tdasilva has joined #openstack-swift | 10:57 | |
| *** k4n0 has joined #openstack-swift | 11:03 | |
| smart_developer | Yes... still not working. Does it matter whether swift_proxy_ip is a public IP address, private IP address, or localhost? | 11:04 |
| smart_developer | acoles : That curl command still seems to return "curl: (7) couldn't connect to host" ........ | 11:06 |
| *** tdasilva has quit IRC | 11:06 | |
| Trixboxer | cschwede: good :) It worked nice, initially I faced some issues myproj/urls.py but then its fine.. I'm testing it against 1.4.8-0ubuntu2 & will let you know if any issues | 11:11 |
| *** cschwede has joined #openstack-swift | 11:13 | |
| acoles | smart_developer: can you successfuly make any other curl commands to your swift proxy? Is it just /info failing? | 11:14 |
| cschwede | smart_developer: maybe you need to set the port as well? for example curl http://swift_proxy_ip/info ? | 11:14 |
| *** dmsimard_away is now known as dmsimard | 11:16 | |
| *** koppara__ has quit IRC | 11:16 | |
| *** kopparam has joined #openstack-swift | 11:16 | |
| smart_developer | what's a default curl command that I could try? | 11:21 |
| smart_developer | Hmm, when I tried using the swift proxy's private IP in the curl command for /info, now it's responding "Authentication required". | 11:24 |
| smart_developer | I have Keystone setup ... so maybe it's referring to the Keystone authentication? | 11:25 |
| acoles | smart_developer: keystone shouldn't make a difference if you curl swift_proxy_ip (NOT keystone ip) | 11:31 |
| acoles | do you have info enabled in /etc/swift/proxy-server.conf (it is enabled by default)? | 11:32 |
| smart_developer | That's right I'm curl-ing the swift_proxy_private_ip_address. | 11:32 |
| cschwede | acoles: thanks for rebasing your patch on mine - saw your patch too late, otherwise i would have abandoned mine | 11:32 |
| smart_developer | How do you enable /info ? | 11:33 |
| smart_developer | when inside proxy-server.conf | 11:33 |
| acoles | It should be enabled by default, see https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L10 | 11:33 |
| smart_developer | It's not there inside my proxy-server.conf | 11:33 |
| smart_developer | The line that you pointed out | 11:34 |
| smart_developer | expose_info = true | 11:34 |
| acoles | smart_developer: ok, well as long as you don't have expose_info = false then it should be working | 11:35 |
| * acoles away for a bit | 11:35 | |
| smart_developer | Thanks. So I added in "expose_info = true" into my proxy-server.conf file, restarted all the services (swift-init all restart), and the curl command "curl http://<my-swift-proxy's-actual-private-ip-here>:8080/info" is still returning "Authentication required" ..... | 11:37 |
| smart_developer | Not sure why this is happening. | 11:38 |
| *** nshaikh has joined #openstack-swift | 11:43 | |
| acoles | smart_developer: i tried to recreate what you are seeing. I removed this line from the [authtoken] section of my proxy-server.conf: | 11:58 |
| acoles | delay_auth_decision = true | 11:58 |
| acoles | and I then get Authentication Required response | 11:58 |
| acoles | smart_developer: so maybe check you have delay_auth_decision = true | 11:58 |
| cschwede | the other possibility (not checked yet) might be to use "swift capabilities", and use the correct keystone credentials. just ruined my devstack, can't check right now :/ | 11:59 |
| acoles | smart_developer: see note in doc for unvalidated requests here http://docs.openstack.org/developer/swift/overview_auth.html#configuring-swift-to-use-keystone | 12:00 |
| * acoles makes not to improve that doc | 12:00 | |
| acoles | note! | 12:01 |
| *** addnull has quit IRC | 12:04 | |
| smart_developer | So, it does seem related to Keystone authentication, then, right? | 12:06 |
| smart_developer | cschwede : How do you get to "swift capabilities" (from your suggestion)? Thanks. | 12:07 |
| cschwede | smart_developer: using the "swift" command itself (from python-swiftclient) | 12:07 |
| smart_developer | Ok, thank you! | 12:11 |
| *** marcusvrn has joined #openstack-swift | 12:11 | |
| smart_developer | By the way acoles, does it matter where within the [authtoken] section of proxy-server.conf one puts the line "delay_auth_decision = true" ? | 12:12 |
| smart_developer | (for instance, at the end of the [authtoken] section? somewhere in the middle? after a particular line / another setting value ? | 12:12 |
| acoles | smart_developer: no (it should not). just put it on a line of its own. example here https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L277 | 12:14 |
| acoles | smart_developer: btw i am going to propose a change to the doc to mention this here http://docs.openstack.org/developer/swift/overview_auth.html#configuring-swift-to-use-keystone | 12:15 |
| acoles | smart_developer: also, the example shows "delay_auth_decision = 1", I have "delay_auth_decision = true", both work | 12:17 |
| openstackgerrit | Alistair Coles proposed a change to openstack/swift: Update documentation for using keystone auth https://review.openstack.org/121481 | 12:19 |
| *** kopparam has quit IRC | 12:19 | |
| *** geaaru has joined #openstack-swift | 12:23 | |
| cschwede | i just checked, "swift capabilities" works for me on a fresh devstack (swift+keystone) | 12:34 |
| *** kopparam has joined #openstack-swift | 12:35 | |
| *** kopparam has quit IRC | 12:39 | |
| *** vr2 has joined #openstack-swift | 12:40 | |
| *** kopparam has joined #openstack-swift | 12:50 | |
| *** kopparam has quit IRC | 12:55 | |
| smart_developer | All right, thank you acoles and cschwede. | 13:02 |
| cschwede | smart_developer: you're welcome! | 13:12 |
| *** nshaikh has left #openstack-swift | 13:12 | |
| *** k4n0 has quit IRC | 13:13 | |
| smart_developer | Here's the result of /info | 13:14 |
| smart_developer | https://gist.github.com/anonymous/d9ff688c4e65e2e70a6e | 13:14 |
| smart_developer | Any explanation as to why the JSON portions for account_quotas, ratelimit, and container_quotas are empty? | 13:14 |
| smart_developer | while the other portions seem descriptive. | 13:16 |
| openstackgerrit | Gerry Drudy proposed a change to openstack/swift: direct_client not passing args between some functions https://review.openstack.org/121535 | 13:19 |
| acoles | smart_developer: according to the changelog the ratelimit parameters only got added to the info json in swift 2.0.0 | 13:23 |
| openstackgerrit | Gerry Drudy proposed a change to openstack/swift: direct_client not passing args between some functions https://review.openstack.org/121535 | 13:23 |
| acoles | the *_quotas are correct - there is no more info reported for those other than that they are configured | 13:24 |
| acoles | smart_developer: ^^ | 13:24 |
| *** mahatic has joined #openstack-swift | 13:28 | |
| smart_developer | hmmm | 13:33 |
| smart_developer | but shouldn't they be reporting what the particular quota is...? | 13:33 |
| smart_developer | (e.g., the quota value). | 13:34 |
| smart_developer | in whatever units that may be involved. | 13:34 |
| acoles | smart_developer: i'm not too familiar with those bits of the code, maybe pose your question here again later when the US is awake | 13:35 |
| smart_developer | all right, thanks! | 13:37 |
| cschwede | smart_developer: no, the quota is unique for each account and container, thus it is not included in /info | 13:38 |
| *** tdasilva has joined #openstack-swift | 13:38 | |
| cschwede | smart_developer: if you want to know the quota of an account or container, simply do a HEAD request against the account/container - the metadata will contain the quota (if set) | 13:38 |
| *** chuck_ has quit IRC | 13:41 | |
| *** r-daneel__ has joined #openstack-swift | 13:48 | |
| *** kopparam has joined #openstack-swift | 13:51 | |
| *** kopparam has quit IRC | 13:56 | |
| smart_developer | Does a user need to thus initially set the quota for the account/container, or is there usually a default initial value that it's set to? | 13:56 |
| cschwede | smart_developer: there is none set by default, thus it needs to be set by the operator/user | 13:59 |
| cschwede | smart_developer: account_quotas: http://docs.openstack.org/developer/swift/middleware.html#module-swift.common.middleware.account_quotas | 13:59 |
| cschwede | smart_developer: container_quotas: http://docs.openstack.org/developer/swift/middleware.html#module-swift.common.middleware.container_quotas | 13:59 |
| *** jroll has quit IRC | 14:00 | |
| *** jroll has joined #openstack-swift | 14:00 | |
| smart_developer | You mean, there's none set by default, even when the middleware has been added into proxy-server.conf / other relevant .conf files ? | 14:02 |
| smart_developer | (none, as in, no default initial value for any account/container quotas). | 14:03 |
| *** wlkely is now known as wkelly | 14:09 | |
| *** kopparam has joined #openstack-swift | 14:10 | |
| *** vr2 has quit IRC | 14:11 | |
| *** vr1 has joined #openstack-swift | 14:12 | |
| smart_developer | sorry, just confirming. | 14:12 |
| *** judd7 has joined #openstack-swift | 14:13 | |
| *** vr2 has joined #openstack-swift | 14:15 | |
| *** vr1 has quit IRC | 14:16 | |
| cschwede | smart_developer: exactly, there is no quota set by default, even with the middleware enabled | 14:17 |
| Trixboxer | Hi, is it possible to not list all objects from container on GET and traverse inside folders as GET ? | 14:20 |
| glange | Trixboxer: http://docs.openstack.org/api/openstack-object-storage/1.0/content/GET_showContainerDetails__v1__account___container__storage_container_services.html | 14:23 |
| glange | check out prefix and marker, they may help | 14:23 |
| tdasilva | Trixboxer, glange: pseudo-hierarchical folders might also help: http://docs.openstack.org/api/openstack-object-storage/1.0/content/folders-directories.html | 14:24 |
| *** vr1 has joined #openstack-swift | 14:25 | |
| *** vr2 has quit IRC | 14:26 | |
| glange | tdasilva: that is a good link, I hadn't seen that before | 14:27 |
| acoles | tdasilva: hi. about the data migration patch - you had suggested moving the upload method to the driver class... | 14:27 |
| acoles | tdasilva: i thought of a use case that might also motivate a change to that interface, which is migrating between swift cluster in same auth domain | 14:28 |
| acoles | when no auth credentials need to be stored in container metadata because the same token can be used to authorise the migration GET. | 14:28 |
| tdasilva | acoles: interesting. do you think that would need yet another driver? different from the current one? | 14:30 |
| acoles | tdasilva: i haven't got too far into detail. it could be same as current but with credentials becoming optional (if container metadata has no credentials then re-use auth-token) | 14:31 |
| *** marcusvrn has quit IRC | 14:31 | |
| acoles | tdasilva: but it ould require passing incoming request env to the driver for it to extract the auth-token, whcih IIRC the interface does not currently do. | 14:31 |
| acoles | tdasilva: anyway, we should consider this before the patch lands before anyone writes other drivers to the current interface | 14:33 |
| tdasilva | acoles: that's right. basically we would pass the whole responsiblity to the driver to get the data and upload to a new place | 14:34 |
| acoles | tdasilva: yup | 14:34 |
| tdasilva | acoles: agreed | 14:34 |
| *** zaitcev has joined #openstack-swift | 14:34 | |
| *** ChanServ sets mode: +v zaitcev | 14:34 | |
| acoles | tdasilva: ok, will mull on it some more. thx | 14:35 |
| tdasilva | acoles: I was planning on taking a look at that review today. I'm hoping I can maybe provide some help with an alternative option for the interface... | 14:35 |
| smart_developer | cschwede : Thank you! | 14:37 |
| acoles | tdasilva: great, if you have cycles then go for it. maybe push a dependent patch that could be squashed in. | 14:37 |
| tdasilva | acoles: sounds good :-) | 14:39 |
| *** kopparam has quit IRC | 14:44 | |
| *** kopparam has joined #openstack-swift | 14:44 | |
| openstackgerrit | A change was merged to openstack/swift: Stop using intersphinx https://review.openstack.org/121318 | 14:45 |
| *** kopparam has quit IRC | 14:48 | |
| *** ppai has quit IRC | 14:54 | |
| acoles | tdasilva: one other thought, if you are looking at the driver interface ... i wonder if each driver should expose a static method to validate the migration headers during container PUT/POST | 14:59 |
| acoles | e.g. fsystem driver could check that the source path has no '..' parts, which at the moment doesn't happen until an object GET | 15:00 |
| tdasilva | acoles: where is that validation happening today? | 15:00 |
| tdasilva | I see | 15:01 |
| acoles | tdasilva: actually, that might be the only example ;) i was just looking at it | 15:01 |
| tdasilva | acoles: and that happens during init, right? | 15:02 |
| acoles | tdasilva: yes, but init is called during GETorHEADmiss i.e. during object request | 15:04 |
| acoles | tdasilva: so you dont get a 400 for having a bad source path until you attempt to GET object | 15:04 |
| acoles | tdasilva: and seems it might be better to get the 400 when PUT/POSTing the container ?? | 15:05 |
| tdasilva | acoles: catching up... :-) So, the modules are loaded in filter_factory, but each driver object is instantiated in remote_driver_resolver, which is part of the object GET code flow | 15:07 |
| *** mahatic has quit IRC | 15:08 | |
| tdasilva | headers are set as part of the container PUT, and some validation is done there | 15:08 |
| tdasilva | basically checking if everything needed is provided, right? | 15:08 |
| *** kopparam has joined #openstack-swift | 15:09 | |
| tdasilva | so if a static method is provided for that driver, you could actually validate params during the container PUT, maybe even check if the token gives access and what not | 15:10 |
| *** kopparam has quit IRC | 15:14 | |
| acoles | tdasilva: yes, thats where i was heading, except for checking access. Stuff that definitely will never work can be checked and rejected on container PUT e.g. a source containing '..'). Stuff that may work at the time of an object GET shouldn't be checked until the object GET e.g. swift credentials, filesystem path existence. | 15:14 |
| acoles | tdasilva: So it would mean resolving the correct driver class during container PUT and calling a static 'check_headers' method passing the X-Container-Migration-* headers (or all headers) | 15:16 |
| *** mahatic has joined #openstack-swift | 15:17 | |
| acoles | tdasilva: to allow for future drivers that might be able to perform other validations on the headers | 15:17 |
| tdasilva | acoles: exactly....seems very reasonable.. | 15:18 |
| acoles | tdasilva: i guess right now i have just the one example (the fsystem source having '..'), but to my mind that should be treated in the same way as specifying no source | 15:19 |
| tdasilva | what about '.' ? | 15:21 |
| portante | centurylink, huh? | 15:21 |
| portante | http://www.crn.com/news/cloud/300073991/centurylink-reportedly-eyeing-a-rackspace-acquisition.htm | 15:21 |
| acoles | tdasilva: the source gets made relative to the driver_fsystem_parent_path but now you mention it i don't think that is prevented form being '.' :/ | 15:25 |
| smart_developer | So I noticed that Keystone does not have a policy.json for Swift | 15:25 |
| smart_developer | That Swift is different from other services in that it doesn't have a policy.json in Keystone, unlike the others. | 15:26 |
| smart_developer | Then how do I establish the right permissions for the two Keystone roles that I created: swift_operator, and swift_user ? | 15:26 |
| acoles | smart_developer: swift keystoneauth middleware does use a policy.json file | 15:26 |
| smart_developer | How do I access/modify the Swift ACL API? | 15:28 |
| acoles | agh s/does/does not/ !! | 15:28 |
| acoles | smart_developer: see doc http://docs.openstack.org/developer/swift/overview_auth.html#configuring-swift-to-use-keystone | 15:28 |
| smart_developer | Okay, I see that there's a way to configure a Keystone role to be a Swift operator_role, but how do I also add/configure just a regular user/member role? | 15:30 |
| acoles | tdasilva: so if there were a check_headers() static method for each driver, then that can also take care of checking that all required headers are present. which removes the need to have the required header keys specified in config file, i think. | 15:30 |
| acoles | tdasilva: which would be nicer imho | 15:31 |
| tdasilva | yes...concerning that...i was trying to figure out why token_url, user and key are stored in the config file | 15:33 |
| tdasilva | ? | 15:33 |
| tdasilva | am i reading that correctly? | 15:33 |
| acoles | smart_developer: a user with operator_role on a project can add/delete/update containers and object in that project's swift account. | 15:34 |
| acoles | smart_developer: what do you mean by a 'regular user/member role'? | 15:35 |
| portante | http://www.crn.com/news/cloud/300073991/centurylink-reportedly-eyeing-a-rackspace-acquisition.htm | 15:35 |
| smart_developer | by regular user role, I was referring to someone who can use the Swift service, but isn't an admin like the operator_role. | 15:37 |
| smart_developer | Basically someone who has fewer privileges than an operator/admin. | 15:39 |
| acoles | smart_developer: ok, but what do you mean by 'use'? read through that doc page and note the reseller_admin_role which may equate to what you are calling 'admin' | 15:39 |
| smart_developer | wait, are you saying that the operator_role is basically just a 'regular' user, not necessarily an admin? | 15:39 |
| acoles | smart_developer: i think i am :) | 15:40 |
| smart_developer | Ah, ok. | 15:41 |
| smart_developer | And you're 100% sure of this ? | 15:41 |
| smart_developer | certain* | 15:41 |
| tdasilva | acoles: nevermind...now I understand your comment about the required headers keys and yes I like it. I was never too much of a fan of having those keys there in the config file | 15:42 |
| cschwede | smart_developer: have a look here: https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L283-L287 | 15:42 |
| smart_developer | Ok so in the description for "operator_role" in that link, it refers to the fact that the operator_role can "..... give ACL to others". | 15:45 |
| smart_developer | This sounds like something admin would do, however. | 15:45 |
| *** rmcall has joined #openstack-swift | 15:46 | |
| smart_developer | Shouldn't a regular user not have the privilege to manage something like ACL(s)? | 15:46 |
| cschwede | smart_developer: think of it more like "admin of that account" | 15:47 |
| acoles | smart_developer: what cschwede said ^^ | 15:47 |
| acoles | smart_developer: the user with operator_role can create container in account, and can grant access to that container to other users (who may not have operator_role for that account) | 15:48 |
| acoles | using an ACL | 15:48 |
| cschwede | smart_developer: so, there are basically three levels of accounts with different permissions: ResellerAdmins, operator_role and a simple user | 15:48 |
| smart_developer | Where in the proxy-server.conf (or any other relevant conf file) would you specify who is a 'simple' user? | 15:50 |
| *** gyee has joined #openstack-swift | 15:50 | |
| smart_developer | It seems clear for operator_role and ResellerAdmin, but 'simple user' doesn't seem to be in the proxy-server.conf-sample | 15:50 |
| *** cschwede has quit IRC | 15:53 | |
| smart_developer | and what would a 'simple user', entail? | 15:53 |
| *** jergerber has joined #openstack-swift | 15:55 | |
| *** jdag____ is now known as jdaggett | 15:55 | |
| Trixboxer | Thanks glange & tdasilva, path variable helped from it | 16:02 |
| tdasilva | Trixboxer: welcome :-) | 16:03 |
| *** tongli has joined #openstack-swift | 16:04 | |
| *** cschwede has joined #openstack-swift | 16:08 | |
| *** mwstorer has joined #openstack-swift | 16:11 | |
| *** cschwede has quit IRC | 16:13 | |
| *** kyles_ne has joined #openstack-swift | 16:14 | |
| *** ledeveloper has joined #openstack-swift | 16:18 | |
| *** judd7 has quit IRC | 16:23 | |
| *** judd7 has joined #openstack-swift | 16:24 | |
| ledeveloper | hello, anyone here whose brain can be picked about swift's eventual-consitency guarantees (or lack thereof) ? | 16:26 |
| acoles | smart_developer: there are only two types of roles or swift: the operator_roles which allow a user to manage an account and the reseller_admin role which allow a user to manage all account | 16:27 |
| acoles | smart_developer: if you want to have a user with lesser privileges that managing an account e.g. access to just one container then you use ACLs | 16:27 |
| acoles | (container ACLs) | 16:28 |
| smart_developer | acoles Thank you. | 16:32 |
| acoles | smart_developer: welcome. i don't know other openstack projects well but I suspect the role descriptions are different? | 16:33 |
| *** zacksh_ is now known as zacksh | 16:33 | |
| *** tab__ has joined #openstack-swift | 16:37 | |
| *** vr1 has quit IRC | 16:46 | |
| openstackgerrit | A change was merged to openstack/swift: Fix internal link to keystoneauth in documentation https://review.openstack.org/121423 | 17:11 |
| notmyname | good morning! | 17:14 |
| tdasilva | ledeveloper: do you have any specific questions? feel free to ask away, even if nobody can answer now, other might join later and see your question and respond... | 17:14 |
| tdasilva | notmyname: good monday :-) | 17:14 |
| *** cdnchris has joined #openstack-swift | 17:17 | |
| *** cdnchris has left #openstack-swift | 17:21 | |
| ledeveloper | ok so lets try the channel then | 17:23 |
| ledeveloper | I have a scenario where a service is required to store multiple objects to a swift storage account concurrently, the container name | 17:24 |
| ledeveloper | derives from the object name (say the first N letters of the object name) with the number of all possible container names being too big for | 17:24 |
| ledeveloper | it to make sense to create them all in advance, so the plan is to try and store the object expecting it's container to already exist and in | 17:24 |
| ledeveloper | case of a 404 error to explicitly PUT the container and then retry PUTting the object (the objects/containers ratio is large enough for this to | 17:24 |
| ledeveloper | be effecient). | 17:24 |
| ledeveloper | The question is - given the eventually-consistent nature of swift, is there a clean way to do this ? that is, something more deterministic than | 17:24 |
| ledeveloper | just retry the "create-container, put-object" sequence untill both succeed ? | 17:24 |
| notmyname | ledeveloper: your proposed plan sounds reasonable and a good way to use swift | 17:25 |
| ledeveloper | the problem is that when I do that I get all sorts of 4xx errors (including ones with python stack traces in them) and while I have unit tests that verify that eventually everything is written correctly it seems like a pain to debug/troubleshoot it later in prod | 17:26 |
| notmyname | ledeveloper: can you pastebin one of the stack traces? you shouldn't ever get that from swift. | 17:27 |
| ledeveloper | I am using soft layers service if that makes any difference so I'm limited to one account - otherwise i'd auto-create at the account level rather than the container | 17:27 |
| ledeveloper | will try to provide a readable pastebin, it is currently heavily escaped by our logging infra | 17:28 |
| notmyname | shouldn't make any difference | 17:28 |
| openstackgerrit | Christian Schwede proposed a change to openstack/swift: Allow filtering by region in swift-recon https://review.openstack.org/121634 | 17:31 |
| *** geaaru has quit IRC | 17:40 | |
| openstackgerrit | Mahati proposed a change to openstack/swift: Added instructions to create a label or UUID to the XFS volume and mount using it. https://review.openstack.org/119193 | 17:49 |
| *** IgnacioCorderi has joined #openstack-swift | 17:53 | |
| *** aix has quit IRC | 17:54 | |
| *** cdnchris has joined #openstack-swift | 18:05 | |
| *** geaaru has joined #openstack-swift | 18:06 | |
| *** nshaikh has joined #openstack-swift | 18:11 | |
| *** cdnchris has quit IRC | 18:15 | |
| *** mitz_ has quit IRC | 18:16 | |
| notmyname | ledeveloper: got your paste. looks like you're getting a 400 in response to a container PUT | 18:18 |
| ledeveloper | yeah, couldnt make sense of the reason/body though | 18:19 |
| ledeveloper | at other times i get an "unsupported method" on container put requests, trying to find a a repro log for this | 18:20 |
| *** cdnchris has joined #openstack-swift | 18:21 | |
| ledeveloper | a 405 Method Not Allowed that is | 18:22 |
| *** gyee has quit IRC | 18:22 | |
| notmyname | ledeveloper: this is interesting | 18:26 |
| notmyname | ledeveloper: what's all the escape sequences in that response? | 18:27 |
| ledeveloper | thats what im tryin to find out as we speak, its either being done by json logging code or arrvies that way from the server | 18:27 |
| notmyname | ledeveloper: looking at the swift source code, I don't see the string "bad request syntax" anywhere. let my try to repro something | 18:28 |
| *** openstackgerrit has quit IRC | 18:32 | |
| notmyname | ledeveloper: interestingly on the 400 response you don't have an x-trans-id header (see the 405 response for comparison). you should have one of those with every response that comes from swift | 18:32 |
| *** openstackgerrit has joined #openstack-swift | 18:33 | |
| ledeveloper | true, might indicate that there is some middleware/proxy involved on the softlayer side | 18:34 |
| notmyname | yeah maybe | 18:34 |
| ledeveloper | on the other hand there is a Server header with "BaseHTTP/0.3", | 18:34 |
| ledeveloper | "Python/2.7.3" values | 18:34 |
| notmyname | yeah, and I don't remember swift setting a server header (but that's a _really_ hard word to grep for in the source ;-) | 18:36 |
| ledeveloper | the escapes in the reason are definitely coming from the server | 18:39 |
| *** IgnacioCorderi has quit IRC | 18:41 | |
| notmyname | ledeveloper: are you doing anything with ACLs? anything that would mean you have more than one user identity trying to do stuff? | 18:45 |
| * notmyname doesn't know how softlayers auth works | 18:45 | |
| notmyname | but even with a non-privileged user, I get a 403 on a container PUT, not 405 | 18:45 |
| ledeveloper | not really, i do a single auth call on service startup and then use the provided X-Auth-Token | 18:46 |
| *** cdnchris has left #openstack-swift | 18:46 | |
| notmyname | ok | 18:46 |
| notmyname | ledeveloper: is this something that you can repro, of does it only happen on some small percentage of requests? | 18:47 |
| ledeveloper | it happens fairly consistently in a test that runs ~50 put requests asynchrnously | 18:48 |
| * notmyname wishes briancline were in here | 18:49 | |
| ledeveloper | will try decoding the escapes and see if it leads me anywhere | 18:53 |
| ledeveloper | anyway thanks for now, will update if I discover anything interesting | 18:54 |
| notmyname | ok. I also asked in #softlayer. sometimes there's some people that can help in there | 18:54 |
| notmyname | but no response yet | 18:55 |
| *** judd7 has quit IRC | 18:56 | |
| ledeveloper | I had a ticket opened with them, they said they cant help without a repro even with X-Trans-Id so I'll see how I can provide them as a stand-alone repro code | 18:57 |
| ledeveloper | the thing is that after enough (several) retries all the operations eventually succeed, which led me to believe it might be something inherent in swifts design | 18:58 |
| notmyname | ledeveloper: anything that may be rarely exposed in swift due to eventual consistency wouldn't be exposed as a 400 or 405 response. so I think there's something else going on here | 18:59 |
| ledeveloper | just from speculation, can a malfunctioning node that is being kept in the cluster cause something like that ? | 19:00 |
| notmyname | depends on how you define "malfunctioning". maybe it means that one server just returns random.choice([HTTPBadRequest(), HTTPMethodNotAllowed()]) ;-) | 19:01 |
| ledeveloper | right :) | 19:02 |
| ahale | randomly malfunctioning stuff tends to produce 503 or slowness, depending how aggressive the clusters been set up | 19:02 |
| notmyname | point is, "malfunctioning" is a huge space. sure something could be there. but reasonably I wouldn't think that it would cause a 400 or 405 in response to a container PUT | 19:02 |
| notmyname | right. what ahale said. he knows :-) | 19:03 |
| *** kyles_ne has quit IRC | 19:04 | |
| *** kyles_ne has joined #openstack-swift | 19:04 | |
| *** kyles_ne has quit IRC | 19:09 | |
| ledeveloper | according to https://dal05.objectstorage.softlayer.net/info the version 1.12.0, is it current/stable ? | 19:10 |
| notmyname | always stable ;-) | 19:21 |
| notmyname | but not quite the newest release. that's a few versions old | 19:22 |
| openstackgerrit | Mahati proposed a change to openstack/swift: Added instructions to create a label or UUID to the XFS volume and mount using it. https://review.openstack.org/119193 | 19:23 |
| *** IgnacioCorderi has joined #openstack-swift | 19:24 | |
| openstackgerrit | Mahati proposed a change to openstack/swift: Added instructions to create a label or UUID to the XFS volume and mount using it. https://review.openstack.org/119193 | 19:25 |
| *** marcusvrn has joined #openstack-swift | 19:25 | |
| *** gyee has joined #openstack-swift | 19:40 | |
| *** Trixboxer has quit IRC | 19:46 | |
| *** kyles_ne has joined #openstack-swift | 19:50 | |
| *** kyles_ne has quit IRC | 19:59 | |
| *** kyles_ne has joined #openstack-swift | 19:59 | |
| portante | curious, has anybody played with vaigrant on Fedora 20? (see https://ttboj.wordpress.com/2014/05/13/vagrant-on-fedora-with-libvirt-reprise/) | 20:02 |
| ekarlso | notmyname: what does 413 mean in swift contex ? | 20:10 |
| notmyname | ekarlso: the 413 response code? | 20:11 |
| ekarlso | ya | 20:11 |
| ekarlso | glance img upload is failing w 413 | 20:11 |
| ekarlso | towards swift | 20:11 |
| notmyname | ekarlso: too big | 20:11 |
| ekarlso | notmyname: weird, object is only < 500 mb | 20:12 |
| notmyname | ekarlso: are you sending a content type or using chunked transfer encoding? | 20:12 |
| ekarlso | notmyname: unsure what it does | 20:12 |
| ekarlso | glance image-upload :p | 20:12 |
| ekarlso | but running swift behind nginx, it used to work before that :| | 20:13 |
| ekarlso | because of ssl | 20:13 |
| notmyname | ekarlso: do you know if it's splitting the image and using object manifests? | 20:15 |
| ekarlso | notmyname: no clue ! | 20:15 |
| notmyname | ekarlso: ok. there aren't a lot of places in swift that return 413. most have to do with sending too many bytes in the body, but there are a couple of other places like when you use static manifests and have too many segments listed in it. | 20:16 |
| ekarlso | notmyname: can it have something to do with running swift behind a nginx reverse proxy ? | 20:17 |
| notmyname | dont' know. I don't think anyone should do that anyway :-) | 20:17 |
| ekarlso | notmyname: what to use for ssl termination then ? | 20:18 |
| notmyname | ekarlso: oh! can you see the headers on the 413 response? do they include the x-trans-id header? | 20:18 |
| ekarlso | can't see nto:| | 20:18 |
| ekarlso | notmyname: | 20:18 |
| notmyname | ah, that's a good sign then. that's an nginx thing. swift always returns the x-trans-id header | 20:19 |
| notmyname | so nginx is probably configured with a max body size (isn't it default to something like 5MB?). that's probably the issue | 20:19 |
| ekarlso | notmyname: yap :D | 20:19 |
| ekarlso | saw that now : p | 20:19 |
| notmyname | I normally recommend haproxy for ssl termination. pound mostly works too (but you lose 100-continue support). stud or stunnel might work too | 20:19 |
| ekarlso | notmyname: which is best ? | 20:20 |
| notmyname | ekarlso: note that nginx spools the request body locally, so if it's in front of swift, then it will have to spool the whole image. that can seriously cause issues with scaling. eg imaging 500 concurrent uploads of 5GB each | 20:21 |
| notmyname | ekarlso: I know that HAProxy is currently being used at very large scale in front of Swift. that's my recommendation | 20:22 |
| ekarlso | notmyname: coolio | 20:22 |
| *** tongli has quit IRC | 20:26 | |
| *** nshaikh has left #openstack-swift | 20:28 | |
| smart_developer | How do I make it so that logrotate only checks a certain file on an hourly basis, while it continues to check the rest of the system files on a daily basis? | 20:51 |
| notmyname | http://d.not.mn/swift_contribs_over_time.png | 20:52 |
| tdasilva | notmyname: very cool...pretty amazing to see the growth | 20:56 |
| notmyname | I'm happy with the trend and that it's consistent | 20:57 |
| notmyname | next up would be to map active contributors | 20:57 |
| notmyname | vs total contibutors | 20:57 |
| *** geaaru has quit IRC | 20:57 | |
| tdasilva | hopefully that's growing too :-) | 20:58 |
| *** fifieldt_ has joined #openstack-swift | 20:59 | |
| *** fifieldt has quit IRC | 21:03 | |
| *** IgnacioCorderi has quit IRC | 21:09 | |
| smart_developer | Does anyone know which user should own the logrotate configuration file for Swift, situated in /etc/logrotate.d/ | 21:35 |
| smart_developer | ? | 21:35 |
| smart_developer | Is it root? swift? | 21:35 |
| smart_developer | (Not only own, but also the group as well). | 21:35 |
| torgomatic | certainly not swift; the swift daemons don't need to read or write stuff in logrotate.d | 21:36 |
| smart_developer | (e.g., root:root, swift:swift, root:swift, swift:root, etc, etc whichever user(s) they may be). | 21:36 |
| torgomatic | probably whatever user:group owns all the other logrotate stuff | 21:36 |
| torgomatic | root:root sounds good to me, but check your local system first | 21:36 |
| smart_developer | root:root seems to be the case. | 21:38 |
| smart_developer | Also, what about the suggestion for the rsyslog configuration file for Swift, situated in /etc/rsyslog.d/ | 21:39 |
| smart_developer | ? | 21:39 |
| smart_developer | also for logging purposes. | 21:39 |
| smart_developer | (namely, to direct the ouput of any programs related to Swift, to the appropriate log path/file). | 21:39 |
| torgomatic | set up syslog how you like. it's flexible enough for most situations; that's why Swift prefers to use syslog in place of some homegrown wacky thing | 21:44 |
| mattoliverau | Morning | 22:13 |
| *** tab__ has quit IRC | 22:15 | |
| notmyname | http://d.not.mn/swift_devs_over_time.png | 22:20 |
| notmyname | tdasilva: ^ | 22:20 |
| tdasilva | humm....interesting...not sure it's what i expected | 22:22 |
| tdasilva | notmyname: actually...what's the definition of active contributor? | 22:23 |
| notmyname | `git shortlog -nes --no-merges --before='@{6 months ago}' --since='@{12 months ago}' | wc -l` is the exact definition :-) | 22:23 |
| tdasilva | haha | 22:23 |
| notmyname | probably could be better labeled as "total contributors" vs "contributors" | 22:24 |
| notmyname | or vs active contributors. whatever | 22:24 |
| torgomatic | 50+ active contributors is still quite a bit, though | 22:25 |
| notmyname | indeed | 22:25 |
| *** mwstorer has quit IRC | 22:26 | |
| notmyname | it's actually a lot higher than I first expected | 22:27 |
| notmyname | I normally look at that number with every release, and we have about 20-30 people in each release. but releases are a lot more often than every six months | 22:28 |
| tdasilva | humm...running just the first part of the command i don't see peluse_away or mattoliverau name's there | 22:28 |
| notmyname | which first part? | 22:28 |
| tdasilva | or mine :\ | 22:28 |
| tdasilva | git shortlog -nes --no-merges --before='@{6 months ago}' --since='@{12 months ago}' | 22:29 |
| *** ryao is now known as ZFS | 22:30 | |
| notmyname | tdasilva: your first patch on master in swift is dated "Fri Jul 11 2014 08:13:52 GMT-0700 (PDT)" | 22:30 |
| *** rmcall has quit IRC | 22:30 | |
| notmyname | therefore it's out of that range | 22:31 |
| tdasilva | ok | 22:31 |
| *** rmcall has joined #openstack-swift | 22:32 | |
| clayg | tdasilva: notmyname is real exclusionary like that - all using "dates" to define "periods of time" | 22:32 |
| clayg | acoles: i was thinking that we do reloading files a bunch in swift already so I was playing with this on the train -> https://gist.github.com/clayg/0db807e42d9b1289118e#file-example-py | 22:33 |
| tdasilva | haha | 22:33 |
| mattoliverau | what.. daes to signafy time.. that's crazy talk :P | 22:33 |
| clayg | acoles: now I just have to figure out how to do a "deep_update" on a bag of dicts | 22:33 |
| mattoliverau | *dates | 22:34 |
| mattoliverau | 6 months ago I was only a few months into my new role in my current rackspace development team, and was working on an infra/nova CI/CD testing database migrations. It wasn't until the last summit that I stumbled into a swift design session and realised how awesome swift was and started reviewing :) | 22:40 |
| mattoliverau | Anyway, coffee run time | 22:40 |
| *** kyles_ne has quit IRC | 22:44 | |
| *** kyles_ne has joined #openstack-swift | 22:48 | |
| *** rmcall has quit IRC | 22:50 | |
| *** IgnacioCorderi has joined #openstack-swift | 22:56 | |
| *** tdasilva has quit IRC | 22:57 | |
| *** dmsimard is now known as dmsimard_away | 23:00 | |
| *** rmcall has joined #openstack-swift | 23:04 | |
| *** mahatic has quit IRC | 23:16 | |
| *** jergerber has quit IRC | 23:16 | |
| *** ZFS is now known as ryao | 23:18 | |
| *** briancline has joined #openstack-swift | 23:35 | |
| briancline | request to Get Weird for a second if anyone's around | 23:42 |
| briancline | what condition(s) could cause all 3 copies of a container db to have differing, days-old md5sums, with .pending and .lock files still hanging around? | 23:42 |
| notmyname | briancline: IO contention on the container drives from lots of concurrent object writes in that container? | 23:43 |
| notmyname | briancline: maybe replication is just having issues walking the drives? | 23:44 |
| notmyname | maybe the container DBs are big and the whole-file sqlite locks are causing issues for updates and replication | 23:45 |
| notmyname | or maybe just replication died sometime | 23:45 |
| *** rmcall_ has joined #openstack-swift | 23:46 | |
| torgomatic | also maybe objects got written in differing orders, resulting in the same logical content but not the same bytes in the .db files | 23:48 |
| notmyname | briancline: oh, earlier today ledeveloper was getting some errors from softlayer. some 400s and 405s on container PUTs | 23:48 |
| notmyname | briancline: torgomatic: but out of order doesn't explain the .pending and .lock files hanging around. could be separate issues, though | 23:49 |
| torgomatic | .pendings don't get consumed until they're big enough, and .lock I think hang around after first use (no? maybe.) | 23:50 |
| *** rmcall has quit IRC | 23:52 | |
| *** r-daneel__ has quit IRC | 23:52 | |
| *** bnelson has quit IRC | 23:52 | |
| *** rmcall_ is now known as rmcall | 23:52 | |
| *** bnelson has joined #openstack-swift | 23:52 | |
| notmyname | torgomatic: the .pending gets consumed, but the file isn't unlinked. the contents goes away fairly quickly | 23:53 |
| torgomatic | notmyname: right, but only on certain requests, and only once it's over a certain size... like a PUT while the .pending is small will just add on | 23:53 |
| notmyname | PUT/POST flushes it. GET doesn't (at least when .pending just has one thing in it) | 23:55 |
| notmyname | also, container-replicator once flushes it | 23:55 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
