| *** haomaiwa_ has quit IRC | 00:01 | |
| *** zhill has quit IRC | 00:01 | |
| *** haomaiwang has joined #openstack-swift | 00:01 | |
| *** okdas has quit IRC | 00:04 | |
| *** eranrom has joined #openstack-swift | 00:09 | |
| *** lpabon has joined #openstack-swift | 00:20 | |
| *** jrichli has joined #openstack-swift | 00:21 | |
| *** wbhuber_ has quit IRC | 00:24 | |
| *** wbhuber has joined #openstack-swift | 00:25 | |
| *** okdas has joined #openstack-swift | 00:27 | |
| *** okdas has joined #openstack-swift | 00:27 | |
| *** wbhuber has quit IRC | 00:29 | |
| *** tongli has quit IRC | 00:30 | |
| *** jerrygb has joined #openstack-swift | 00:31 | |
| *** jerrygb has quit IRC | 00:32 | |
| *** eranrom has quit IRC | 00:33 | |
| clayg | whoa - did you know that argparse automatically lets you use short aliases for arguments? I added an option to ring-check for --save-builder and accidently envoked it with --save but it totally *works* | 00:40 |
|---|---|---|
| *** lpabon has quit IRC | 00:40 | |
| klrmn1 | argparse is pretty smart. it often does what you mean it to do, even when you didn't say it right | 00:41 |
| *** m_kazuhiro has joined #openstack-swift | 00:41 | |
| *** diogogmt has quit IRC | 00:42 | |
| *** haomaiwang has quit IRC | 00:45 | |
| *** sileht has quit IRC | 00:48 | |
| *** willsama has joined #openstack-swift | 00:52 | |
| *** gyee has quit IRC | 00:59 | |
| *** km has joined #openstack-swift | 01:02 | |
| *** km is now known as Guest33144 | 01:02 | |
| *** Guest33988 has quit IRC | 01:03 | |
| jrichli | question: if there is now only one key, can we still have the keymaster interface allow for 3 different keys: object, container, and account levels? | 01:04 |
| jrichli | the encrypter and decrypter middleware use the container key to encrypt what will be in the container listing | 01:04 |
| jrichli | if we take that away, it means that you cannot plug-in a keymaster with more keys | 01:05 |
| notmyname | jrichli: I could imagine container_key = sha1_hmac(container_name, cluster_key) or something like that. not sure if an hmac would actually be needed, but my point is the ability to make a contianer key from the cluster key | 01:05 |
| jrichli | oh, right | 01:06 |
| notmyname | probably would mix in the account key at some point too | 01:06 |
| jrichli | i am sorry. getting mixed up | 01:06 |
| notmyname | container_key = cluster_key + account_key # or something | 01:06 |
| jrichli | of course | 01:06 |
| notmyname | container_key = cluster_key + account_name + container_name # or something | 01:06 |
| notmyname | that's what I meant ;-) | 01:06 |
| *** hrou has joined #openstack-swift | 01:07 | |
| jrichli | so, I am starting to wonder why one cluster key in config file is better than what the trivial_keymaster does now. maybe i am missing something simple ... | 01:10 |
| jrichli | oh, nevermind | 01:11 |
| jrichli | it was silly. | 01:11 |
| jrichli | ok, i am gonna stop talking in channel now | 01:11 |
| hrou | Noo don't stop ; ) Well it was one key for every cluster in the entire world ; ) That'd be cool for us though. | 01:13 |
| jrichli | hrou: seriously, lets not talk about my silliness from tonight | 01:13 |
| jrichli | hrou: i am just clearly having an air-head sort of night | 01:13 |
| hrou | jrichli, oh my bouncer died a day or so ago, did I miss other entertaining comments ? : ) | 01:14 |
| *** blmartin has quit IRC | 01:15 | |
| jrichli | hrou: just 2 silly thoughts. | 01:15 |
| *** willsama has quit IRC | 01:17 | |
| pchng_ | jrichli: I welcome your silly thoughts, since I am still learning how different keys are derived in Swift :) | 01:18 |
| jrichli | well, technically, it wouldn't be one key for every cluster, just derived (predictable) from the path name. | 01:18 |
| pchng_ | btw, thank you for the welcome yesterday hrou :) | 01:19 |
| jrichli | but we will change that to have an actual SECRET from the config file | 01:19 |
| hrou | Yea jrichli is right, we had an older key master that had just one hard coded yet | 01:19 |
| jrichli | maybe some wrapping scheme later at some level, but first patch will not have wrapping. | 01:19 |
| *** pchng_ is now known as pchng | 01:19 | |
| hrou | pchng, oh anytime ! I think I learned your desk is across from mine ; ) | 01:20 |
| *** badari has quit IRC | 01:21 | |
| pchng | hrou: Yep, we're both in A2! | 01:23 |
| *** badari has joined #openstack-swift | 01:23 | |
| pchng | Possibly naive/noob question: I was looking at the fast-POST functionality, and was wondering why POST was used only for create/update object metadata and PUT was used to create or replace object data and metadata? | 01:31 |
| *** diogogmt has joined #openstack-swift | 01:33 | |
| clayg | pchng: well in REST the PUT is a pretty good way to describe "here is the whole version of the resource that belongs at this uri" | 01:35 |
| hrou | pchng, there are no silly questions ! (Only jrichli asks those ; ) jk, we showed her question wasn't silly at all): More of a http PUT generally is the entire object and POST to update it (and in swift its metadata per say) | 01:36 |
| clayg | pchng: there's not really a good verb for "modify the resource a just a little bit" - but we do have POST - which means "i have no idea wft is even going on with this verb - it's anything the app wants to do" | 01:36 |
| clayg | so that seemed like we could make it work for "update metadata" | 01:36 |
| *** hezhiqiang has joined #openstack-swift | 01:37 | |
| pchng | clayg, hrou: Thanks for the explanations. I've read various arguments between PUT vs POST and the exact semantics so this makes sense | 01:37 |
| *** janonymous__ has joined #openstack-swift | 01:37 | |
| hrou | Yea you could in theory have defined an API that uses POST instead really, its more of a swift specific implementation but you'll find many http api's are quite similar | 01:38 |
| pchng | The documentation shows a clear delineation between what PUT and POST does so there's no ambiguity, was just wondering why | 01:38 |
| *** jlhinson has quit IRC | 01:40 | |
| *** yuan has quit IRC | 01:40 | |
| *** yuan has joined #openstack-swift | 01:40 | |
| *** km has joined #openstack-swift | 01:41 | |
| *** Guest33144 has quit IRC | 01:41 | |
| *** km is now known as Guest95465 | 01:41 | |
| *** eranrom has joined #openstack-swift | 01:58 | |
| *** Guest95465 has quit IRC | 01:59 | |
| *** garthb has quit IRC | 02:00 | |
| *** breitz1 has joined #openstack-swift | 02:08 | |
| *** breitz has quit IRC | 02:11 | |
| *** breitz2 has joined #openstack-swift | 02:11 | |
| *** klrmn1 has quit IRC | 02:12 | |
| *** breitz1 has quit IRC | 02:13 | |
| *** janonymous__ has quit IRC | 02:14 | |
| *** wbhuber has joined #openstack-swift | 02:17 | |
| *** dmorita has quit IRC | 02:17 | |
| *** badari has quit IRC | 02:29 | |
| *** badari has joined #openstack-swift | 02:46 | |
| *** haomaiwang has joined #openstack-swift | 02:55 | |
| *** haomaiwang has quit IRC | 03:00 | |
| *** rohit_ has quit IRC | 03:04 | |
| *** badari has quit IRC | 03:10 | |
| *** haomaiwang has joined #openstack-swift | 03:19 | |
| *** jrichli has quit IRC | 03:30 | |
| *** haomaiwang has quit IRC | 03:48 | |
| *** wbhuber has quit IRC | 04:07 | |
| *** wbhuber has joined #openstack-swift | 04:07 | |
| *** aagrawal has joined #openstack-swift | 04:09 | |
| *** SkyRocknRoll has joined #openstack-swift | 04:11 | |
| *** wbhuber has quit IRC | 04:11 | |
| *** badari has joined #openstack-swift | 04:23 | |
| *** links has joined #openstack-swift | 04:26 | |
| *** janonymous has quit IRC | 04:27 | |
| *** mahatic has joined #openstack-swift | 04:27 | |
| *** haomaiwa_ has joined #openstack-swift | 04:32 | |
| *** hezhiqiang has quit IRC | 04:42 | |
| *** sileht has joined #openstack-swift | 04:44 | |
| *** janonymous has joined #openstack-swift | 04:59 | |
| *** haomaiwa_ has quit IRC | 05:01 | |
| *** haomaiwa_ has joined #openstack-swift | 05:01 | |
| *** hezhiqiang has joined #openstack-swift | 05:08 | |
| *** badari has quit IRC | 05:31 | |
| *** km has joined #openstack-swift | 05:32 | |
| *** km is now known as Guest60383 | 05:33 | |
| *** haomaiwa_ has quit IRC | 05:44 | |
| *** venkat_p has joined #openstack-swift | 06:19 | |
| *** sileht has quit IRC | 06:25 | |
| *** sileht has joined #openstack-swift | 06:42 | |
| openstackgerrit | Hisashi Osanai proposed openstack/swift: Add functional test for access control (RBAC) with Keystone https://review.openstack.org/202411 | 06:48 |
| *** hrou has quit IRC | 06:57 | |
| *** asettle has quit IRC | 06:59 | |
| *** pchng_ has joined #openstack-swift | 06:59 | |
| *** pchng has quit IRC | 07:03 | |
| *** mac_ified has quit IRC | 07:05 | |
| *** haomaiwang has joined #openstack-swift | 07:11 | |
| *** sileht has quit IRC | 07:15 | |
| *** eranrom has quit IRC | 07:17 | |
| *** eranrom has joined #openstack-swift | 07:20 | |
| *** ppai has joined #openstack-swift | 07:22 | |
| *** aagrawal has quit IRC | 07:32 | |
| *** aagrawal has joined #openstack-swift | 07:32 | |
| *** hseipp has joined #openstack-swift | 07:50 | |
| *** hseipp has quit IRC | 07:50 | |
| *** hseipp has joined #openstack-swift | 07:50 | |
| *** zigo has quit IRC | 07:53 | |
| *** jamielennox is now known as jamielennox|away | 07:54 | |
| *** jamielennox|away is now known as jamielennox | 07:54 | |
| *** zigo has joined #openstack-swift | 07:56 | |
| *** jamielennox is now known as jamielennox|away | 07:57 | |
| *** sileht has joined #openstack-swift | 07:59 | |
| *** haomaiwang has quit IRC | 08:01 | |
| *** haomaiwang has joined #openstack-swift | 08:01 | |
| *** hseipp has quit IRC | 08:01 | |
| *** hseipp has joined #openstack-swift | 08:02 | |
| *** aagrawal has quit IRC | 08:10 | |
| *** janonymous has quit IRC | 08:14 | |
| *** openstackgerrit has quit IRC | 08:16 | |
| *** openstackgerrit has joined #openstack-swift | 08:16 | |
| *** rledisez has joined #openstack-swift | 08:17 | |
| *** peterlisak has joined #openstack-swift | 08:24 | |
| *** mfalatic has quit IRC | 08:27 | |
| *** mfalatic has joined #openstack-swift | 08:28 | |
| *** openstack has joined #openstack-swift | 08:35 | |
| *** jordanP has joined #openstack-swift | 08:39 | |
| *** ho has quit IRC | 08:43 | |
| *** venkat_p has quit IRC | 08:44 | |
| *** haomaiwang has quit IRC | 09:01 | |
| *** venkat_p has joined #openstack-swift | 09:01 | |
| *** haomaiwa_ has joined #openstack-swift | 09:07 | |
| *** janonymous has joined #openstack-swift | 09:08 | |
| *** acoles_ is now known as acoles | 09:16 | |
| acoles | good morning | 09:20 |
| acoles | ho: looks like that latest version of patch 202411 has all the scenarios added in one patch? | 09:22 |
| patchbot | acoles: https://review.openstack.org/#/c/202411/ - Add functional test for access control (RBAC) with... | 09:22 |
| *** jmccarthy has joined #openstack-swift | 09:23 | |
| *** jistr has joined #openstack-swift | 09:24 | |
| acoles | mahatic: hi. you here? | 09:35 |
| mahatic | acoles: hello, yes | 09:35 |
| acoles | mahatic: did you have any more comments on patch 203310 | 09:35 |
| patchbot | acoles: https://review.openstack.org/#/c/203310/ - Encrypting/Decrypting object metadata | 09:35 |
| *** venkat_p has quit IRC | 09:36 | |
| acoles | just wondering if you are looking at it still? | 09:36 |
| mahatic | acoles: ah, could you give me a few mins? Looking right away | 09:37 |
| acoles | mahatic: i can give you a few hours :) | 09:37 |
| acoles | mahatic: no rush, but if we can merge it this week that would be good. | 09:38 |
| mahatic | acoles: :D cool! I was trying to update the spec actually, (with the content-type crypto metadata change) but found a few more things that maybe need to be corrected. Will push something and put comments i think | 09:38 |
| mahatic | acoles: yes sure, I just want to take a look mainly at the encrypter/decrypter changes | 09:38 |
| acoles | ok thanks | 09:39 |
| *** hseipp has quit IRC | 09:41 | |
| *** hseipp has joined #openstack-swift | 09:43 | |
| *** haomaiwa_ has quit IRC | 09:46 | |
| *** haomaiwang has joined #openstack-swift | 09:46 | |
| mahatic | acoles: done. Also, we're not using this anywhere anymore right? -> X-Object-Sysmeta-Crypto-Content-Type | 09:52 |
| mahatic | I like the idea of appending etag crypto metadata to the header as well. Don't see why we shouldn't. Make it simpler and also avoids that conflict in question | 09:55 |
| mahatic | Makes* | 09:55 |
| *** hezhiqiang has quit IRC | 09:57 | |
| *** haomaiwang has quit IRC | 10:01 | |
| *** haomaiwang has joined #openstack-swift | 10:01 | |
| *** jamielennox|away is now known as jamielennox | 10:01 | |
| *** haomaiwang has quit IRC | 10:04 | |
| *** aagrawal has joined #openstack-swift | 10:06 | |
| *** zaitcev has quit IRC | 10:23 | |
| openstackgerrit | Catherine Northcott proposed openstack/swift: Add support for storage policies to have more than one name https://review.openstack.org/241978 | 10:38 |
| *** jerrygb has joined #openstack-swift | 10:58 | |
| *** Guest60383 has quit IRC | 11:04 | |
| *** aagrawal has quit IRC | 11:16 | |
| *** aagrawal has joined #openstack-swift | 11:17 | |
| *** venkat_p has joined #openstack-swift | 11:20 | |
| *** hezhiqiang has joined #openstack-swift | 11:24 | |
| venkat_p | Hi all | 11:28 |
| venkat_p | please review this change | 11:28 |
| venkat_p | https://review.openstack.org/#/c/241527/6 | 11:28 |
| *** venkat_p has quit IRC | 11:35 | |
| *** kei_yama has quit IRC | 11:38 | |
| *** lpabon has joined #openstack-swift | 11:42 | |
| *** SandeepBazar has joined #openstack-swift | 11:47 | |
| *** SandeepBazar has left #openstack-swift | 11:48 | |
| *** lpabon has quit IRC | 11:48 | |
| *** lpabon has joined #openstack-swift | 11:48 | |
| acoles | mahatic: makes sense | 12:03 |
| mahatic | acoles: yay, two more patches +2'd! Thanks for your work! | 12:03 |
| acoles | progress! | 12:04 |
| mahatic | yup :) | 12:04 |
| *** jerrygb has quit IRC | 12:10 | |
| *** jordanP has quit IRC | 12:18 | |
| *** jordanP has joined #openstack-swift | 12:18 | |
| *** jordanP has quit IRC | 12:19 | |
| openstackgerrit | Merged openstack/swift: Get rid of contextlib.nested() for py3 https://review.openstack.org/238237 | 12:20 |
| *** jordanP has joined #openstack-swift | 12:20 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Update container on fast-POST https://review.openstack.org/135380 | 12:24 |
| *** mahatic_ has joined #openstack-swift | 12:27 | |
| *** 17SADZP6R has joined #openstack-swift | 12:27 | |
| *** sileht has quit IRC | 12:28 | |
| *** SkyRocknRoll has quit IRC | 12:29 | |
| *** mahatic has quit IRC | 12:30 | |
| *** haypo has joined #openstack-swift | 12:32 | |
| haypo | acoles: hi. i'm sorry, i'm unable to reproduce your issue with old versions of pbr & tox, for the review https://review.openstack.org/#/c/217423/ | 12:32 |
| acoles | haypo: thanks for looking into it. | 12:34 |
| haypo | acoles: i expected that we all use tox :) | 12:35 |
| acoles | haypo: sure, i use tox for tests | 12:37 |
| acoles | haypo: setup.py to install. i'll play with it some more | 12:38 |
| *** sileht has joined #openstack-swift | 12:39 | |
| haypo | acoles: since it looks like you don't care of being synchronized with openstack global requirements, we can probably start with pbr>=1.0 | 12:40 |
| haypo | acoles: and maybe even leave setup.py unchanged | 12:40 |
| haypo | acoles: what do you think? | 12:40 |
| openstackgerrit | Peter Lisák proposed openstack/swift: swift-init return codes https://review.openstack.org/230352 | 12:41 |
| haypo | acoles: the purpose of the patch is not to upgrade pbr, it's to install dnspython3 on python3, but we need to the support for environment markers for that | 12:41 |
| acoles | haypo: i'm confused. the commit messsage says we need pbr>=1.8 to support environment markers in requirements, for dnspython. if so how does pbr>=1.0 work? | 12:44 |
| acoles | haypo: afk | 12:44 |
| *** m_kazuhiro has quit IRC | 12:48 | |
| *** daemontool has quit IRC | 12:49 | |
| openstackgerrit | Victor Stinner proposed openstack/swift: On py3, use dnspython3 dependency, not dnspython https://review.openstack.org/217423 | 12:49 |
| haypo | acoles: ^^ here you have, it would be great if you could test it | 12:52 |
| *** jistr has quit IRC | 12:52 | |
| *** 17SADZP6R has quit IRC | 13:01 | |
| *** haomaiwang has joined #openstack-swift | 13:01 | |
| *** arnox has joined #openstack-swift | 13:03 | |
| *** jamielennox is now known as jamielennox|away | 13:08 | |
| *** mahatic__ has joined #openstack-swift | 13:09 | |
| *** jistr has joined #openstack-swift | 13:11 | |
| *** jerrygb has joined #openstack-swift | 13:11 | |
| *** mahatic_ has quit IRC | 13:13 | |
| *** eranrom has quit IRC | 13:16 | |
| *** jerrygb has quit IRC | 13:16 | |
| *** pdardeau has joined #openstack-swift | 13:25 | |
| *** NM has quit IRC | 13:33 | |
| *** hrou has joined #openstack-swift | 13:34 | |
| *** pdardeau has quit IRC | 13:48 | |
| *** pdardeau has joined #openstack-swift | 13:49 | |
| *** pdardeau has quit IRC | 13:54 | |
| *** pdardeau has joined #openstack-swift | 13:54 | |
| *** daemontool has joined #openstack-swift | 13:59 | |
| *** haomaiwang has quit IRC | 14:01 | |
| *** links has quit IRC | 14:01 | |
| *** haomaiwa_ has joined #openstack-swift | 14:01 | |
| *** jerrygb has joined #openstack-swift | 14:04 | |
| *** pdardeau has quit IRC | 14:06 | |
| *** NM has joined #openstack-swift | 14:06 | |
| *** david-lyle has joined #openstack-swift | 14:08 | |
| *** hseipp has quit IRC | 14:09 | |
| *** diogogmt has quit IRC | 14:13 | |
| *** aagrawal has quit IRC | 14:14 | |
| *** daemontool has quit IRC | 14:15 | |
| *** daemontool has joined #openstack-swift | 14:16 | |
| *** diogogmt has joined #openstack-swift | 14:16 | |
| *** ppai has quit IRC | 14:30 | |
| *** diogogmt has quit IRC | 14:31 | |
| *** janonymous_ has joined #openstack-swift | 14:32 | |
| *** badari has joined #openstack-swift | 14:39 | |
| *** dustins has joined #openstack-swift | 14:39 | |
| *** kota_ has quit IRC | 14:42 | |
| *** amandap has quit IRC | 14:43 | |
| *** cebruns has quit IRC | 14:43 | |
| *** janonymous has quit IRC | 14:43 | |
| *** cebruns has joined #openstack-swift | 14:44 | |
| *** janonymous_ has quit IRC | 14:45 | |
| *** amandap has joined #openstack-swift | 14:45 | |
| *** dfg_ has quit IRC | 14:46 | |
| acoles | haypo: jenkins doesn't like your last change :( its the change in pbr that caused me problems, the pbr version in requirements is ok i think | 14:46 |
| acoles | argh. its the change in pbr *in setup.py* that caused me problems | 14:46 |
| *** petertr7_away is now known as petertr7 | 14:46 | |
| *** dfg has joined #openstack-swift | 14:46 | |
| *** peluse_ has quit IRC | 14:47 | |
| *** kota_ has joined #openstack-swift | 14:48 | |
| *** peluse has joined #openstack-swift | 14:51 | |
| haypo | acoles: vm-saio-probe looks to use an old version of pip -- https://8b86aea46fb38e6450f2-0e5f4c086da474abc1df58826577db2f.ssl.cf1.rackcdn.com/217423/731/probetests/console.txt | 14:52 |
| haypo | RequirementParseError: Expected ',' or end-of-list in dnspython>=1.12.0;python_version<'3.0' at ;python_version<'3.0' | 14:52 |
| haypo | it doesn't support environment markers | 14:52 |
| haypo | acoles: i don't think that changing setup.py would fix vm-saio-probe | 14:53 |
| acoles | haypo: i was referring to gate-swift-requirements | 14:54 |
| acoles | vm-saio-probe isn't voting | 14:54 |
| haypo | acoles: ah | 14:54 |
| acoles | haypo: its weird that failure doesnt show in the top level jenkins report | 14:55 |
| acoles | so you have to look in the comment | 14:55 |
| openstackgerrit | Victor Stinner proposed openstack/swift: On py3, use dnspython3 dependency, not dnspython https://review.openstack.org/217423 | 14:56 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Re-organise ssync tests https://review.openstack.org/220198 | 14:56 |
| acoles | review, merge, rebase, repeat... | 14:56 |
| haypo | acoles: jenkins failed because i wrote "pbr>=1.0" whereas global requirements require to write "pbr>=1.6" | 14:56 |
| acoles | yes | 14:57 |
| haypo | acoles: i restored "pbr>=1.6" in requirements.txt | 14:57 |
| haypo | acoles: i wrote pbr>=1.0 because you wrote that you got issues with an older version pbr version, but you had pbr >= 1.0 | 14:57 |
| *** pdardeau has joined #openstack-swift | 14:57 | |
| haypo | acoles: in fact, i have no idea of what i'm doing :-D i'm trying random changes until it works... | 14:58 |
| *** hezhiqiang has quit IRC | 14:58 | |
| acoles | haypo: ah, good, so its not just me that has no ideas :) | 14:58 |
| haypo | haha | 14:58 |
| haypo | acoles: pbr is a blackbox for me | 14:58 |
| haypo | (distutils, setuptools, pip, pbr) | 14:58 |
| acoles | haypo: so, i agree we need pbr>=1.0 to get the env marker support, and openstack dictates therefore it must be pbr>=1.6.0 in requirements.txt | 14:59 |
| acoles | haypo: ditto! | 14:59 |
| acoles | haypo: i don't understand the pbr>=1.8.1 requirement in setup.py, i just know it broke omm. | 15:00 |
| *** haomaiwa_ has quit IRC | 15:01 | |
| *** haomaiwang has joined #openstack-swift | 15:01 | |
| acoles | haypo: after playing some more, the pbr>=1.8.1 requirement in setup.py breaks when i have pbr==1.6.0 | 15:02 |
| acoles | or older | 15:02 |
| acoles | so it seems incompatible with having pbr>=1.6 in requirements.txt. either that or i am dumb! | 15:04 |
| *** david-lyle has quit IRC | 15:06 | |
| haypo | acoles: lifeless tried once to explain me this issue. in short, setuptools is unable to update itself. so indirectly setup.py cannot update pbr | 15:06 |
| haypo | acoles: it's not an incompatibility in requirements, but an annoying issue: you have to update _manually_ pbr on your system | 15:07 |
| *** janonymous has joined #openstack-swift | 15:09 | |
| acoles | haypo: so it seems. i am just aware that swift doc recommends sudo pip install -r requirements.txt ; sudo python setup.py develop and that may no longer be sufficient | 15:11 |
| *** janonymous_ has joined #openstack-swift | 15:11 | |
| acoles | http://docs.openstack.org/developer/swift/development_saio.html#getting-the-code | 15:11 |
| *** rohit_ has joined #openstack-swift | 15:12 | |
| acoles | which makes me wonder (like cschwede did on the review) why requirements.txt does not have same pbr version as setup.py | 15:12 |
| haypo | acoles: "why requirements.txt does not have same pbr version as setup.py" it's mandatory, see global requirements | 15:13 |
| openstackgerrit | Merged openstack/swift: Encrypting/Decrypting object metadata https://review.openstack.org/203310 | 15:13 |
| haypo | acoles: initially, i just ran "tox -e update ../swift" from the requirements project | 15:13 |
| *** blmartin has joined #openstack-swift | 15:14 | |
| *** wbhuber has joined #openstack-swift | 15:16 | |
| acoles | haypo: http://docs.openstack.org/developer/requirements/ doesn't help me undertand what goes into setup.py | 15:17 |
| *** sanchitmalhotra has joined #openstack-swift | 15:22 | |
| *** sanchitmalhotra has quit IRC | 15:24 | |
| *** jrichli has joined #openstack-swift | 15:27 | |
| *** lcurtis has joined #openstack-swift | 15:28 | |
| haypo | acoles: it looks like setup.py is simply hardcoded here, https://github.com/openstack/requirements/blob/master/openstack_requirements/cmds/update.py#L38 | 15:30 |
| *** NM has quit IRC | 15:31 | |
| acoles | haypo: interesting. thanks for link | 15:33 |
| *** NM has joined #openstack-swift | 15:34 | |
| *** hseipp has joined #openstack-swift | 15:35 | |
| *** hseipp has quit IRC | 15:35 | |
| *** hseipp has joined #openstack-swift | 15:36 | |
| *** diogogmt has joined #openstack-swift | 15:37 | |
| *** juzuluag has joined #openstack-swift | 15:38 | |
| *** diogogmt has quit IRC | 15:49 | |
| *** diogogmt has joined #openstack-swift | 15:49 | |
| *** jlhinson has joined #openstack-swift | 15:51 | |
| *** peterlisak has quit IRC | 15:53 | |
| *** jistr is now known as jistr|afkmtg | 15:53 | |
| *** janonymous has quit IRC | 15:58 | |
| *** vinsh has quit IRC | 16:00 | |
| *** haomaiwang has quit IRC | 16:01 | |
| jrichli | acoles: thanks for the merge! | 16:01 |
| *** haomaiwang has joined #openstack-swift | 16:01 | |
| acoles | jrichli: ok. i am just working on the container listing patch | 16:01 |
| jrichli | acoles: you rock! | 16:02 |
| hrou | jrichli, so I was half way through and with just 1 silly comment, so +1 great work ; ) If anything interesting I'll just propose a patch | 16:02 |
| acoles | hrou: ah, half way through which? | 16:03 |
| hrou | acoles, oh the big one you just did patch 203310 (but really don't worry at all, it looked great) | 16:04 |
| patchbot | hrou: https://review.openstack.org/#/c/203310/ - Encrypting/Decrypting object metadata | 16:04 |
| acoles | hrou: oh sorry, yeah just propose patches now. | 16:04 |
| briancline | even in the sid packages? | 16:04 |
| briancline | oops | 16:04 |
| briancline | wrong window | 16:04 |
| hrou | acoles, yep! Oh no worries at all, its really great work | 16:04 |
| jordanP | notmyname, hi. Any chance you had time to review https://review.openstack.org/#/c/235933/ (Fix testing issues) ? | 16:05 |
| *** minwoob has joined #openstack-swift | 16:05 | |
| acoles | hrou jrichli it beats filing expenses! | 16:05 |
| jrichli | yea, I still gotta do that too | 16:06 |
| hrou | jrichli, acoles errrr don't remind me, its actually a pretty big pain because of all the silly subway recites I have : ) The rest is pretty easy | 16:07 |
| hrou | given we do per diem | 16:08 |
| *** jerrygb has quit IRC | 16:13 | |
| *** tongli has joined #openstack-swift | 16:14 | |
| *** klrmn1 has joined #openstack-swift | 16:15 | |
| *** jerrygb has joined #openstack-swift | 16:15 | |
| *** janonymous has joined #openstack-swift | 16:16 | |
| *** jerrygb has quit IRC | 16:31 | |
| *** jerrygb has joined #openstack-swift | 16:31 | |
| *** jrichli has quit IRC | 16:34 | |
| haypo | acoles: i'm not sure that i understood your last comment on my patch set 5, but the patch set 6 got a +1 for Jenkins: https://review.openstack.org/#/c/217423/ | 16:38 |
| notmyname | good morning | 16:40 |
| briancline | does anyone else see the memcache client in proxy-server irreversibly crapping out under high load, and only fixed with a proxy reload? | 16:41 |
| haypo | notmyname: hi. today i noticed that for the case of cinder, we have the complete list of patches to port cinder to python 3: https://blueprints.launchpad.net/cinder/+spec/cinder-python3 | 16:41 |
| haypo | notmyname: maybe it can give you a better estimation of how much work is required to port swift. cinder is almost fully ported to py3 (at least, unit tests) | 16:42 |
| notmyname | thanks | 16:42 |
| haypo | hum, it looks like i'm working on porting cinder to py3 since june, so 6 months | 16:43 |
| haypo | most changes are small. today i wrote a change which only adds a single line :) | 16:43 |
| haypo | https://review.openstack.org/#/c/242142/1/cinder/tests/unit/test_tintri.py | 16:43 |
| janonymous | i could register a blueprint just for tracking purpose if it's ok ? | 16:44 |
| *** ahale_ has joined #openstack-swift | 16:51 | |
| *** chrisnelson_ has joined #openstack-swift | 16:52 | |
| *** _hrou_ has joined #openstack-swift | 16:52 | |
| notmyname | janonymous: I'm not sure if that would help at this point. | 16:52 |
| *** nadeem has joined #openstack-swift | 16:53 | |
| *** dosaboy_ has joined #openstack-swift | 16:54 | |
| *** nadeem has quit IRC | 16:54 | |
| *** balajir has joined #openstack-swift | 16:54 | |
| *** torgomatic_ has joined #openstack-swift | 16:54 | |
| *** StevenK_ has joined #openstack-swift | 16:54 | |
| *** hrou has quit IRC | 16:54 | |
| *** nadeem has joined #openstack-swift | 16:55 | |
| *** diazjf has joined #openstack-swift | 16:56 | |
| *** barra204 has joined #openstack-swift | 16:58 | |
| *** aerwin3_ has joined #openstack-swift | 16:59 | |
| *** dhellmann_ has joined #openstack-swift | 17:00 | |
| *** diazjf has quit IRC | 17:00 | |
| *** haomaiwang has quit IRC | 17:01 | |
| janonymous | Okay. | 17:01 |
| *** haomaiwang has joined #openstack-swift | 17:01 | |
| *** chsc has joined #openstack-swift | 17:03 | |
| *** dhellmann has quit IRC | 17:04 | |
| *** aerwin3 has quit IRC | 17:04 | |
| *** balajir_ has quit IRC | 17:04 | |
| *** chrisnelson has quit IRC | 17:04 | |
| *** torgomatic has quit IRC | 17:04 | |
| *** StevenK has quit IRC | 17:04 | |
| *** dosaboy has quit IRC | 17:04 | |
| *** shakamunyi has quit IRC | 17:04 | |
| *** ahale has quit IRC | 17:04 | |
| *** dhellmann_ is now known as dhellmann | 17:05 | |
| *** hseipp has quit IRC | 17:06 | |
| *** garthb has joined #openstack-swift | 17:08 | |
| *** hseipp has joined #openstack-swift | 17:09 | |
| *** lpabon has quit IRC | 17:11 | |
| *** jistr|afkmtg is now known as jistr | 17:14 | |
| *** minwoob has quit IRC | 17:15 | |
| *** rledisez has quit IRC | 17:24 | |
| clayg | heyoh! | 17:31 |
| *** jordanP has quit IRC | 17:32 | |
| mahatic__ | hello! | 17:32 |
| clayg | briancline: how high a load - I think there's some error limiting | 17:33 |
| clayg | briancline: there's some timeouts that are adjustable now too | 17:33 |
| clayg | briancline: tracebacks? | 17:33 |
| clayg | briancline: no | 17:33 |
| *** CaioBrentano has joined #openstack-swift | 17:33 | |
| mahatic__ | acoles: 232572 is failing at the gate because the metadata dict is ordering itself into this {"cipher": "AES_CTR_256", "iv": base64.b64encode(iv)} and failing the assertion. | 17:33 |
| clayg | acoles: bah expense reports :\ | 17:34 |
| clayg | mahatic__: assert on sorted(dict.items()) ?? | 17:34 |
| mahatic__ | clayg: we don't want it sorted. It should be this {"iv": base64.b64encode(iv), "cipher": "AES_CTR_256"} | 17:35 |
| acoles | mahatic__: yeah there's a bad test in there #willfix | 17:35 |
| mahatic__ | acoles: :) maybe using the crypto meta directly in the assertion i guess? | 17:36 |
| acoles | mahatic__: i don't think we can know the order of the dumped dict | 17:36 |
| mahatic__ | acoles: actually it orders itself even before the dumps | 17:36 |
| mahatic__ | >>> {"iv": base64.b64encode(iv), "cipher": "AES_CTR_256"} will result in {"cipher": "AES_CTR_256", "iv": base64.b64encode(iv)} | 17:37 |
| acoles | mahatic__: ? where's that | 17:38 |
| mahatic__ | acoles: just on the console | 17:38 |
| mahatic__ | apparently "Items stored in a dictionary do not have any inherent order. The order they are printed out is entirely down to the hash values for each of the keys and the other items in the dictionary." | 17:39 |
| acoles | mahatic__: exactly. and the order may vary from machine to machine | 17:39 |
| mahatic__ | acoles: oh, so that's the reason it passed the first time and didn't the second! | 17:39 |
| acoles | so works omm but not jenkins. | 17:39 |
| mahatic__ | works on my machine too | 17:39 |
| acoles | i guess! | 17:40 |
| acoles | mahatic__: i haven't had chance to look at the detail but something like what clayg said is likely to be the fix | 17:41 |
| *** wbhuber_ has joined #openstack-swift | 17:41 | |
| acoles | mahatic__: i want to get the container listing patch clean and pushed before end of day, i'll look at that failing test tomorrow (if you don't fix it first :D) | 17:42 |
| mahatic__ | clayg: I now get what you said :D my bad! (I completedly blinded out 'assert on') | 17:43 |
| mahatic__ | acoles: sure! good luck on the container listing goal :D | 17:44 |
| acoles | mahatic__: its looking good so far... | 17:44 |
| *** wbhuber has quit IRC | 17:44 | |
| mahatic__ | nice | 17:45 |
| acoles | ugh, xml :( | 17:47 |
| mahatic__ | yup, that's the tricky part | 17:48 |
| clayg | i think today is the day to review fast-POST - wish me luck | 17:50 |
| notmyname | good luck! | 17:50 |
| mahatic__ | good luck | 17:51 |
| *** arnox has quit IRC | 17:55 | |
| *** diazjf has joined #openstack-swift | 17:58 | |
| *** marcusvrn_ has joined #openstack-swift | 17:59 | |
| *** jistr has quit IRC | 18:00 | |
| *** haomaiwang has quit IRC | 18:01 | |
| *** haomaiwang has joined #openstack-swift | 18:01 | |
| *** wbhuber_ has quit IRC | 18:02 | |
| *** wbhuber has joined #openstack-swift | 18:02 | |
| *** wbhuber has quit IRC | 18:07 | |
| *** wbhuber has joined #openstack-swift | 18:08 | |
| *** daemontool has quit IRC | 18:09 | |
| lifeless | acoles: you could add pbr to your requirements.txt which would address that | 18:09 |
| lifeless | acoles: but also be aware that we've been recommending to use pip install -e . rather than setup.py develop for a year or more now | 18:10 |
| acoles | lifeless: pbr is in requirements.txt. the piece thats confusing me (and maybe others) is why we have (proposed) pbr>=1.6 in requirements.txt and pbr>=1.8 in setup.py | 18:11 |
| acoles | lifeless: ack re. pip install -e | 18:12 |
| lifeless | acoles: ah, because setup.py describes build requirements, and requirements.txt describes the runtime API | 18:12 |
| lifeless | acoles: the runtime API hasn't had bugfixes since 1.6 | 18:13 |
| notmyname | isn't pbr just a build-time requirement? | 18:13 |
| lifeless | notmyname: not for many projects. It has both a runtime and build time API | 18:14 |
| lifeless | notmyname: the runtime API abstracts out the difference between pkg_resources version metadata (used for installed cases) and git based version metadata (used for uninstalled cases) | 18:14 |
| lifeless | notmyname: [and needless to say, super stable because of its use everywhere....] | 18:15 |
| * mahatic__ calls it a night | 18:15 | |
| * acoles wishes clayg luck, and braces himself | 18:15 | |
| notmyname | mahatic__: good night | 18:15 |
| acoles | mahatic__: g'night | 18:16 |
| notmyname | mahatic__: oh wait a sec | 18:16 |
| lifeless | notmyname: we're working on the issue with setup-requires in setuptools with the new build system PEP for Python | 18:16 |
| mahatic__ | thanks, good day/evening | 18:16 |
| mahatic__ | notmyname: what's up? | 18:16 |
| *** nadeem has quit IRC | 18:16 | |
| notmyname | mahatic__: pm sent | 18:16 |
| lifeless | notmyname: once thats done everything should work a -whole lot- better | 18:16 |
| *** diazjf has quit IRC | 18:19 | |
| clayg | anyone care about useless micro-optimization bike-shedding want to sanity check that my results that everything in python is slower than you expect -> https://review.openstack.org/#/c/241527/ | 18:24 |
| clayg | acoles: yeah pip install -e is the bomb? what is this setup.py you speak of? | 18:24 |
| clayg | like I mean there was this one time (it was more like a decade) the entire python community got by with setup.py develop - but then openstack came to python - and the world had to move | 18:26 |
| clayg | ^ redbo for you bra | 18:26 |
| notmyname | my muscle memory still uses `python ./setup.py develop` | 18:27 |
| *** mahatic__ has quit IRC | 18:27 | |
| *** haomaiwa_ has joined #openstack-swift | 18:27 | |
| *** haomaiwang has quit IRC | 18:28 | |
| acoles | notmyname: our docs do too http://docs.openstack.org/developer/swift/development_saio.html#getting-the-code | 18:34 |
| acoles | clayg: i'm so last year ;) | 18:35 |
| *** proteusguy_ has quit IRC | 18:36 | |
| *** mzhou has joined #openstack-swift | 18:36 | |
| notmyname | clayg: https://gist.github.com/notmyname/19a9ff6c8a037f471e87 | 18:37 |
| *** petertr7 is now known as petertr7_away | 18:37 | |
| clayg | notmyname: do you know what any of that means!? | 18:39 |
| notmyname | clayg: I just updated it so the any() call is a little more clear in the disassembly | 18:40 |
| clayg | i mean the any one looks like less instructions - but i'm pretty sure my timeit demonstrates it's slower? | 18:40 |
| notmyname | ie split it into more lines | 18:40 |
| notmyname | yeah | 18:40 |
| notmyname | BUILD_TUPLE and MAKE_CLOSURE and especially the 2 CALL_FUNCTION look really suspicious for the slow any() | 18:41 |
| lifeless | notmyname: so the issue with that isn't coming from pbr btw | 18:41 |
| notmyname | clayg: received wisdom for python is that "function calls are slow" | 18:42 |
| lifeless | notmyname: its a general upstream python packaging consensus | 18:42 |
| *** amoturi has joined #openstack-swift | 18:44 | |
| *** zhill has joined #openstack-swift | 18:44 | |
| *** dabukalam has quit IRC | 18:45 | |
| *** hseipp has quit IRC | 18:45 | |
| clayg | notmyname: yeah I guess you've got the <genexp>.__next__ called multiple times? | 18:45 |
| *** mandarine has quit IRC | 18:46 | |
| clayg | notmyname: hey and I know since iterating generators is super fast let's build a co-routine runtime around the idea! | 18:46 |
| *** dabukalam has joined #openstack-swift | 18:46 | |
| *** mandarine has joined #openstack-swift | 18:46 | |
| clayg | notmyname: so which do you find more readable? | 18:46 |
| notmyname | clayg: cool! and maybe we could have a convenient short name for it. every()? or all_of_them()? or any_of_them()? surely there's a shorter one ;-) | 18:47 |
| clayg | notmyname: I think if you make it a helper function it goes back to being the same as any :\ | 18:49 |
| *** proteusguy_ has joined #openstack-swift | 18:49 | |
| notmyname | yeah :-) | 18:49 |
| notmyname | that's a good short name | 18:49 |
| clayg | lol | 18:50 |
| notmyname | I don't have any preference between any() or the for loop. if forced, I'd probably slightly prefer the for loop, but I really don't think it matters | 18:53 |
| notmyname | but looking at that patch, the any() is probably worse if it's actually calling unquote() multiple times (ie len(forbidden chars) times) | 18:53 |
| *** mfalatic_ has joined #openstack-swift | 18:56 | |
| *** jerrygb has quit IRC | 18:57 | |
| *** jerrygb has joined #openstack-swift | 18:58 | |
| *** mfalatic has quit IRC | 19:00 | |
| *** haomaiwa_ has quit IRC | 19:01 | |
| *** haomaiwang has joined #openstack-swift | 19:01 | |
| clayg | I don't think it's a lambda there - it only gets invoked once | 19:01 |
| notmyname | wow | 19:02 |
| notmyname | $ python ./speed.py test_loop | 19:02 |
| notmyname | 0.835028171539 | 19:02 |
| notmyname | $ python ./speed.py test_any | 19:02 |
| notmyname | 1.45459890366 | 19:02 |
| clayg | oh... no - it's probably every time | 19:02 |
| notmyname | $ pypy ./speed.py test_loop | 19:02 |
| notmyname | 0.174441099167 | 19:02 |
| notmyname | $ pypy ./speed.py test_any | 19:02 |
| notmyname | 0.423096895218 | 19:02 |
| clayg | well that's an order of magnitude | 19:03 |
| *** jerrygb has quit IRC | 19:03 | |
| clayg | well - i guess not quite | 19:03 |
| clayg | I CAN MATH! | 19:03 |
| notmyname | that was with pypy 2.2.1. installing pypy 4.0.0 now | 19:04 |
| clayg | i guess in either case it'd be useful to assign the unquoted path to temp var | 19:05 |
| acoles | notmyname: clayg i think i know what you are discussing, i nearly dinged that patch for having the unquote(req.path) in the any, maybe i should have :/ | 19:05 |
| notmyname | yeah. I added an identity function to do just that in my test | 19:05 |
| clayg | acoles: lol! | 19:06 |
| notmyname | pypy 4 is about 2x faster than pypy 2.2.1 | 19:06 |
| clayg | acoles: this code was working *fine* before - why is *this* keeping me from fast-POST! WHYYYYY?! | 19:06 |
| notmyname | on the any() | 19:06 |
| clayg | (hontestly i'm planing with my ring-check spread sheets - but still) | 19:07 |
| notmyname | $ pypy ./speed.py test_loop | 19:07 |
| notmyname | 0.140868902206 | 19:07 |
| notmyname | $ pypy ./speed.py test_any | 19:07 |
| notmyname | 0.203679084778 | 19:07 |
| notmyname | not too much difference between the two with latest pypy. or at least not nearly as much as with python or earlier pypy | 19:07 |
| *** diazjf has joined #openstack-swift | 19:07 | |
| notmyname | all tests run on my mac | 19:08 |
| *** dustins has quit IRC | 19:08 | |
| *** jrichli has joined #openstack-swift | 19:10 | |
| clayg | what does STM buy you again? When you do a thread they can both run on different cpus at the same time or something? | 19:16 |
| lifeless | yes | 19:17 |
| lifeless | the idea is you run speculatively and throw away invalidated results | 19:18 |
| *** _hrou_ is now known as hrou | 19:19 | |
| acoles | jrichli: with the container listing patch, did you see additional func test failures? | 19:19 |
| * acoles hopes for a yes | 19:20 | |
| jrichli | I haven't tested with the latest container listing patch yet. is that what you mean? or do you want to know the number of failures before you started working on it? | 19:20 |
| acoles | jrichli: i see container listing func tests fail. i know why, i'm just not sure if i broke them with my changes or it was always the case. i suspect always the case, but idk | 19:22 |
| jrichli | the last time I had tested that patch, it did not break more tests from the patch it depended on. | 19:23 |
| jrichli | but there had been several rebases since then | 19:23 |
| acoles | oh yes! | 19:23 |
| jrichli | eureka! | 19:23 |
| jrichli | :-) | 19:23 |
| acoles | jrichli: ? what happened? | 19:24 |
| clayg | goal #3 for STM sounds interesting - http://pypy.org/tmdonate2.html#goal-3 - transparently adapting to evented framework to managed threads for multi-core parallelization | 19:24 |
| jrichli | acoles: nothing, just in the spirit of the "oh yes" | 19:24 |
| jrichli | sorry to get you excited | 19:24 |
| acoles | jrichli: i was saying 'oh yes there have been many rebases (sigh)' | 19:25 |
| jrichli | :-) that is one bad thing about not having the new middleware in the gate. but then everything will fail anyway. is there a way to give the gate a baseline number of failures :-p | 19:26 |
| acoles | jrichli: so the way we append crypto_meta to content-type as a param breaks in the container server, because it tries to parse the content-type and throws out bad param fields. | 19:28 |
| acoles | meaning the crypto_meta gets lost from the listing response :/ | 19:28 |
| jrichli | acoles: if I recall, there is some logic somewhere in the backend that removes anything after a ";" in content-type | 19:29 |
| briancline | is that prioritized right? it's only got 218 objects in there... | 19:29 |
| briancline | ack | 19:29 |
| briancline | wrong window again | 19:29 |
| acoles | briancline: heh twice today ;) | 19:29 |
| briancline | acoles: yeah, I'm doing great with computers today | 19:30 |
| jrichli | acoles: i mean in the proxy obj controller | 19:31 |
| jrichli | acoles: nm. i think that is only for startswith('swift_bytes=') | 19:33 |
| *** petertr7_away is now known as petertr7 | 19:34 | |
| acoles | jrichli: ^^ yes but in container server. while parsing and looking for swift_bytes it throws away our crypto_meta :/ so all we see in listing is "<content-type>;meta=" | 19:34 |
| jrichli | darn swift_bytes! | 19:35 |
| pdardeau | clayg: what is STM? | 19:39 |
| clayg | pdardeau: http://doc.pypy.org/en/latest/stm.html#software-transactional-memory | 19:40 |
| pdardeau | clayg: ah, nice. thx | 19:40 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Decrypting Container Listing https://review.openstack.org/214438 | 19:44 |
| acoles | jrichli: ^^ thats as far as i got for today, will have to revisit the meta handling tomorrow. there will be a way. | 19:44 |
| jrichli | acoles: ok, thanks. i will think about the content-type formating | 19:45 |
| acoles | jrichli: its still worth reviewing the patch, it should be otherwise sane and ready. | 19:47 |
| acoles | hrou mahati ^^ | 19:47 |
| hrou | acoles, this time I will review before anyone else lol probably not | 19:47 |
| jrichli | got it, will do | 19:47 |
| *** dustins has joined #openstack-swift | 19:51 | |
| *** acoles is now known as acoles_ | 19:51 | |
| *** jerrygb has joined #openstack-swift | 19:53 | |
| *** sujit has joined #openstack-swift | 19:56 | |
| openstackgerrit | Paul Dardeau proposed openstack/swift: Add unit tests to cover print_item_locations https://review.openstack.org/240961 | 19:57 |
| *** sujit is now known as sfs_ | 19:59 | |
| *** sfs_ is now known as suythf_ | 19:59 | |
| *** haomaiwang has quit IRC | 20:01 | |
| *** haomaiwang has joined #openstack-swift | 20:01 | |
| *** alejandrito has joined #openstack-swift | 20:02 | |
| *** jrichli has quit IRC | 20:16 | |
| openstackgerrit | Ondřej Nový proposed openstack/swift: Compare Swift config checksum in swift-recon --all https://review.openstack.org/240722 | 20:18 |
| *** jrichli has joined #openstack-swift | 20:20 | |
| openstackgerrit | Ondřej Nový proposed openstack/swift: Compare Swift config checksum in swift-recon --all https://review.openstack.org/240722 | 20:21 |
| *** janonymous has quit IRC | 20:21 | |
| *** daemontool has joined #openstack-swift | 20:22 | |
| *** rohit_ has quit IRC | 20:24 | |
| *** diazjf has quit IRC | 20:25 | |
| *** jlhinson_ has joined #openstack-swift | 20:30 | |
| *** jlhinson has quit IRC | 20:30 | |
| *** suythf_ has quit IRC | 20:32 | |
| *** CaioBrentano has quit IRC | 20:33 | |
| *** janonymous has joined #openstack-swift | 20:35 | |
| *** amoturi has quit IRC | 20:37 | |
| clayg | i really need to come up with a way to "find a part that can go in this hole" - instead of gather up some random parts form devices that have too many and then try and see if you can fit them somewhere better than where you got them from | 20:44 |
| clayg | I'm sure I can come up with replica2part2dev that will never balance if you only gather the parts from full devices - like there is *no* part on a full device that could possibly fit onto the hungry dev w/o giving up dispersion | 20:45 |
| clayg | I think | 20:45 |
| clayg | maybe *sure* is too strong | 20:46 |
| clayg | anyway - in my dataset I've got ~25 stupid rings that won't balance (worst is at %7 and the required overload is %4 so it's not a great topology) - but it keeps reassinging %5 of it's parts every crank and not really making any progress | 20:47 |
| clayg | *generally* this is fine, if the balance doesn't improve by > %1 you won't even save the changes w/o -f | 20:48 |
| clayg | but it just bothers me to no end :'( | 20:49 |
| clayg | also the real values are stupid low - that %3 difference on the worst device from the required overload and the current balance works out to ~11 parts across the entire cluster that "could" be on better homes. | 20:52 |
| openstackgerrit | Paul Dardeau proposed openstack/swift: Added unit tests for ringbuilder command-line utility https://review.openstack.org/240076 | 20:53 |
| clayg | no wait... I think i'm counting that wrong :\ | 20:55 |
| *** lpabon has joined #openstack-swift | 20:56 | |
| openstackgerrit | Paul Dardeau proposed openstack/swift: Add unit tests to cover print_item_locations https://review.openstack.org/240961 | 20:59 |
| *** haomaiwang has quit IRC | 21:01 | |
| *** haomaiwang has joined #openstack-swift | 21:01 | |
| *** mzhou_ has joined #openstack-swift | 21:03 | |
| *** mzhou has quit IRC | 21:03 | |
| *** mzhou_ is now known as mzhou | 21:03 | |
| *** NM has quit IRC | 21:09 | |
| openstackgerrit | Bill Huber proposed openstack/swift: Add unit tests for swift.account.reaper https://review.openstack.org/239105 | 21:20 |
| *** diazjf has joined #openstack-swift | 21:21 | |
| *** NM has joined #openstack-swift | 21:27 | |
| openstackgerrit | Paul Dardeau proposed openstack/swift: Added unit tests for ringbuilder command-line utility https://review.openstack.org/240076 | 21:34 |
| *** mwheckmann has joined #openstack-swift | 21:37 | |
| notmyname | clayg: do you have a --force-save-this-even-if-not-much-moved option in your version of the balancer? I know wer | 21:39 |
| notmyname | I know we've talked about it before | 21:39 |
| clayg | notmyname: it's already in there | 21:40 |
| clayg | notmyname: and I didn't break it | 21:40 |
| notmyname | ah, cool | 21:40 |
| *** NM has quit IRC | 21:40 | |
| mwheckmann | hi all. I'm in process of upgrading to Liberty and needed to make some ring changes. I'm seeing that errors in swift-ring-builder rebalance: | 21:41 |
| *** NM has joined #openstack-swift | 21:42 | |
| mwheckmann | File "/usr/lib/python2.7/site-packages/swift/common/ring/builder.py", line 1065, in _gather_reassign_parts for tier in tfd[dev['id']]: | 21:42 |
| mwheckmann | TypeError: 'NoneType' object has no attribute '__getitem__' | 21:42 |
| mwheckmann | This is what my account ring looked like: | 21:45 |
| mwheckmann | 0 1 1 172.16.0.11 6002 172.16.0.11 6002 SSD1 100.00 196608 0.00 | 21:45 |
| mwheckmann | 1 1 1 172.16.0.12 6002 172.16.0.12 6002 SSD2 100.00 196608 0.00 | 21:45 |
| mwheckmann | 2 1 1 10.131.0.13 6002 10.131.0.13 6002 SSD3 0.00 0 0.00 | 21:45 |
| mwheckmann | The only change I made was to set the weight of the SSD3 device to 25 (I also tried other weights but no difference). | 21:46 |
| mwheckmann | I can't seem to find any known/related bug in launchpad | 21:48 |
| notmyname | mwheckmann: are the errors with the upgraded code or the old code? and what version of swift are you using? | 21:49 |
| mwheckmann | notmyname: It's from the RDO Liberty packages: openstack-swift-2.5.0-1.el7.noarch | 21:50 |
| notmyname | ok | 21:50 |
| mwheckmann | do you want a Launchpad bug for this? | 21:51 |
| openstackgerrit | Bill Huber proposed openstack/swift: ObjectControllers return application errors as 499 on bad read https://review.openstack.org/236007 | 21:52 |
| notmyname | mwheckmann: you added the drive with a zero weight, then you tried to set the weight up. is that right? | 21:52 |
| wbhuber | clayg: ^^ i think the same that we need to leave timeout there until further review | 21:53 |
| notmyname | I want to try to duplicate it | 21:53 |
| mwheckmann | notmyname: close. I decreased the weight to get data off so that I could re-add it with another IP address. | 21:53 |
| mwheckmann | However, remove doesn't seem to work. | 21:54 |
| mwheckmann | I then discovered the "set_info" command and used that to change the IP. | 21:54 |
| mwheckmann | both IPs (replication + regular) | 21:54 |
| mwheckmann | All I wanted to do now was re-add weight and re-balance | 21:55 |
| notmyname | ok, I've got a 3-device ring locally now. you set the weight to zero, rebalanced, then set it higher again and then the rebalance failed? | 21:58 |
| mwheckmann | exactly but with an IP change using set_info before setting the weight higher | 21:58 |
| mwheckmann | notmyname: replica count is 3 | 21:59 |
| * clayg wonders if this bug is already fixed on fix-rings-for-realz | 22:00 | |
| clayg | mwheckmann: can you publish your builder? | 22:00 |
| *** lpabon has quit IRC | 22:00 | |
| *** haomaiwang has quit IRC | 22:01 | |
| mwheckmann | clayg: sure. Where should I do that? | 22:01 |
| *** haomaiwang has joined #openstack-swift | 22:01 | |
| clayg | mwheckmann: good question | 22:01 |
| mwheckmann | I'll just do it on my Google Drive | 22:01 |
| clayg | notmyname: ^ ? | 22:01 |
| mwheckmann | give me a minute | 22:02 |
| notmyname | nope. I don't know. google drive is fine | 22:02 |
| notmyname | maybe if you had a swift cluster.... ;-) | 22:02 |
| notmyname | mwheckmann: FWIW, my first attempt to recreate it didn't have any errors | 22:02 |
| clayg | notmyname: oh that's an ok point - I could make a PUT tempurl | 22:02 |
| notmyname | aside from "hey you want 3 replicas but now you only have 2 drives" | 22:03 |
| clayg | notmyname: mwheckmann: the only way that dev's end up being None is if you *remove* them setting to zero weight shouldn't have been sufficient to trigger that bug | 22:03 |
| clayg | acoles_: I don't understand the TODO in the commit about allowed_headers metadata - if it gets written to the .meta file it should be fine (the e.g. content-disposition should be written to the .meta file) | 22:06 |
| *** petertr7 is now known as petertr7_away | 22:07 | |
| mwheckmann | notmyname: https://goo.gl/pIR5Z5 --> On Swift :) | 22:10 |
| mwheckmann | clayg: notmyname: I did attempt to remove the device as well. But after running rebalance the device was still in the ring with zero weight as before. | 22:12 |
| clayg | mwheckmann: idk, sounds broken | 22:13 |
| notmyname | mwheckmann: yup. rebalance on that one locally gives me an error | 22:14 |
| mwheckmann | good. I'm not crazy :) | 22:15 |
| *** jrichli has quit IRC | 22:16 | |
| *** wbhuber has quit IRC | 22:17 | |
| *** petertr7_away is now known as petertr7 | 22:17 | |
| *** wbhuber has joined #openstack-swift | 22:17 | |
| clayg | mwheckmann: well the device is definately marked for deletion :\ | 22:20 |
| *** daemontool has quit IRC | 22:21 | |
| clayg | it's getting removed and things go nuts | 22:21 |
| mwheckmann | ok. That would explain it. Then the real bug is the fact that it doesn not actually get removed during rebalance | 22:21 |
| notmyname | I've got a meeting in about 5 minutes, so I won't be able to look at this for much longer | 22:21 |
| *** wbhuber has quit IRC | 22:22 | |
| *** mzhou has quit IRC | 22:22 | |
| mwheckmann | I'm about to go pick up my kids. But I don't mind catching up later. | 22:22 |
| clayg | mwheckmann: you have 2 more mins? | 22:23 |
| mwheckmann | yes | 22:23 |
| mwheckmann | at least 15 more :) | 22:23 |
| clayg | mwheckmann: well it *does* get removed, that's what causes the bug :\ | 22:23 |
| clayg | I think maybe you tried to rebalance after the remove and things went south because the current code doesn't prevent 3 replicas with 2 devices | 22:23 |
| clayg | that left the ring in a wonky state | 22:23 |
| mwheckmann | ok. | 22:24 |
| mwheckmann | but on my object.builder where I have many more devices, I could not remove the device either | 22:24 |
| clayg | anyway - to work around it you need to let it get remvoed - but you'll probably have a add a dummy device | 22:24 |
| clayg | mwheckmann: why not? same traceback? | 22:25 |
| mwheckmann | I didn't try re-adding some weight yet. Shall I ? | 22:25 |
| *** mzhou has joined #openstack-swift | 22:25 | |
| mwheckmann | I'm just pointing out that after running rebalance (after removing the devices), they still showed up in the list | 22:26 |
| *** yuan has quit IRC | 22:26 | |
| *** dustins has quit IRC | 22:26 | |
| mwheckmann | let me try to add some weight to my object ring | 22:27 |
| clayg | well - what I had to do was add a new device with ip 10.131.0.14 then rebalance to let 10.131.0.13 remove (and parts get assigned to 10.131.0.14) | 22:27 |
| clayg | then I fixed 10.131.0.14 with set_info | 22:27 |
| mwheckmann | what to you mean by "fixed 10.131.0.14 with set_info" ? | 22:28 |
| clayg | swift-ring-builder test.builder set_info --id 3 --change-ip 10.131.0.13 | 22:28 |
| mwheckmann | ok. Got it. | 22:28 |
| mwheckmann | same error in the object ring :( | 22:29 |
| clayg | the NoneType thing? | 22:29 |
| mwheckmann | yes. | 22:29 |
| clayg | ok - let me pull down your ring again and switch baack to master | 22:30 |
| mwheckmann | even though I have 24 devices that are available. | 22:30 |
| mwheckmann | I'm assuming that a fix for the object ring would be to add 12 dummy devices and do the same as you did with the account ring? | 22:30 |
| *** gyee has joined #openstack-swift | 22:32 | |
| clayg | did you *remove* 12 devices? | 22:33 |
| mwheckmann | yes. | 22:34 |
| clayg | idk, doesn't look the dummy fix works on master | 22:34 |
| mwheckmann | shit. | 22:34 |
| mwheckmann | I tried to remove the 12 devices because I thought that would be the only way to change the IP of a device. That was before I discovered "set_info" | 22:35 |
| clayg | mwheckmann: ok it worked for me - let me start over from scratch and capture in a paste bin | 22:35 |
| mwheckmann | ok. cool. Much appreciated | 22:35 |
| clayg | mwheckmann: something like this strategy may work for the other ring with the 12 devices -> https://gist.github.com/clayg/6b1ffbb0fec28344ff50 | 22:38 |
| openstackgerrit | Merged openstack/swift: Optimize the code performance https://review.openstack.org/241527 | 22:39 |
| *** badari has quit IRC | 22:40 | |
| *** badari has joined #openstack-swift | 22:40 | |
| mwheckmann | clayg: this was on master? | 22:40 |
| clayg | where's that swiftclient change that outputs content-type with -l already! | 22:40 |
| clayg | mwheckmann: oh crap... yeah I think so? | 22:40 |
| *** petertr7 is now known as petertr7_away | 22:40 | |
| clayg | mwheckmann: reflog says it was -> aae5810 HEAD@{0}: checkout: moving from master to review/alistair_coles/p-fast-post-rebase | 22:41 |
| *** alejandrito has quit IRC | 22:41 | |
| mwheckmann | ok. I will try it later on this evening. I must go. | 22:41 |
| mwheckmann | thanks for your help! | 22:41 |
| clayg | gl | 22:42 |
| clayg | notmyname: fix-rings-for-real messed up on the has three devices but one of them is going to be removed rebalance | 22:43 |
| *** diogogmt has quit IRC | 22:44 | |
| clayg | i added a commit for the fix, but I need to write a test. | 22:45 |
| clayg | ... but I'm reviewing fast-POST GD! | 22:45 |
| clayg | well acctually I just udpated content-type with a POST that didn't have to do a copy and the upate made it's way into the object listing - so I think I'm done right? | 22:45 |
| clayg | :shipit: | 22:45 |
| hrou | Hey clayg for bug 1509429 we ended up writing some func tests to validate most of the APIs (namely the headers returned), would you like to see those uploaded to swift, some unit / func tests to validate some of these headers but these would be a bit more clearer I'd wager ? | 22:47 |
| openstack | bug 1509429 in OpenStack Object Storage (swift) "X-Timestamp missing from object PUT/COPY response headers" [Low,Confirmed] https://launchpad.net/bugs/1509429 | 22:47 |
| *** mwheckmann has quit IRC | 22:48 | |
| clayg | hrou: I love me some functests! | 22:48 |
| hrou | clayg, awesome and the only other little debate we were having is if unit tests were more appropriate in this context ? | 22:49 |
| *** klrmn2 has joined #openstack-swift | 22:50 | |
| *** klrmn1 has quit IRC | 22:50 | |
| *** asettle has joined #openstack-swift | 22:51 | |
| clayg | functest is fine - it's not something that's so likely to regress as much as it's good documentation of the expected behaviors of the api | 22:51 |
| clayg | hrou: if we had some direct to storage server blackbox tests I'd probably... well idk... if it's going out to the client functests are fine | 22:51 |
| clayg | hrou: just push up what you have and then ask me to look at it - it's hard speaking in abstracts | 22:52 |
| hrou | clayg, gotcha yea that makes sense, but nah its all the external APIs really so all through proxy, and agreed they're not likely to change / regress much. | 22:52 |
| hrou | thanks! | 22:52 |
| clayg | thanks for working on it! | 22:52 |
| hrou | clayg, anytime!, not me personally though, new hire on our team ; ) | 22:54 |
| *** km has joined #openstack-swift | 22:55 | |
| *** km is now known as Guest7190 | 22:55 | |
| *** gyee has quit IRC | 22:57 | |
| *** jamielennox|away is now known as jamielennox | 22:59 | |
| *** jlhinson_ has quit IRC | 22:59 | |
| *** haomaiwang has quit IRC | 23:01 | |
| *** haomaiwa_ has joined #openstack-swift | 23:01 | |
| *** jerrygb has quit IRC | 23:08 | |
| openstackgerrit | Paul Dardeau proposed openstack/swift: Added unit tests for ringbuilder command-line utility https://review.openstack.org/240076 | 23:10 |
| *** pdardeau has quit IRC | 23:11 | |
| *** zaitcev has joined #openstack-swift | 23:12 | |
| *** ChanServ sets mode: +v zaitcev | 23:12 | |
| *** diazjf has quit IRC | 23:14 | |
| *** diazjf has joined #openstack-swift | 23:15 | |
| *** janonymous has quit IRC | 23:15 | |
| *** blmartin has quit IRC | 23:19 | |
| *** gyee has joined #openstack-swift | 23:19 | |
| *** StevenK_ is now known as StevenK | 23:21 | |
| *** diazjf has quit IRC | 23:22 | |
| *** kei_yama has joined #openstack-swift | 23:24 | |
| *** hrou has quit IRC | 23:25 | |
| *** jerrygb has joined #openstack-swift | 23:33 | |
| mattoliverau | Hey all from Takayama Japan, yup still here :) | 23:35 |
| *** ujjain- has quit IRC | 23:38 | |
| *** chsc has quit IRC | 23:39 | |
| *** ujjain has joined #openstack-swift | 23:39 | |
| *** ujjain has joined #openstack-swift | 23:39 | |
| briancline | clayg: oops, just saw your question. i'll grab a sample of the tracebacks and error messages | 23:41 |
| *** daemontool has joined #openstack-swift | 23:46 | |
| *** lcurtis has quit IRC | 23:47 | |
| *** jrichli has joined #openstack-swift | 23:47 | |
| *** tongli has quit IRC | 23:48 | |
| *** jerrygb_ has joined #openstack-swift | 23:49 | |
| *** jerrygb has quit IRC | 23:50 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!