| *** hezhiqiang has quit IRC | 00:01 | |
| kairo | ho: http://pastebin.com/GwdDdci3 Do you want I update in the ask? | 00:02 |
|---|---|---|
| ho | kairo: thanks! no it's fine. you got 401 not 403. wait a minutes | 00:03 |
| *** ccavanna has joined #openstack-swift | 00:07 | |
| *** kevinc___ has quit IRC | 00:08 | |
| *** ccavanna has quit IRC | 00:09 | |
| ho | kairo: there is no info in the token. L6 in the debug log and code is here: https://github.com/openstack/keystonemiddleware/blob/master/keystonemiddleware/auth_token/__init__.py#L716 | 00:09 |
| ho | kairo: I'm not sure but your authtoken section in proxy-server.conf doesn't follow https://github.com/openstack/swift/blob/stable/kilo/etc/proxy-server.conf-sample#L284. | 00:14 |
| ho | kairo: can you try this? http://paste.openstack.org/show/421196/ | 00:17 |
| kairo | ho: sure. I got it on restart of proxy-server http://paste.openstack.org/show/421217/ | 00:24 |
| kairo | ho: I'll try. one minute. | 00:24 |
| kairo | ho: the same error. 401 | 00:30 |
| ho | kairo: you replaced the section and got 401. how about "L6 in the debug log"? | 00:37 |
| kairo | ho: sorry, I'm confused. https://github.com/openstack/keystonemiddleware/blob/master/keystonemiddleware/auth_token/__init__.py#L716 My __init__.py don't have this same line. | 00:41 |
| ho | kairo: sorry. in kilo https://github.com/openstack/keystonemiddleware/blob/stable/kilo/keystonemiddleware/auth_token/__init__.py#L632 is the one. | 00:44 |
| *** lyrrad has quit IRC | 00:46 | |
| ho | kairo: I checked the installation guide and realized your configuration is latest way (proxy-server.conf is old...) | 00:46 |
| ho | kairo: please re-replace it. | 00:48 |
| *** nadeem has quit IRC | 00:48 | |
| kairo | ho: this https://git.openstack.org/cgit/openstack/swift/plain/etc/proxy-server.conf-sample?h=stable/kilo | 00:50 |
| kairo | ho: I used this sample. | 00:51 |
| *** lpabon has joined #openstack-swift | 00:53 | |
| *** gyee has quit IRC | 00:55 | |
| kairo | ho: Do you have some proxy-server.conf sample updated? | 00:57 |
| *** lpabon has quit IRC | 00:58 | |
| *** darrenc is now known as darrenc_afk | 01:07 | |
| ho | kairo: i created curl version of head account. http://paste.openstack.org/show/421275/ | 01:11 |
| ho | kairo: is my saio's proxy-server.conf sample fine? | 01:12 |
| *** _hrou_ has joined #openstack-swift | 01:13 | |
| ho | kairo: you use right part of X-Subject-Token: as a token. | 01:13 |
| *** hrou has quit IRC | 01:14 | |
| openstackgerrit | Merged openstack/swift: Make swift-ring-builder filename usage more consistent https://review.openstack.org/209787 | 01:20 |
| kairo | ho: I'll try :) Thanks. | 01:21 |
| *** darrenc_afk is now known as darrenc | 01:21 | |
| kairo | ho: the curl worked well | 01:27 |
| *** sanchitmalhotra1 has joined #openstack-swift | 01:33 | |
| *** nexusz99_ has joined #openstack-swift | 01:35 | |
| *** nexusz99 has quit IRC | 01:35 | |
| ho | kairo: means "head account worked well"? | 01:35 |
| *** sanchitmalhotra has quit IRC | 01:35 | |
| ho | kairo: my saio's setting regarding auth. http://paste.openstack.org/show/421276/ | 01:36 |
| *** wbhuber has joined #openstack-swift | 01:38 | |
| kairo | ho: yeah. the curl version of head account. But curl -i http://ctl1srv:8080/v1/AUTH_dc182f2c475846889c7cb68602439d84 -I -H "X-Auth-Token: <token>" not. | 01:41 |
| kairo | ho: I'll try this auth | 01:41 |
| ho | kairo: did you set right part of X-Subject-Token for <token>? | 01:42 |
| *** nakagawamsa has joined #openstack-swift | 01:44 | |
| ho | kairo: it's better to validate token first. use: http://paste.openstack.org/show/421295/ | 01:45 |
| kairo | ho: yes. but I received the same 401. | 01:46 |
| ho | kairo: if there are info in the token, there is a problem with swift configuration. if not, there is a problem with keystone configuration. | 01:55 |
| clayg | g'night | 01:58 |
| openstackgerrit | Merged openstack/swift: Doc instructions for post rebase steps https://review.openstack.org/208669 | 02:01 |
| kairo | ho: I'll review the the keystone configuration. | 02:01 |
| *** haomaiwang has joined #openstack-swift | 02:05 | |
| *** jrichli has joined #openstack-swift | 02:06 | |
| ho | clayg: good night! | 02:07 |
| *** baojg has joined #openstack-swift | 02:15 | |
| openstackgerrit | Tim Burke proposed openstack/python-swiftclient: flake8 ignores same hacks as swift https://review.openstack.org/202909 | 02:22 |
| *** annegentle has quit IRC | 02:25 | |
| mattoliverau | notmyname: Melbourne Australia voted worlds most livable city (I think 5th year in a row) which is why you need to visit ;) http://www.cnn.com/2015/08/17/travel/most-liveable-city-2015/ | 02:26 |
| *** annegentle has joined #openstack-swift | 02:26 | |
| *** annegentle has quit IRC | 02:32 | |
| *** baojg has quit IRC | 02:35 | |
| ho | kairo: I summarized the script: http://paste.openstack.org/show/421329/ . hope it works :-) | 02:36 |
| ho | torgomatic: thanks for the review! yay! :-) | 02:38 |
| jrichli | mattoliverau hrou: wow, both Vancouver and Toronto made it to the top 5. Go Canada! Of course, 'Oz' is in there twice as well ;-) | 02:42 |
| _hrou_ | jrichli, mattoliverau, yea Toronto is always 4th in these. I was going to say notmyname is a lot closer to 3 out of the 5, but I guess there's nothing like #1 ; ) | 02:45 |
| *** baojg has joined #openstack-swift | 02:48 | |
| hugokuo | @clayg @timburke Keystone V3 works with new version of keystone middleware https://github.com/openstack/keystonemiddleware. This middleware includes keystone domain concept. | 02:49 |
| hugokuo | Anyone needs information about keystone V3 + Swift. Feel free to ping me | 02:50 |
| ho | hugokuo: hi, swiftclient + keystone v3 works? i usually use curl so i don't know it | 02:51 |
| *** dbarman1 has quit IRC | 02:53 | |
| *** _hrou_ is now known as hrou | 02:55 | |
| *** wbhuber has quit IRC | 02:56 | |
| timburke | hugokuo: so is an old version of keystonemiddleware the most likely explanation for https://ask.openstack.org/en/question/80290/swift-account-head-failed-401-unauthorized-kilo-centos-7/ ? | 02:57 |
| *** haomaiwang has quit IRC | 03:01 | |
| *** jrichli has quit IRC | 03:01 | |
| *** jrichli has joined #openstack-swift | 03:02 | |
| *** haomaiwang has joined #openstack-swift | 03:02 | |
| hugokuo | curl -i http://ctl1srv:8080/v1/AUTH_dc182f2c475846889c7cb68602439d84 -I -H "X-Auth-Token: 706f5084859f46c0a85218b14d3fd76c" | 03:03 |
| hugokuo | INFO:swiftclient:RESP STATUS: 401 Unauthorized | 03:03 |
| hugokuo | It is. | 03:03 |
| hugokuo | The debug information indicates the toekn sent to proxy but proxy either can't validate it with Keystone or the token was wrong. It's more like proxy can't validate the token due to 1) The user is not in default domain 2) The service admin user's credential is wrong | 03:05 |
| hugokuo | For this case, the user can try to validate a user in default domain. If it works that mean he has old version of keystone middleware | 03:05 |
| hugokuo | 1) Ensure the service admin's credential is correct in middleware 2) To access Swift by a user in default domain via keystone v2 API. 3) Check the middleware version. | 03:07 |
| *** kei_yama has quit IRC | 03:07 | |
| *** sanchitmalhotra has joined #openstack-swift | 03:11 | |
| *** kei_yama has joined #openstack-swift | 03:12 | |
| *** sanchitmalhotra1 has quit IRC | 03:13 | |
| *** rodrigods has quit IRC | 03:16 | |
| *** csmart has quit IRC | 03:18 | |
| *** rodrigods has joined #openstack-swift | 03:19 | |
| *** csmart has joined #openstack-swift | 03:21 | |
| *** sanchitmalhotra1 has joined #openstack-swift | 03:24 | |
| *** sanchitmalhotra has quit IRC | 03:26 | |
| *** raginbajin has quit IRC | 03:27 | |
| *** raginbajin has joined #openstack-swift | 03:29 | |
| *** logan2 has quit IRC | 03:31 | |
| *** ccavanna has joined #openstack-swift | 03:37 | |
| *** haigang has joined #openstack-swift | 03:47 | |
| openstackgerrit | Janie Richling proposed openstack/swift: Decrypting Container Listing https://review.openstack.org/214438 | 03:48 |
| *** fandi_ has joined #openstack-swift | 03:48 | |
| fandi_ | hi all | 03:49 |
| fandi_ | how to install swift-stack with dev env on virtual box ? | 03:49 |
| *** fandi_ has quit IRC | 03:49 | |
| jrichli | fandi_ : if you want a one-node swift cluster, you can use vagrant https://github.com/swiftstack/vagrant-swift-all-in-one | 03:51 |
| *** hezhiqiang has joined #openstack-swift | 03:52 | |
| openstackgerrit | paul luse proposed openstack/swift: EC GET path: require fragments to be of same set https://review.openstack.org/212187 | 03:58 |
| *** haomaiwang has quit IRC | 04:01 | |
| *** km has quit IRC | 04:01 | |
| *** haomaiwang has joined #openstack-swift | 04:02 | |
| *** km has joined #openstack-swift | 04:02 | |
| *** kei_yama has quit IRC | 04:03 | |
| *** kei_yama has joined #openstack-swift | 04:03 | |
| *** jlhinson has joined #openstack-swift | 04:05 | |
| *** jamespag` has quit IRC | 04:05 | |
| notmyname | good evening | 04:10 |
| *** raginbajin has quit IRC | 04:10 | |
| *** ppai has joined #openstack-swift | 04:13 | |
| *** raginbajin has joined #openstack-swift | 04:28 | |
| *** jrichli has quit IRC | 04:54 | |
| *** hrou has quit IRC | 04:55 | |
| *** baojg has quit IRC | 04:59 | |
| *** haomaiwang has quit IRC | 05:01 | |
| *** haomaiwa_ has joined #openstack-swift | 05:02 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:05 | |
| *** mahatic has joined #openstack-swift | 05:06 | |
| *** mahatic_ has joined #openstack-swift | 05:07 | |
| *** mahatic has quit IRC | 05:07 | |
| ho | notmyname: good evening! | 05:11 |
| notmyname | ho: hi! I was just working on my travel to tokyo | 05:11 |
| ho | notmyname: great! | 05:12 |
| ho | notmyname: do you have a plan to put some holiday after it? | 05:12 |
| notmyname | no. while I enjoy traveling, I don't like to be gone longer than that if my wife isn't with me. I don't want to leave her alone with the kids for that long | 05:13 |
| *** kei_yama has quit IRC | 05:14 | |
| *** km has quit IRC | 05:15 | |
| *** kei_yama has joined #openstack-swift | 05:15 | |
| ho | notmyname: yeah, I joined the hack-a-thon, then back I feel some cool atomosphere in my house :-) in tokyo we have some good sighseeing place such as asakusa, tukiji and so on. | 05:16 |
| notmyname | but I am getting in a day early | 05:16 |
| notmyname | yeah, I'd like to see what I can while I'm there :-) | 05:16 |
| ho | notmyname: yay! | 05:16 |
| *** silor has joined #openstack-swift | 05:17 | |
| *** silor1 has joined #openstack-swift | 05:19 | |
| mahatic_ | notmyname: I'm seeing you around here for the first time, at this hour! | 05:20 |
| *** silor has quit IRC | 05:21 | |
| *** silor1 is now known as silor | 05:21 | |
| notmyname | if this is the first time you've seen me online at this time... ;-) | 05:25 |
| notmyname | mahatic_: how are things? :-) | 05:26 |
| mahatic_ | notmyname: hehe, yeah. Everything good! It was hilarious to read on your knowledge about cricket on IRC :P | 05:28 |
| notmyname | heh | 05:28 |
| *** jith_ has joined #openstack-swift | 05:29 | |
| mahatic_ | I'm also actually doing some tokyo travel work - documents | 05:29 |
| mahatic_ | documents for visa | 05:29 |
| notmyname | oh good | 05:29 |
| *** jrichli has joined #openstack-swift | 05:31 | |
| jith_ | hi all i have done with SAIO setup.. But certain test cases are getting failed in unittests, functests and probetests.. is it mean that swift is not configured properly.. how to see the log for these testcases | 05:32 |
| notmyname | jith_: correct. all tests should pass in a SAIO | 05:33 |
| notmyname | jith_: you can use `./.unittests -vv` to get more info about what tests are being run. | 05:33 |
| notmyname | but the failure message should be printed when the test run finishes | 05:34 |
| jith_ | notmyname: thanks | 05:35 |
| jrichli | mahatic: rebase on patch 214438 instead of the one I had mentioned earlier. It adds handling of container listings. | 05:35 |
| patchbot | jrichli: https://review.openstack.org/#/c/214438/ | 05:35 |
| *** jrichli has quit IRC | 05:35 | |
| *** jlhinson has quit IRC | 05:38 | |
| mahatic_ | jrichli: sure, will do | 05:38 |
| jith_ | notmyname: how to change the maximum uploading size of 5GB in swift?? i mean without segmenting i need to upload the data above with size 7 GB | 05:40 |
| ho | hugokuo: cool. one question. how to confirm "validate a user in default domain"? | 05:42 |
| notmyname | jith_: it's set in swift.conf. see the message at https://github.com/openstack/swift/blob/master/etc/swift.conf-sample#L89 | 05:42 |
| jith_ | notmyname: thanks :) | 05:44 |
| *** hezhiqiang has quit IRC | 05:45 | |
| hugokuo | ho: Two steps : 1) Use service admin's token to fetch that user's information of the incoming token via V3 and V2 API. 2) V3 suppose to return the domain information to you | 05:45 |
| *** hezhiqiang has joined #openstack-swift | 05:45 | |
| zaitcev | jith_: But you really should not bother, because no matter how far you change it, there's going to be a longer object. My advice would be to learn to live with segmentation. I think the CLI segments automatically for you. | 05:46 |
| ho | hugokuo: is it same understanding with my previous post? (3) in http://paste.openstack.org/show/421329/ | 05:47 |
| hugokuo | ho: correct | 05:50 |
| ho | hugokuo: thanks! | 05:53 |
| *** sanchitmalhotra has joined #openstack-swift | 05:57 | |
| *** baojg has joined #openstack-swift | 05:58 | |
| *** sanchitmalhotra1 has quit IRC | 06:00 | |
| *** haomaiwa_ has quit IRC | 06:01 | |
| *** haomaiwa_ has joined #openstack-swift | 06:02 | |
| notmyname | some pics from last week | 06:13 |
| notmyname | http://d.not.mn/hackathon_pics/bbq.jpg | 06:14 |
| notmyname | http://d.not.mn/hackathon_pics/cookies.jpg | 06:14 |
| notmyname | http://d.not.mn/hackathon_pics/dinner.jpg | 06:14 |
| notmyname | http://d.not.mn/hackathon_pics/golf1.jpg | 06:14 |
| notmyname | http://d.not.mn/hackathon_pics/golf2.jpg | 06:14 |
| notmyname | http://d.not.mn/hackathon_pics/lunch.jpg | 06:14 |
| notmyname | http://d.not.mn/hackathon_pics/panicroom.jpg | 06:14 |
| notmyname | http://d.not.mn/hackathon_pics/people1.jpg | 06:14 |
| notmyname | http://d.not.mn/hackathon_pics/people2.jpg | 06:14 |
| notmyname | http://d.not.mn/hackathon_pics/sign.jpg | 06:14 |
| notmyname | http://d.not.mn/hackathon_pics/topics1.jpg | 06:14 |
| notmyname | http://d.not.mn/hackathon_pics/topics2.jpg | 06:14 |
| notmyname | and good night | 06:15 |
| ho | the bbq was so delicious :-) | 06:18 |
| mahatic_ | nice! lots of fun and work! | 06:23 |
| *** sanchitmalhotra1 has joined #openstack-swift | 06:27 | |
| *** akle|2 has joined #openstack-swift | 06:28 | |
| *** sanchitmalhotra has quit IRC | 06:29 | |
| *** wshao has joined #openstack-swift | 06:40 | |
| *** wshao has quit IRC | 06:41 | |
| jith_ | zaitcev: thanks really... ya but i made swift as glance backend... so i cant able to upload images(>5GB) to glance using glance client and also using dashboard | 06:43 |
| zaitcev | jith_: that makes no sense | 06:43 |
| zaitcev | jith_: I'm not familiar with Glance intimately, but I know for sure that it segments VM images automatically and transparently. You don't need to configure anything for it to happen. | 06:44 |
| jith_ | zaitcev: i m not getting error when i upload images greater than 5GB in glance... but when i launch instance from it.. its not getting launched... other images below 5GB is getting launched | 06:46 |
| jith_ | zaitcev: will check again | 06:46 |
| zaitcev | Hmm. | 06:46 |
| zaitcev | jith_: That may be a problem. However, it could be something like Swift updater not working right and Glance using DLO that needs up-to-date container listings. | 06:48 |
| zaitcev | Sadly I cannot investigate it for you. | 06:49 |
| jith_ | zaitcev: thanks for the informative answer... Any way to check the swift processes(like swift updater) seperately ? | 06:51 |
| zaitcev | I'd start by using Glance credentials with Swift CLI to look at images it uploaded, including those that get segmented and see if CLI can download them. | 06:52 |
| zaitcev | But I don't know where to go further | 06:52 |
| zaitcev | Well, Glance has its own CLI, so you can pretend to be Nova | 06:53 |
| zaitcev | BTW, make sure Nova's compute nodes aren't out of temporary space. | 06:53 |
| zaitcev | It's going to be the height of irony if they only have 5 GB left in /var by a gigantic coincidence... | 06:54 |
| *** ppai_ has joined #openstack-swift | 06:54 | |
| *** ppai has quit IRC | 06:56 | |
| *** StevenK has quit IRC | 06:58 | |
| *** StevenK has joined #openstack-swift | 06:59 | |
| *** haomaiwa_ has quit IRC | 07:01 | |
| *** haomaiwang has joined #openstack-swift | 07:01 | |
| *** rledisez has joined #openstack-swift | 07:02 | |
| *** geaaru has joined #openstack-swift | 07:07 | |
| *** sileht has joined #openstack-swift | 07:15 | |
| *** SkyRocknRoll has quit IRC | 07:30 | |
| *** ppai_ has quit IRC | 07:35 | |
| jith_ | zaitcev: thanks will check it... sorry for late reply | 07:45 |
| *** ppai_ has joined #openstack-swift | 07:48 | |
| *** zaitcev has quit IRC | 07:54 | |
| *** rledisez1 has joined #openstack-swift | 07:58 | |
| *** rledisez has quit IRC | 07:58 | |
| *** haomaiwang has quit IRC | 08:01 | |
| openstackgerrit | Kazuhiro MIYAHARA proposed openstack/swift: Fix dispersion-reports error message https://review.openstack.org/213697 | 08:01 |
| *** haomaiwang has joined #openstack-swift | 08:02 | |
| *** jordanP has joined #openstack-swift | 08:20 | |
| *** acoles_ is now known as acoles | 08:25 | |
| *** jistr has joined #openstack-swift | 08:34 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 08:39 |
| *** ho has quit IRC | 08:41 | |
| *** sanchitmalhotra has joined #openstack-swift | 08:44 | |
| *** sanchitmalhotra1 has quit IRC | 08:47 | |
| *** haigang has quit IRC | 08:52 | |
| *** haigang has joined #openstack-swift | 08:52 | |
| *** haigang has quit IRC | 08:52 | |
| *** haigang has joined #openstack-swift | 08:53 | |
| *** haigang has quit IRC | 08:53 | |
| *** haigang has joined #openstack-swift | 08:53 | |
| *** aix has joined #openstack-swift | 08:54 | |
| *** kei_yama has quit IRC | 08:55 | |
| *** haomaiwang has quit IRC | 09:01 | |
| *** haomaiwa_ has joined #openstack-swift | 09:02 | |
| *** joeljwright has joined #openstack-swift | 09:11 | |
| *** ChanServ sets mode: +v joeljwright | 09:11 | |
| *** hezhiqiang has quit IRC | 09:15 | |
| *** hezhiqiang has joined #openstack-swift | 09:15 | |
| *** SkyRocknRoll has joined #openstack-swift | 09:21 | |
| *** sanchitmalhotra1 has joined #openstack-swift | 09:32 | |
| *** ppai_ has quit IRC | 09:33 | |
| *** sanchitmalhotra has quit IRC | 09:34 | |
| *** sanchitmalhotra has joined #openstack-swift | 09:44 | |
| *** sanchitmalhotra1 has quit IRC | 09:46 | |
| *** ppai_ has joined #openstack-swift | 09:47 | |
| *** mahatic_ has quit IRC | 09:52 | |
| *** haomaiwa_ has quit IRC | 10:01 | |
| *** haomaiwang has joined #openstack-swift | 10:02 | |
| openstackgerrit | Merged openstack/swift: Keep user id and project id in subrequests env https://review.openstack.org/212149 | 10:06 |
| *** nakagawamsa has quit IRC | 10:07 | |
| *** hezhiqiang has quit IRC | 10:18 | |
| *** baojg has quit IRC | 10:26 | |
| *** baojg has joined #openstack-swift | 10:27 | |
| *** baojg has quit IRC | 10:31 | |
| *** mahatic has joined #openstack-swift | 10:39 | |
| *** mahatic has quit IRC | 10:47 | |
| *** nexusz99_ has quit IRC | 10:48 | |
| *** mahatic has joined #openstack-swift | 10:54 | |
| *** mahatic has quit IRC | 10:58 | |
| acoles | kairo: did you get your keystone auth sorted? looks like ho was helping. i left a comment on your ask, think problem may be that authtoken middleware is failing to auth with keystone, which means it cannot validate the x-auth-token in the account HEAD request. | 11:08 |
| *** haomaiwang has quit IRC | 11:12 | |
| *** cdelatte has joined #openstack-swift | 11:31 | |
| *** delattec has joined #openstack-swift | 11:31 | |
| openstackgerrit | paul luse proposed openstack/swift: EC GET path: require fragments to be of same set https://review.openstack.org/212187 | 11:44 |
| acoles | peluse: here? | 11:48 |
| openstackgerrit | paul luse proposed openstack/swift: EC GET path: support use of alternate fragments on object servers https://review.openstack.org/213147 | 11:55 |
| peluse | hi | 11:55 |
| acoles | peluse: i will review your same set frags patch today | 11:55 |
| peluse | cool, thanks | 11:56 |
| peluse | they are both up to date/ready | 11:57 |
| acoles | k | 11:57 |
| acoles | peluse: you're #2 on list, lunch is #1 ;) | 11:58 |
| peluse | food should always come first! | 11:59 |
| jith_ | hi all.. what is this error.. when i try to start/restart the processes. Unable to modify max process limit. Running as non-root? | 12:01 |
| peluse | jith_, see https://wiki.openstack.org/wiki/TroubleshootingSwift - it OK | 12:06 |
| * peluse off for breakfast... | 12:07 | |
| *** robefran has quit IRC | 12:10 | |
| jith_ | peluse: thanks | 12:14 |
| jith_ | hi all, in saio setup i m able to start only the server of proxy,object,container and account processes.. what i have to do to start the other process such as container-updater, container auditor, object auditor etc?? | 12:25 |
| *** chlong has joined #openstack-swift | 12:26 | |
| jordanP | jith_, try swift-init all start | 12:27 |
| *** wbhuber has joined #openstack-swift | 12:33 | |
| *** logan2 has joined #openstack-swift | 12:39 | |
| *** haomaiwang has joined #openstack-swift | 12:39 | |
| *** haigang has quit IRC | 12:39 | |
| *** mahatic has joined #openstack-swift | 12:46 | |
| *** SkyRocknRoll has quit IRC | 12:49 | |
| *** ccavanna has quit IRC | 12:58 | |
| *** wbhuber has quit IRC | 12:59 | |
| *** bill_az has joined #openstack-swift | 13:01 | |
| acoles | peluse: am i being dumb, or does test_GET_with_misplaced_frags have nothing to do with the etag buckets change? in patch 212187 | 13:03 |
| patchbot | acoles: https://review.openstack.org/#/c/212187/ | 13:03 |
| *** petertr7_away is now known as petertr7 | 13:11 | |
| *** hrou has joined #openstack-swift | 13:11 | |
| *** lpabon has joined #openstack-swift | 13:13 | |
| *** annegentle has joined #openstack-swift | 13:13 | |
| *** esker has joined #openstack-swift | 13:14 | |
| *** lpabon has quit IRC | 13:16 | |
| *** ppai_ has quit IRC | 13:17 | |
| *** dustins has joined #openstack-swift | 13:19 | |
| *** lcurtis has joined #openstack-swift | 13:22 | |
| *** NM has joined #openstack-swift | 13:22 | |
| *** ppai_ has joined #openstack-swift | 13:30 | |
| *** robefran has joined #openstack-swift | 13:37 | |
| *** annegentle has quit IRC | 13:37 | |
| kairo | acoles: sorry I was out. Thank you. | 13:43 |
| kairo | acoles: I found the problem. The problem is the users services passwords (swift, glance, nova etc) has the special caracters and in all services the password are between "" or ''. But on proxy-server.conf if I use "" or '' doesn't work. | 13:43 |
| *** mriedem has joined #openstack-swift | 13:46 | |
| mriedem | is there someone around that is able to create a stable/juno branch for python-swiftclient? there are some capped/uncapped dependency issues involving swiftclient on stable/juno we need to resolve per bug 1486576 | 13:49 |
| openstack | bug 1486576 in python-swiftclient "version 2.3.1 on stable/juno has uncapped futures which breaks dep resolution with capped taskflow" [Undecided,New] https://launchpad.net/bugs/1486576 | 13:49 |
| mriedem | the stable/juno branch would be created from the 2.3.1 tag of python-swiftclient | 13:49 |
| *** blmartin has joined #openstack-swift | 13:51 | |
| peluse | acoles, it does sorta :) | 13:53 |
| peluse | acoles, it was written before the patch but the idea is to simulate the actual behavior of the bug that started all of this | 13:55 |
| peluse | acoles, and the other new test routine is specically based on the implementation of the fix | 13:55 |
| acoles | kairo: ok glad you found it. could be difference in way other services read config using oslo.config vs swift ConfigParser :/ | 13:55 |
| acoles | peluse: its a cool test, just had me scratching my ehad :) | 13:56 |
| acoles | head even | 13:56 |
| peluse | acoles, there is a sorta wording challenge in that test code though, tere's a var named 'handoff_node' that actually isn't a handoff, or even a parity node, because of the shuffling of the nodes in node_iter. It's just 'another node' | 13:56 |
| *** wbhuber has joined #openstack-swift | 13:56 | |
| acoles | peluse: yeah i got a comment about that. | 13:58 |
| peluse | acoles, but yeah I guess it's not totally tied to etag_buckets, it was part of the last larger patch set that included some fixes and tets from both myself and clayg :) | 13:58 |
| *** mahatic has quit IRC | 13:58 | |
| acoles | peluse: the handoff nodes are parity nodes i think, but then you make a data node return content from a parity node | 13:58 |
| *** mahatic has joined #openstack-swift | 13:58 | |
| peluse | acoles, BTW I have to take off for a few hours. You'll probably be gone when I get back but thanks again for looking through these! | 13:59 |
| acoles | and all the etags are good right? in that test? | 13:59 |
| *** ccavanna has joined #openstack-swift | 13:59 | |
| *** mriedem is now known as mriedem_meeting | 13:59 | |
| peluse | acoles, they're not even parity because of node shuffling... they can be anything! | 13:59 |
| acoles | peluse: k. i'm confused then :) | 13:59 |
| peluse | acoles, yes, all the etags are good in that case. I was thinking of the probe tests that does an overwrite when I was tying this one to the etag buckets fix | 13:59 |
| acoles | k i will leave comments on gerrit in case i miiss you later | 14:00 |
| jith_ | jordanP: thanks... it was working | 14:00 |
| peluse | acoles, yeah, I know... so primary nodes in this case would be 0-14 then they're shuffled in the node_iter because that's how we do things. So slicing the first data or last parity doesn't get you what you might think | 14:00 |
| mattoliverau | morning | 14:00 |
| peluse | acoles, you end up getting 'something' from the first 10 (slicing off data) and 'something' for the last 4 (slicing parity) but any node can be in any order at this point... | 14:01 |
| peluse | acoles, OK, gotta go.... | 14:01 |
| wbhuber | acoles: i've several questions related to the low durable quorum comments you've left. i'll pan them out later in the morning - need some time to think through them. | 14:01 |
| *** robefran has quit IRC | 14:02 | |
| acoles | wbhuber: sure. it was a bit of a brain dump, but basically your patch is good but we need to do more :) | 14:02 |
| mattoliverau | peluse: I saw your email in intel/rackspace phone meeting today, are you attending it? I will be :) | 14:02 |
| *** robefran has joined #openstack-swift | 14:02 | |
| acoles | mattoliverau: morning !?! | 14:02 |
| mattoliverau | acoles: evening then :P (nah I'm still in Texas, so morning) | 14:04 |
| wbhuber | acoles: yes, "we need to do more" is what i'm trying to dissect to get the patch up and running. | 14:04 |
| mattoliverau | I fly back tomorrow afternoon | 14:04 |
| *** petertr7 is now known as petertr7_away | 14:10 | |
| kairo | acoles: can I report this for future people using the installation doc? | 14:12 |
| acoles | kairo: yes, you can report a bug on that install doc from link at bottom of the pages (swift team do not maintain those docs) and/or report a bug on swift project https://bugs.launchpad.net/swift | 14:14 |
| acoles | kairo: if you can include characters you found that broke in the bug report that would help | 14:15 |
| acoles | kairo: thanks! | 14:15 |
| *** thumpba has joined #openstack-swift | 14:15 | |
| kairo | acoles: I'll do that. thanks! | 14:17 |
| hrou | kairo: What led you to find the issue ? Debugging these swift / keystone always seems to be a pain, ho did a really good job yesterday with you (and I took notes for my own future use) : ) | 14:17 |
| mahatic | acoles: hello, query on "encrypt" flag, how would the existing failing probe tests know not to use encryption? Modify them to check a client header? (the client header- which I am to add in TVM). Advice please | 14:21 |
| notmyname | good morning | 14:22 |
| mahatic | good morning! | 14:23 |
| acoles | mahatic: notmyname good morning | 14:23 |
| acoles | mahatic: yes the existing probe tests would need to be modified to send the disable encrypt header | 14:24 |
| mahatic | good evening to me ;) | 14:24 |
| acoles | oops | 14:24 |
| acoles | mahatic: i'm still not certain whether we have encryption off by default (so no mods to existing tests) and 'enable' it for new probe tests by sending a key in a header, or on by default and disable in existing tests where necessary. | 14:25 |
| kairo | hrou: hrou: well. I made all tests passed by ho. The errors appears only using the proxy (this really helped me). Now in the morning, I saw the msg from acoles (in my ask). I review the user password. First time I put the same password. But after I thought in try a simple password. Didn't work with "" and I thought in try without and works well. I went back and tried recreate the problem again. And | 14:25 |
| kairo | I found this. :) | 14:25 |
| *** jrichli has joined #openstack-swift | 14:26 | |
| hrou | kairo, that's awesome, great works. Thanks for the clarification ! I'll add that to my 'debugging keystone' issue list. | 14:26 |
| kairo | thanks everybody! Sorry my bad english :) | 14:26 |
| kairo | hrou: I like troubleshootings. I'm sysadmin and I need do it everyday :P | 14:27 |
| mattoliverau | notmyname: morning, hows the OPs meetup going? | 14:27 |
| acoles | kairo: nice. | 14:27 |
| hrou | acoles, mahatic - can we just always include the header in those probe tests (assuming they're ignored if crypto is not in the pipeline) ? | 14:28 |
| notmyname | mahatic: going well enough. not much talk on swift (other than "yeah, swift doesn't give us any problems") | 14:28 |
| hrou | kairo, yea that's the issue, I'm not at all a sys admin but need to pretend to be at times ; ) Thanks again ! | 14:28 |
| notmyname | mattoliverau: ^ | 14:28 |
| mattoliverau | \o/ | 14:28 |
| notmyname | yesterday i talked to Time Warner Cable (a large US cable provider). they are using swift and have no problems | 14:29 |
| acoles | hrou: mahatic yes we should be able to always send a header an it be ignored | 14:29 |
| notmyname | acoles: yesterday I talked to Steve who says he sits next to you in bristol | 14:29 |
| mattoliverau | thats some of the feedback I've gotten here.. some ops in RPC want more swift experience but the damn thing is too stable (my paraphrase) | 14:29 |
| acoles | notmyname: Steve P! yeah that makes sense | 14:30 |
| hrou | acoles, mahatic - right so I wonder just pass in the "no encrypt" headers for now always in those 2 probe tests. But yea longer term may re evaluate depending on the approach taken. | 14:30 |
| notmyname | acoles: yeah, I had a good conversation with him and a couple of other people (someone on the "tools" side, IIRC) from HP | 14:30 |
| acoles | notmyname: he's a smart guy | 14:31 |
| notmyname | I talked to dhellmann about some potential things we can do wrt hacking/flake8/whatever to get back to a whitelist of style checks (very low priority right now though) | 14:32 |
| hrou | notmyname, re TWC, not sure if you recall but vinsh a month ago or so from there : ) | 14:32 |
| notmyname | hrou: I think you accidentally a word | 14:32 |
| vinsh | Swift. thats my full time job at TWC. | 14:33 |
| notmyname | vinsh: ah! great :-) | 14:33 |
| acoles | vinsh: hi | 14:33 |
| notmyname | vinsh: you're not at the ops summit, are you? | 14:33 |
| openstackgerrit | Merged openstack/swift: move global statement up a few lines https://review.openstack.org/214000 | 14:33 |
| vinsh | I skipped out on it, have 5 team members there though. | 14:33 |
| notmyname | right. I had lunch with them yesterday | 14:33 |
| hrou | vinsh, ah, hey ! : ), notmyname yep sorry missed the "vinsh was online a month ago ..." | 14:34 |
| kairo | hrou: it's my first time with OpenStack. On really I work with IBM Power/Linux/VMWare. I create this poor tool http://poweradm.org using Python (https://github.com/kairoaraujo/PowerAdm). In future I look contribute with openstack. | 14:34 |
| vinsh | I'm busy scoping a large erasure code swift cluster that took priority. | 14:34 |
| mahatic | acoles: hrou would it not need any workaround (the headers) if crypto is not in the pipeline? as in a check? | 14:34 |
| notmyname | vinsh: cool. the EC support in swift is very new, so please ask questions and share experiences related to it in here :-) | 14:34 |
| hrou | mahatic, you meaning adding the extra headers ? Would they not just be ignored ? jrichli is online too so she can chime in as well now. | 14:35 |
| vinsh | I have 8 object and 6 proxy nodes high horsepower on the way to setup and test it. | 14:35 |
| vinsh | 216TB/object 36 disks each. | 14:35 |
| vinsh | 6 10gig ports per node. | 14:35 |
| *** thurloat_isgone is now known as thurloat | 14:35 | |
| mahatic | hrou: right, I mean the extra headers. Not sure if I figured it completely, will check back on that. | 14:36 |
| hrou | kairo, that's really great ! Fairly new to openstack and swift as well. | 14:36 |
| hrou | mahatic, yea its the lightest weight option for now (assuming they're ignored), but longer term who knows if we go with byok or what not. | 14:37 |
| tdasilva | notmyname, vinsh: can you share how you are using swift? i'm assuming to store videos???? can you share any info like number of objects, cluster size, etc (a la HP and rackspace) ? | 14:37 |
| *** NM has quit IRC | 14:37 | |
| *** mriedem_meeting is now known as mriedem | 14:37 | |
| vinsh | Might as well throw a question out there... Our proxy logs have almost a 1-1 "invalid token rejected downstream" message for every client request. I feel this might be because I have 'v3' as part of my auth_uri.. but we needed to use v3 so that it can request that a service catalog is not returned in token validation. | 14:38 |
| acoles | mahatic: hrou unexpected headers will mostly be ignored | 14:38 |
| *** jlhinson has joined #openstack-swift | 14:38 | |
| hrou | acoles, perfect great. | 14:38 |
| vinsh | tdasilva: Our current production swift is multi-region. 4 zones. 4 replicas. (2 in each) It backs twc internal users desktop backup clients as well as serves web traffic for internal and extrenal sites. it also does glance image backups and about 100 other things. Mostly large image backups and any range of data stored. | 14:39 |
| mahatic | acoles: okay, thanks (I still don't think I understood the flow, will look into it more) | 14:39 |
| notmyname | vinsh: I'm told that TWC isn't yet using it for streaming videos, but it's used for a lot of the content for their web properties. oh cool! lots of backups too | 14:40 |
| vinsh | tdasilva: For videos, we are exploring erasure coding (due to cost of the massive volume to store) | 14:40 |
| vinsh | notmyname: those 8object 6proxies are what I will test streaming on. | 14:40 |
| notmyname | cool | 14:40 |
| vinsh | the proxies are all nvme drives and 6 10gig ports 2.6Ghzx2 18 core cpus. | 14:40 |
| *** petertr7_away is now known as petertr7 | 14:40 | |
| vinsh | We don't know what sort of cpu erasure coding will eat up for video streaming.. so will start big and see. | 14:41 |
| tdasilva | vinsh: nice! | 14:41 |
| wbhuber | vinsh, will you be able to share high level benchmarking numbers as a result of running workloads (video streaming) on EC? | 14:41 |
| vinsh | Heck yes! | 14:42 |
| tdasilva | vinsh: last week there were some discussions about making some enhancements to global replication and one of the questions clayg had was how are people using multi-region | 14:42 |
| vinsh | for production we have used multi-region from the start. | 14:42 |
| acoles | vinsh: sounds like a problem with authtoken talking to keystone, can you paste proxy logs somewhere. keystone v3 should work | 14:43 |
| acoles | vinsh: also bump log level to debug to see more from authtoken | 14:44 |
| vinsh | wbhuber: I have learned how to use ssbench so far, so I plan to use that against the EC cluster.. see if the numbers I get there make sense compared to what I got out of current production. | 14:44 |
| tdasilva | vinsh: we listed basically two use cases for multi-region. one is using a region as a DR site and another is where people are actually writing and reading data from both regions | 14:44 |
| vinsh | (yes I benchmarked against prod :-o ) | 14:44 |
| vinsh | acoles: Good call on debug. gathering some logs for ya. | 14:45 |
| vinsh | tdasilva: That is pretty right on for how we use it... DR was primary concern. | 14:45 |
| acoles | vinsh: and if you can paste your proxy-server.conf | 14:45 |
| *** annegentle has joined #openstack-swift | 14:46 | |
| *** NM has joined #openstack-swift | 14:47 | |
| vinsh | acoles: http://paste.openstack.org/show/421904/ is proxy-server.conf | 14:47 |
| *** annegentle has quit IRC | 14:47 | |
| hrou | acoles, vinsh, kairo - getting my notepad back out for another keystone/swift debug session ; ) | 14:47 |
| *** annegentle has joined #openstack-swift | 14:47 | |
| vinsh | acoles: and here are some log snips: http://paste.openstack.org/show/421905/ | 14:48 |
| vinsh | every get seems to work.. but prints the invalid user token message as well. | 14:48 |
| acoles | vinsh: oh right, so no 401's just log noise? | 14:49 |
| notmyname | I've got to drive down to the ops meetup now. I'll be online off and on throughout the day (and yes there's still the team meeting today) | 14:49 |
| vinsh | yeah | 14:49 |
| vinsh | hrou: hopefully this is more obvious of a thing :) | 14:50 |
| hrou | vinsh, I put my notebook away, once I read the not 401 comment ; ) jk | 14:51 |
| *** ppai_ has quit IRC | 14:52 | |
| *** kevinc_ has joined #openstack-swift | 14:52 | |
| *** minwoob has joined #openstack-swift | 14:58 | |
| *** zaitcev has joined #openstack-swift | 15:01 | |
| *** ChanServ sets mode: +v zaitcev | 15:01 | |
| *** logan2 has quit IRC | 15:01 | |
| acoles | vinsh: hmmm, not sure. if token is invalid but access permitted then must be anonymous auth, which looks like either its a container sync request or you have public ACL | 15:04 |
| acoles | vinsh: debug log may give you more clues as to why the token is not validated | 15:05 |
| *** logan2 has joined #openstack-swift | 15:05 | |
| vinsh | I think its all web traffic | 15:06 |
| vinsh | so image loads of web pages | 15:06 |
| *** zul has joined #openstack-swift | 15:07 | |
| *** chenhuayi has joined #openstack-swift | 15:09 | |
| acoles | vinsh: i haven't played with anonymous auth much but a container ACL can grant access such that no auth token is required. that could explain the log pattern. | 15:10 |
| acoles | http://docs.openstack.org/developer/swift/misc.html#module-swift.common.middleware.acl <-- ACL format | 15:11 |
| *** blmartin_ has joined #openstack-swift | 15:11 | |
| vinsh | acoles: debug log http://paste.openstack.org/show/421960/ | 15:12 |
| vinsh | Yeah, its always 'Authorizing as anonymous" then transaction works.. then invalid token message | 15:12 |
| *** chsc has joined #openstack-swift | 15:13 | |
| *** blmartin has quit IRC | 15:14 | |
| vinsh | acoles: Would you expect the proxy node to log an invalid token message for every anonymous request? | 15:15 |
| acoles | heh i was going to suggest a grep for 'via referer ACL ' | 15:15 |
| acoles | vinsh: from reading authtoken code, yes, it appears to log every time | 15:15 |
| vinsh | Right on, so its just annoying log chatter then :) | 15:16 |
| vinsh | Confusing to the operator.. but no harm done. | 15:16 |
| acoles | looks that way - authtoken is not swift maintained - its from keystonemiddleware project - so don't blame us :) | 15:17 |
| vinsh | lol, I hear ya :) | 15:17 |
| acoles | vinsh: some of the *debug* level you see is coming from keystoneauth https://github.com/openstack/swift/blob/master/swift/common/middleware/keystoneauth.py#L565 | 15:19 |
| vinsh | acoles: Very interesting. Thank you for helping me understand this one, appears to be a cosmetic issue only... user gets its data. | 15:21 |
| acoles | vinsh: np. will be v interesting to hear how your EC cluster comes along. | 15:21 |
| *** zul has quit IRC | 15:23 | |
| *** petertr7 is now known as petertr7_away | 15:26 | |
| *** petertr7_away is now known as petertr7 | 15:27 | |
| petertr7 | Hey torgomatic, I had a question about Tempurls. The docs say that signatures are checked against all of the 4 secret keys (2 account level, 2 container level). I'm currently getting invalid tempurls when I create signatures with the container key but no issues with the account key. Any thoughts? | 15:31 |
| portante | peluse: optane drives, are you working with them? http://www.technologyreview.com/news/540411/intels-reinvention-of-the-hard-drive-could-make-all-kinds-of-computers-faster/ | 15:31 |
| vinsh | acoles: I'll keep you all well informed, I won't be able to get it going alone :) | 15:31 |
| *** haomaiwang has quit IRC | 15:40 | |
| *** sanchitmalhotra has quit IRC | 15:40 | |
| *** dustins has quit IRC | 15:44 | |
| *** zul has joined #openstack-swift | 15:45 | |
| *** petertr7 is now known as petertr7_away | 15:49 | |
| *** sanchitmalhotra has joined #openstack-swift | 15:52 | |
| *** zul has quit IRC | 15:55 | |
| *** cebruns_ is now known as cebruns | 16:01 | |
| *** SkyRocknRoll has joined #openstack-swift | 16:08 | |
| *** amandap has quit IRC | 16:09 | |
| *** dustins has joined #openstack-swift | 16:09 | |
| *** SkyRocknRoll has quit IRC | 16:10 | |
| *** amandap has joined #openstack-swift | 16:12 | |
| *** SkyRocknRoll has joined #openstack-swift | 16:12 | |
| *** lyrrad has joined #openstack-swift | 16:17 | |
| *** gyee has joined #openstack-swift | 16:20 | |
| *** SkyRocknRoll has quit IRC | 16:21 | |
| *** lyrrad has quit IRC | 16:22 | |
| *** rledisez1 has quit IRC | 16:24 | |
| *** petertr7_away is now known as petertr7 | 16:24 | |
| *** lyrrad has joined #openstack-swift | 16:27 | |
| *** jistr has quit IRC | 16:30 | |
| *** jordanP has quit IRC | 16:30 | |
| notmyname | mattoliverau: seems that there is a lot of desire (or at least interest) in ansible modules for deploying swift. is the stuff you've worked on public? | 16:30 |
| wbhuber | notmyname: that's what i have been hearing too. chef seems to be running in the back mirror view of the ansible. folks around here are suggesting ansible as the deployment automation tool. | 16:31 |
| wbhuber | because of the convenience and easy-to-create scripts using ansible | 16:32 |
| *** thumpba has quit IRC | 16:32 | |
| notmyname | yeah, I'd like to see some publicly available (and officially referenced in upstream docs) tools for ansible/chef/puppet/salt/etc. stuff to remove barriers for people to install/deploy swift | 16:33 |
| *** petertr7 is now known as petertr7_away | 16:34 | |
| hrou | wbhuber, notmyname I've used this i nthe past: https://github.com/Open-I-Beam/swift-install, but I should check on the status. | 16:34 |
| *** nadeem has joined #openstack-swift | 16:35 | |
| *** SkyRocknRoll has joined #openstack-swift | 16:35 | |
| jrichli | I just asked around here, and we have some that we are using that are in a github, but it is probably not something for an "official" reference at this point. I will find out about the plans, though. | 16:38 |
| *** zul has joined #openstack-swift | 16:40 | |
| tdasilva | notmyname: https://github.com/matthewoliver/swift-ansible | 16:42 |
| notmyname | thanks | 16:42 |
| tdasilva | notmyname: i'm planning to use that too for some tests i'd like to run here, so being able to setup a cluster and tear it down after tests is really key | 16:42 |
| tdasilva | notmyname: mattoliverau added the ability to even specify a patch in case you want to test that | 16:43 |
| notmyname | cool | 16:45 |
| *** zul has quit IRC | 16:47 | |
| *** zul has joined #openstack-swift | 16:49 | |
| *** mriedem has left #openstack-swift | 16:49 | |
| *** dustins has quit IRC | 16:52 | |
| *** thurloat is now known as thurloat_isgone | 16:54 | |
| *** thurloat_isgone is now known as thurloat | 16:54 | |
| *** dustins has joined #openstack-swift | 16:56 | |
| jith_ | hi all.. what all i should reset if my VM (SAIO setup) restarted... because when i start the services using swift-init main start.. it is getting hanged | 16:58 |
| *** sanchitmalhotra has quit IRC | 17:00 | |
| *** sanchitmalhotra has joined #openstack-swift | 17:01 | |
| *** resker has joined #openstack-swift | 17:04 | |
| *** wbhuber is now known as wbhuber_ | 17:05 | |
| *** petertr7_away is now known as petertr7 | 17:07 | |
| *** dbarman has joined #openstack-swift | 17:08 | |
| *** esker has quit IRC | 17:08 | |
| torgomatic | petertr7: I'd make sure your Swift is new enough to support container tempurls; it's a pretty recent feature | 17:10 |
| petertr7 | torgomatic, Thanks a bunch. Unfortunately I'm on Juno =( | 17:12 |
| torgomatic | petertr7: so I'm going to assume that you've verified the presence of the tempurl keys on the container via a HEAD request; the only other thing to check is that the object requested is within the container that has the keys | 17:14 |
| *** geaaru has quit IRC | 17:14 | |
| torgomatic | it's a scoping thing; account keys get you objects in the account, but container keys only get you objects in the container | 17:15 |
| petertr7 | Hi torgomatic, I confirmed that indeed the container has the proper header ("X-Container-Meta-Temp-URL-Key") | 17:16 |
| petertr7 | And the object is definitely in the container (deleted object, reuploaded it) | 17:16 |
| acoles | torgomatic: wasn't container tempurl new in kilo? | 17:19 |
| torgomatic | acoles: maybe? I have a hard time keeping track | 17:19 |
| torgomatic | what tag was juno again? | 17:19 |
| acoles | 2.2.0 ? | 17:20 |
| *** dbarman has quit IRC | 17:21 | |
| torgomatic | 2.2.0 doesn't have it but 2.3.0 does | 17:21 |
| torgomatic | notmyname: what tag did we call "Juno"? | 17:22 |
| *** resker has quit IRC | 17:22 | |
| acoles | yup so says the changelog | 17:22 |
| torgomatic | and by "we" I mean "you" | 17:22 |
| notmyname | tempurl was in 2.3.0 and 2.3.0 == kilo | 17:22 |
| petertr7 | Ohh I see. Thanks for clearing this up everyone, much appreciated! | 17:22 |
| *** zul has quit IRC | 17:23 | |
| torgomatic | oh here we go, correlate these dates https://wiki.openstack.org/wiki/Releases with dates on the tagged commits, and it makes sense | 17:24 |
| petertr7 | that's so well organized | 17:25 |
| *** joeljwright has quit IRC | 17:30 | |
| *** hezhiqiang has joined #openstack-swift | 17:37 | |
| notmyname | we squeezed some swift stuff into the ops summit just now. questions about deployment and any issues therein | 17:38 |
| notmyname | first line on the etherpad is "swift is the most solid part of our openstack deployment+1+1" | 17:38 |
| notmyname | :-) | 17:38 |
| egon | That's a great add, since we've got some operational comments about running it. | 17:39 |
| *** kevinc_ has quit IRC | 17:42 | |
| openstackgerrit | Samuel Merritt proposed openstack/swift: Add OpenStack release names to changelog https://review.openstack.org/214728 | 17:44 |
| mattoliverau | notmyname: yup my stuff is in 2 forms. 1 was merged and refactored into OSAD, and I still have a simpler version (tdasilva is referring to) which I have been maintaining which I use for my own test clusters. | 17:46 |
| *** jordanP has joined #openstack-swift | 17:49 | |
| openstackgerrit | Samuel Merritt proposed openstack/swift: Add OpenStack release names to changelog https://review.openstack.org/214728 | 17:49 |
| mattoliverau | OSAD is used by Rackspace RPC and can be found here: https://github.com/stackforge/os-ansible-deployment | 17:50 |
| mattoliverau | That does all of openstack including swift. | 17:51 |
| mattoliverau | NOTE: that OSAD is moving from stackforge to openstack if it hasn't already. | 17:51 |
| *** breitz has quit IRC | 17:51 | |
| acoles | torgomatic: for some reason i also struggle with ordering those release names :) number ordering i can do! | 17:52 |
| *** breitz has joined #openstack-swift | 17:52 | |
| torgomatic | :) | 17:52 |
| mattoliverau | The simpler one I use, that tdasilva was talking about is: https://github.com/matthewoliver/swift-ansible it isn't feture complete as I've only been adding to it as I've needed it. | 17:53 |
| *** ccavanna_ has joined #openstack-swift | 17:54 | |
| *** aix has quit IRC | 17:55 | |
| *** ccavanna has quit IRC | 17:57 | |
| *** brnelson has quit IRC | 17:57 | |
| *** brnelson has joined #openstack-swift | 17:58 | |
| *** dbarman has joined #openstack-swift | 18:03 | |
| *** janonymous has joined #openstack-swift | 18:04 | |
| *** kevinc_ has joined #openstack-swift | 18:04 | |
| *** annegentle has quit IRC | 18:04 | |
| *** acoles is now known as acoles_ | 18:05 | |
| *** jordanP has quit IRC | 18:08 | |
| *** dbarman has quit IRC | 18:08 | |
| *** dbarman has joined #openstack-swift | 18:09 | |
| *** dbarman has quit IRC | 18:15 | |
| *** dbarman has joined #openstack-swift | 18:16 | |
| *** jith_ has quit IRC | 18:24 | |
| *** dbarman has quit IRC | 18:25 | |
| *** dbarman has joined #openstack-swift | 18:25 | |
| *** dbarman has quit IRC | 18:29 | |
| *** bapalm has quit IRC | 18:31 | |
| clayg | i like mattoliverau being around in the daytime | 18:32 |
| *** bapalm has joined #openstack-swift | 18:32 | |
| *** dmorita has joined #openstack-swift | 18:36 | |
| *** dmorita has quit IRC | 18:36 | |
| peluse | acoles_, scratch what I said earlier about that patch wrt data vs parity & shuffle. Totally wrong and based on a bogus mental model of being able to tell a data node from a parity node via - that you can't do w/default shuffle mode but clearly the first 10 (whatever they are) in that case would be data. I'll change the name of the var in the test... | 18:37 |
| peluse | via node index that is... why did this keyboard skip a whole word?? :) | 18:37 |
| clayg | peluse: which test? i'm looking at patch 212187 now | 18:42 |
| patchbot | clayg: https://review.openstack.org/#/c/212187/ | 18:42 |
| *** sanchitmalhotra1 has joined #openstack-swift | 18:46 | |
| *** esker has joined #openstack-swift | 18:46 | |
| peluse | clayg, one that you wrote :) one sec.... | 18:48 |
| *** sanchitmalhotra has quit IRC | 18:48 | |
| peluse | clayg, test_GET_with_misplaced_frags() the name of 'handoff_node' should be 'parity_node' | 18:49 |
| openstackgerrit | Tim Burke proposed openstack/python-swiftclient: Drop flake8 ignores for already-passing tests https://review.openstack.org/214752 | 18:50 |
| *** zul has joined #openstack-swift | 18:50 | |
| peluse | clayg, acoles_, OK I'm changing my tune yet again on the statement that "the first ndata nodes from node_iter are data nodes even though tis shuffled" because the first ndata nodes on initial PUT will likely be different from a subsequent GET - yes?? | 18:52 |
| *** sanchitmalhotra1 has quit IRC | 18:52 | |
| clayg | peluse: yeah I'm not sure I beleive that the first ndata nodes from the node_iter would be data nodes (vs. parity) - but does any code in the proxy really care? it's all primaries to him... | 18:54 |
| clayg | peluse: acoles_: also, sorry my tests suck | 18:55 |
| *** kevinc_ has quit IRC | 18:55 | |
| clayg | peluse: so i've got a few things I'm going to clarify for myself before I push comments/diffs | 18:55 |
| clayg | peluse: acoles_ has some good fixups on there already I think - do you wanna push those first and then see if anything I have is still relevant? | 18:56 |
| peluse | clayg, the only 'thing' in the proxy that cares (sorta) is that a GET to all data will result in a concat in the EC lib instead of a rebuild. A mix of data+parity will always require rebuild | 19:00 |
| peluse | clayg, sure to your last comment | 19:01 |
| peluse | clayg, so if EC didn't shuffle we'd get a free perf opt on GET | 19:01 |
| clayg | peluse: I think most people run with node_sort to timing? | 19:02 |
| * peluse rememembers a patch somewere that someone proposed to add parms to policies and shuffle policy was one of them, wonder what hapened to that.... | 19:02 | |
| clayg | peluse: yeah we really need per-policy proxy app tunings like that :\ | 19:02 |
| peluse | clayg, really? I would have assumed the def (shuffle) | 19:02 |
| clayg | peluse: oh yeah maybe that's the *default* | 19:03 |
| peluse | clayg, I know there was a patch at one point. WOuld be nice for EC to not shuffle since it buys us nothing | 19:03 |
| peluse | other than confusion :) | 19:03 |
| clayg | peluse: well... it spreads the load around the disks | 19:03 |
| openstackgerrit | Victor Stinner proposed openstack/swift: pep8: fix warning F812 https://review.openstack.org/213496 | 19:03 |
| openstackgerrit | Victor Stinner proposed openstack/swift: pep8: replace deprecated calls to assert_() https://review.openstack.org/213491 | 19:03 |
| openstackgerrit | Victor Stinner proposed openstack/swift: pep8: Don't override '_' symbol https://review.openstack.org/213495 | 19:03 |
| openstackgerrit | Victor Stinner proposed openstack/swift: pep8: Fix usage of the l10n _('...') function https://review.openstack.org/213494 | 19:03 |
| peluse | spread shmed | 19:03 |
| clayg | peluse: i'd have to be a significant hit on cpu to be worth giving up all the other things the proxy is doing right | 19:04 |
| clayg | peluse: 100ms of latency waiting on a backend connection - or an extra stream on an already busy disk - is going to have a big impact - i'm don't know rebuilt vs. data conact? | 19:04 |
| clayg | plus there's a ttfb vs overall throughput tradeoff | 19:05 |
| openstackgerrit | Victor Stinner proposed openstack/swift: pep8: fix warning F812 https://review.openstack.org/213496 | 19:05 |
| openstackgerrit | Victor Stinner proposed openstack/swift: pep8: replace deprecated calls to assert_() https://review.openstack.org/213491 | 19:05 |
| openstackgerrit | Victor Stinner proposed openstack/swift: pep8: Fix hacking H232 warnings (octal) https://review.openstack.org/206149 | 19:05 |
| openstackgerrit | Victor Stinner proposed openstack/swift: pep8: Don't override '_' symbol https://review.openstack.org/213495 | 19:05 |
| openstackgerrit | Victor Stinner proposed openstack/swift: pep8: Fix usage of the l10n _('...') function https://review.openstack.org/213494 | 19:05 |
| clayg | peluse: you can try doing different sorting - if we had sorting per policy it'd be interesting to add a "strict ring order" sorting (maybe?) | 19:06 |
| clayg | peluse: if you have a hot storage tier strict ring order + concurrent gets may do better at warming up the first primary of each part with the data for that part - leaving the other nodes more room to warm up their page cache with data from the other parts they are the primaries for | 19:07 |
| haypo | cschwede: hi. i chose to write a patch serie because otherwise will conflict because all of the them modify the same line in tox.ini, the ignore= line | 19:08 |
| haypo | cschwede: i modified my patch serie to remove rejected patches | 19:08 |
| *** sanchitmalhotra has joined #openstack-swift | 19:08 | |
| peluse | clayg, agreed | 19:08 |
| *** annegentle has joined #openstack-swift | 19:13 | |
| *** esker has quit IRC | 19:16 | |
| *** silor has quit IRC | 19:16 | |
| peluse | portante, I wish they'd tell us how to map marketing names of stuff they do press releases on to internal project names... there are a few things that PR could be about but it doesn't have enough detail (or the internal codename) for me to tell :) | 19:26 |
| *** hezhiqiang has quit IRC | 19:38 | |
| *** petertr7 is now known as petertr7_away | 19:41 | |
| *** petertr7_away is now known as petertr7 | 19:41 | |
| *** mahatic has quit IRC | 19:42 | |
| *** gyee has quit IRC | 19:47 | |
| *** esker has joined #openstack-swift | 19:50 | |
| *** dbarman has joined #openstack-swift | 19:52 | |
| *** esker has quit IRC | 19:54 | |
| *** esker has joined #openstack-swift | 19:55 | |
| *** gyee has joined #openstack-swift | 19:56 | |
| *** alejandrito has joined #openstack-swift | 19:59 | |
| *** annegentle has quit IRC | 19:59 | |
| *** alejandrito_ has joined #openstack-swift | 20:00 | |
| *** alejandrito has quit IRC | 20:00 | |
| *** thurloat is now known as thurloat_isgone | 20:05 | |
| *** alejandrito_ has quit IRC | 20:18 | |
| openstackgerrit | paul luse proposed openstack/swift: EC GET path: require fragments to be of same set https://review.openstack.org/212187 | 20:24 |
| *** janonymous has quit IRC | 20:25 | |
| *** SkyRocknRoll has quit IRC | 20:28 | |
| *** mahatic has joined #openstack-swift | 20:30 | |
| *** mahatic has quit IRC | 20:31 | |
| *** robefran has quit IRC | 20:32 | |
| *** blmartin_ has quit IRC | 20:32 | |
| openstackgerrit | paul luse proposed openstack/swift: EC GET path: support use of alternate fragments on object servers https://review.openstack.org/213147 | 20:34 |
| *** petertr7 is now known as petertr7_away | 20:34 | |
| *** petertr7_away is now known as petertr7 | 20:37 | |
| *** annegentle has joined #openstack-swift | 20:45 | |
| notmyname | swift team meeting in 15 minutes. | 20:45 |
| mattoliverau | notmyname: peluse and I are in a meeting | 20:46 |
| notmyname | ack | 20:46 |
| *** ho has joined #openstack-swift | 20:46 | |
| clayg | neat | 20:46 |
| notmyname | I think this seems to be mostly a summary of the hackathon, and also ho has a topic he proposed | 20:46 |
| *** aerwin has joined #openstack-swift | 20:47 | |
| ho | good morning! | 20:48 |
| aerwin | o/ | 20:50 |
| peluse | mattoliverau, but only one meeting :) there's room for at least one other | 20:50 |
| *** dustins has quit IRC | 20:50 | |
| mattoliverau | lol, true | 20:50 |
| *** annegentle has quit IRC | 20:53 | |
| *** ccavanna_ has quit IRC | 20:53 | |
| *** mjfork has joined #openstack-swift | 20:53 | |
| *** annegentle has joined #openstack-swift | 20:54 | |
| *** mahatic has joined #openstack-swift | 20:54 | |
| mjfork | I am debugging long running requests and am seeing GET requests fail with 20s timeouts talking to the container servers. What would cause a get to go to a container server? | 20:55 |
| *** blmartin has joined #openstack-swift | 20:56 | |
| *** acoles_ is now known as acoles | 20:57 | |
| ctennis | I think it makes an initial request to determine ACL settings if you have the rights to read the object | 20:58 |
| ctennis | it should then cache it in memcache for a period of time | 20:58 |
| *** esker has quit IRC | 20:58 | |
| mattoliverau | peluse: just in time for the swift meeting | 20:59 |
| notmyname | mattoliverau: wait, you're already done with your meeting? | 20:59 |
| notmyname | that was fast | 20:59 |
| mattoliverau | well it helps when intel and rackspace (peluse and I) had the same list :) | 20:59 |
| notmyname | wonder how that happened ;-) | 21:00 |
| *** zul has quit IRC | 21:00 | |
| notmyname | ok, meeting time | 21:00 |
| mattoliverau | good communication within the swift communuty | 21:00 |
| notmyname | exactly! | 21:00 |
| *** barker has joined #openstack-swift | 21:01 | |
| *** petertr7 is now known as petertr7_away | 21:07 | |
| acoles | peluse: sorry i didnt get to the second patch, alt frags, will do tomorrow | 21:08 |
| acoles | peluse: had to go walk the dog! | 21:08 |
| *** hezhiqiang has joined #openstack-swift | 21:09 | |
| *** hezhiqiang has quit IRC | 21:14 | |
| *** robefran has joined #openstack-swift | 21:15 | |
| *** blmartin_ has joined #openstack-swift | 21:15 | |
| *** blmartin has quit IRC | 21:18 | |
| *** hrou has quit IRC | 21:20 | |
| *** jrichli has quit IRC | 21:26 | |
| *** jrichli has joined #openstack-swift | 21:26 | |
| peluse | acoles, heh, thanks!!! | 21:27 |
| *** barker has quit IRC | 21:37 | |
| *** nadeem has quit IRC | 21:38 | |
| *** nadeem has joined #openstack-swift | 21:39 | |
| *** nadeem has quit IRC | 21:39 | |
| *** nadeem has joined #openstack-swift | 21:40 | |
| openstackgerrit | Tim Burke proposed openstack/python-swiftclient: Drop flake8 ignores for already-passing tests https://review.openstack.org/214752 | 21:55 |
| clayg | redbo: yeah per-policy constraints would be pretty safe/sane I think | 21:57 |
| redbo | that would probably be the easiest solution | 21:57 |
| *** mahatic has quit IRC | 21:58 | |
| ho | mattoliverau: could you check patch 138342? | 21:58 |
| patchbot | ho: https://review.openstack.org/#/c/138342/ | 21:58 |
| jrichli | notmyname: awe, you didn't say the ending phrase. | 21:59 |
| *** acoles is now known as acoles_ | 21:59 | |
| *** dbarman has quit IRC | 21:59 | |
| *** blmartin_ has quit IRC | 21:59 | |
| notmyname | jrichli: I did | 22:00 |
| notmyname | jrichli: ah, I didn't it was in my buffer, but then I said somethign else :-) | 22:00 |
| jrichli | :-) at least I know you are still feeling thankful! | 22:01 |
| ho | today i overslept so i parked my car in front of the office. i have to move it to right place before colleagues come to here ;-) | 22:02 |
| *** esker has joined #openstack-swift | 22:03 | |
| * notmyname closes his laptop to get back to the ops summit | 22:04 | |
| *** hrou has joined #openstack-swift | 22:04 | |
| *** resker has joined #openstack-swift | 22:07 | |
| *** resker has quit IRC | 22:08 | |
| *** esker has quit IRC | 22:09 | |
| *** tonyb has joined #openstack-swift | 22:10 | |
| *** annegentle has quit IRC | 22:17 | |
| *** jrichli has quit IRC | 22:18 | |
| *** esker has joined #openstack-swift | 22:21 | |
| clayg | peluse: wow, test_GET_with_mismatch_etag is a fair bit more readable this go round - thanks! | 22:28 |
| *** sanchitmalhotra1 has joined #openstack-swift | 22:28 | |
| *** sanchitmalhotra has quit IRC | 22:30 | |
| *** wbhuber_ is now known as wbhuber | 22:34 | |
| *** esker has quit IRC | 22:54 | |
| *** jlhinson has quit IRC | 22:54 | |
| *** esker has joined #openstack-swift | 22:54 | |
| *** NM has quit IRC | 22:55 | |
| *** lyrrad_ has joined #openstack-swift | 22:58 | |
| *** lyrrad has quit IRC | 22:59 | |
| *** lyrrad_ is now known as lyrrad | 22:59 | |
| *** bill_az_ has joined #openstack-swift | 23:00 | |
| *** bill_az has quit IRC | 23:01 | |
| *** km has joined #openstack-swift | 23:02 | |
| *** lcurtis has quit IRC | 23:04 | |
| *** zul has joined #openstack-swift | 23:06 | |
| *** zul has quit IRC | 23:12 | |
| clayg | so if someone sends a request with if-match: not-the-etag how many nodes do we think we should hit before we give up? | 23:12 |
| torgomatic | clayg: one that doesn't 404? | 23:13 |
| clayg | I think if we get get a 416 (range not satisfiable) we return it immediately - but if we get a 412 - we'll keep trying until we exahust the node iter | 23:13 |
| torgomatic | well, that's unfortunate | 23:13 |
| clayg | torgomatic: is it? i'm not really sure what's the expected behavior | 23:14 |
| clayg | torgomatic: like if someone is trying to do some sort of optomistic overwrite protection maybe trying more nodes is a good idea? | 23:14 |
| torgomatic | clayg: I mean, if the proxy learns that the object (a) exists and (b) this copy doesn't match, that's good enough to return early IMO | 23:14 |
| clayg | torgomatic: yeah there has to be some diminishing returns, after you get back one or two 412's | 23:15 |
| clayg | the behavior is sort of insane with EC, the resumming getters just go go go! I think the fake ring limits my test cases requests to "only" 28 | 23:16 |
| clayg | I'm not sure 2 x replicas was really ever what anyone wanted - I think someone wanted 6 handoffs and they just happened to think "hey that's twice as many as my replica count! maybe that's useful" and then it turned out for replica counts != 3 it was not useful :'( | 23:18 |
| torgomatic | yeah, the goal was enough nodes for read-after-write consistency with new objects; 2*primaries is definitely overdoing it for EC | 23:19 |
| clayg | torgomatic: so request_node_count is the total number of nodes, not the number of handoffs? | 23:22 |
| torgomatic | clayg: I think so? | 23:22 |
| clayg | yeah i guess it is | 23:22 |
| *** kei_yama has joined #openstack-swift | 23:25 | |
| *** chlong has quit IRC | 23:28 | |
| *** aerwin has quit IRC | 23:29 | |
| *** nadeem has quit IRC | 23:31 | |
| *** dbarman has joined #openstack-swift | 23:35 | |
| clayg | peluse: bah, so I was convinced that the handling of bad requests was suboptimal with your change and rewrote a bunch of the handling in when iterating the response pile to break when len(bad_responses) >= min_gets | 23:39 |
| clayg | I wrote some tests and not only did it work out what you were doing was fine (because you only spawn when 200's >= num_gets) - but my code ended up making an extra request if you only get 9 out of 10 bad_responses | 23:39 |
| clayg | so it was basically *exactly* the opposite of what I set out to prove; you were right, i was wrong, your code was better, my code stunk :'( | 23:40 |
| *** zul has joined #openstack-swift | 23:41 | |
| clayg | still I'd bet getting ndata 416's is almost never needed | 23:42 |
| *** barker has joined #openstack-swift | 23:54 | |
| *** sanchitmalhotra has joined #openstack-swift | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
