| *** chsc has quit IRC | 00:02 | |
| *** sneti has quit IRC | 00:03 | |
| *** devlaps has quit IRC | 00:15 | |
| *** zhill has quit IRC | 00:21 | |
| kota_ | wow, bug #1496636 seems what I wanted to dig about chunked transfer protocol between proxy and object-server. | 00:22 |
|---|---|---|
| openstack | bug 1496636 in OpenStack Object Storage (swift) "EC: Chunked transfer/commit protocol is *not* HTTP" [Undecided,New] https://launchpad.net/bugs/1496636 | 00:22 |
| *** zhill has joined #openstack-swift | 00:31 | |
| *** breitz has quit IRC | 00:35 | |
| *** zhill has quit IRC | 00:35 | |
| *** hezhiqiang has joined #openstack-swift | 00:38 | |
| clayg | kota_: yeah you totally turned me on to it with the test gists you did for peluse's patch | 00:41 |
| *** m_kazuhiro has joined #openstack-swift | 00:41 | |
| clayg | kota_: way to point out that everything we think is wrong! | 00:41 |
| clayg | briancline: whoot - just got your recon fixups on master - congrats! | 00:42 |
| kota_ | clayg: yey, seems I had do good work? But awsome you found it in detail!! | 00:43 |
| *** hezhiqia_ has joined #openstack-swift | 00:44 | |
| clayg | kota_: all you brother - awesome work | 00:44 |
| *** garthb has quit IRC | 00:45 | |
| *** hezhiqiang has quit IRC | 00:48 | |
| *** nakagawamsa has joined #openstack-swift | 00:54 | |
| clayg | who are the guys working on py3 in swift - it's like janonymous and the python core guy | 01:03 |
| clayg | victor | 01:03 |
| clayg | ayway - i'm getting annoyed about never knowing which file is going to want print 'blah' vs print('blah') - I wish there was only one way to do it in the whole swift code base so I could get used it already! | 01:04 |
| *** baojg has joined #openstack-swift | 01:29 | |
| *** sneti has joined #openstack-swift | 01:33 | |
| mattoliverau | clayg: this multplie patches that hit everything every now and again is frustrating, we kinda just want to line up all py3 patches, apply them all and be done with it. | 01:35 |
| * mattoliverau thinks this could just be me being fedup being sick | 01:36 | |
| clayg | mattoliverau: no way - that is a totally reasonable thing to do! | 01:36 |
| mattoliverau | then only one more rebase for all patches and it's done | 01:37 |
| *** baojg has quit IRC | 01:38 | |
| mattoliverau | Once we're py3 compat we can go tell someone to go investigate asyncio and see if it is better then greenlet ;) | 01:39 |
| clayg | mattoliverau: done that - it's not | 01:40 |
| mattoliverau | damn | 01:40 |
| clayg | mattoliverau: golang! | 01:41 |
| mattoliverau | \o/ golang | 01:41 |
| *** 7F1AALPHR has joined #openstack-swift | 01:43 | |
| *** chlong has joined #openstack-swift | 01:46 | |
| *** chlong_ has joined #openstack-swift | 01:53 | |
| *** baojg has joined #openstack-swift | 01:54 | |
| *** 7F1AALPHR has quit IRC | 02:01 | |
| *** haomaiwang has joined #openstack-swift | 02:01 | |
| *** barker has joined #openstack-swift | 02:02 | |
| *** thurloat is now known as thurloat_isgone | 02:06 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Add a client disconnect test at multhiphase commit https://review.openstack.org/224387 | 02:19 |
| clayg | kota_: did you just beat me to it? I was about to attach a failing test to the bug :) | 02:20 |
| kota_ | clayg: Are you already working? | 02:21 |
| kota_ | clayg: I was reading your comment for missing container updates. | 02:21 |
| clayg | kota_: nm, we're working on differnt things - i was writing a failing test for lp bug #1496636 | 02:21 |
| openstack | Launchpad bug 1496636 in OpenStack Object Storage (swift) "EC: Chunked transfer/commit protocol is *not* HTTP" [Undecided,New] https://launchpad.net/bugs/1496636 | 02:21 |
| clayg | ... and you're looking at container updates :) | 02:21 |
| clayg | to team swift! | 02:21 |
| kota_ | clayg: And i noticed I got misunderstading for object-server behavior. | 02:22 |
| kota_ | oh, you mean that chunked transfer, wait... | 02:22 |
| clayg | does patch 224387 prove my comments on the container-update change wrong? | 02:22 |
| patchbot | clayg: https://review.openstack.org/#/c/224387/ | 02:22 |
| kota_ | no, you are true on the container-update change. | 02:23 |
| clayg | damn | 02:23 |
| kota_ | What I was misunderstanding is "object-server can update container when .data lands" | 02:24 |
| kota_ | but it's wrong. | 02:24 |
| kota_ | object-server updates the container only when .durable landed correctly. | 02:24 |
| clayg | yeah because the object server doesn't know how many *other* servers successfully wrote the data | 02:24 |
| kota_ | patch 224387 is for making sure the behavior. | 02:25 |
| patchbot | kota_: https://review.openstack.org/#/c/224387/ | 02:25 |
| clayg | sending the container update from one node (if he was the only one that got the bytes) is almost as bad ad missing the update :'( | 02:25 |
| clayg | kota_: cool - descriptive tests are always *so* welcome | 02:25 |
| kota_ | clayg: exactly | 02:25 |
| kota_ | And I will update my patch for container updates according to your comment. | 02:26 |
| clayg | brute force! | 02:27 |
| kota_ | maybe ndata + quorum_size(conainer_updates) might be good? | 02:27 |
| kota_ | to push the header into object-server, anyways, I'll add comments during today. | 02:28 |
| clayg | kota_: maybe indeed! If the one and only object server that acked was the only one that successfully wrote the container async and his disk dies | 02:29 |
| clayg | the other's could propogate the .durable - but the container may never be updated :'( | 02:29 |
| clayg | %^&* it - just send it to all of em! | 02:29 |
| kota_ | that's too simple :\ | 02:30 |
| clayg | LOL! | 02:31 |
| clayg | grep -i simple CONTRIBUTING.md ;) | 02:31 |
| kota_ | LOL | 02:32 |
| *** david-lyle has joined #openstack-swift | 02:34 | |
| *** barker has quit IRC | 02:36 | |
| *** barker has joined #openstack-swift | 02:37 | |
| clayg | shit - wrong bug :'( | 02:37 |
| *** mahatic has joined #openstack-swift | 02:40 | |
| kota_ | clayg: which one? | 02:40 |
| clayg | i was working on a test for lp bug #1496205 - but attached it to the commit transfer is not HTTP bug | 02:40 |
| openstack | Launchpad bug 1496205 in OpenStack Object Storage (swift) "EC: Client Disconnect leaves inaccessible data on disk" [Undecided,New] https://launchpad.net/bugs/1496205 | 02:40 |
| clayg | although - I think it's all the same bug - we're going to fix everything in patch 211338 ;) | 02:41 |
| patchbot | clayg: https://review.openstack.org/#/c/211338/ | 02:41 |
| kota_ | clayg: I see, thx. | 02:42 |
| clayg | I *hope* that the object server can detect if the "end" (really the middle) of the chunked request isn't a MIME document boundary it can know it was disconnected | 02:42 |
| kota_ | yup | 02:43 |
| clayg | it may even be possible to fix eventlet to yeild out the chunks of a chunked transfer as it recieves them instead of that buffer up and block until network read size or eof nonsense it has now that caused all that silly content_length/0\r\n\r\n nonsense in the first place | 02:43 |
| clayg | but I don't know when we'll get to take out self.queue.put('') from the proxy since object servers with old eventlet would hang forever waiting for 0\r\n\r\n before sending the commit confirm headers | 02:44 |
| *** patchbot has quit IRC | 02:44 | |
| clayg | I mean... not forever - node_timeout would tear down the connection - hopefully only a few requests would 503 while you're restarting your object servers - we'll have to patch eventlet first tho | 02:45 |
| clayg | oh well - done for tonite i 'spose | 02:45 |
| *** patchbot has joined #openstack-swift | 02:46 | |
| clayg | shit - i still need to work on that ring change | 02:46 |
| mahatic | jrichli: extended weekend here, we have a holiday today, taking off on Friday. Will be on and off. Will update if there is anything on trello. | 02:46 |
| clayg | I'm sorta boo berries on it now that I know how often we're making these crap rings now tho :'( | 02:46 |
| mahatic | clayg: and review my patch :P kidding, no hurry | 02:47 |
| *** barker has quit IRC | 02:50 | |
| kota_ | makes me sense about eventlet thing | 02:50 |
| *** chlong_ has quit IRC | 02:50 | |
| *** haomaiw__ has joined #openstack-swift | 02:52 | |
| *** haomaiwang has quit IRC | 02:53 | |
| *** jkugel has quit IRC | 02:54 | |
| *** haomaiw__ has quit IRC | 03:01 | |
| *** mahatic has quit IRC | 03:02 | |
| *** haomaiwa_ has joined #openstack-swift | 03:02 | |
| *** bill_az has quit IRC | 03:02 | |
| notmyname | good evening | 03:03 |
| notmyname | just got through reading over the day's playback buffer | 03:03 |
| mattoliverau | notmyname: o/ | 03:03 |
| notmyname | now for the 209 unread emails.... | 03:05 |
| *** kota_ has quit IRC | 03:06 | |
| *** baojg has quit IRC | 03:18 | |
| *** baojg has joined #openstack-swift | 03:23 | |
| *** baojg has quit IRC | 03:23 | |
| *** kei_yama has quit IRC | 03:27 | |
| *** kei_yama has joined #openstack-swift | 03:28 | |
| *** baojg has joined #openstack-swift | 03:33 | |
| *** sneti has quit IRC | 03:35 | |
| notmyname | great that the per disk rsync landed! | 03:40 |
| *** zhill has joined #openstack-swift | 03:43 | |
| *** zhill has quit IRC | 03:48 | |
| *** trifon has quit IRC | 03:59 | |
| *** haomaiwa_ has quit IRC | 04:01 | |
| *** haomaiwa_ has joined #openstack-swift | 04:02 | |
| *** wbhuber has joined #openstack-swift | 04:07 | |
| *** zhill has joined #openstack-swift | 04:11 | |
| *** wbhuber has quit IRC | 04:11 | |
| *** zhill has quit IRC | 04:15 | |
| *** chlong has quit IRC | 04:16 | |
| clayg | mahatic: which one? | 04:24 |
| clayg | yay per-disk rsync! | 04:24 |
| *** chlong has joined #openstack-swift | 04:33 | |
| *** baojg has quit IRC | 04:39 | |
| *** kota_ has joined #openstack-swift | 04:43 | |
| *** ChanServ sets mode: +v kota_ | 04:43 | |
| *** baojg has joined #openstack-swift | 04:43 | |
| *** SkyRocknRoll has joined #openstack-swift | 04:44 | |
| *** baojg has quit IRC | 04:47 | |
| *** chlong has quit IRC | 04:48 | |
| *** baojg has joined #openstack-swift | 04:48 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Add a client disconnect test at multhiphase commit https://review.openstack.org/224387 | 04:50 |
| *** baojg has quit IRC | 04:54 | |
| *** SkyRocknRoll has quit IRC | 04:57 | |
| *** breitz has joined #openstack-swift | 05:00 | |
| *** haomaiwa_ has quit IRC | 05:01 | |
| *** haomaiw__ has joined #openstack-swift | 05:04 | |
| *** breitz has quit IRC | 05:05 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:10 | |
| *** haomaiw__ has quit IRC | 05:15 | |
| *** haomaiwang has joined #openstack-swift | 05:15 | |
| *** hrou has quit IRC | 05:20 | |
| eranrom | kota_, Greetigs, good time to discuss EC global multi-region | 05:27 |
| *** trifon has joined #openstack-swift | 05:28 | |
| kota_ | eranrom: hello | 05:29 |
| kota_ | eranrom: I have an hour to stay at IRC. | 05:30 |
| kota_ | eranrom: so what's up? | 05:30 |
| eranrom | kota_: all is good. thanks | 05:31 |
| *** zhill has joined #openstack-swift | 05:32 | |
| eranrom | kota_: So the first question I have is whether you have made up your mind whether to use container sync or to extend the 'placement' mechanism to make sure fragments are disctributed appropriately between regions. | 05:32 |
| eranrom | s/disctributed/distributed | 05:32 |
| kota_ | I'd like to bet later one, right now. | 05:33 |
| kota_ | I'm writing a poc code for expanding placement mechanism | 05:33 |
| eranrom | kota_: sounds resonable to me, IMO its the right way to go. | 05:34 |
| kota_ | but not yet enough testing whether it's really fitting to actuall use case. | 05:34 |
| kota_ | quickly, reporting | 05:34 |
| kota_ | I've done small testing my code with normal statement. | 05:35 |
| kota_ | ( | 05:35 |
| kota_ | (no failure case) | 05:35 |
| kota_ | my poc code worked well and GET performance increased around 2x | 05:35 |
| eranrom | sounds good! | 05:35 |
| eranrom | what code are you using? | 05:35 |
| kota_ | but PUT performance I assumed 3x increasement resuleted in what I expected. | 05:35 |
| kota_ | around 1.2~1.3x increasement. | 05:36 |
| kota_ | currently digging the reason. | 05:36 |
| kota_ | wait | 05:36 |
| *** zhill has quit IRC | 05:36 | |
| kota_ | eranrom: here, https://review.openstack.org/#/c/219165/ | 05:36 |
| kota_ | note that it is still WIP code, no enough error handling. | 05:37 |
| *** baojg has joined #openstack-swift | 05:39 | |
| *** baojg has quit IRC | 05:41 | |
| kota_ | ah, I got mistake to write down, PUT result doesn't fit the expected result. | 05:41 |
| eranrom | sure, actually my question is on a higher level (I am not currently familiar with the EC at all so I am not in a position to review it), I am trying to understand what is the right code to be used for e.g. 2 regions vs. 3 regions etc. | 05:43 |
| eranrom | and probably suggest that in terms of performance tests it would be good to concentrate on codes that are suitable for the global situaltion | 05:44 |
| eranrom | e.g. for 3 regions you may want to have a 6:6 code | 05:44 |
| kota_ | eranrom: sure | 05:45 |
| kota_ | eranrom: my recomendation for 3 region is 11:7 | 05:45 |
| eranrom | and for 2? | 05:45 |
| kota_ | I don't think so. | 05:46 |
| eranrom | I mean what would you recommend for 2 regions? | 05:46 |
| kota_ | ah...nice question. | 05:46 |
| kota_ | around 10:4 * 2? | 05:47 |
| eranrom | you mean replicate 10:4? | 05:47 |
| kota_ | if we can take more ndata, 20:4 * 2 is more efficient. | 05:47 |
| kota_ | depends on use case. | 05:47 |
| kota_ | If we don't consider about DR, I mean DR is a region might be gone completely. | 05:48 |
| eranrom | I am not sure I understand what you mean by 10:4 * 2? (what's the *2 means) | 05:48 |
| eranrom | ? | 05:48 |
| *** zhill has joined #openstack-swift | 05:48 | |
| kota_ | we must set enough parities more than 2x of original data, i.e. 6:6:, 10:10 or so. | 05:48 |
| kota_ | *2 is what I am developing for global ec cluster. | 05:49 |
| *** zhill has quit IRC | 05:49 | |
| kota_ | sorry | 05:49 |
| eranrom | replication? | 05:49 |
| kota_ | that means 10:18 if setting as parities. | 05:49 |
| eranrom | ok gotcha | 05:50 |
| kota_ | but the data volume will be 2.8x of original data on it. | 05:51 |
| kota_ | almost similar with triple replicated. | 05:51 |
| eranrom | I was just about to say that :-) | 05:51 |
| kota_ | if we can employ 20:4 *2 i.e. 20:28 | 05:52 |
| kota_ | that will be 2.4x of original. | 05:52 |
| kota_ | seems reasonable. | 05:52 |
| kota_ | However, increasing ndata is expensive for erasure coding calculation. | 05:54 |
| kota_ | (increasing ndata and nparity, strictly) | 05:55 |
| kota_ | And currently PyECLib support k+m up to 32. | 05:55 |
| eranrom | right. In the spec you were mentioning a mechanism that will be aware of the fragment index and place accordingly. From the codes you mention it seems that you are not going that way - Am I right? I mean with such a mechanism you could also do sonething like 7:10 - Am I missing something? | 05:57 |
| kota_ | That is why I propose new scheme like encoding 10:4 and place 28 (10:4 * 2(duplicated replicas)) fragments into Swift. | 05:57 |
| kota_ | I am writing according specs... | 05:59 |
| kota_ | according to | 05:59 |
| kota_ | Coud you point out the weird point in my code by ... like making a comment on gerrit? | 06:00 |
| *** haomaiwang has quit IRC | 06:01 | |
| eranrom | Nothing weird here :-) and it might be my EC ignorance. Do you mean point witha comment in the spec? | 06:01 |
| *** haomaiwang has joined #openstack-swift | 06:01 | |
| kota_ | code and spec, either is fine. | 06:01 |
| kota_ | we can ask/answer anytime in gerrit :-) | 06:02 |
| eranrom | sure will do, I could not find the spec yesterday. Any chance you have the link? | 06:02 |
| eranrom | I guess I need to sharpen my search capabilities... | 06:03 |
| kota_ | Oh, the spec landed, wait creating new thread... | 06:03 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift-specs: Add questions section into global_ec_cluster https://review.openstack.org/224439 | 06:05 |
| kota_ | done | 06:05 |
| eranrom | Thanks!!! and thanks for your time. I hope I am not waisting your time with EC ignorance :-) | 06:05 |
| kota_ | eranrom: no problem. Thanks for bringing up it. | 06:06 |
| *** zhill has joined #openstack-swift | 06:06 | |
| kota_ | eranrom: anytime I'm waiting comments to improve my spec and codes ;) | 06:07 |
| *** zhill has quit IRC | 06:07 | |
| kota_ | It's time to leave here. See you tomorrow. | 06:17 |
| *** kota_ has quit IRC | 06:17 | |
| *** chlong has joined #openstack-swift | 06:17 | |
| *** baojg has joined #openstack-swift | 06:18 | |
| *** baojg has quit IRC | 06:23 | |
| *** baojg has joined #openstack-swift | 06:26 | |
| *** silor has joined #openstack-swift | 06:37 | |
| *** haomaiwang has quit IRC | 06:51 | |
| *** dmorita has joined #openstack-swift | 06:52 | |
| *** haomaiwang has joined #openstack-swift | 06:53 | |
| *** zhill has joined #openstack-swift | 06:54 | |
| *** haomaiwang has quit IRC | 07:01 | |
| *** haomaiwang has joined #openstack-swift | 07:01 | |
| *** chenhuayi has joined #openstack-swift | 07:36 | |
| chenhuayi | hello, I have meet a problem about query with 'delimiter'. | 07:37 |
| chenhuayi | curl -k -i -X GET -H 'X-Auth-Token: TEMPAUTH_tk253eee65cac64ea097f4494aeee194e3' http://127.0.0.1:8080/v1/TEMPAUTH_test/testcontain2?delimiter=a | 07:38 |
| chenhuayi | HTTP/1.1 200 OK | 07:38 |
| chenhuayi | Content-Length: 30 | 07:38 |
| chenhuayi | X-Container-Object-Count: 9 | 07:38 |
| chenhuayi | Accept-Ranges: bytes | 07:38 |
| chenhuayi | X-Storage-Policy: Policy-0 | 07:38 |
| chenhuayi | X-Container-Bytes-Used: 91287 | 07:38 |
| chenhuayi | X-Timestamp: 1442473867.67031 | 07:38 |
| chenhuayi | Content-Type: text/plain; charset=utf-8 | 07:38 |
| chenhuayi | X-Trans-Id: tx352861a0605242528163f-0055fa6d83 | 07:38 |
| chenhuayi | Date: Thu, 17 Sep 2015 07:36:35 GMT | 07:38 |
| chenhuayi | 1 | 07:38 |
| chenhuayi | 12 | 07:38 |
| chenhuayi | 123 | 07:38 |
| chenhuayi | 2 | 07:38 |
| chenhuayi | LICENSE | 07:38 |
| chenhuayi | a | 07:38 |
| chenhuayi | ab | 07:38 |
| chenhuayi | abc | 07:38 |
| chenhuayi | b | 07:38 |
| chenhuayi | set the delimiter=a, but the result return all the objects. | 07:39 |
| chenhuayi | Anybody can help? thanks | 07:40 |
| *** rohit_ has quit IRC | 07:40 | |
| *** rledisez has joined #openstack-swift | 07:55 | |
| *** haomaiwang has quit IRC | 08:01 | |
| *** haomaiwa_ has joined #openstack-swift | 08:01 | |
| -openstackstatus- NOTICE: Gate is currently stuck, failing grenade upgrade tests due the release of oslo.utils 1.4.1 for Juno. | 08:02 | |
| *** geaaru has joined #openstack-swift | 08:06 | |
| *** jistr has joined #openstack-swift | 08:20 | |
| *** silor has quit IRC | 08:22 | |
| *** silor has joined #openstack-swift | 08:23 | |
| *** acoles_ is now known as acoles | 08:25 | |
| *** slo_ has joined #openstack-swift | 08:29 | |
| *** slo_ has quit IRC | 08:29 | |
| acoles | clayg: http://make-everything-ok.com/ | 08:39 |
| acoles | notmyname: new link for channel topic ^^ :-) | 08:39 |
| * acoles tries new gerrit screen | 08:39 | |
| * acoles reverts to old | 08:40 | |
| * acoles discovers gerrit option 'show person name in review category' | 08:42 | |
| mattoliverau | acoles: lol | 08:50 |
| *** hezhiqia_ has quit IRC | 08:52 | |
| *** zhill has quit IRC | 08:53 | |
| *** silor has quit IRC | 08:56 | |
| *** chenhuayi has quit IRC | 08:57 | |
| *** jordanP has joined #openstack-swift | 08:59 | |
| *** haomaiwa_ has quit IRC | 09:01 | |
| *** haomaiwang has joined #openstack-swift | 09:01 | |
| *** silor has joined #openstack-swift | 09:04 | |
| mattoliverau | chenhuayi: the delimiter isn't counted if its the first character of the object so delimiter=b or 2 should work in your example. Further, please use a paste.openstack.org, gist etc rather then pasting the whole output into irc. :) | 09:05 |
| acoles | mattoliverau: +1 i was just looking at that too. i guess the intent may have been to roll up dir objects named 'a/b', 'a/c' to a single listing 'a' by using delimiter = '/'. but yeah it doesn't work on the first char. | 09:12 |
| *** aix has quit IRC | 09:14 | |
| mattoliverau | acoles: and the find(delimilter, len(prefix)) worries me, it might need to be find(delimiter, max(len(prefix),1)) | 09:19 |
| mattoliverau | There might be an edge case bug there | 09:19 |
| acoles | mattoliverau: so the swiftclient usage for delimiter says 'go read the swift API doc' and the swift API doc is kinda vague about it :) | 09:23 |
| mattoliverau | Lol, I feel a doc bug coming.. But I guess we need to decide the correct behaviour first | 09:24 |
| *** flwang has quit IRC | 09:27 | |
| acoles | mattoliverau: yup. maybe one of the old-timers will remember the intent.i'm away for a bit. | 09:32 |
| *** bkopilov has quit IRC | 09:40 | |
| *** flwang has joined #openstack-swift | 09:41 | |
| *** luksaur has joined #openstack-swift | 09:41 | |
| *** silor has quit IRC | 09:42 | |
| *** silor has joined #openstack-swift | 09:43 | |
| *** hezhiqiang has joined #openstack-swift | 09:43 | |
| *** aix has joined #openstack-swift | 09:45 | |
| luksaur | hi, i'm thinking in deploy my swift environment in a cloud infrastructure, i did not find any case of a swift production deployment in virtual machines on internet | 09:45 |
| *** silor has quit IRC | 09:46 | |
| luksaur | noting that I'll use this environment as a backend-storage for a local docker-registry, so high performance is really not a problem here | 09:46 |
| *** silor has joined #openstack-swift | 09:46 | |
| luksaur | I just need a distributed and reliable env to place the docker images | 09:46 |
| luksaur | has someone using swift like that? | 09:47 |
| luksaur | oh, forgot to say, it is a private cloud structure, everything on the same local network :-) | 09:49 |
| *** bkopilov has joined #openstack-swift | 09:49 | |
| *** silor1 has joined #openstack-swift | 09:54 | |
| *** silor has quit IRC | 09:55 | |
| *** silor1 is now known as silor | 09:55 | |
| *** bkopilov has quit IRC | 09:58 | |
| *** silor has quit IRC | 10:00 | |
| *** silor has joined #openstack-swift | 10:01 | |
| *** haomaiwang has quit IRC | 10:01 | |
| *** haomaiwang has joined #openstack-swift | 10:01 | |
| *** silor has quit IRC | 10:05 | |
| *** silor has joined #openstack-swift | 10:10 | |
| *** bkopilov has joined #openstack-swift | 10:11 | |
| *** nakagawamsa has quit IRC | 10:12 | |
| *** silor has quit IRC | 10:13 | |
| *** silor has joined #openstack-swift | 10:14 | |
| -openstackstatus- NOTICE: Gate back to normal, thanks to the backlisting of the problematic version | 10:16 | |
| *** silor has quit IRC | 10:18 | |
| *** haomaiwang has quit IRC | 10:18 | |
| *** haomaiwang has joined #openstack-swift | 10:18 | |
| *** baojg has quit IRC | 10:22 | |
| *** baojg has joined #openstack-swift | 10:24 | |
| *** baojg has quit IRC | 10:28 | |
| *** silor has joined #openstack-swift | 10:29 | |
| *** silor has quit IRC | 10:30 | |
| *** silor has joined #openstack-swift | 10:31 | |
| *** silor1 has joined #openstack-swift | 10:37 | |
| *** silor has quit IRC | 10:37 | |
| *** silor1 is now known as silor | 10:37 | |
| *** mfalatic_ has quit IRC | 11:00 | |
| *** mfalatic has joined #openstack-swift | 11:00 | |
| *** haomaiwang has quit IRC | 11:01 | |
| *** haomaiwang has joined #openstack-swift | 11:01 | |
| *** KLuka__ has joined #openstack-swift | 11:09 | |
| *** slo__ has joined #openstack-swift | 11:10 | |
| *** jordanP has quit IRC | 11:40 | |
| *** m_kazuhiro has quit IRC | 11:45 | |
| *** haomaiwang has quit IRC | 11:54 | |
| *** silor has quit IRC | 12:14 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 12:17 |
| *** km has quit IRC | 12:19 | |
| *** haomaiwang has joined #openstack-swift | 12:23 | |
| *** silor has joined #openstack-swift | 12:30 | |
| *** jkugel has joined #openstack-swift | 12:52 | |
| *** aix has quit IRC | 12:56 | |
| *** janonymous has joined #openstack-swift | 12:58 | |
| *** dustins has joined #openstack-swift | 12:58 | |
| *** aix has joined #openstack-swift | 12:59 | |
| *** haomaiwang has quit IRC | 13:01 | |
| *** haomaiwang has joined #openstack-swift | 13:01 | |
| *** haypo has joined #openstack-swift | 13:14 | |
| *** kei_yama has quit IRC | 13:19 | |
| *** KLuka__ has quit IRC | 13:22 | |
| *** janonymous has quit IRC | 13:22 | |
| *** hrou has joined #openstack-swift | 13:23 | |
| *** wbhuber has joined #openstack-swift | 13:26 | |
| *** jrichli has joined #openstack-swift | 13:30 | |
| *** bill_az has joined #openstack-swift | 13:30 | |
| *** openstackgerrit has quit IRC | 13:31 | |
| *** openstackgerrit has joined #openstack-swift | 13:31 | |
| *** tongli has joined #openstack-swift | 13:31 | |
| *** oddsam91 has joined #openstack-swift | 13:31 | |
| *** jsuchome has joined #openstack-swift | 13:48 | |
| *** pberis has joined #openstack-swift | 13:48 | |
| jsuchome | hi, could anyone please take a look at https://review.openstack.org/#/c/189258/ ? | 13:49 |
| *** Fin1te has joined #openstack-swift | 13:49 | |
| *** slo__ has quit IRC | 13:54 | |
| *** oddsam91 has left #openstack-swift | 13:56 | |
| *** lcurtis has joined #openstack-swift | 14:00 | |
| *** haomaiwang has quit IRC | 14:01 | |
| *** 5EXAB5K66 has joined #openstack-swift | 14:01 | |
| *** sneti has joined #openstack-swift | 14:07 | |
| *** Fin1te has quit IRC | 14:09 | |
| *** pberis has quit IRC | 14:20 | |
| *** pberis has joined #openstack-swift | 14:23 | |
| *** barker has joined #openstack-swift | 14:27 | |
| *** jordanP has joined #openstack-swift | 14:27 | |
| *** esker has joined #openstack-swift | 14:30 | |
| *** amoturi has joined #openstack-swift | 14:32 | |
| *** jlhinson has joined #openstack-swift | 14:36 | |
| *** trifon has quit IRC | 14:40 | |
| *** tsimp06 has quit IRC | 14:40 | |
| *** minwoob has joined #openstack-swift | 14:42 | |
| *** barker has quit IRC | 14:43 | |
| *** nadeem has joined #openstack-swift | 14:53 | |
| *** nadeem has quit IRC | 14:55 | |
| *** nadeem has joined #openstack-swift | 14:56 | |
| *** nadeem has quit IRC | 14:57 | |
| *** 5EXAB5K66 has quit IRC | 15:01 | |
| *** haomaiwang has joined #openstack-swift | 15:01 | |
| *** hezhiqiang has quit IRC | 15:01 | |
| *** nadeem has joined #openstack-swift | 15:05 | |
| *** janonymous has joined #openstack-swift | 15:07 | |
| *** mc_nair has joined #openstack-swift | 15:16 | |
| *** DericHorn-HP has joined #openstack-swift | 15:18 | |
| *** tongli has quit IRC | 15:19 | |
| *** mfalatic_ has joined #openstack-swift | 15:36 | |
| *** mfalatic has quit IRC | 15:36 | |
| *** tongli has joined #openstack-swift | 15:39 | |
| *** mc_nair has quit IRC | 15:40 | |
| *** devlaps has joined #openstack-swift | 15:40 | |
| *** aix has quit IRC | 15:43 | |
| *** annegentle has joined #openstack-swift | 15:47 | |
| *** annegentle has quit IRC | 15:52 | |
| *** annegentle has joined #openstack-swift | 15:53 | |
| *** mc_nair has joined #openstack-swift | 15:54 | |
| *** haomaiwang has quit IRC | 16:01 | |
| *** haomaiwang has joined #openstack-swift | 16:01 | |
| *** Fin1te has joined #openstack-swift | 16:07 | |
| *** evilrob has joined #openstack-swift | 16:11 | |
| *** bkopilov has quit IRC | 16:16 | |
| *** tsimp06 has joined #openstack-swift | 16:21 | |
| *** bkopilov has joined #openstack-swift | 16:23 | |
| tsimp06 | is there a way to use the swift CLI with a swift compatible API on the backend? I have nova pointing to the compatible api, but can't find where my 404's are being generated from | 16:24 |
| *** silor has quit IRC | 16:24 | |
| *** silor1 has joined #openstack-swift | 16:25 | |
| *** silor1 has quit IRC | 16:29 | |
| tsimp06 | nvm, figured it out. | 16:29 |
| *** rledisez has quit IRC | 16:30 | |
| *** jordanP has quit IRC | 16:33 | |
| *** garthb has joined #openstack-swift | 16:33 | |
| *** jistr has quit IRC | 16:35 | |
| *** david-lyle has quit IRC | 16:36 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Enable object server to return non-durable data https://review.openstack.org/215276 | 16:38 |
| *** david-lyle has joined #openstack-swift | 16:47 | |
| *** DericHorn-HP has quit IRC | 16:47 | |
| *** breitz has joined #openstack-swift | 16:48 | |
| *** acoles is now known as acoles_ | 16:50 | |
| clayg | acoles_: thanks for that link | 16:50 |
| clayg | minwoob: while the make-everything-ok button is in progress is there anything I can merge that would make ec better? | 16:52 |
| *** DericHorn-HP has joined #openstack-swift | 16:56 | |
| *** jsuchome has quit IRC | 16:56 | |
| *** sneti has quit IRC | 16:57 | |
| wbhuber | clayg: minwoob is out for lunch, but should be back shortly. | 16:58 |
| clayg | wbhuber: what about you - got any patches for me!? | 16:58 |
| wbhuber | clayg: actually i'm working on the dancing etag bug that you opened not so long ago | 16:59 |
| clayg | wbhuber: heh, great description! | 16:59 |
| *** haomaiwang has quit IRC | 17:01 | |
| *** haomaiwa_ has joined #openstack-swift | 17:01 | |
| *** hezhiqiang has joined #openstack-swift | 17:02 | |
| *** david-lyle has quit IRC | 17:03 | |
| *** Fin1te has quit IRC | 17:04 | |
| *** hezhiqiang has quit IRC | 17:06 | |
| *** SkyRocknRoll has quit IRC | 17:20 | |
| *** SkyRocknRoll has joined #openstack-swift | 17:21 | |
| *** geaaru has quit IRC | 17:23 | |
| *** SkyRocknRoll has quit IRC | 17:24 | |
| *** SkyRocknRoll has joined #openstack-swift | 17:24 | |
| clayg | gah, this resetting of content-length by set_hundered_continue_headers is making it terribly difficult to demonstrate the behaviors I want to see in tests (as in the tests are instead demonstrating that our services do not have the behaviors we want) :'( | 17:26 |
| *** SkyRocknRoll has quit IRC | 17:27 | |
| *** SkyRocknRoll has joined #openstack-swift | 17:28 | |
| *** haomaiwa_ has quit IRC | 17:28 | |
| *** abaranos has joined #openstack-swift | 17:29 | |
| *** zhill has joined #openstack-swift | 17:30 | |
| abaranos | I'm able to connect to our Swift instance using the swift command client, but having trouble connecting via raw HTTP. (The application is in Clojure so needs to be able to hit the http API directly) | 17:30 |
| *** chsc has joined #openstack-swift | 17:31 | |
| abaranos | This is what I'm doing w/ the swift client: | 17:32 |
| abaranos | export OS_AUTH_URL=https://us-texas-3.foo.com:5000/v2.0 | 17:32 |
| abaranos | export OS_TENANT_ID=asdasdasdsadsdadadasdsaddasdasd | 17:32 |
| abaranos | export OS_TENANT_NAME="my-tenant-nae" | 17:32 |
| abaranos | export OS_USERNAME=$SWIFT_ACCOUNT | 17:32 |
| abaranos | export OS_PASSWORD=$SWIFT_KEY | 17:32 |
| abaranos | export OS_REGION_NAME="us-texas-3" | 17:32 |
| clayg | whoa - good morning | 17:32 |
| abaranos | swift list ## => foo | 17:32 |
| clayg | abaranos: well all of those options are for keystone - which is a distinct REST service from Swift with it's own api to deliver the storage-url and token | 17:33 |
| *** mfalatic_ has quit IRC | 17:33 | |
| abaranos | clayg: that's what confuses me: I thought keystone was swift version 2+ | 17:33 |
| clayg | I think the command line client has a --debug flag - it might show you something about the keystone api requests | 17:33 |
| abaranos | but nowhere is OS_AUTH_VERSION set to 2 | 17:34 |
| clayg | abaranos: well it figures it out based on all the other options being there | 17:34 |
| clayg | e.g. the v2.0 in the auth url | 17:34 |
| clayg | but that's just the keystone api version - swift only has v1 | 17:34 |
| abaranos | clayg: ugh. I read through the client code for 40 minute probably combined, and couldn't find that inference . lol. figured it must be | 17:34 |
| abaranos | clayg: yup v2 in url was very suspicious :) | 17:35 |
| clayg | I mean I think mainly swiftclient just passes all of that off to the keystone client to get the storage-url and api-token | 17:35 |
| clayg | so I don't really know for sure what sort of sillyness it's doing | 17:35 |
| clayg | but w/o the ST_[AUTH|USER|KEY] options it's not going to be able to make a v1 auth request - so if it's working it's definately passing the OS_* options of to keystone and from there the keystone client just magics! | 17:36 |
| abaranos | clayg: ok, then if I had to authenticate with a v2.0 url what might that look lik eit raw http? | 17:36 |
| clayg | abaranos: but I'm sure Keystone has some api docs somewhere that will show you how to make a v2 auth request and get back the stuff you need to talk to swift | 17:36 |
| clayg | abaranos: it's sorta crappy - you hve to POST some json - well that's not *too* crappy | 17:37 |
| clayg | but like I don't do it with curl that much ;) | 17:37 |
| *** mragupat has joined #openstack-swift | 17:37 | |
| ctennis | abaranos: just add "-v --debug" to your swift command it will show the curl commands it's using behind the scenes | 17:37 |
| abaranos | ctennis: I add --verbose and it showed me nothing | 17:38 |
| clayg | abaranos: maby this? http://developer.openstack.org/api-ref-identity-v2.html#authenticate-v2.0 | 17:38 |
| ctennis | abaranos: add --debug too | 17:38 |
| clayg | abaranos: yeah I think --d ^ what ctennis said | 17:38 |
| *** trifon has joined #openstack-swift | 17:39 | |
| minwoob | clayg: We could try to close these out | 17:41 |
| minwoob | https://review.openstack.org/#/c/196848/ | 17:41 |
| minwoob | https://review.openstack.org/#/c/221956/ | 17:41 |
| *** janonymous has quit IRC | 17:41 | |
| jrichli | clayg: patch 218490 would probably be a quick review. it has a -1 for a race condition, which will be fixed. but your feedback would be great to have! | 17:42 |
| patchbot | jrichli: https://review.openstack.org/#/c/218490/ | 17:42 |
| clayg | minwoob: ok | 17:42 |
| *** abaranos has quit IRC | 17:42 | |
| ajiang | Is it safe to use 2.4 ring builder to add weight to take advantage of https://github.com/openstack/swift/commit/2328983b7535d24d08942e62095b4346f2626566 and push the ring to the 2.2.2 cluster? | 17:42 |
| clayg | jrichli: i'm not sure i'll get to that one today - really trying to move the needle on EC :'( | 17:42 |
| jrichli | clayg: ok, np | 17:43 |
| clayg | jrichli: but it looks like an awesome patch! thanks for pointing it out! Would be very nice to have indeed! | 17:43 |
| clayg | minwoob: jrichli: thanks | 17:43 |
| *** breitz has quit IRC | 17:43 | |
| minwoob | clayg: Actually for the first one that I posted, there is probably a better way to check that a frag has completely reverted (sanity check that none of the .data, .durable files exist anymore). | 17:44 |
| *** sneti has joined #openstack-swift | 17:45 | |
| *** breitz has joined #openstack-swift | 17:45 | |
| wbhuber | clayg: are you looking into merging all EC-related patches out there or looking for a specific patch from minwoob (i.e. a helper method that takes lisdirs and groups files, etc.)? | 17:45 |
| evilrob | in connecting to our VPN, abaranos got disconnected. He wanted me to thank you clayg and ctennis for responding. The -v --debug shows him exactly what he needed. Thanks | 17:46 |
| *** pberis has quit IRC | 17:47 | |
| *** pberis1 has joined #openstack-swift | 17:47 | |
| ctennis | evilrob: cool, thanks | 17:47 |
| evilrob | I guess now I have to teach him about IRC bouncers | 17:47 |
| evilrob | :) | 17:47 |
| *** pberis1 is now known as pberis | 17:49 | |
| ajiang | Sorry to ask it again: Is it safe to use 2.4 ring builder to add weight to take advantage of https://github.com/openstack/swift/commit/2328983b7535d24d08942e62095b4346f2626566 and push the ring to the 2.2.2 cluster? | 17:50 |
| clayg | wbhuber: I've got a couple of EC issues I'm tracking and frustrated with my progress so I'm looking for some other things that "make EC better" that I've been ignoring so I can feel like we made some progress this week. | 17:50 |
| clayg | ajiang: bah! sorry I missed it - yup | 17:51 |
| clayg | ajiang: on disk ring format hasn't changed *in a while* | 17:51 |
| ajiang | clayg: Thanks for confirmation ! | 17:51 |
| clayg | ajiang: but I guess it *sorta* depends on just how old your current cluster is | 17:51 |
| clayg | oh 2.2.2 - no you're fine | 17:51 |
| ajiang | clayg: thanks! | 17:51 |
| minwoob | clayg: I've been looking through peluse and acoles_ make-everything-ok patches to try to help push those along. | 17:52 |
| clayg | minwoob: much appreciated | 17:52 |
| clayg | minwoob: I haven't heard from peluse in awhile - I think he must be elbow deep in graphs | 17:53 |
| clayg | minwoob: but he had a banch trying to catch some ValueErrors that needs some tighting up | 17:53 |
| *** sneti has quit IRC | 17:53 | |
| clayg | minwoob: all of acoles_ stuff is pretty far out there - maybe the simple DiskFile refactors would help | 17:53 |
| minwoob | clayg: It seems that the low and medium EC bugs on LP wouldn't be release blockers or anything, so have been focusing on reviewing for those big ones. | 17:53 |
| minwoob | I see. | 17:54 |
| clayg | minwoob: yeah I've been trying to focus on the big stuff as well | 17:54 |
| clayg | minwoob: there was one I had for fixing revert jobs of tombstones or something? | 17:54 |
| clayg | minwoob: I don't know if it just needs more reviews or testing or a rebase - I haven't checked on it | 17:54 |
| clayg | minwoob: thanks for the help! good luck today! | 17:54 |
| minwoob | clayg: Yeah, if there are any other release blocker ones then please send them to me. I'm hoping to finish the review for peluse pretty soon. | 17:57 |
| *** sneti has joined #openstack-swift | 18:00 | |
| *** rohit_ has joined #openstack-swift | 18:00 | |
| *** jamielennox is now known as jamielennox|away | 18:05 | |
| *** Fin1te has joined #openstack-swift | 18:13 | |
| tsimp06 | is swift sending the POST of /v1/tokens, or is keystone? | 18:21 |
| *** NM has joined #openstack-swift | 18:37 | |
| *** Fin1te has quit IRC | 18:38 | |
| briancline | it's up to the user to get a token, but swift will ask keystone to verify it if swift doesn't have it cached | 18:42 |
| *** Fin1te has joined #openstack-swift | 18:46 | |
| *** greghaynes has quit IRC | 18:49 | |
| cschwede | because of the DHT this reminds me a *lot* to Swift: https://ipfs.io/ipns/ipfs.io/ | 18:54 |
| cschwede | like giving read-access directly to storage nodes | 18:54 |
| tsimp06 | my problem was that I had overridden the endpoint to a non-keystone box, and it was still sending a /v1/tokens request, which was failing, but routing through keystone and v2, it works fine | 19:00 |
| *** Fin1te has quit IRC | 19:04 | |
| *** devlaps has quit IRC | 19:08 | |
| *** david-lyle has joined #openstack-swift | 19:11 | |
| *** trifon has quit IRC | 19:14 | |
| *** trifon has joined #openstack-swift | 19:15 | |
| *** delattec has joined #openstack-swift | 19:19 | |
| *** cdelatte has quit IRC | 19:22 | |
| *** david-lyle has quit IRC | 19:25 | |
| wbhuber | peluse: clayg: with the graphs ready, would we use them to determine EC is awesome or say EC needs more work? or we'd ship EC as is regardless of the indicators in the graphs? | 19:25 |
| *** nadeem has quit IRC | 19:27 | |
| *** delatte has joined #openstack-swift | 19:43 | |
| *** delattec has quit IRC | 19:46 | |
| *** nadeem has joined #openstack-swift | 19:48 | |
| clayg | wbhuber: I don't think graphs will provide a deciscive impact on EC readiness like that - it's just one input | 19:49 |
| clayg | wbhuber: that said - I'm very much looking forward to peluse's collective thoughts and experiences from his work - I'm sure it will be facisnating and provide invaluable input to the community! | 19:49 |
| clayg | further, because I value peluse's objective technical evaluation so highly - his opinion of EC readiness based on his work in the lab is much more likely to have an impact on my views regarding our current progress with EC than his ability to demonstrate excicting things with pictures (same to ctennis!) | 19:52 |
| clayg | so far both of them have told me - "fix the bugs" - more than "make the graphs look better" | 19:52 |
| * clayg goes back to hole | 19:52 | |
| *** barker has joined #openstack-swift | 19:54 | |
| openstackgerrit | Michael Barton proposed openstack/swift: go: add a way to terminate replication cleanly https://review.openstack.org/224881 | 19:57 |
| *** sneti has quit IRC | 19:57 | |
| *** sneti has joined #openstack-swift | 19:57 | |
| *** amoturi has quit IRC | 20:00 | |
| *** mc_nair has quit IRC | 20:00 | |
| *** dustins has quit IRC | 20:06 | |
| *** rubal has joined #openstack-swift | 20:11 | |
| rubal | hi.. i am new to swift | 20:11 |
| rubal | i would like someone's guidance on the installation procedure | 20:11 |
| eranrom | rubal: Are you looking at an installation for dev & tests or trying a real deployment install? | 20:12 |
| rubal | eranrom : i am using a book, by oreilly | 20:13 |
| rubal | and trying on my vm | 20:13 |
| rubal | every thing went well, until getting the verification on the account | 20:14 |
| eranrom | rubal: details? | 20:15 |
| *** proteusguy_ has quit IRC | 20:15 | |
| rubal | eranrom : i am receiving a 404 error to the query curl -v -H 'X-Storage-Token: AUTH_mytoken' http://127.0.0.1:8080/v1/AUTH_myaccount/ | 20:16 |
| *** trifon has quit IRC | 20:18 | |
| rubal | eranrom: i got a 200 acknowledgement before for trying out with my username and password | 20:19 |
| *** fifieldt has quit IRC | 20:19 | |
| eranrom | rubal: and I assume that along with that 200 you also got the X-Auth-Token and X-Storage-Url right? | 20:19 |
| eranrom | rubal, that is in the response headers | 20:20 |
| rubal | eranrom: yes, and i used the same for my next query | 20:20 |
| rubal | eranrom: for which i received 404 | 20:20 |
| *** fifieldt has joined #openstack-swift | 20:20 | |
| eranrom | rubal: your curl shows X-Storage-Token shouldn't it be X-Auth-Token? | 20:21 |
| rubal | eranrom: The book said Storage, i have also tried using Auth, it returned the same response | 20:22 |
| rubal | eranrom: 404 | 20:22 |
| eranrom | rubal: Also, AFAIK, curl with no -X <method> defauts to POST, and I assume that in this case you wanted to do -X HEAD (that is to do the verification) | 20:22 |
| clayg | eranrom: without a body default is GET | 20:23 |
| clayg | eranrom: id you do the auth request again - did you get back the same token? | 20:24 |
| rubal | yeah, we are trying to retrieve the status of the account with that | 20:24 |
| rubal | eranrom: i should recieve 204 | 20:24 |
| clayg | rubal: well the -v will show you the response headers so that's about good enough | 20:24 |
| rubal | eranrom: not 404 | 20:24 |
| eranrom | clayg, good to know...(GET default) | 20:25 |
| clayg | rubal: if you do the auth request again - did you get back the same token? | 20:25 |
| rubal | +clayg: yes, same token. memcache is working fine | 20:25 |
| rubal | clayg: what can cause this error? | 20:28 |
| rubal | anyclue? | 20:28 |
| *** proteusguy_ has joined #openstack-swift | 20:28 | |
| *** haomaiwang has joined #openstack-swift | 20:29 | |
| *** mragupat has quit IRC | 20:29 | |
| *** SkyRocknRoll has quit IRC | 20:30 | |
| *** gyee has joined #openstack-swift | 20:31 | |
| eranrom | rubal: I wonder if this can be a problem with the device. Can you identify the swift device and see if it has an account created on it at all? look for an 'account' dir and look under it | 20:31 |
| *** trifon has joined #openstack-swift | 20:33 | |
| *** haomaiwang has quit IRC | 20:33 | |
| *** barker has quit IRC | 20:34 | |
| *** mac_ified has joined #openstack-swift | 20:36 | |
| *** rubal_ has joined #openstack-swift | 20:38 | |
| *** esker has quit IRC | 20:41 | |
| *** rubal_ has quit IRC | 20:42 | |
| *** Fin1te has joined #openstack-swift | 20:52 | |
| *** annegentle has quit IRC | 20:52 | |
| *** tongli has quit IRC | 20:55 | |
| rubal | eranrom: okey | 20:57 |
| rubal | eranrom: lemme check | 20:57 |
| openstackgerrit | Tim Burke proposed openstack/swift: Add the ability to specify ranges for SLO segments https://review.openstack.org/211726 | 20:58 |
| *** dipe has quit IRC | 21:04 | |
| *** david-lyle has joined #openstack-swift | 21:06 | |
| *** david-lyle has quit IRC | 21:09 | |
| *** sneti has quit IRC | 21:12 | |
| tsimp06 | does swift work with keystone v3? | 21:16 |
| hrou | tsimp06, yep! But from the swift client you need to specify the version through '-V 3' | 21:21 |
| hrou | e.g. swift -V 3 list | 21:21 |
| *** jrichli has quit IRC | 21:21 | |
| hrou | It defaults to V2 | 21:22 |
| tsimp06 | ok, thanks | 21:22 |
| hrou | np! | 21:22 |
| *** jamielennox|away is now known as jamielennox | 21:24 | |
| openstackgerrit | Minwoo Bae proposed openstack/swift: EC GET path: support use of alternate fragments on object servers https://review.openstack.org/213147 | 21:25 |
| tsimp06 | what is the most common swift load generation tool? | 21:26 |
| tsimp06 | what i have is swift cli using keystone to validate the user and writing to cleversafe storage | 21:28 |
| tsimp06 | which works, but i don't know which endpoint to use for continuous I/O | 21:28 |
| *** wbhuber has quit IRC | 21:43 | |
| *** jst_ has joined #openstack-swift | 21:44 | |
| *** jst_ has quit IRC | 21:48 | |
| *** rubal has quit IRC | 21:50 | |
| *** pberis has quit IRC | 21:51 | |
| *** Fin1te has quit IRC | 21:54 | |
| *** DericHorn-HP has quit IRC | 21:54 | |
| *** pberis has joined #openstack-swift | 21:57 | |
| *** pberis has quit IRC | 21:58 | |
| clayg | tsimp06: swift-bench was sorta 'standard' albiet not very sophisticated, the industry wants cosbench to be the thing because it isn't swift specific per-say (but every time I know someone use it it ends up not generating the load they expected), ss-bench came out of SwiftStack's requirements for a decent load generation tool - and it is in fact... pretty decent as far as these things go. | 22:00 |
| *** fbo has quit IRC | 22:03 | |
| *** fbo has joined #openstack-swift | 22:04 | |
| peluse | tsimp06, I've been using both for the last month or so... have been able to confirm same results w/either and there are pros and cons to both depending on what you are trying to do and personal preference | 22:07 |
| *** annegentle has joined #openstack-swift | 22:08 | |
| *** jkugel has quit IRC | 22:09 | |
| *** lcurtis has quit IRC | 22:17 | |
| *** man-a-nickname has joined #openstack-swift | 22:22 | |
| *** hezhiqiang has joined #openstack-swift | 22:23 | |
| tsimp06 | +clayg: Thanks, I have cosbench, don't have an endpoint that's not internal to our application to point it to, I'll check out ss-bench | 22:24 |
| *** mfalatic has joined #openstack-swift | 22:25 | |
| man-a-nickname | I am using keystone auth; Can a user with read/write (does not have a role in operator_roles) to a container list the containers in the account? | 22:26 |
| peluse | tsimp06 you'll point either to your proxy | 22:26 |
| peluse | tsimp06, but you'll likely get more swift specific support using ssbench... and you are likely to find it easier to setup and get running as well | 22:27 |
| peluse | support meaning people helping you use it :) | 22:27 |
| *** delatte has quit IRC | 22:28 | |
| mattoliverau | Morning | 22:28 |
| man-a-nickname | The account DB is would need (part of) the metadata of the container in the listing, I guess. | 22:28 |
| *** hezhiqiang has quit IRC | 22:28 | |
| tsimp06 | alrighty :) | 22:28 |
| *** annegentle has quit IRC | 22:30 | |
| man-a-nickname | any ideas? | 22:31 |
| tsimp06 | man-a-nickname is it not working? i would assume list containers requires the admin role | 22:32 |
| tsimp06 | unless you own the container that is | 22:33 |
| man-a-nickname | +tsimp06 There is a header to set for the "owner" of that container? | 22:36 |
| *** annegentle has joined #openstack-swift | 22:37 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Fix client disconnect during multhiphase commit https://review.openstack.org/224387 | 22:38 |
| clayg | ^ kota: I sorta ransacked your change :\ | 22:38 |
| clayg | kota: but it was a really great idea! | 22:38 |
| clayg | MOAR TESTS! | 22:39 |
| *** dipe has joined #openstack-swift | 22:40 | |
| peluse | clayg, nice - will check it out! | 22:40 |
| clayg | peluse: oh thanks bro! | 22:42 |
| clayg | peluse: I think the ChunkReadError is key - I'm going to look at adding that over the ValueError catching change - I need to track down all the calls to wsgi_input.read and make sure IOError/ValueError get translated - then I think we're golden to find except ClientDisconnect and make it (ClientDisconnect, ChunkReadError) - maybe? torgomatic? | 22:43 |
| clayg | peluse: but I'm probably not going to get to that this afternoon :\ | 22:43 |
| peluse | clayg, still running the EC perf tests on the last ver... once I get another run done I can update as it would hit these errors quite frequently based on load | 22:44 |
| torgomatic | isn't ClientDisconnect what *we* raise when we get a ChunkReadError? | 22:44 |
| clayg | peluse: a) I have an idea for lp bug #1496205 that I want to flesh out | 22:44 |
| openstack | Launchpad bug 1496205 in OpenStack Object Storage (swift) "EC: Client Disconnect leaves inaccessible data on disk" [Undecided,New] https://launchpad.net/bugs/1496205 | 22:44 |
| torgomatic | ChunkReadError just means "hey, there was supposed to be some chunked-encoding stuff here and now there's not, so, uh, wtf" | 22:44 |
| clayg | peluse: b) I want to see if I can get a +2 on minwoob's changes | 22:44 |
| clayg | patch 196848 | 22:44 |
| patchbot | clayg: https://review.openstack.org/#/c/196848/ | 22:44 |
| clayg | patch 221956 | 22:44 |
| patchbot | clayg: https://review.openstack.org/#/c/221956/ | 22:44 |
| peluse | clayg, I saw that fly by but haven't read it yet, have been sucked away from work almost entirely for the last week or so | 22:44 |
| man-a-nickname | +tsimp06 I as a non-admin can HEAD/GET the container just fine. But cannot GET/HEAD the account to find out what containers I have to GET/HEAD. Am I missing something here? | 22:45 |
| *** hrou has quit IRC | 22:45 | |
| peluse | man, a lot has happened lately :) | 22:45 |
| clayg | torgomatic: I don't know for sure where we raise ClientDisconnect from? maybe I"m saying it wrong - maybe we catch mostly ChunkReadTimeout | 22:45 |
| torgomatic | clayg: that'd make sense | 22:46 |
| clayg | torgomatic: that's probably it - but sometimes instead of ChunkReadTimeout we get some bs protocol error because - well I guess because sock.recv can return ''? that doesn't sound quite right either... make it's all the makefile stuff that eventlet.wsgi does to get readline - and then readline will return '' to indicate EOF and eventlet.wsgi is just stupid | 22:47 |
| peluse | clayg, we were raiing ValueError because that's what the older (still not up to date for us I don't think) eventlet would do when wsgi input went away in the middle of a ready | 22:47 |
| clayg | torgomatic: there's also some IOError that it raises - I remember seeing that | 22:47 |
| torgomatic | clayg: could be an error if someone's counting bytes and doesn't get enough | 22:47 |
| clayg | torgomatic: maybe - i sorta think that *would* end up looking like a ChunkReadTimeout - I mean read(network_chunk_size) is going to block if there's nothing on the wire - fuck I forget how makefile works | 22:48 |
| clayg | torgomatic: anyway I have some really good (IMHO) real socket tests in patch 224387 - if you could review that'd be like steller! | 22:48 |
| patchbot | clayg: https://review.openstack.org/#/c/224387/ | 22:48 |
| torgomatic | I think makefile() just dup()-s the file descriptor and wraps a Python file wrapper around it | 22:48 |
| clayg | torgomatic: yeah but there has to be some symantic difference with a raw socked and wrapped file for EOF and re: read and readline | 22:49 |
| torgomatic | yeah, probably | 22:49 |
| clayg | hrm.. I don't think patch 224387 had an up-to-date rebase - I'm going to carry on work/fix for lp bug #1496205 ontop of it anyway tho | 22:53 |
| openstack | Launchpad bug 1496205 in OpenStack Object Storage (swift) "EC: Client Disconnect leaves inaccessible data on disk" [Undecided,New] https://launchpad.net/bugs/1496205 | 22:53 |
| patchbot | clayg: https://review.openstack.org/#/c/224387/ | 22:53 |
| tsimp06 | man-a-nickname check out http://docs.openstack.org/developer/swift/overview_auth.html it states that read/write can't access account info | 22:53 |
| tsimp06 | i'm not sure what the affects of not having the role will do to the requests | 22:54 |
| man-a-nickname | It will just let you have r/w to the objects in the container and GET/HEAD of the container. But the user must know the container exists first. | 22:55 |
| tsimp06 | in that case, wouldn't the user who created the container provide that info? | 22:56 |
| *** [1]eranrom has joined #openstack-swift | 22:58 | |
| *** eranrom has quit IRC | 22:58 | |
| *** [1]eranrom is now known as eranrom | 22:58 | |
| man-a-nickname | In this case it would be two parts of an application. The parts that works on the containers cannot go discover them; it has to be told. | 22:58 |
| *** jlhinson has quit IRC | 22:59 | |
| *** david-lyle has joined #openstack-swift | 22:59 | |
| man-a-nickname | To have a scoped container listing would be kind of expensive for swift to do. | 23:00 |
| *** eranrom has quit IRC | 23:02 | |
| *** annegentle has quit IRC | 23:03 | |
| *** km has joined #openstack-swift | 23:04 | |
| *** DericHorn-HP has joined #openstack-swift | 23:04 | |
| *** breitz has quit IRC | 23:04 | |
| clayg | torgomatic: you know pyeclib stuff - would it be possible for the auditor to do some sort of checks on the ec frags in the archive to validate some of the EC metadata (like X-Object-Sysmeta-Ec-Frag-Index or right now I'd be most inerested in X-Object-Sysmeta-Ec-Content-Length) | 23:04 |
| torgomatic | clayg: what kind of validations? -Ec-Content-Length is just the length of the original object, so all you could really validate is that it's >= 0, right? | 23:05 |
| torgomatic | or I guess you could take the fragment archive's size and turn that into a bound on the object's size, ± half a segment | 23:06 |
| clayg | torgomatic: yeah that's what I'm talking about! | 23:06 |
| *** breitz has joined #openstack-swift | 23:07 | |
| torgomatic | clayg: what are you trying to do? | 23:07 |
| clayg | working on lp bug #1496205 staring at these objects that obviously are too small to solve the X-Object-Sysmeta-Ec-Content-Length - but the auditor don't care - Content-Length matches os.stat so he's happy | 23:07 |
| openstack | Launchpad bug 1496205 in OpenStack Object Storage (swift) "EC: Client Disconnect leaves inaccessible data on disk" [Undecided,New] https://launchpad.net/bugs/1496205 | 23:07 |
| peluse | clayg, wonder if it makes sense to dust off the algebraic signature stuff we talked about a long long time ago.... would confirm integrity of a frag set and not just spot check length | 23:09 |
| *** zhill has quit IRC | 23:10 | |
| torgomatic | clayg: does the fragment archive's etag match its md5sum? | 23:11 |
| clayg | yeah swift-object-info says it's valid | 23:12 |
| clayg | peluse: that requires talking to the other nodes right? | 23:12 |
| torgomatic | huh, wacky; sounds like the proxy goofed up when writing it | 23:12 |
| clayg | peluse: could be useful as an out of band tool | 23:12 |
| peluse | clayg, yeah but HEAD only I believe | 23:12 |
| peluse | but we never fully explored it | 23:12 |
| clayg | peluse: I'd still be worried about the connections per object in a background process - even if it was just stat the file and read metadata - no way we could just do that all the time | 23:13 |
| clayg | peluse: but as a out of band ops tool like swift-object-info or something like that it'd probably be pretty sweet | 23:13 |
| clayg | I love those kinds of tools for digging around on stuff on live data | 23:14 |
| clayg | torgomatic: yeah I think i'm troubleshooting a proxy bug - but having the auditor able to protect against this kind of invalid data would be useful at least in this case | 23:15 |
| clayg | torgomatic: like peluse's pointed out I'm curious what other sort of validation we might ahve available if we went looking for it | 23:15 |
| peluse | clayg, we have some old trello discussions on it, I'll check it out. | 23:16 |
| peluse | need to read that bug report / patch as well, curious how things got messed up | 23:16 |
| peluse | having dinner with your execs tonight though so maybe tomorrow :) | 23:17 |
| clayg | their generally friendly folks - have a good time | 23:18 |
| peluse | yeah, I think I've broken bread with most at one point or another... | 23:19 |
| peluse | clayg, are you still banging on the kinetic stuff now and again? | 23:20 |
| clayg | peluse: now and again | 23:22 |
| peluse | clayg, FYI trying to get us signed up for the new Open Kinetic working group thingy under the foundation. having some legal challenges (go figure) so curious how hot that stuff has been for you lately... can chat later | 23:24 |
| peluse | clayg, torgomatic, notmyname - please see email | 23:26 |
| *** kei_yama has joined #openstack-swift | 23:29 | |
| *** chsc has quit IRC | 23:30 | |
| *** breitz has quit IRC | 23:37 | |
| *** man-a-nickname has quit IRC | 23:37 | |
| *** CR7 has joined #openstack-swift | 23:39 | |
| openstackgerrit | Minwoo Bae proposed openstack/swift: EC GET path: support use of alternate fragments on object servers https://review.openstack.org/213147 | 23:40 |
| clayg | man, stupid proxy is so stupid | 23:50 |
| *** DericHorn-HP has quit IRC | 23:52 | |
| *** hezhiqiang has joined #openstack-swift | 23:55 | |
| *** nadeem has quit IRC | 23:56 | |
| *** minwoob has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!