| *** bi_fa_fu has joined #openstack-swift | 00:12 | |
| *** bi_fa_fu has quit IRC | 00:14 | |
| *** chlong has quit IRC | 00:18 | |
| *** flwang has joined #openstack-swift | 00:20 | |
| *** kevinc_ has quit IRC | 00:21 | |
| *** tellesnobrega_ has joined #openstack-swift | 00:22 | |
| *** tellesnobrega_ has quit IRC | 00:23 | |
| *** tellesnobrega_ has joined #openstack-swift | 00:23 | |
| *** tellesnobrega_ has quit IRC | 00:24 | |
| *** onovy has quit IRC | 00:25 | |
| *** zul has joined #openstack-swift | 00:34 | |
| *** onovy has joined #openstack-swift | 00:35 | |
| *** flwang has quit IRC | 00:40 | |
| *** zaitcev has quit IRC | 00:46 | |
| *** zaitcev has joined #openstack-swift | 00:49 | |
| *** ChanServ sets mode: +v zaitcev | 00:49 | |
| *** nakagawamsa has joined #openstack-swift | 00:49 | |
| ho | kota_: I would like to get in touch with kazuriho. could you tell my irc name to him? | 00:59 |
|---|---|---|
| *** 64MADOHP4 has joined #openstack-swift | 01:12 | |
| *** hrou has joined #openstack-swift | 01:15 | |
| *** flwang has joined #openstack-swift | 01:21 | |
| *** 64MADOHP4 has quit IRC | 01:24 | |
| *** miya-test has joined #openstack-swift | 01:27 | |
| *** miya-test has quit IRC | 01:28 | |
| *** kota_ has joined #openstack-swift | 01:28 | |
| *** ChanServ sets mode: +v kota_ | 01:28 | |
| kota_ | hello | 01:28 |
| kota_ | I'm back from vacation. | 01:28 |
| ho | kota_: welcome back! did you spend good vacation? | 01:29 |
| *** gyee has quit IRC | 01:29 | |
| timburke | hi kota_! i hope vacation was nice :) | 01:29 |
| kota_ | ho: definitely, it was nice. | 01:30 |
| ho | kota_: great! | 01:30 |
| kota_ | timburke: hi, nice work about swift3, I am going to start a tons of reviewer work from today :/ | 01:30 |
| timburke | kota_: heh, no rush. actually, i think that queue isn't even very long | 01:31 |
| openstackgerrit | Matthew Oliver proposed openstack/swift: Follow up patch to fix a multiline import NITPIC https://review.openstack.org/216479 | 01:32 |
| mattoliverau | kota_: morning! | 01:32 |
| kota_ | timburke: ok, thanks | 01:33 |
| kota_ | mattoliverau: morning! | 01:33 |
| kota_ | mattoliverau: could you review my spec about global ec cluster? (about patch 209447) | 01:35 |
| patchbot | kota_: https://review.openstack.org/#/c/209447/ | 01:35 |
| *** asd112z has joined #openstack-swift | 01:35 | |
| mattoliverau | kota_: hai! But it'll have to be after lunch cause I'm about to go out to a lunch meeting | 01:36 |
| kota_ | mattoliverau: it's ok, not so urgent to me, thanks! | 01:37 |
| openstackgerrit | Merged openstack/python-swiftclient: flake8 ignores same hacks as swift https://review.openstack.org/202909 | 01:38 |
| csmart | mattoliverau: stop spying on me | 01:40 |
| * csmart slaps tonyb | 01:40 | |
| mattoliverau | csmart: mwahahaha, never! | 01:40 |
| csmart | :-) | 01:40 |
| tonyb | Well that escaleted quickly | 01:41 |
| mattoliverau | lol | 01:41 |
| csmart | :-D | 01:43 |
| mattoliverau | sigh, spent ages trying to figure out why I couldn't start kota_'s change so I could come back later.. to realise I couldn't see it cause I wasn't logged in via the web-UI (I use gertty alot).. I think I need more caffine :P | 01:51 |
| mattoliverau | s/start/star/ | 01:52 |
| zaitcev | You totally do, it appears. | 01:53 |
| *** miyahara has joined #openstack-swift | 01:53 | |
| kota_ | miyahara: welocome to IRC! | 01:54 |
| miyahara | kota_: thank you. | 01:55 |
| openstackgerrit | Merged openstack/python-swiftclient: Add minimal working service token support. https://review.openstack.org/182640 | 02:02 |
| *** haomaiwang has joined #openstack-swift | 02:05 | |
| *** kota_ has quit IRC | 02:07 | |
| ho | miyahara: welcome! | 02:19 |
| miyahara | ho: thank you! | 02:20 |
| *** _hrou_ has joined #openstack-swift | 02:20 | |
| ho | miyahara: I would like to discuss about your patch. | 02:20 |
| miyahara | OK. | 02:21 |
| ho | miyahara: my concern now is raise at L181 in swift-dispersion-report. is it necessary? | 02:22 |
| ho | miyahara: i wrote it in the previous comment. | 02:22 |
| *** hrou has quit IRC | 02:23 | |
| *** nexusz99 has joined #openstack-swift | 02:23 | |
| ho | miyahara: this is a command so i prefer to not print traceback in this case. | 02:24 |
| *** flwang has quit IRC | 02:26 | |
| ho | miyahara: and i prefer to replace "raise at L185 in internal_client.py" to ClientException. The exception is already handled so it's better to wrap here. | 02:27 |
| ho | miyahara: here is my memo (use for the review) : http://paste.openstack.org/show/426557/ | 02:30 |
| ho | miyahara: what do you think? | 02:30 |
| miyahara | ho: Do you mean that "if err has http_status, it is not necessary to raise err" ? | 02:31 |
| *** jrichli has joined #openstack-swift | 02:34 | |
| *** bkopilov has quit IRC | 02:34 | |
| miyahara | If so, I agree to your proposal "raise at L181 in swift-dispersion-report is unnecessary." | 02:36 |
| ho | miyahara: yeah, i'm not sure why exceptions raise there. | 02:37 |
| ho | miyahara: if the code got 403, a traceback will be outputted. | 02:38 |
| miyahara | ho: I see. | 02:41 |
| ho | miyahara: if you fix my second comment (replace to clientexception), it's better to split except block for ClientEx and Ex | 02:56 |
| ho | miyahara: i will wirte above comments on gerrit. | 02:56 |
| *** kota_ has joined #openstack-swift | 02:58 | |
| *** ChanServ sets mode: +v kota_ | 02:58 | |
| *** haomaiwang has quit IRC | 03:01 | |
| miyahara | ho: Now, I discussed this topic with kota_ , and I got new conclusion. | 03:01 |
| *** haomaiwang has joined #openstack-swift | 03:02 | |
| miyahara | ho: Can you discuss this topic 1 hour later? | 03:05 |
| ho | miyahara: great! | 03:05 |
| ho | miyahara: i think it's not necessary. you discussed my concerns with kota_, i just wanted to tell them to reduce our mis-understanding in gerrit because of our english. :-) | 03:10 |
| ho | miyahara: i'm looking foward to reading your next patch :-) | 03:11 |
| *** sanchitmalhotra has joined #openstack-swift | 03:13 | |
| *** links has joined #openstack-swift | 03:24 | |
| *** ppai has joined #openstack-swift | 03:26 | |
| *** asd112z has quit IRC | 03:31 | |
| *** sanchitmalhotra1 has joined #openstack-swift | 03:37 | |
| *** sanchitmalhotra has quit IRC | 03:39 | |
| miyahara | ho: OK. I make new patch. Thank you. | 03:45 |
| notmyname | good evening | 03:47 |
| kota_ | notmyname: good evening | 03:47 |
| miyahara | notmyname: good evening | 03:47 |
| *** ppai has quit IRC | 03:48 | |
| ho | miyahara: you are welcome :-) btw should I put "-san" after your irc name? your irc name is same as family name so i'm not sure i write your irc name without "-san" | 03:48 |
| ho | notmyname: good evening! | 03:49 |
| miyahara | ho: I don't care. Please call me "miyahara". | 03:56 |
| ho | miyahara: sure! | 03:57 |
| zaitcev | Do you guys remember the term for addressing without honorifics? I think it sounded like "yobishite", but I can't find it. | 03:58 |
| kota_ | zaitcev: maybe you want "yobisute" | 03:59 |
| ho | zaitcev: yobisute is right word | 03:59 |
| kota_ | lol | 04:00 |
| ho | you won :-) | 04:00 |
| zaitcev | kota_, ho: thanks a lot, that was what I was looking for. | 04:00 |
| *** haomaiwang has quit IRC | 04:01 | |
| *** 17WAAILMU has joined #openstack-swift | 04:01 | |
| *** ppai has joined #openstack-swift | 04:02 | |
| ho | zaitcev: you are welcome. you study japanese hard :P | 04:06 |
| zaitcev | ho: Sorry, not really hard. | 04:06 |
| zaitcev | ho: But hopefuly it comes handy in October. I got a room in Ueno, so I'm going to commute. | 04:07 |
| ho | zaitcev: study japanese from tv (animation) is a good idea. oh, you can have experience for "rush hour in tokyo" | 04:09 |
| mattoliverau | kota_: I had a look at your spec, looks great, there are spelling errors etc, which I assume others will start -1 | 04:12 |
| mattoliverau | kota_: -ing so had a first draft at correcting http://paste.openstack.org/show/426660/ | 04:12 |
| kota_ | mattoliverau: Thanks! | 04:13 |
| mattoliverau | I'm happy with how it is now (minus some spelling corrections) but ^^ had a go at rewording just in case it becomes stuck in review hell.. happy to push it up if your happy for me to. | 04:13 |
| mattoliverau | kota_: ^ | 04:14 |
| notmyname | my plan for tokyo is to not let kota_ and ho out of my sight | 04:14 |
| mattoliverau | notmyname: good plan ;) | 04:14 |
| jrichli | +1 | 04:14 |
| kota_ | lol | 04:16 |
| ho | usuful info. i don't live in tokyo, i'm afraid to get there (too big city for me) :-) | 04:17 |
| notmyname | new plan: ho is staying in my hotel the week of the summit ;-) | 04:17 |
| kota_ | i guess dmorita and miyahara could be volunteers, too. | 04:17 |
| notmyname | :-) | 04:17 |
| ho | lol | 04:17 |
| zaitcev | Come on. Spread your wings in The International Edo. | 04:18 |
| notmyname | in seriousness, though, I will be grateful for any help in Japan you can provide while we're there :-) | 04:18 |
| mattoliverau | kota_: I like the way you think ;) | 04:18 |
| *** bkopilov has joined #openstack-swift | 04:18 | |
| ho | i have to study about tokyo | 04:19 |
| zaitcev | Besides, the Summit is in Shinagawa, which is a kind of a foreigner's corral. It's right next door to Roppongi, where all the embassies are. | 04:19 |
| mattoliverau | kota_: maybe ntt will need to hold a swift party :) | 04:19 |
| openstackgerrit | Merged openstack/swift: Quorum on durable response is too low https://review.openstack.org/213822 | 04:20 |
| notmyname | mattoliverau: we (swiftstack) will be doing a low-key swift one like we did in vancouver. or that's the plan so far :-) | 04:20 |
| kota_ | mattoliverau: I hope it, too and now planning but not yet determined I can do so. | 04:20 |
| notmyname | but NTT should do one too! | 04:20 |
| kota_ | yup | 04:21 |
| mattoliverau | notmyname: I want to go to many swift parties! | 04:21 |
| zaitcev | Maybe I can still come... | 04:21 |
| mattoliverau | zaitcev: maybe.. | 04:21 |
| mattoliverau | :P | 04:21 |
| jrichli | me too ... more swift parties | 04:22 |
| kota_ | notmyname: have you plan the date already? | 04:22 |
| mattoliverau | Okonomiyaki party, a suhi train party | 04:22 |
| kota_ | I dont't want the ones hold in the same day. | 04:22 |
| notmyname | kota_: that would be bad! | 04:23 |
| notmyname | we haven't set any day yet. have you? | 04:23 |
| kota_ | notmyname: not yet. | 04:23 |
| mattoliverau | Well make sure your both in communication when you do! | 04:23 |
| notmyname | actually I'm a little confused on how it's working this time. seems that everything starts on tuesday morning? and the confenrence and summit 100% overlap for tuesday through friday | 04:24 |
| kota_ | notmyname: I'm not sure when almost of attendees would land tokyo... | 04:24 |
| notmyname | kota_: yeah, that's another issue | 04:24 |
| redbo | I'm hoping it turns out everyone actually speaks english but they're angry about it, like in Paris. | 04:25 |
| kota_ | notmyname: yes, all of summit starts from tuesday. | 04:25 |
| notmyname | I am getting in early, but leaving on friday night. reasonable days seem like they'd be monday-thursday nights | 04:25 |
| jrichli | redbo: +1 | 04:25 |
| notmyname | redbo: except the taxi drivers. then they're just angry with you about it | 04:25 |
| mattoliverau | yeah good point, I'd assume Monday for me then as I don't need a jetlag day.. but maybe I could come on the weekend.. I'll have to discuss it with the bosses | 04:25 |
| zaitcev | notmyname: It was a surprised for me too. I arrive on Saturday, because I did not even think to examine the schedule. | 04:26 |
| notmyname | zaitcev: I know, right? | 04:26 |
| notmyname | yeah, I'm getting in saturday night too (at about 10:30pm local time) | 04:26 |
| zaitcev | Well, then.... Sunday night is Akiba night then. | 04:27 |
| kota_ | zaitcev: lol | 04:27 |
| mattoliverau | I'm looking forward to it, maybe I'll need to try and get in on the weekend then (tho can't promise anything) :) | 04:28 |
| kota_ | nights through monday-thursday seems better. ok, I am going to plan around there (but can't promise for now) | 04:30 |
| kota_ | thanks | 04:31 |
| notmyname | kota_: ok, great. If you come up with something, let me know and I'll work around it | 04:31 |
| kota_ | notmyname: sure | 04:31 |
| *** sanchitmalhotra has joined #openstack-swift | 04:35 | |
| *** trifon has quit IRC | 04:35 | |
| *** sanchitmalhotra1 has quit IRC | 04:37 | |
| *** abhirc has quit IRC | 04:37 | |
| *** abhirc has joined #openstack-swift | 04:38 | |
| *** SkyRocknRoll has joined #openstack-swift | 04:42 | |
| *** abhirc has quit IRC | 04:42 | |
| *** abhirc has joined #openstack-swift | 04:47 | |
| *** abhirc has quit IRC | 04:48 | |
| *** abhirc has joined #openstack-swift | 04:48 | |
| *** jrichli has quit IRC | 04:54 | |
| *** 17WAAILMU has quit IRC | 05:01 | |
| *** haomaiwang has joined #openstack-swift | 05:02 | |
| *** abhirc has quit IRC | 05:04 | |
| *** silor has joined #openstack-swift | 05:16 | |
| *** trifon has joined #openstack-swift | 05:18 | |
| *** silor has quit IRC | 05:20 | |
| *** silor has joined #openstack-swift | 05:25 | |
| *** asd112z has joined #openstack-swift | 05:32 | |
| *** kota_ has quit IRC | 05:47 | |
| openstackgerrit | Merged openstack/swift: Fix typo of a comment in replicator https://review.openstack.org/216453 | 05:49 |
| *** asd112z has quit IRC | 05:51 | |
| *** links has quit IRC | 05:57 | |
| *** links has joined #openstack-swift | 06:00 | |
| *** haomaiwang has quit IRC | 06:01 | |
| *** haomaiwang has joined #openstack-swift | 06:01 | |
| *** zaitcev has quit IRC | 06:07 | |
| *** baojg has joined #openstack-swift | 06:10 | |
| *** serverascode has quit IRC | 06:12 | |
| *** mahatic has joined #openstack-swift | 06:13 | |
| openstackgerrit | Samuel Merritt proposed openstack/swift: Allow pep8 of a single file https://review.openstack.org/216545 | 06:16 |
| *** baojg has quit IRC | 06:16 | |
| *** serverascode has joined #openstack-swift | 06:17 | |
| *** baojg has joined #openstack-swift | 06:19 | |
| *** _hrou_ has quit IRC | 06:34 | |
| nakagawamsa | Hi all, could you tell me about version confliction of requrement packages? | 06:41 |
| nakagawamsa | I think, Ubuntu 14.04 cannot be satisfied with requirement version of six and eventlet,because 14.04's repository doesn't have required version of those packages. | 06:41 |
| nakagawamsa | So we need to add latest repository if we deploy swift in Ubuntu 14.04. | 06:41 |
| nakagawamsa | But SAIO document doesn't reffer to it, so I think we should modify SAIO document. | 06:41 |
| nakagawamsa | Could you tell me that my worrying is incorrect? or this is already discussed matter? | 06:41 |
| *** baojg has quit IRC | 06:42 | |
| *** baojg has joined #openstack-swift | 06:43 | |
| lifeless | nakagawamsa: how are you installing ? | 06:47 |
| *** mahatic has quit IRC | 06:48 | |
| *** mahatic has joined #openstack-swift | 06:48 | |
| nakagawamsa | lifeless: following SAIO document, <http://docs.openstack.org/developer/swift/development_saio.html> | 06:49 |
| lifeless | huh those docs are old | 06:55 |
| lifeless | we recommend pip install -e . | 06:55 |
| lifeless | not setup.py develop | 06:55 |
| lifeless | pip will install what you need | 06:55 |
| lifeless | anyhow, what error are you hving? | 06:55 |
| nakagawamsa | In my understanding, pip cannot update packages which installed by OS (like apt-get) | 06:58 |
| lifeless | I want to know | 06:58 |
| lifeless | are you theorising about a problem | 06:58 |
| lifeless | or have you experienced a problem ? | 06:58 |
| nakagawamsa | theorising | 06:59 |
| nakagawamsa | "pip -r requirement.txt" can not update six/eventlet packages with error | 07:00 |
| *** haomaiwang has quit IRC | 07:01 | |
| *** haomaiwang has joined #openstack-swift | 07:02 | |
| *** rledisez has joined #openstack-swift | 07:02 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-swiftclient: Updated from global requirements https://review.openstack.org/89250 | 07:11 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 07:12 |
| lifeless | nakagawamsa: with appropriate privileges it can | 07:13 |
| nakagawamsa | pip install is run with sudo. | 07:18 |
| nakagawamsa | and pip community said OS installed packages will not updated by pip | 07:19 |
| nakagawamsa | <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=771794> | 07:22 |
| openstack | Debian bug 771794 in python-pip "pip silently removes/updates system provided python packages" [Serious,Fixed] | 07:22 |
| nakagawamsa | if my understanding is mistaken, please tell me. (my english skill is poor) | 07:24 |
| lifeless | nakagawamsa: I think you should test this | 07:26 |
| lifeless | rather than theorising | 07:26 |
| lifeless | Debian have patched pip, so its not necessarily going to do what pip would normally | 07:26 |
| *** mahatic has quit IRC | 07:30 | |
| *** silor has quit IRC | 07:34 | |
| nakagawamsa | lifeless: ok, thanks a lot. I retry check now. | 07:34 |
| nakagawamsa | I think I need to retry two process. | 07:35 |
| nakagawamsa | 1.original SAIO document process | 07:35 |
| nakagawamsa | 2.using pip install -e process which you gave me | 07:35 |
| nakagawamsa | Could you give me document which uses "pip install -e" ? | 07:35 |
| lifeless | sorry, not really - its written down in mailing list posts and so on, but not yet in the swift docs | 07:36 |
| nakagawamsa | ok, i try to seach it. | 07:38 |
| nakagawamsa | And if i find it and needed, I would like to try to modify SAIO document, ok? | 07:38 |
| lifeless | sure | 07:45 |
| *** akle has joined #openstack-swift | 07:52 | |
| *** jordanP has joined #openstack-swift | 07:52 | |
| *** bkopilov has quit IRC | 07:54 | |
| *** mahatic has joined #openstack-swift | 07:55 | |
| *** haomaiwang has quit IRC | 08:01 | |
| *** haomaiwang has joined #openstack-swift | 08:01 | |
| *** asd112z has joined #openstack-swift | 08:03 | |
| *** asd112z has quit IRC | 08:08 | |
| openstackgerrit | Merged openstack/swift: New troubleshooting case in documentation. https://review.openstack.org/215744 | 08:10 |
| openstackgerrit | Merged openstack/swift: Minor cleanup handoff mode warnings https://review.openstack.org/215858 | 08:11 |
| *** jistr has joined #openstack-swift | 08:12 | |
| *** acoles_ is now known as acoles | 08:23 | |
| openstackgerrit | Merged openstack/swift: Use correct Storage-Policy header for REPLICATE requests https://review.openstack.org/215857 | 08:31 |
| *** joeljwright has joined #openstack-swift | 08:31 | |
| *** ChanServ sets mode: +v joeljwright | 08:31 | |
| *** haomaiwang has quit IRC | 08:41 | |
| *** haomaiwang has joined #openstack-swift | 08:42 | |
| *** akle has quit IRC | 08:46 | |
| openstackgerrit | Alistair Coles proposed openstack/python-swiftclient: Stop Connection class modifying os_options parameter https://review.openstack.org/216240 | 08:47 |
| *** ho has quit IRC | 08:54 | |
| *** akle has joined #openstack-swift | 09:05 | |
| *** haomaiwang has quit IRC | 09:09 | |
| *** haomaiwang has joined #openstack-swift | 09:10 | |
| *** marzif has joined #openstack-swift | 09:21 | |
| *** marzif has quit IRC | 09:31 | |
| *** marzif has joined #openstack-swift | 09:31 | |
| *** sanchitmalhotra has quit IRC | 09:31 | |
| *** joeljwright1 has joined #openstack-swift | 09:34 | |
| *** jordanP has quit IRC | 09:36 | |
| *** joeljwright has quit IRC | 09:37 | |
| *** joeljwright has joined #openstack-swift | 09:41 | |
| *** ChanServ sets mode: +v joeljwright | 09:41 | |
| *** joeljwright1 has quit IRC | 09:43 | |
| *** joeljwright has quit IRC | 09:46 | |
| *** joeljwright has joined #openstack-swift | 09:50 | |
| *** ChanServ sets mode: +v joeljwright | 09:50 | |
| *** aix has joined #openstack-swift | 09:50 | |
| *** aix has quit IRC | 09:52 | |
| *** aix has joined #openstack-swift | 09:53 | |
| openstackgerrit | Kazuhiro MIYAHARA proposed openstack/swift: Fix dispersion-reports error message https://review.openstack.org/213697 | 09:58 |
| *** nexusz99 has quit IRC | 10:05 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Fix swob.Range docstring https://review.openstack.org/216649 | 10:09 |
| *** haomaiwang has quit IRC | 10:09 | |
| *** haomaiwa_ has joined #openstack-swift | 10:10 | |
| *** baojg has quit IRC | 10:11 | |
| openstackgerrit | Merged openstack/swift: Follow up patch to fix a multiline import NITPIC https://review.openstack.org/216479 | 10:28 |
| *** eandersson has joined #openstack-swift | 10:29 | |
| *** miyahara has quit IRC | 10:37 | |
| *** SkyRocknRoll has quit IRC | 10:39 | |
| *** bwall has quit IRC | 10:49 | |
| *** dmorita has quit IRC | 10:50 | |
| *** SkyRocknRoll has joined #openstack-swift | 10:57 | |
| *** lpabon has joined #openstack-swift | 11:01 | |
| *** asd112z has joined #openstack-swift | 11:03 | |
| openstackgerrit | Merged openstack/swift: Fix use of delimiter in account listings https://review.openstack.org/216427 | 11:04 |
| openstackgerrit | Merged openstack/swift: pep8: Fix usage of the l10n _('...') function https://review.openstack.org/213494 | 11:04 |
| *** asd112z has quit IRC | 11:08 | |
| *** haomaiwa_ has quit IRC | 11:09 | |
| *** haomaiwa_ has joined #openstack-swift | 11:10 | |
| *** joeljwright has quit IRC | 11:23 | |
| onovy | notmyname, clayg, mattoliverau: hi, do you have time for small review pls? https://review.openstack.org/#/c/210736/ | 11:25 |
| *** nakagawamsa has quit IRC | 11:44 | |
| *** kairo_ has quit IRC | 11:48 | |
| *** lpabon has quit IRC | 11:50 | |
| *** marcusvrn_ has joined #openstack-swift | 11:53 | |
| *** km has quit IRC | 12:07 | |
| *** haomaiwa_ has quit IRC | 12:09 | |
| *** haomaiwang has joined #openstack-swift | 12:10 | |
| *** petertr7_away is now known as petertr7 | 12:11 | |
| *** annegentle has joined #openstack-swift | 12:28 | |
| *** bill_az has joined #openstack-swift | 12:31 | |
| *** kei_yama has quit IRC | 12:31 | |
| *** lpabon has joined #openstack-swift | 12:31 | |
| *** jordanP has joined #openstack-swift | 12:32 | |
| *** petertr7 is now known as petertr7_away | 12:33 | |
| *** ppai has quit IRC | 12:34 | |
| *** petertr7_away is now known as petertr7 | 12:35 | |
| *** SkyRocknRoll has quit IRC | 12:42 | |
| *** gustavo has joined #openstack-swift | 12:45 | |
| *** annegentle has quit IRC | 12:50 | |
| gustavo | Hi guys, I'm running a production Swift cluster setup with authentication using swauth. I'm not using any other openstack component. Swift version: Folsom (1.7.4). I would like to upgrade to Swift 2.2.0. Should I migrate to keystone ? | 12:51 |
| *** dustins has joined #openstack-swift | 12:52 | |
| *** joeljwright has joined #openstack-swift | 12:56 | |
| *** ChanServ sets mode: +v joeljwright | 12:56 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 12:56 |
| *** chlong has joined #openstack-swift | 12:56 | |
| onovy | gustavo, not needed | 13:01 |
| onovy | we have 2.3.0 swift cluster with swauth in production | 13:01 |
| onovy | it always depends on size of cluster. for many users and many auth request it's better to have Keystone | 13:02 |
| onovy | for other usecases is swauth fine | 13:02 |
| openstackgerrit | Merged openstack/swift: Fix 500 for bogus Range request to 0-byte object. https://review.openstack.org/215317 | 13:07 |
| gustavo | onovy, Thanks for the reply! I'm using php-cloudfiles to access to swift cluster but this library is deprecated and not available any more. I read that php-opencloud is the replacement, but it library only works with keystone isn't it? | 13:08 |
| onovy | not using php-* at all, sry | 13:08 |
| onovy | but i think it should be compatible with any auth backend | 13:09 |
| onovy | must go, bye | 13:09 |
| gustavo | onovy, thanks very much ! | 13:09 |
| *** haomaiwang has quit IRC | 13:09 | |
| *** haomaiwang has joined #openstack-swift | 13:10 | |
| *** annegentle has joined #openstack-swift | 13:11 | |
| *** annegentle has quit IRC | 13:12 | |
| *** annegentle has joined #openstack-swift | 13:14 | |
| *** breitz has joined #openstack-swift | 13:19 | |
| *** pgbridge has joined #openstack-swift | 13:20 | |
| *** changbl has quit IRC | 13:21 | |
| *** links has quit IRC | 13:23 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 13:24 |
| *** bill_az has quit IRC | 13:26 | |
| *** hrou has joined #openstack-swift | 13:29 | |
| *** breitz has quit IRC | 13:34 | |
| *** lpabon has quit IRC | 13:38 | |
| *** lpabon has joined #openstack-swift | 13:42 | |
| *** bapalm has quit IRC | 13:44 | |
| *** bapalm_ is now known as bapalm | 13:51 | |
| *** breitz has joined #openstack-swift | 13:52 | |
| *** thumpba has joined #openstack-swift | 13:53 | |
| *** trifon has quit IRC | 13:56 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Fix EC GET backend stream iteration state https://review.openstack.org/199043 | 13:58 |
| acoles | kota clayg ^^ i just fixed some minor stuff on this patch before +2'ing | 13:58 |
| *** petertr7 is now known as petertr7_away | 14:00 | |
| *** haomaiwang has quit IRC | 14:09 | |
| *** chsc has joined #openstack-swift | 14:10 | |
| *** haomaiwang has joined #openstack-swift | 14:10 | |
| *** petertr7_away is now known as petertr7 | 14:13 | |
| *** jlhinson has joined #openstack-swift | 14:21 | |
| *** asd112z has joined #openstack-swift | 14:32 | |
| *** asd112z has quit IRC | 14:33 | |
| *** asd112z has joined #openstack-swift | 14:33 | |
| *** asd112z has quit IRC | 14:34 | |
| *** asd112z has joined #openstack-swift | 14:34 | |
| *** petertr7 is now known as petertr7_away | 14:41 | |
| *** annegentle has quit IRC | 14:41 | |
| *** annegentle has joined #openstack-swift | 14:41 | |
| *** petertr7_away is now known as petertr7 | 14:42 | |
| *** links has joined #openstack-swift | 14:43 | |
| *** ccavanna has quit IRC | 14:46 | |
| *** ccavanna has joined #openstack-swift | 14:46 | |
| mahatic | acoles: looks like I need this patch 216240 merged to push mine | 14:47 |
| patchbot | mahatic: https://review.openstack.org/#/c/216240/ | 14:47 |
| acoles | mahatic: do you depend on that? | 14:47 |
| mahatic | acoles: I have changes on the same file (client.py) | 14:48 |
| acoles | python | 14:49 |
| acoles | oops! | 14:49 |
| acoles | mahatic: do they conflict? if so you couldl rebase your change onto the other patch | 14:49 |
| mahatic | acoles: actually I don't think they do. But then if that gets merged before, I will not have the latest code on my code | 14:51 |
| mahatic | latest code on the pushed patch* | 14:52 |
| *** jrichli has joined #openstack-swift | 14:53 | |
| acoles | mahatic: its fine, push your patch, gerrit takes care of rebasing on latest master as patches get merged, unless a rebase fails, in which case it will let you know. | 14:54 |
| acoles | mahatic: you only need to rebase on other patch if you need its change | 14:54 |
| mahatic | acoles: oh cool. thanks for letting me know, will do | 14:55 |
| *** minwoob has joined #openstack-swift | 14:59 | |
| *** redbo has quit IRC | 15:06 | |
| *** kevinc_ has joined #openstack-swift | 15:07 | |
| *** dustins has quit IRC | 15:08 | |
| *** redbo has joined #openstack-swift | 15:09 | |
| *** ChanServ sets mode: +v redbo | 15:09 | |
| *** haomaiwang has quit IRC | 15:09 | |
| *** haomaiwang has joined #openstack-swift | 15:10 | |
| *** lpabon has quit IRC | 15:13 | |
| *** lpabon has joined #openstack-swift | 15:17 | |
| *** thurloat is now known as thurloat_isgone | 15:23 | |
| *** nadeem has joined #openstack-swift | 15:24 | |
| *** tsg has joined #openstack-swift | 15:25 | |
| *** ccavanna has quit IRC | 15:26 | |
| *** joeljwright has quit IRC | 15:27 | |
| *** tsg_ has joined #openstack-swift | 15:35 | |
| *** tsg has quit IRC | 15:35 | |
| *** tsg_ has quit IRC | 15:36 | |
| *** links has quit IRC | 15:44 | |
| *** lpabon has quit IRC | 15:44 | |
| *** ccavanna has joined #openstack-swift | 15:52 | |
| *** changbl has joined #openstack-swift | 15:55 | |
| *** thurloat_isgone is now known as thurloat | 15:57 | |
| *** petertr7 is now known as petertr7_away | 15:58 | |
| *** jordanP has quit IRC | 16:00 | |
| *** aix has quit IRC | 16:08 | |
| *** haomaiwang has quit IRC | 16:09 | |
| *** haomaiwa_ has joined #openstack-swift | 16:10 | |
| *** thurloat is now known as thurloat_isgone | 16:12 | |
| *** marzif has quit IRC | 16:12 | |
| *** lpabon has joined #openstack-swift | 16:16 | |
| *** jistr has quit IRC | 16:17 | |
| *** lpabon has quit IRC | 16:25 | |
| *** marzif has joined #openstack-swift | 16:28 | |
| *** vinsh_ is now known as Vinsh | 16:31 | |
| *** bkopilov has joined #openstack-swift | 16:32 | |
| *** rledisez has quit IRC | 16:36 | |
| *** bwall has joined #openstack-swift | 16:39 | |
| *** andrew_____ has joined #openstack-swift | 16:40 | |
| *** andrew_____ has quit IRC | 16:40 | |
| *** trifon has joined #openstack-swift | 16:41 | |
| *** gyee has joined #openstack-swift | 16:43 | |
| *** takotuesday has joined #openstack-swift | 16:45 | |
| takotuesday | Hey everyone, I need some help in figuring out the best way to verify if a file exists within my swift container. I am trying to do this in python. I've been looking at the python swiftclient but the best Ive been able to come up with in that API is get_object | 16:46 |
| ctennis | I'm not particularly sure what you're asking takotuesday, if you're trying to verify if an object exists you can just do an HTTP head to the object | 16:48 |
| *** lpabon has joined #openstack-swift | 16:50 | |
| *** lpabon has quit IRC | 16:55 | |
| peluse | torgomatic, clayg - chatted a bit about this one at the hackathon... error handling change for Ec patch 211338 | 17:06 |
| patchbot | peluse: https://review.openstack.org/#/c/211338/ | 17:06 |
| openstackgerrit | Tushar Gohad proposed openstack/swift: Restrict PyECLib version to 1.0.7 https://review.openstack.org/214054 | 17:07 |
| openstackgerrit | Tim Burke proposed openstack/python-swiftclient: Accept gzip-encoded API responses https://review.openstack.org/184956 | 17:08 |
| *** silor has joined #openstack-swift | 17:09 | |
| *** silor has quit IRC | 17:09 | |
| *** haomaiwa_ has quit IRC | 17:09 | |
| *** haomaiwang has joined #openstack-swift | 17:10 | |
| *** silor has joined #openstack-swift | 17:11 | |
| *** rjaiswal has joined #openstack-swift | 17:14 | |
| rjaiswal | notmyname: Hi thr, a question regarding swift proxy server | 17:18 |
| *** andrew_____ has joined #openstack-swift | 17:19 | |
| andrew_____ | hi, are there a grok pattern for swift logs publicly available (for logstash)? | 17:20 |
| *** petertr7_away is now known as petertr7 | 17:20 | |
| *** esker has joined #openstack-swift | 17:24 | |
| *** themadcanudist has joined #openstack-swift | 17:29 | |
| themadcanudist | hey guys, can you safely "sqlite X vacuum" a container db (and all its replicas)? | 17:30 |
| ctennis | andrew_____: perhaps this? http://docs.openstack.org/developer/swift/_sources/logs.txt | 17:32 |
| ctennis | https://github.com/openstack/swift/blob/master/doc/source/logs.rst for a slightly nicer version | 17:32 |
| *** lpabon has joined #openstack-swift | 17:35 | |
| *** esker has quit IRC | 17:37 | |
| rjaiswal | understand that the proxy server might issue HEAD after a specific timeout, can this be controlled via configuration? | 17:39 |
| *** mahatic has quit IRC | 17:39 | |
| rjaiswal | anyone ^^^^ | 17:41 |
| *** dustins has joined #openstack-swift | 17:43 | |
| torgomatic | rjaiswal: the proxy doesn't spontaneously make requests; it's all done in response to client requests | 17:45 |
| *** themadcanudist has left #openstack-swift | 17:46 | |
| rjaiswal | torgomatic: yes, so when a client issues a swift upload command, the proxy might issue a HEAD if the previous command for the same container was before some time limit? | 17:48 |
| torgomatic | rjaiswal: yes, the proxy may HEAD the container to confirm that it exists. same for the account. | 17:49 |
| torgomatic | it is configurable | 17:49 |
| *** dustins has quit IRC | 17:50 | |
| rjaiswal | ok, so i was told that for all such HEAD requests that originate from middleware, there will always be a swift.source in the env | 17:50 |
| rjaiswal | however, when i tested in devstack with a swift upload <con-name> <obj> command, i saw HEAD requests with no swift.source | 17:51 |
| torgomatic | provided that the middleware sets swift.source, this is true; I believe all the ones in the Swift source tree do, but third-party middlewares don't automatically get 8t | 17:51 |
| torgomatic | rjaiswal: well, the proxy server isn't middleware | 17:51 |
| rjaiswal | yeah, i meant the proxy server itself | 17:51 |
| clayg | morning | 17:52 |
| rjaiswal | is there a scenario when the swift upload command could result in a HEAd without a swift.source? | 17:52 |
| peluse | clayg, morning | 17:52 |
| acoles | clayg: i left some comments on patch 215360 plus a diff with some ideas | 17:52 |
| patchbot | acoles: https://review.openstack.org/#/c/215360/ | 17:52 |
| peluse | clayg, just completed a cool test.... | 17:52 |
| torgomatic | right... so you've asserted (a) requests from middleware have swift.source in the environment, (b) the proxy can make account/container HEAD requests, and (c) you see requests logged with no swift.soruce | 17:52 |
| torgomatic | *source | 17:53 |
| torgomatic | I see no inconsistencies here | 17:53 |
| peluse | (1) write (2) take a pile of drives offline (3) stop ec recon (4) overwrite (5) read and confirm no client errors and 2 etag buckets used on ocacssion (6) run ec recon (7) re-read and confirm all single etag bucket | 17:54 |
| rjaiswal | torgomatic: so when i do swift upload, i do see one HEAD request with swift.source set to RL, followed by a PUT, but then i see another HEAD without a swift.source set | 17:54 |
| rjaiswal | torgomatic: i was thinking that the HEADs due to an upload would have the swift.source set | 17:55 |
| clayg | acoles: I had a diff that turned the node_iter into a instance of a NodeIter class that had a nodes_left property | 17:56 |
| torgomatic | rjaiswal: only if they're done by middleware; if the proxy server (swift.proxy.server.Application) does it, then there's no swift.source | 17:56 |
| acoles | clayg: is that up on gerrit? | 17:56 |
| clayg | acoles: in the diff you have doesn't that go back to only spawning one GET waiting on, then seeing we need another, spawning it, waiting on it, seeing we need another.... | 17:57 |
| rjaiswal | torgomatic: ok | 17:57 |
| clayg | acoles: here was NodeIter -> https://gist.github.com/clayg/e4c54ff9ab1331cc67d3 | 17:57 |
| clayg | acoles: the 416's are tricky, if you get enough "bad gets" the current behavior is to return the 416 (enough in this context is ec_ndata for some reason) | 17:58 |
| rjaiswal | torgomatic: which is that configurable option in proxy server to control the timeout? | 17:58 |
| torgomatic | rjaiswal: it's something like recheck_account and recheck_container, I think | 17:59 |
| rjaiswal | in the proxy-server.conf? | 17:59 |
| torgomatic | yes | 17:59 |
| *** mahatic has joined #openstack-swift | 17:59 | |
| rjaiswal | ok, checking | 18:00 |
| clayg | peluse: awesome! you put your etag_bucket patch on there to get the no client error behavior!? | 18:00 |
| rjaiswal | torgomatic: maybe this?https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L91 | 18:01 |
| peluse | clayg, correct | 18:01 |
| torgomatic | rjaiswal: yeah, that looks rihg | 18:01 |
| torgomatic | *right | 18:01 |
| rjaiswal | torgomatic: thanks | 18:01 |
| peluse | clayg, pretty cool to see it work on a real cluster. if you have an updated version from your end this week we can run it as well | 18:01 |
| torgomatic | np | 18:01 |
| rjaiswal | torgomatic: one last q, is there anyway to find out if a request originated in the proxy and not middleware? | 18:03 |
| rjaiswal | like those HEADs that the proxy would do the check for account/container existence | 18:04 |
| acoles | clayg: i don't think so, *every* time a get completes it evaluates if another may be needed. So the number of gets "in-flight" should always be enough to satisfy ec_ndata if they all land in best_etag bucket. As soon as a get lands in a different etag bucket or is bad, another gets spawned. | 18:04 |
| * acoles intended that at least :) | 18:04 | |
| torgomatic | rjaiswal: absence of swift.source? I think that's what you determined earlier. | 18:04 |
| acoles | clayg: nodes_left will be useful! | 18:05 |
| *** arringtp_ has quit IRC | 18:05 | |
| rjaiswal | torgomatic: but thats also for requests that originate from clients, right | 18:05 |
| torgomatic | rjaiswal: yes, that's true | 18:05 |
| torgomatic | rjaiswal: correlate by txid; if you see multiple requests with the same txid, you can figure out what happened when and why | 18:06 |
| acoles | peluse: nice! | 18:06 |
| rjaiswal | torgomatic: i am trying to fix a bug in ceilometermilldeware.swift, its a middleware that emits events for swift api requests, the problem is that it ignores the swift.source, it should only emit event for requetss that originate from client | 18:07 |
| clayg | acoles: ah, hrm.. ok thanks for that pointer - i was obviously misreading it | 18:07 |
| rjaiswal | so i am trying to figure out a way to distinguish client requests from ones that originate in Swift, swift.source can tell us those that originate in middleware | 18:07 |
| torgomatic | rjaiswal: a middleware won't *see* HTTP requests made by the proxy, so don't worry about those | 18:07 |
| clayg | acoles: still not quite sure about 416's - if we don't do *something* with len(bad_gets) I think we'd have to exahust the node_iter :\ | 18:08 |
| torgomatic | it'll only see requests made by clients or by middlewares to its left in the pipeline | 18:08 |
| rjaiswal | torgomatic: yes, that makes sense, but i saw 2 HEADS, with the latter one without a swift.source, so confused | 18:09 |
| *** haomaiwang has quit IRC | 18:09 | |
| torgomatic | rjaiswal: try requests with curl; swiftclient might be making more than one request per invocation | 18:09 |
| acoles | clayg: yeah, agree, need to keep track of bad_gets to give up early rather than exhaust the iter pointlessly. | 18:10 |
| *** haomaiwang has joined #openstack-swift | 18:10 | |
| clayg | acoles: so what's the right number to return 416's at? | 18:10 |
| clayg | torgomatic: ^ | 18:10 |
| rjaiswal | torgomatic, ok, so you suspecting the second HEAD originated from client | 18:10 |
| clayg | how many 416 response needed on EC GET to return 416? | 18:10 |
| *** marzif has quit IRC | 18:11 | |
| clayg | torgomatic: in the "ideal" | 18:11 |
| clayg | acoles: how about parity + 1 !? | 18:11 |
| rjaiswal | torgomatic: got it, so relying on swift.source to filter out requests that have a value for it should be a good enuf | 18:11 |
| acoles | clayg: hmmm | 18:11 |
| torgomatic | clayg: uh... I dunno | 18:12 |
| torgomatic | rjaiswal: theoretically, yes | 18:12 |
| clayg | acoles: bah - with handoffs you never quite know! there could be enough 206's out there somewhere - if you just keep looking! | 18:12 |
| clayg | you could hit 5 416's and still find 10 206's on the rest of the primaries and one handoff | 18:12 |
| torgomatic | clayg: one? Then you know that there exists (or existed) a version of the object for which the client's range is not satisfiable | 18:12 |
| torgomatic | so blah blah eventual consistency, it's something clients should expect | 18:13 |
| rjaiswal | torgomatic: so, if swiftclient makes more than 1 request for a upload, is it incorrect, in theory, by swiftclient | 18:13 |
| acoles | clayg: we have 2*(parity+data) nodes so don't we need 4xx from 2*parity + data before giving up? | 18:13 |
| acoles | 4xx from 2*parity + data + 1 I mean | 18:13 |
| acoles | clayg: or 1 as torgomatic says ?!? :P | 18:14 |
| clayg | acoles: well that's the weird thing 4XX will just be skipped in the node iter - it's only 416 that leeks into the ECObjController | 18:14 |
| *** lpabon has quit IRC | 18:14 | |
| torgomatic | rjaiswal: that's not what I'm saying at all. I'm saying that swiftclient might make more than one request for an upload, so to test the hypothesis that swift.source is sufficient to distinguish client requests from non-client requests, you should use curl so you know how many client requests were made | 18:14 |
| acoles | clayg: aaaahhh | 18:15 |
| clayg | well call them "bad" responses but it's really *just* 416 | 18:15 |
| *** joeljwright has joined #openstack-swift | 18:15 | |
| *** ChanServ sets mode: +v joeljwright | 18:15 | |
| clayg | or something like that - the ResummingGetter has an "is_success" qualifier that's like 2XX or 416 (IIRC) | 18:15 |
| acoles | clayg: got it | 18:15 |
| rjaiswal | torgomatic: got it | 18:16 |
| clayg | acoles: "is_good_source" | 18:16 |
| *** lpabon has joined #openstack-swift | 18:16 | |
| clayg | rjaiswal: got it | 18:16 |
| clayg | ^ torgomatic you do acoles now! | 18:16 |
| acoles | clayg: i gotta run. will think on 416s some more and take a look at your NodeIter class. | 18:19 |
| clayg | acoles: ok bye bye! | 18:19 |
| *** tab has joined #openstack-swift | 18:20 | |
| *** tab is now known as Guest55946 | 18:20 | |
| *** Guest55946 has quit IRC | 18:20 | |
| *** tab___ has joined #openstack-swift | 18:21 | |
| *** acoles is now known as acoles_ | 18:22 | |
| clayg | mattoliverau: doesn't concurrent GET's expose GreenAsyncPile._inflight or something? | 18:27 |
| clayg | mattoliverau: heck yeah it do! @proerpty inflight! | 18:28 |
| *** jrichli has quit IRC | 18:31 | |
| *** dustins has joined #openstack-swift | 18:33 | |
| rjaiswal | torgomatic: or i could use the --debug option when i use swiftclient to post or upload and i could possibly see the different curl requests being made | 18:34 |
| *** haomaiwang has quit IRC | 18:42 | |
| *** flwang has joined #openstack-swift | 18:44 | |
| *** haomaiwang has joined #openstack-swift | 18:45 | |
| clayg | mattoliverau: _infligth is interesting - it turned out it wasn't really what I was looking for. | 18:47 |
| clayg | I wanted to know how many times do I have to call next to get all of the responses I've spawned - which is different from _inflight (since _inflight is decremented when the response finishes, not when I consume it) | 18:48 |
| clayg | so _inflight could be zero, but I have 3 "pending" responses still "outstanding" (they're done, I just haven't eaten 'em yet) | 18:48 |
| *** changbl has quit IRC | 18:49 | |
| *** marzif has joined #openstack-swift | 18:52 | |
| clayg | i think _inflight works for concurrent GET's tho because you just need the first one that finished - if it finished and inflight got decremented the next wait will break | 18:53 |
| *** changbl has joined #openstack-swift | 18:59 | |
| *** takotuesday has left #openstack-swift | 19:00 | |
| *** dustins has quit IRC | 19:04 | |
| clayg | acoles_: I know you're off - but man - I am *digging* on your diff - I added in the node_iter.nodes_left and it looks good | 19:09 |
| *** haomaiwang has quit IRC | 19:09 | |
| clayg | acoles_: I don't know how I missed it before the spawning the of the extra requests as soon as your pending requests drop below the amount that could come to reach ec_ndata - so good | 19:09 |
| clayg | acoles_: THANKS! | 19:09 |
| *** haomaiwa_ has joined #openstack-swift | 19:10 | |
| *** jlk has joined #openstack-swift | 19:15 | |
| jlk | Has anybody been able to run swift-proxy in such a way that you can pdb walk through the keystone middleware auth routine? I've got a bug somewhere in there where it's trying ot read a cert file I'm passing along, and it's failing hard. | 19:15 |
| jlk | I can't get pdb to attach, and epdb isn't exactly being helpful either. | 19:16 |
| jlk | (or I'm not using it right) | 19:16 |
| *** lpabon has quit IRC | 19:17 | |
| *** andrew_____ has quit IRC | 19:22 | |
| *** lpabon has joined #openstack-swift | 19:24 | |
| *** mfalatic has quit IRC | 19:33 | |
| *** gyee has quit IRC | 19:42 | |
| *** marzif_ has joined #openstack-swift | 19:44 | |
| *** haomaiw__ has joined #openstack-swift | 19:46 | |
| *** haomaiwa_ has quit IRC | 19:46 | |
| *** marzif has quit IRC | 19:46 | |
| clayg | jlk: try to set workers = 0 (no fork) - then start it with swift-proxy-server /etc/swift/proxy-server.conf(.d) verbose | 19:49 |
| clayg | jlk: i'm not 100% sure that will work - but it might getyou close | 19:49 |
| clayg | torgomatic: acoles_: so I'm not sure exactly the correct number of 416 - but it has to be <= ec_ndata | 19:53 |
| clayg | torgomatic: acoles_: if it's good enough for a 200 it's got to be good enough for a 416! | 19:53 |
| peluse | clayg, so FYI on this cluster with 3x repl we get a few .5 sec connection TO at the proxy under decent 64MB write load. Same thing under EC we see quite a few more (but not a crazy amount). Any thoughts on the origin of the .5 sec val and if it makes sense to consider adjusting the default? | 19:55 |
| *** proteusguy has joined #openstack-swift | 19:55 | |
| *** dustins has joined #openstack-swift | 19:56 | |
| peluse | clayg, (BTW if we turn it up the TO's go away, we only tried a gross value though, from .5 to 5) | 19:56 |
| jlk | clayg: thanks. I finally figured out the issue. | 19:58 |
| clayg | peluse: connection TO - that's weird | 20:02 |
| * clayg thinks | 20:02 | |
| *** pberis has joined #openstack-swift | 20:02 | |
| clayg | peluse: are you running replication_ip/replication_port? | 20:03 |
| clayg | peluse: it could be that ssync is chewing up some responsiveness in the obj server (compared to rsync repliation?) | 20:03 |
| clayg | peluse: you could throw this theory out by turning off ec_recon | 20:03 |
| peluse | clayg, yeah, we can try that | 20:04 |
| clayg | peluse: other possibility is cpu work blocking up the eventlet hub in the proxy | 20:04 |
| peluse | clayg, cpu is not heavily loaded during test | 20:05 |
| clayg | peluse: eventlet has the (weird?) behavior that if you set a timeout and then starve the hub - if the t/o is spent by the time you get back around to service the gt (even if it's ready!) it'll blow the timeout | 20:05 |
| openstackgerrit | Nadeem Syed proposed openstack/swift: go: Collecting runtime metrics https://review.openstack.org/216855 | 20:05 |
| peluse | clayg, maybe 40% on each proxy | 20:06 |
| clayg | peluse: well it may not be just the cpu *load* - eventlet might not be using the cpu entirely efficiently | 20:07 |
| clayg | peluse: even if pyeclib releases the GIL (i have no idea if it does) - since we're not doing the cpu in an os.thread (we don't want to because of context-switching overhead) the hub is "blocked" while doing the call to c library is doing math | 20:08 |
| clayg | i wouldn't think normally it'd be noticeable (compared to a half-dozen python function calls) - unless maybe the segment_size is really high? | 20:08 |
| clayg | peluse: i'm just trying to think of ways that a proxy might timeout a backend connection - hub starvation comes to mind | 20:09 |
| *** haomaiw__ has quit IRC | 20:09 | |
| clayg | one thing that can starve a hub is blocking call to c library - one thing that's different between replication is a blocking call to pyeclib - so it's an idea - need to think about how we could quantify or rule it out | 20:10 |
| peluse | clayg, yeah, BTW running the same test with kota's not_client_connection patch seems to result in fewer (so far none) timeouts under same conditions | 20:10 |
| *** haomaiwang has joined #openstack-swift | 20:10 | |
| peluse | maybe the logging?? | 20:10 |
| *** petertr7 is now known as petertr7_away | 20:11 | |
| peluse | clayg, yeah these are all PUTs so I could dummy that call up pretty easy | 20:11 |
| clayg | maybe - what's your syslog configuration? I thought we foced it to use udp - so that should be ok - i think | 20:11 |
| peluse | defaults | 20:11 |
| *** flwang has quit IRC | 20:12 | |
| peluse | I'm just sayin' - they're few and far between to begin with but so far still none w/kota's patch. will let it run for ten min and see (that resulted in a dozen before) | 20:12 |
| clayg | peluse: maybe something in kota's patch is causing backend connections to get cleaned up better - so the object servers are more responsive - this is a good line of testing I think! | 20:12 |
| *** flwang has joined #openstack-swift | 20:12 | |
| clayg | ok, good observations - can you point me at *which* of kota's patches we're talking about? | 20:13 |
| clayg | I remember he had a few that had to do with response codes, or content-length, client disconnect logging, ecappiter generator exit error - i can't remember the specifics | 20:13 |
| peluse | patch 199043 | 20:14 |
| patchbot | peluse: https://review.openstack.org/#/c/199043/ | 20:14 |
| peluse | ahhh, just had a few. so maybe helped a little but not gone (again we see with repl as well). Will try w/EC recon turned off on all nodes | 20:16 |
| clayg | oh - you see the connection timeouts on replication too? yeah that could just be average run of the mill object-server disk wait starvation | 20:17 |
| clayg | yeah i misread what you said earlier - thanks for clarifying | 20:18 |
| peluse | clayg, yeah could be. we just noticed more with EC so wanted to mention it | 20:18 |
| clayg | peluse: turn up your object server works to like 4 x #of-disks | 20:18 |
| clayg | peluse: or go server per port!! whoooowhooooo | 20:19 |
| wbhuber | peluse: how recent is proxy/controllers/obj.py on your cluster? the bug, 1488610, seems to indicate that the lines are a bit off from what i am having right now. | 20:19 |
| clayg | wbhuber: he's master++ | 20:20 |
| peluse | clayg, right now workers is 32 and disks is 10 (plus OS + SSD) | 20:20 |
| *** joeljwright has quit IRC | 20:21 | |
| clayg | peluse: well that's not bad then - 40 would be ok too tho! or more maybe!? burn those cpus to the floor! | 20:22 |
| peluse | wbhuber, yeah sorry I'm not sure which patch I was on but for that bug it doesn't matter. you can repro on master by shutting down a storage node and then doing some IO | 20:22 |
| peluse | clayg, I'll try 40000 and see how that works :) | 20:22 |
| peluse | clayg, FYI shutting off EC recon doesn't affect things. Will try > workers then just blow it off and move onto something more important... | 20:23 |
| clayg | peluse: k, I don't think kota's patch should really be having an impact - there's probably just some variance - do the same test 10 times w/ and w/o and it might turn out to be about the same regardless | 20:24 |
| openstackgerrit | Merged openstack/swift: Add container sync probe test to SAIO default set https://review.openstack.org/205804 | 20:25 |
| peluse | clayg, agreed | 20:26 |
| *** dustins has quit IRC | 20:27 | |
| *** pberis has quit IRC | 20:33 | |
| clayg | wbhuber: I'm not sure why common.utils.LogAdapter.exception is catching that exception log and translating it to a plain error log message (w/o traceback) | 20:33 |
| wbhuber | %s, not %(path)s? | 20:34 |
| wbhuber | just spitting out an idea | 20:34 |
| wbhuber | don't have the tools to test that one out | 20:34 |
| *** lpabon has quit IRC | 20:34 | |
| torgomatic | clayg: so for EC GET, you have <total> nodes, and they're all either unused, 2xx, or not-2xx | 20:35 |
| peluse | clayg, FYI 64 workers didn't help. seems like kind of a nit so we're moving on to reconstructor tests... | 20:35 |
| *** MVenesio has joined #openstack-swift | 20:35 | |
| torgomatic | and I'm going to call not-2xx "4xx" because it doesn't have a minus sign in it and I'm about to use it as a variable | 20:35 |
| torgomatic | so total = unused + 2xx + 4xx | 20:35 |
| torgomatic | and total - 4xx = unused + 2xx | 20:36 |
| clayg | wbhuber: I think you'll want to try a tight unittest around ECPutter.connect with http_connect mocked out to raise a socket.error with ECONNREFUSED | 20:36 |
| torgomatic | then observe that if unused + 2xx < ec_ndata, then we're done. We can't possibly get a successful reconstruction as we don't have enough pieces. | 20:36 |
| clayg | wbhuber: well even that may not be entirely enough because the debug_logger might behave slightly differently than the real adapted logger | 20:36 |
| torgomatic | so we can stop trying more nodes once unused + 2xx < ec_ndata, or equivalently, once total - 4xx < ec_ndata | 20:36 |
| clayg | torgomatic: wow - you totally just highschool algebra'd me | 20:37 |
| torgomatic | I don't get to break out the algebra stick often | 20:38 |
| wbhuber | clayg: why don't we use self.logger.exception instead of self.app.exception_occurred? i guess real.adapted.logger = self.logger.exception whereas debug_logger exists where it is now | 20:39 |
| clayg | torgomatic: *but* if I have my request_node_count set crazy high - and I get ec_ndata + nparity 416's - like how many 404's do I really want to consume before I give you back your 416? | 20:39 |
| clayg | torgomatic: I mean - it *could* be "out there" "reconstructable" - or maybe it is *toally* reconstructed and on the primaries - it just can't service your stupid range request? | 20:40 |
| clayg | wbhuber: the benifit of app.exception_occurred is really all about the node erorr limit tracking | 20:40 |
| torgomatic | clayg: well, if you want to be 100% certain that there's no way to make this GET work, then the answer is total_nodes - ec_ndata + 1 | 20:40 |
| torgomatic | giving up earlier is probably a good idea, though, because that's a lot of work every time someone fatfingers an extra 0 onto their range request | 20:41 |
| *** pberis has joined #openstack-swift | 20:41 | |
| *** gyee has joined #openstack-swift | 20:41 | |
| notmyname | yo | 20:41 |
| clayg | I was just suggesting that in a unittest what you observe from debug_logger may not be a perfect proxy for what the real logger would do in the server - but if there's a differce it's a bug in our fake and should probably be fixed - i'm just thinking through how you might try and duplicate the issue - it *seems* like that code path should go through that adapted logger's .excpetion method and peluse should have never seen th | 20:42 |
| * notmyname is a an OpenStack Trove day today | 20:42 | |
| wbhuber | clayg: yeah, i am going to see if i can create something of a mock out of this | 20:42 |
| clayg | torgomatic: yeah that's why I figured once you have ec_ndata 416's that's probably good enough - if we had that many 206's we'd return it - I think 416's are just another kind of etag bucket - if you get enough to fill a "rebuildable" - take it and don't look back | 20:43 |
| clayg | wbhuber: good luck! | 20:43 |
| *** lpabon has joined #openstack-swift | 20:43 | |
| *** lpabon has quit IRC | 20:43 | |
| clayg | peluse: quit using our software and you wouldn't find all these arcane bugs | 20:44 |
| wbhuber | clayg: lol | 20:44 |
| clayg | torgomatic: anyway - i have a patch - i'll push it up | 20:44 |
| torgomatic | clayg: sounds good | 20:44 |
| wbhuber | clayg: it could be just how the format is mapped up from adapter logger's exception method... | 20:45 |
| clayg | wbhuber: that would be an unexpected surprise! | 20:46 |
| clayg | torgomatic: did you see peluse's note about connection timeout's in his benchmark run? *connection* timeouts. i wish we had that "better eventlet logging" patch in for him so we look at if the object-server is getting getting gummed up in eventlet land | 20:49 |
| *** changbl has quit IRC | 20:53 | |
| clayg | charz: bah! you just can't please some people! => lp bug #1488608 | 20:53 |
| openstack | Launchpad bug 1488608 in OpenStack Object Storage (swift) "stats output in reconstructor.py gives wrong device count" [Undecided,New] https://launchpad.net/bugs/1488608 | 20:53 |
| *** eranrom has joined #openstack-swift | 20:53 | |
| *** ccavanna has quit IRC | 20:53 | |
| *** tsg has joined #openstack-swift | 20:53 | |
| eranrom | Greetings. acoles, clayg thanks for reviewing the probe tests additions. | 20:55 |
| notmyname | there's not been anything added to the meeting agenda by anyone. other than a "what's the status of ...", I don't have anything either | 20:55 |
| *** tsg_ has joined #openstack-swift | 20:55 | |
| notmyname | anyone have something that needs to be in the meeting tomorrow? | 20:55 |
| clayg | notmyname: oh oh oh - let's skip! | 20:55 |
| *** tsg has quit IRC | 20:55 | |
| notmyname | clayg: maybe. let's see if there is something important first :-) | 20:55 |
| *** tsg_ is now known as tsg | 20:55 | |
| notmyname | also, I'll want to check/alert the people who are asleep right now | 20:56 |
| eranrom | you got me alerted and I am definately sleeping | 20:56 |
| notmyname | eranrom: :-) | 20:57 |
| notmyname | eranrom: actually, I talked about that with Rowen last week. I'm sorry about keeping you up late. I honestly didn't realize it was so late for you. if you do have stuff in a meeting, I want to make sure it's the first agenda item so you can get to bed | 20:58 |
| eranrom | yep, Ronen updated me, thanks! I guess IL is still not in the worst time zone... | 20:59 |
| notmyname | (oh, sorry. ronen) | 20:59 |
| notmyname | yeah, mahatic gets that award in bangalore | 21:00 |
| eranrom | anyway, I do not think I have anything for the agenda tomorrow, but will have a fresh patch soon fixing Clays good comments. | 21:00 |
| notmyname | great | 21:00 |
| clayg | eranrom: awesome! and please do poke me before it gets to be another long cycle w/o feedback - i'll try to stay on top - but I'm basically a terrible person :'( | 21:03 |
| notmyname | kota_: ho: mattoliverau: cschwede: acoles_: please ping me if you have anything to bring up in the meeting | 21:04 |
| eranrom | clayg: sure will do! Thanks! | 21:04 |
| *** tab___ has quit IRC | 21:08 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: EC GET path: require fragments to be of same set https://review.openstack.org/212187 | 21:08 |
| clayg | peluse: torgomatic: ^ after acoles showed me the light it's basically poetry in code form | 21:09 |
| *** haomaiwang has quit IRC | 21:09 | |
| *** haomaiwang has joined #openstack-swift | 21:10 | |
| *** hrou has quit IRC | 21:14 | |
| openstackgerrit | Eran Rom proposed openstack/swift: Container-Sync to iterate only over synced containers https://review.openstack.org/205803 | 21:15 |
| *** nadeem has quit IRC | 21:30 | |
| openstackgerrit | Samuel Merritt proposed openstack/swift: Make the object auditor's run-once mode run once. https://review.openstack.org/216449 | 21:31 |
| openstackgerrit | Minwoo Bae proposed openstack/swift: EC: Handoff node to push existing fragment to the correct location. https://review.openstack.org/196848 | 21:37 |
| *** rjaiswal has quit IRC | 21:40 | |
| openstackgerrit | Merged openstack/swift: test/(functional/probe):Replace python print operator with print function (pep H233, py33) https://review.openstack.org/206603 | 21:43 |
| *** thumpba has quit IRC | 21:48 | |
| *** eranrom has left #openstack-swift | 21:54 | |
| *** tsg has quit IRC | 21:56 | |
| *** pgbridge has quit IRC | 21:57 | |
| *** MVenesio has quit IRC | 22:00 | |
| clayg | minwoob: ping! | 22:00 |
| minwoob | clayg: Hi! | 22:01 |
| minwoob | clayg: I think this is right. | 22:02 |
| minwoob | clayg: partner_primary (node) is placed with the extra fragment. | 22:03 |
| minwoob | (Which simulates ring rebalance causing fragments of the same object to be placed on the same node). | 22:04 |
| minwoob | clayg: Reconstructor then fixes the situation on the "new primary". | 22:05 |
| *** rjaiswal has joined #openstack-swift | 22:07 | |
| minwoob | Or vice versa -- it has now become a handoff node. | 22:07 |
| *** aix has joined #openstack-swift | 22:07 | |
| clayg | minwoob: yeah... I think that's only one half of the story - not bad to have a test for it - but... I think your change had more to do with what happens when you run the reconstructor on say the downed primaries left hand partner (who would discover the frag missing from the downed primary already exists on the *other* primary and *not* rebuild it) | 22:08 |
| clayg | we would expect that after running *someone's* reconstructor with the change in the patch that it would *not* rebuild the object (GET should still work) but instead *wait* for the other reconstructor to run. | 22:08 |
| clayg | so maybe the test is more than halfway there | 22:08 |
| clayg | minwoob: I think there's a helper method used in one of the other ec probe tests that will find anodes left and right hand partners | 22:09 |
| *** haomaiwang has quit IRC | 22:09 | |
| clayg | if you enumerate them explicitly you may be have to move the handoff frag to the left hand partner, run reconstructor on right hand partner (verify it's *not* reconstrcuted) then run reconstructor on the right hand partner and verify it's reverted | 22:10 |
| *** haomaiwa_ has joined #openstack-swift | 22:10 | |
| clayg | minwoob: also, for my own personal edification - can you tell me what shutil.move will do if you give it a path that overlaps with another? does it just *merge* the two? I feel like if I tried to `mv /srv/node1/sdb1/objects/0 /srv/node2/sdb2/objects/0` and /srv/node2/sdb2/objects/0 already exists I'd get an EEXIST | 22:11 |
| clayg | but the mv shell utility may have different behavior than the shutil.move | 22:12 |
| *** chsc has quit IRC | 22:12 | |
| minwoob | clayg: It depends on whether dest is a file or folder. | 22:12 |
| minwoob | clayg: If it's a file, I think it will overwrite it. | 22:12 |
| minwoob | clayg: If it's a folder, I believe it will place src in dest. | 22:13 |
| clayg | hrmm... actually 412 might not be quite right - you should `import pdb; pdb.set_trace()` right there - pause the test - and the go run find on your file system until you're convinced it moved things in the right place like you think | 22:13 |
| *** tsg has joined #openstack-swift | 22:14 | |
| minwoob | Got it. | 22:17 |
| mattoliverau | Morning | 22:23 |
| *** flwang has quit IRC | 22:23 | |
| *** flwang has joined #openstack-swift | 22:24 | |
| *** pberis has quit IRC | 22:24 | |
| *** tsg has quit IRC | 22:24 | |
| *** tsg has joined #openstack-swift | 22:28 | |
| clayg | minwoob: i'm not sure if you'd played with pdb & nosetests much - if not be sure to give nose the --verbose --no-caputure (I always just use -vsx) so you can see your debugger prompt - but if you run `nosetests /path/to/probe/test_thing.py:TestCaseName -vsx` it should work (mostly) | 22:30 |
| clayg | mattoliverau: good morning! | 22:30 |
| minwoob | clayg: Thanks for the tip! | 22:30 |
| minwoob | mattoliverau: morning! | 22:30 |
| minwoob | clayg: Yeah, alternatively I normally use the PyCharm debugger as it's the environment I've been using, but with probe tests for some reason they freeze even when I attach the processes. | 22:33 |
| minwoob | clayg: Right now might be a good opportunity though, to familiarize with pdb & nosetests. | 22:34 |
| *** silor1 has joined #openstack-swift | 22:35 | |
| *** jlhinson has quit IRC | 22:35 | |
| clayg | minwoob: oh that's cool tho - you'll have to show me that at the next hack-a-thon! | 22:36 |
| minwoob | clayg: Sure. We're all using it over here, btw. | 22:37 |
| minwoob | Kind of as our default tool. | 22:37 |
| *** silor has quit IRC | 22:38 | |
| *** silor1 is now known as silor | 22:38 | |
| minwoob | Well, not kind of. For real. | 22:38 |
| *** ho has joined #openstack-swift | 22:44 | |
| clayg | notmyname: I think there should be a limit to the number of things we can have on the priority review queue :\ | 22:46 |
| *** tsg has quit IRC | 22:47 | |
| *** asd112z has quit IRC | 22:59 | |
| *** km has joined #openstack-swift | 23:00 | |
| *** wbhuber has quit IRC | 23:07 | |
| ho | good morning! | 23:07 |
| clayg | ho: good morning | 23:08 |
| *** haomaiwa_ has quit IRC | 23:09 | |
| *** 64MADOR5X has joined #openstack-swift | 23:10 | |
| *** silor has quit IRC | 23:11 | |
| *** minwoob has quit IRC | 23:13 | |
| ho | clayg: hello | 23:14 |
| mattoliverau | ho: morning | 23:18 |
| ho | mattoliverau: morning! | 23:19 |
| *** kei_yama has joined #openstack-swift | 23:25 | |
| *** dmorita has joined #openstack-swift | 23:26 | |
| *** minwoob has joined #openstack-swift | 23:26 | |
| *** bitblt has joined #openstack-swift | 23:28 | |
| *** bitblt has quit IRC | 23:28 | |
| *** changbl has joined #openstack-swift | 23:29 | |
| *** marzif_ has quit IRC | 23:29 | |
| *** kevinc_ has quit IRC | 23:30 | |
| *** minwoob has quit IRC | 23:32 | |
| *** pberis has joined #openstack-swift | 23:33 | |
| *** ccavanna has joined #openstack-swift | 23:35 | |
| ho | notmyname: +1 for clayg's comment and I would like to know registered by "who" in Starred Patches (by core), if possible. | 23:35 |
| clayg | notmyname: and a pony | 23:36 |
| *** tsg has joined #openstack-swift | 23:37 | |
| *** annegent_ has joined #openstack-swift | 23:42 | |
| *** marcusvrn_ has quit IRC | 23:45 | |
| *** annegentle has quit IRC | 23:45 | |
| *** CR7 has joined #openstack-swift | 23:46 | |
| *** kota_ has joined #openstack-swift | 23:49 | |
| *** ChanServ sets mode: +v kota_ | 23:49 | |
| kota_ | good morning | 23:52 |
| *** pberis has quit IRC | 23:52 | |
| tonyb | notmyname: Can you -W and then +W https://review.openstack.org/#/c/215786/ it seems to be stuck :/ | 23:52 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!