| *** tonanhngo has quit IRC | 00:10 | |
| *** caiobrentano has joined #openstack-swift | 00:13 | |
| *** openstackstatus has quit IRC | 00:13 | |
| *** openstackstatus has joined #openstack-swift | 00:16 | |
| *** ChanServ sets mode: +v openstackstatus | 00:16 | |
| *** ndk- has quit IRC | 00:16 | |
| *** portante has quit IRC | 00:17 | |
| *** ndk_ has joined #openstack-swift | 00:19 | |
| *** portante has joined #openstack-swift | 00:19 | |
| *** tonanhngo has joined #openstack-swift | 00:21 | |
| *** ndk_ has quit IRC | 00:23 | |
| *** ndk_ has joined #openstack-swift | 00:26 | |
| *** tonanhngo has quit IRC | 00:26 | |
| *** tovin07_ has joined #openstack-swift | 00:34 | |
| *** dmellado has joined #openstack-swift | 00:37 | |
| *** zhugaoxiao has quit IRC | 00:39 | |
| *** Administrator__ has quit IRC | 00:39 | |
| *** portante has quit IRC | 01:00 | |
| *** caiobrentano has quit IRC | 01:01 | |
| *** ndk_ has quit IRC | 01:01 | |
| *** dmellado has quit IRC | 01:06 | |
| *** portante has joined #openstack-swift | 01:07 | |
| *** ndk_ has joined #openstack-swift | 01:07 | |
| *** itlinux_ has joined #openstack-swift | 01:12 | |
| *** hoonetorg has quit IRC | 01:35 | |
| *** hoonetorg has joined #openstack-swift | 01:52 | |
| openstackgerrit | Kazuhiro MIYAHARA proposed openstack/swift master: Symlink support in container-sync https://review.openstack.org/421669 | 01:54 |
|---|---|---|
| *** zhurong has joined #openstack-swift | 02:10 | |
| *** itlinux_ has quit IRC | 02:24 | |
| kota_ | hello world | 02:27 |
| *** caiobrentano has joined #openstack-swift | 02:56 | |
| *** zhurong has quit IRC | 02:58 | |
| *** zhurong has joined #openstack-swift | 03:00 | |
| *** SkyRocknRoll has joined #openstack-swift | 03:19 | |
| mattoliverau | kong: hey sorry, been away with the family. Ahh yes the delete affinity patch. I'll find some time this arvo to jump back in. Thanks for bringing it up. Also sorry it's taken me a while. | 03:26 |
| mattoliverau | I'm not working on Swift fulltime atm so can only review/write when I have free moments. | 03:27 |
| *** gkadam has joined #openstack-swift | 03:43 | |
| *** zhurong has quit IRC | 04:02 | |
| *** psachin has joined #openstack-swift | 04:08 | |
| *** caiobrentano has quit IRC | 04:11 | |
| *** links has joined #openstack-swift | 04:15 | |
| *** qwertyco has joined #openstack-swift | 04:30 | |
| *** zhurong has joined #openstack-swift | 04:35 | |
| *** klrmn has quit IRC | 04:41 | |
| *** m_kazuhiro has joined #openstack-swift | 05:10 | |
| *** qwertyco has quit IRC | 05:17 | |
| *** qwertyco has joined #openstack-swift | 05:18 | |
| *** ChubYann has quit IRC | 05:19 | |
| *** psachin has quit IRC | 05:32 | |
| * kota_ is grubing increase part power patch to look at | 05:33 | |
| *** psachin has joined #openstack-swift | 05:39 | |
| *** qwertyco has quit IRC | 05:46 | |
| *** qwertyco has joined #openstack-swift | 05:47 | |
| -openstackstatus- NOTICE: unknown issue with the git farm, everything broken - we're investigating | 06:05 | |
| *** ChanServ changes topic to "unknown issue with the git farm, everything broken - we're investigating" | 06:05 | |
| *** gyee has quit IRC | 06:08 | |
| *** m_kazuhiro has quit IRC | 06:20 | |
| mattoliverau | ^^ ahh git farm is broken, so I'm not going crazy. #silverlining :) | 06:21 |
| *** qwertyco has quit IRC | 06:22 | |
| *** qwertyco has joined #openstack-swift | 06:22 | |
| *** rcernin_ has joined #openstack-swift | 06:32 | |
| *** hseipp has joined #openstack-swift | 06:41 | |
| mahatic | mattoliverau: :D | 06:41 |
| *** cschwede has joined #openstack-swift | 07:02 | |
| *** cschwede has quit IRC | 07:03 | |
| *** cschwede has joined #openstack-swift | 07:04 | |
| *** ChanServ sets mode: +v cschwede | 07:04 | |
| *** cschwede has quit IRC | 07:05 | |
| *** cschwede has joined #openstack-swift | 07:05 | |
| *** ChanServ sets mode: +v cschwede | 07:05 | |
| *** tovin07__ has joined #openstack-swift | 07:07 | |
| *** tovin07_ has quit IRC | 07:09 | |
| *** skudlik has joined #openstack-swift | 07:14 | |
| *** pcaruana has joined #openstack-swift | 07:18 | |
| *** tesseract has joined #openstack-swift | 07:45 | |
| acoles | good morning | 07:56 |
| -openstackstatus- NOTICE: ok git.openstack.org is working again, you can recheck failed jobs | 08:04 | |
| *** ChanServ changes topic to "ok git.openstack.org is working again, you can recheck failed jobs" | 08:04 | |
| mahatic | acoles: o/ | 08:35 |
| acoles | mahatic: hi | 08:45 |
| *** SkyRocknRoll has quit IRC | 08:59 | |
| *** hseipp has quit IRC | 09:07 | |
| *** vvargaszte has joined #openstack-swift | 09:07 | |
| *** aselius has quit IRC | 09:10 | |
| *** ChanServ changes topic to "Swift is 7 years old! | Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/ | Meetings: https://wiki.openstack.org/wiki/Meetings/Swift | Review Dashboard: http://not.mn/reviews.html" | 09:11 | |
| *** skudlik has quit IRC | 09:25 | |
| *** zhurong has quit IRC | 09:37 | |
| *** skudlik has joined #openstack-swift | 09:48 | |
| *** chlong has quit IRC | 10:03 | |
| *** tovin07__ has quit IRC | 10:04 | |
| *** bkopilov_ has joined #openstack-swift | 10:43 | |
| *** bkopilov has quit IRC | 10:47 | |
| *** cnf has left #openstack-swift | 10:50 | |
| *** kei_yama has quit IRC | 11:15 | |
| *** dmellado_ has joined #openstack-swift | 11:17 | |
| *** dmellado_ is now known as dmellado | 11:19 | |
| *** hseipp has joined #openstack-swift | 11:29 | |
| *** openstackgerrit has quit IRC | 11:33 | |
| *** jistr is now known as jistr|errand | 12:00 | |
| *** caiobrentano has joined #openstack-swift | 12:08 | |
| *** lucasxu has joined #openstack-swift | 12:56 | |
| *** qwertyco has quit IRC | 12:58 | |
| *** qwertyco has joined #openstack-swift | 12:58 | |
| *** catintheroof has joined #openstack-swift | 13:00 | |
| *** silor has joined #openstack-swift | 13:03 | |
| *** silor has quit IRC | 13:05 | |
| *** Dinesh_Bhor has quit IRC | 13:10 | |
| *** gkadam has quit IRC | 13:21 | |
| *** jistr|errand is now known as jistr | 13:25 | |
| *** silor has joined #openstack-swift | 13:34 | |
| *** qwertyco has quit IRC | 13:35 | |
| *** qwertyco has joined #openstack-swift | 13:35 | |
| *** psachin has quit IRC | 13:36 | |
| *** MVenesio has joined #openstack-swift | 13:41 | |
| *** qwertyco has quit IRC | 13:43 | |
| *** qwertyco has joined #openstack-swift | 13:43 | |
| *** qwertyco has quit IRC | 13:49 | |
| *** cbartz has joined #openstack-swift | 13:49 | |
| *** oshritf has joined #openstack-swift | 13:54 | |
| *** chlong has joined #openstack-swift | 13:55 | |
| *** jeremyfreudberg has joined #openstack-swift | 14:14 | |
| jeremyfreudberg | hey swift guys, does swift have support for generating temp urls at the level of container? or only per-object? | 14:15 |
| acoles | jeremyfreudberg: just objects, but you can generate a tempurl with an object name prefix https://docs.openstack.org/developer/swift/middleware.html#tempurl | 14:21 |
| jeremyfreudberg | acoles, cool. kind of hard to tell from the docs, does the prefix have to be "/"-separated (pseudo folder) or can it be any string? | 14:29 |
| *** cbartz has quit IRC | 14:30 | |
| jeremyfreudberg | acoles, never mind my question, it just got through my head that it works for the whole container | 14:37 |
| *** silor has quit IRC | 14:44 | |
| *** silor1 has joined #openstack-swift | 14:44 | |
| *** noark9 has joined #openstack-swift | 14:44 | |
| *** jeremyfreudberg has left #openstack-swift | 14:46 | |
| *** silor1 is now known as silor | 14:46 | |
| *** skudlik has quit IRC | 14:48 | |
| acoles | jeremyfreudberg: sorry, got distracted. I see what you mean - the doc examples show prefix/rest-of-path but as far as I can tell from the code prefixrest-of-path works i.e. no requirement for a / delimiter after prefix. (disclaimer: I have not used the prefix form in practice) | 14:53 |
| *** jistr is now known as jistr|mtg | 14:54 | |
| *** rcernin_ has quit IRC | 15:01 | |
| *** ukaynar has joined #openstack-swift | 15:07 | |
| *** aselius has joined #openstack-swift | 15:11 | |
| *** links has quit IRC | 15:16 | |
| *** ukaynar has quit IRC | 15:23 | |
| *** ukaynar has joined #openstack-swift | 15:23 | |
| *** klrmn has joined #openstack-swift | 15:25 | |
| *** ukaynar has quit IRC | 15:26 | |
| *** ukaynar has joined #openstack-swift | 15:27 | |
| *** jistr|mtg is now known as jistr | 15:30 | |
| *** ukaynar has quit IRC | 15:31 | |
| *** oshritf has quit IRC | 15:33 | |
| *** openstackgerrit has joined #openstack-swift | 15:35 | |
| openstackgerrit | Christian Schwede proposed openstack/swift master: Make swift-dispersion-report importable https://review.openstack.org/476953 | 15:35 |
| *** MVenesio has quit IRC | 15:37 | |
| openstackgerrit | Christian Schwede proposed openstack/swift master: Make swift-dispersion-report importable https://review.openstack.org/476953 | 15:40 |
| openstackgerrit | Christian Schwede proposed openstack/swift master: Add bin/swift-dispersion-report https://review.openstack.org/476955 | 15:40 |
| notmyname | good morning | 15:57 |
| *** vvargaszte has quit IRC | 16:07 | |
| *** links has joined #openstack-swift | 16:17 | |
| *** pcaruana has quit IRC | 16:18 | |
| *** klrmn has quit IRC | 16:18 | |
| *** lucasxu has quit IRC | 16:19 | |
| timburke | good morning | 16:20 |
| *** gyee has joined #openstack-swift | 16:21 | |
| *** hseipp has quit IRC | 16:28 | |
| *** cschwede has quit IRC | 16:31 | |
| notmyname | hmm... quite a few bugs filed overnight (from a small number of people) | 16:35 |
| *** silor has quit IRC | 16:40 | |
| *** skudlik has joined #openstack-swift | 16:53 | |
| *** MVenesio has joined #openstack-swift | 16:55 | |
| *** itlinux has joined #openstack-swift | 17:00 | |
| *** noark9 has quit IRC | 17:00 | |
| *** tesseract has quit IRC | 17:11 | |
| *** links has quit IRC | 17:15 | |
| *** itlinux has quit IRC | 17:16 | |
| *** lucasxu has joined #openstack-swift | 17:20 | |
| *** NM has joined #openstack-swift | 17:26 | |
| *** ChubYann has joined #openstack-swift | 17:27 | |
| *** klrmn has joined #openstack-swift | 17:37 | |
| *** silor has joined #openstack-swift | 17:41 | |
| *** ukaynar has joined #openstack-swift | 18:01 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: Add probe tests to better characterize If-None-Match support https://review.openstack.org/476992 | 18:04 |
| *** ukaynar has quit IRC | 18:42 | |
| *** ukaynar has joined #openstack-swift | 18:42 | |
| openstackgerrit | Alistair Coles proposed openstack/swift master: WIP: Rebalance respects others builders' last_part_moves https://review.openstack.org/477000 | 18:44 |
| acoles | clayg: ^^ just noticed an intermittent test failure but getting what I have off my machine | 18:45 |
| clayg | np | 18:46 |
| acoles | heh, I like that patch number | 18:46 |
| notmyname | nice | 18:46 |
| *** ukaynar has quit IRC | 18:47 | |
| clayg | for some reason this reminders me of a class clown making a joke in algebra in middle school... | 18:47 |
| clayg | "Mrs. Sims - how come everytime you do this stuff you always have to use some ungodly numbers like that?" | 18:48 |
| clayg | "What do you mean... ungodly?" | 18:48 |
| clayg | "Any number large than 4... or with a decimal... or fractions..." | 18:48 |
| clayg | I don't think I ever really had a sense of one number being better than another - so I always found it immensely enjoyable | 18:49 |
| acoles | clayg: I had lecturer who infomed us before our finals that if we derived an answer to his questions that did not conform to those ^^ constraints (plus being negative) then we should recheck our workings :) | 18:50 |
| clayg | love it | 18:51 |
| acoles | I think he worked quite hard to design quite complex questions with answers in {1,2,3} | 18:51 |
| acoles | good night | 18:52 |
| *** silor has quit IRC | 18:58 | |
| *** MVenesio has quit IRC | 19:37 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: Make If-None-Match:* work properly with 0-byte PUTs https://review.openstack.org/476992 | 19:46 |
| *** ukaynar has joined #openstack-swift | 20:00 | |
| *** tonanhngo has joined #openstack-swift | 20:15 | |
| *** vint_bra has joined #openstack-swift | 20:25 | |
| *** esnyder has joined #openstack-swift | 20:28 | |
| esnyder | does anyone have advice/pointers for doing per-account bandwidth ratelimiting for swift clusters? i realize that this is hard to do, what with the various proxy nodes not easily having access to what traffic is doing on other proxies :) | 20:30 |
| notmyname | esnyder: no. you're right that the hard part is finding a summary across all nodes | 20:35 |
| notmyname | if you need real-time ratelimiting (eg to protect the cluster), the ratelimit middleware in swift can be used. but that's for limiting writes per second instead of bytes per second | 20:36 |
| notmyname | if you're ok with not-real-time, then you can aggregate the logged bandwidth used and make limits based on that | 20:36 |
| esnyder | notmyname: thanks for confirmation. yes, we have issues w/ our total traffic (bytes) through loadbalancer rather than total requests/sec (or rather, we can limit req/sec w/ ratelimit already) | 20:37 |
| notmyname | however, for that, you'll need to do log processing (fairly easy) and some feedback code to limit based on what you got from logs (somewhat more complicated) | 20:37 |
| notmyname | I imagine that some load balancer that understands the request might be able to be configured to do what you want. but that's toally a SEP for me, so I have no idea | 20:38 |
| notmyname | http://hitchhikers.wikia.com/wiki/Somebody_Else%27s_Problem_field | 20:38 |
| esnyder | notmyname: yes, exploring LB side possiblities. but, when thinking about swift side, am also having a hard time seeing how to implement bandwidth limit in middleware (assuming i fix the summary-accross-nodes thing). am i just missing it, or would that need to be somewhere else? | 20:40 |
| notmyname | if I were writing middleware to ratelimit by bandwidth, I'd do somethng like the existing ratelimit middleware: I'd add the current request's bytes to a total stored in the memcache pool, and i'd check that value before every reuqest | 20:41 |
| esnyder | so, does that help if reads are mostly of really big objects? isn't it all or nothing per request? | 20:42 |
| notmyname | the bytes actually sent on the wire are tracked as they're sent. so a 5GB download that is stopped 2GB through will only track 2GB of bytes | 20:44 |
| notmyname | see the InputProxy in utils.py and how it's used in proxy_logging.py | 20:46 |
| esnyder | thanks, i'll go read code and ponder some | 20:47 |
| notmyname | or the bytes_sent in iter_response() in proxy_logging | 20:47 |
| esnyder | i'm just having a hard time seeing, in ratelimit.py, how it helps in large object situation. for instance, you have client making single request for 100GB file (for example). at the time that ratelimit can decide whether to sleep or not, everything is good. as soon as it decides not to sleep, the client is going to read that 100GB as fast as it can, right? | 20:49 |
| esnyder | (or even, as soon as your sleep is over) | 20:50 |
| notmyname | oh, I was thinking of some completely new middleware | 20:52 |
| esnyder | yes, me too, just analogizing | 20:52 |
| notmyname | and it would probably not do the exact same thing as the existing ratelimit | 20:52 |
| notmyname | wait + fast-as-possible is ok when it's doing req/sec | 20:53 |
| esnyder | to rephrase: how can i, in middleware, slow down a large object GET? or does it have to happen outside middleware? | 20:53 |
| notmyname | the new bw_ratelimit would be more about yielding the iterator for the response body (or request body?) a little at a time | 20:53 |
| *** lucasxu has quit IRC | 20:53 | |
| esnyder | i see | 20:53 |
| notmyname | yeah, you'd wrap the app iter and yield it as fast or as slow as you want | 20:53 |
| esnyder | ok, thanks for the help | 20:54 |
| *** cschwede has joined #openstack-swift | 20:57 | |
| *** ChanServ sets mode: +v cschwede | 20:57 | |
| torgomatic | esnyder: protip: don't forget about iterator closing; if someone calls .close() on your iterator, you then have to .close() the app iter, otherwise you leak file descriptors | 20:57 |
| * torgomatic has written that bug a couple different times now | 20:57 | |
| esnyder | torgomatic: thanks. i still struggle w/ wsgi mware conventions, so every little hard won tip like that is much appreciated! | 20:58 |
| *** tonanhngo has quit IRC | 20:58 | |
| notmyname | torgomatic: do you remember if we have a handy wrapper for that? | 20:58 |
| timburke | with closing_if_possible(inner_iter): for chunk in inner_iter: ... ? | 20:59 |
| *** tonanhngo has joined #openstack-swift | 20:59 | |
| timburke | on the test side, FakeSwift's unclosed_requests is great -- i just went through closing all the subrequest iters in versioned_writes in https://review.openstack.org/#/c/307937/ | 21:00 |
| patchbot | patch 307937 - swift - Close all versioned_writes subrequests' app_iters | 21:00 |
| esnyder | awesome, thx | 21:01 |
| clayg | awww... no zaitcev :'( | 21:01 |
| notmyname | so if you're in the inner iter and out in the outer iter you get a close, make sure in the inner iter ends the iter | 21:02 |
| notmyname | that's hard for me to say out loud | 21:02 |
| notmyname | it's the swift sobriety test! please clearly enunciate the following: ... | 21:03 |
| *** tonanhngo has quit IRC | 21:04 | |
| timburke | basically, as soon as you get an app_iter, you want to be thinking about a finally block that closes it | 21:04 |
| timburke | either in your middleware, or returning it so it's someone else's problem | 21:05 |
| esnyder | (one last q... is there a handy example of some middleware of this general form of "process each piece of the data that would have been returned, had this middleware not been in the pipeline"?) | 21:06 |
| esnyder | i guess in the middleware/crypto? | 21:07 |
| esnyder | looks like. thanks all for the help, much appreciated. | 21:10 |
| *** catinthe_ has joined #openstack-swift | 21:10 | |
| *** catintheroof has quit IRC | 21:12 | |
| *** tonanhngo has joined #openstack-swift | 21:22 | |
| *** tonanhngo has quit IRC | 21:26 | |
| *** skudlik has quit IRC | 21:27 | |
| *** tonanhngo has joined #openstack-swift | 21:31 | |
| *** NM has quit IRC | 21:36 | |
| *** tonanhngo has quit IRC | 21:36 | |
| *** caiobrentano has quit IRC | 21:37 | |
| *** tonanhngo has joined #openstack-swift | 21:38 | |
| *** tonanhngo has quit IRC | 21:42 | |
| *** tonanhngo has joined #openstack-swift | 21:50 | |
| *** ukaynar has quit IRC | 21:50 | |
| *** catinthe_ has quit IRC | 21:52 | |
| *** tonanhngo has quit IRC | 21:55 | |
| clayg | why don't we land patch 406111 - both changes were just minor cleanups to patch 346475 | 22:15 |
| patchbot | https://review.openstack.org/#/c/406111/ - swift - Initial placement gets one go | 22:15 |
| patchbot | https://review.openstack.org/#/c/346475/ - swift - For any part, only one replica can move in a rebal... (MERGED) | 22:15 |
| clayg | patch 428408 is also pretty sure is good/obvious - I think rledisez' clusters are basically already doing this (sort of) | 22:17 |
| patchbot | https://review.openstack.org/#/c/428408/ - swift - Don't rehash primaries in reconstructor handoffs_o... | 22:17 |
| *** tonanhngo has joined #openstack-swift | 22:30 | |
| timburke | clayg: re 406111, i remember taking a look at it (i even have a draft about how the bitmap buddy makes sense), but got lost trying to figure out why marking parts moved during initial rebalance resulted in such worse rings. initial dispersion going from ~40 to ~60 seems bad, no? i would assume there's similar weirdness in balance though i don't remember whether i investigated what the current code produces... | 22:30 |
| timburke | are we just assuming that people are going to rebalance their rings several times before pushing out that inital set? | 22:31 |
| timburke | on 428408, i already +2ed, what are the rest of you waiting for? :P | 22:32 |
| *** tonanhngo_ has joined #openstack-swift | 22:32 | |
| *** infurno has joined #openstack-swift | 22:33 | |
| *** tonanhngo has quit IRC | 22:34 | |
| *** tonanhngo_ has quit IRC | 22:37 | |
| *** tonanhngo has joined #openstack-swift | 22:38 | |
| *** tonanhngo has quit IRC | 22:43 | |
| *** tonanhngo has joined #openstack-swift | 22:45 | |
| *** chlong has quit IRC | 22:45 | |
| *** cschwede has quit IRC | 22:45 | |
| *** tonanhngo has quit IRC | 22:49 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: Make If-None-Match:* work properly with 0-byte PUTs https://review.openstack.org/476992 | 22:50 |
| *** tonanhngo has joined #openstack-swift | 22:51 | |
| *** tonanhngo has quit IRC | 22:55 | |
| *** tonanhngo has joined #openstack-swift | 22:57 | |
| *** tonanhngo has quit IRC | 23:02 | |
| *** tonanhngo has joined #openstack-swift | 23:03 | |
| *** tonanhngo has quit IRC | 23:08 | |
| *** caiobrentano has joined #openstack-swift | 23:10 | |
| *** NM has joined #openstack-swift | 23:32 | |
| *** NM has quit IRC | 23:51 | |
| *** gyee has quit IRC | 23:52 | |
| *** NM has joined #openstack-swift | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!