| *** tsg has joined #openstack-swift | 00:00 | |
| *** yuanzz has joined #openstack-swift | 00:05 | |
| *** yuanz has quit IRC | 00:08 | |
| *** dmorita has joined #openstack-swift | 00:27 | |
| *** shri has quit IRC | 00:56 | |
| *** addnull has joined #openstack-swift | 01:05 | |
| *** nexusz99 has joined #openstack-swift | 01:18 | |
| *** kota_ has quit IRC | 01:31 | |
| *** shakamunyi has joined #openstack-swift | 01:33 | |
| *** haomaiwa_ has joined #openstack-swift | 01:46 | |
| *** nosnos has joined #openstack-swift | 01:47 | |
| *** tsg has quit IRC | 01:48 | |
| *** mitz has quit IRC | 01:48 | |
| *** dmsimard_away has quit IRC | 01:48 | |
| *** kevinc_ has quit IRC | 01:54 | |
| *** tsg has joined #openstack-swift | 01:57 | |
| *** dmsimard_away has joined #openstack-swift | 01:57 | |
| *** morganfainberg is now known as morganfainberg_Z | 02:14 | |
| *** addnull has quit IRC | 02:14 | |
| *** Krast_ has quit IRC | 02:26 | |
| *** Krast has joined #openstack-swift | 02:26 | |
| *** annegentle has joined #openstack-swift | 02:34 | |
| *** addnull has joined #openstack-swift | 02:45 | |
| *** addnull has quit IRC | 02:49 | |
| openstackgerrit | A change was merged to openstack/swift: add the account management config values to swift info https://review.openstack.org/114878 | 02:50 |
|---|---|---|
| * notmyname is looking through git history and open reviews and thinking about the next swift release | 03:02 | |
| *** dmorita has quit IRC | 03:03 | |
| *** dmorita has joined #openstack-swift | 03:04 | |
| *** shakayumi has joined #openstack-swift | 03:07 | |
| mattoliverau | notmyname: is it getting clearer? | 03:08 |
| notmyname | mattoliverau: ? | 03:08 |
| mattoliverau | notmyname: thinking about the next swift release :) | 03:09 |
| notmyname | that's what thinking about it is supposed to do. I just started :-) | 03:09 |
| *** shakamunyi has quit IRC | 03:09 | |
| *** shakayumi has quit IRC | 03:10 | |
| mattoliverau | lol | 03:10 |
| *** addnull has joined #openstack-swift | 03:18 | |
| *** addnull has quit IRC | 03:18 | |
| *** addnull has joined #openstack-swift | 03:19 | |
| *** haomaiwa_ has quit IRC | 03:20 | |
| notmyname | mattoliverau: going through what's landed, I'm starting to think that 2.1.0 is better than 2.0.1 | 03:21 |
| notmyname | a few defaults were changed and some log stuff changed (at least) | 03:21 |
| mattoliverau | ahh yeah, logging and defaults changes might require a bigger release number. otherwise operators might get grumpy | 03:23 |
| *** dmorita has quit IRC | 03:28 | |
| *** dmorita has joined #openstack-swift | 03:29 | |
| *** gyee_ has quit IRC | 03:32 | |
| *** haomaiwang has joined #openstack-swift | 03:34 | |
| *** tsg has quit IRC | 03:47 | |
| *** annegentle has quit IRC | 03:51 | |
| *** addnull has quit IRC | 03:58 | |
| *** xianghuihui has quit IRC | 04:03 | |
| *** xianghuihui has joined #openstack-swift | 04:03 | |
| *** shakamunyi has joined #openstack-swift | 04:07 | |
| notmyname | mattoliverau: you're added to the AUTHORS file in this release! | 04:12 |
| mattoliverau | notmyname: woo!! That's awesome! | 04:12 |
| *** addnull has joined #openstack-swift | 04:13 | |
| *** xianghuihuihui has joined #openstack-swift | 04:15 | |
| *** xianghuihui has quit IRC | 04:18 | |
| *** xianghuihuihui has quit IRC | 04:20 | |
| *** xianghui has joined #openstack-swift | 04:21 | |
| *** haomaiwang has quit IRC | 04:23 | |
| *** haomaiwang has joined #openstack-swift | 04:23 | |
| openstackgerrit | John Dickinson proposed a change to openstack/swift: authors and changelog updates for 2.1.0 release https://review.openstack.org/115167 | 04:24 |
| notmyname | interesting. | 04:25 |
| notmyname | sometimes in the last couple of months I feel that dev work has slowed on swift (not as many people contributing). but that's actually not true at all | 04:27 |
| notmyname | https://gist.github.com/notmyname/7146d6ccdeb54d16ca38 | 04:27 |
| notmyname | we're actually slightly higher on the number of people participating in a release | 04:28 |
| notmyname | and we've got 7 new contributors in this one, which is about normal | 04:28 |
| *** haomaiw__ has joined #openstack-swift | 04:29 | |
| *** haomaiwang has quit IRC | 04:32 | |
| mattoliverau | well there you go then, the swift project, still going strong. :) I get the impression that many cores are working on big awesome features (like SP and EC) while other devs are extending parts they use (scratching thier itches) (tempurl, logging, formpost, etc.).. So nice work on the healthy developer community and friendly dev environment. I know I'm enjoying it :) | 04:36 |
| notmyname | mattoliverau: great to hear. and I think you're right | 04:36 |
| openstackgerrit | John Dickinson proposed a change to openstack/swift: authors and changelog updates for 2.1.0 release https://review.openstack.org/115167 | 04:41 |
| notmyname | perfect. all authors properly tracked now | 04:42 |
| zaitcev | A big chunk of core like EC is going to create a distortion. Also, a ton of work is in PyECLib. | 04:43 |
| *** haomaiw__ has quit IRC | 04:43 | |
| notmyname | yup | 04:43 |
| *** haomaiwang has joined #openstack-swift | 04:43 | |
| notmyname | AUTHORS has 173 people in it. git shows 170 people with commits. which is exactly right. we have 3 contributors acknowledged with no commits in swift | 04:45 |
| notmyname | one did code work before it was open-sourced. one did doc work. and the last intentionally didn't do any coding, but was the initial designer of the system at rackspace | 04:46 |
| notmyname | and if you're curious and don't want to do the archaeology yourself, they are Ed Leafe, Stephen Milton, and Will Reese (respectively) | 04:48 |
| notmyname | and I just published the tool I use to keep track of these things at every release. https://github.com/notmyname/git-stats | 04:50 |
| notmyname | it's got hard-coded path names and stuff, but it's useful to me | 04:50 |
| *** tsg has joined #openstack-swift | 04:52 | |
| *** kopparam has joined #openstack-swift | 04:52 | |
| notmyname | "* Various other minor bug fixes and improvements" is my favorite changelog entry | 04:54 |
| StevenK | notmyname: It may as well as include "Please read the diff for details" :-) | 04:57 |
| notmyname | StevenK: what!? you, the deployer, didn't read the code!?! | 04:57 |
| StevenK | Haha | 04:57 |
| mattoliverau | lol | 04:58 |
| notmyname | I strongly believe that `git shortlog >changelog` is very unhelpful to anyone and hand-curated changelogs are the way to go. so if we (I) miss something there, we're letting the deployers down about what's important | 04:59 |
| notmyname | (shade-grown, artisanal, organic changelogs. written with love.) | 05:00 |
| notmyname | oh, and free-range too. that's important | 05:00 |
| *** zaitcev has quit IRC | 05:02 | |
| mattoliverau | lol, very important, and don't forget gluten free | 05:03 |
| notmyname | naturally | 05:03 |
| StevenK | low sodium, too | 05:03 |
| notmyname | and naturally gluten free | 05:03 |
| StevenK | Now with more buzzwords! | 05:04 |
| notmyname | StevenK: nah, we're a bunch of salty old devs ;-) | 05:04 |
| notmyname | ok, as a tentative framework, here's what I'm proposing for the rest of the juno cycle | 05:05 |
| notmyname | 2.1.0 rc next monday (aug 25) with a final release on monday sept 1 | 05:06 |
| notmyname | then swift-next (either 2.2.0 or 2.1.1) RC on monday october 6. this will be the release included in the juno integrated release on october 16 | 05:07 |
| notmyname | I'll review these dates with ttx in the morning and see how they work with his schedule | 05:07 |
| *** Krast has quit IRC | 05:08 | |
| notmyname | and if there is any change (or even if there isn't), I'll also make sure this is brought up in the team meeting on wednesday | 05:09 |
| mattoliverau | notmyname: awesome, sounds like good plan to me :) | 05:09 |
| *** shakamunyi has quit IRC | 05:13 | |
| notmyname | ...and with that, I'm off to bed. I've got an early start tomorrow | 05:14 |
| mattoliverau | notmyname: good night! | 05:17 |
| *** ppai has joined #openstack-swift | 05:30 | |
| *** k4n0 has joined #openstack-swift | 05:36 | |
| *** nottrobin has quit IRC | 05:36 | |
| *** nottrobin has joined #openstack-swift | 05:37 | |
| *** Alex_Gaynor has quit IRC | 05:37 | |
| *** morganfainberg_Z is now known as morganfainberg | 05:41 | |
| *** Alex_Gaynor_ has joined #openstack-swift | 05:41 | |
| *** Alex_Gaynor_ is now known as Alex_Gaynor | 05:55 | |
| *** Alex_Gaynor has quit IRC | 05:55 | |
| *** Alex_Gaynor has joined #openstack-swift | 05:55 | |
| *** Krast has joined #openstack-swift | 06:14 | |
| *** haomaiw__ has joined #openstack-swift | 06:27 | |
| *** Krast has quit IRC | 06:28 | |
| *** haomaiwang has quit IRC | 06:30 | |
| *** nshaikh has joined #openstack-swift | 06:33 | |
| *** joeljwright has joined #openstack-swift | 06:52 | |
| *** chandankumar has joined #openstack-swift | 07:04 | |
| *** kopparam has quit IRC | 07:18 | |
| *** kopparam has joined #openstack-swift | 07:19 | |
| *** kopparam has quit IRC | 07:23 | |
| *** mitz has joined #openstack-swift | 07:49 | |
| *** kopparam has joined #openstack-swift | 07:53 | |
| *** aix has joined #openstack-swift | 07:57 | |
| *** kopparam has quit IRC | 07:58 | |
| *** fifieldt_ has joined #openstack-swift | 07:58 | |
| openstackgerrit | A change was merged to openstack/python-swiftclient: Do not create an empty directory 'pseudo/' https://review.openstack.org/115141 | 07:59 |
| *** bkopilov has joined #openstack-swift | 08:17 | |
| *** nexusz99 has quit IRC | 08:18 | |
| *** kopparam has joined #openstack-swift | 08:19 | |
| *** Krast has joined #openstack-swift | 08:22 | |
| *** tanee has joined #openstack-swift | 08:31 | |
| *** aix has quit IRC | 08:31 | |
| *** tsg has quit IRC | 08:32 | |
| *** joeljwright has quit IRC | 08:47 | |
| *** joeljwright has joined #openstack-swift | 08:48 | |
| *** aix has joined #openstack-swift | 08:59 | |
| *** mkollaro has joined #openstack-swift | 09:13 | |
| *** tanee has quit IRC | 09:15 | |
| *** tanee has joined #openstack-swift | 09:15 | |
| *** fifieldt_ has quit IRC | 09:18 | |
| *** bkopilov has quit IRC | 09:24 | |
| *** addnull has quit IRC | 09:44 | |
| *** geaaru has joined #openstack-swift | 09:44 | |
| *** ujjain has quit IRC | 09:46 | |
| *** addnull has joined #openstack-swift | 09:48 | |
| *** ujjain has joined #openstack-swift | 09:48 | |
| geaaru | hi, i'm a newbie of swift. In all examples that I found I see that rings are created all with name object.builder, container.builder and account.builder. Are mandatary these names ? If not what is options to set for define use of correct ring ? thanks in advance | 09:49 |
| *** ujjain has quit IRC | 09:51 | |
| *** ujjain has joined #openstack-swift | 09:51 | |
| *** kopparam has quit IRC | 10:21 | |
| *** kopparam has joined #openstack-swift | 10:22 | |
| *** kopparam has quit IRC | 10:26 | |
| *** zul has quit IRC | 10:36 | |
| *** Midnightmyth has joined #openstack-swift | 10:50 | |
| *** mitz has quit IRC | 10:51 | |
| *** mitz has joined #openstack-swift | 10:53 | |
| *** kopparam has joined #openstack-swift | 10:53 | |
| *** kopparam has quit IRC | 11:33 | |
| *** dmsimard_away is now known as dmsimard | 11:39 | |
| *** nexusz99 has joined #openstack-swift | 11:55 | |
| *** ppai has quit IRC | 11:58 | |
| *** Krast has quit IRC | 12:02 | |
| nexusz99 | Hi. I'm using swift3 middleware and running proxy-server on apache web server. If i request 'PUT BUCKET', swift3 middleware response 'Location: /[bucket]' in response header. And then Apache redirect to '/bucket' by GET Method. But keystone response 403 forbidden. is there any solution to solve this problem? | 12:04 |
| *** dmorita has quit IRC | 12:17 | |
| *** nexusz99 has quit IRC | 12:29 | |
| *** annegentle has joined #openstack-swift | 12:30 | |
| *** addnull has quit IRC | 12:33 | |
| *** kopparam has joined #openstack-swift | 12:44 | |
| *** kopparam has quit IRC | 12:46 | |
| *** annegentle has quit IRC | 12:52 | |
| *** nosnos has quit IRC | 12:56 | |
| *** miqui has joined #openstack-swift | 13:19 | |
| *** mrsnivvel has quit IRC | 13:21 | |
| notmyname | geaaru: you should use those names. the swift-ring-builder edits the .builder files and generates .ring.gz files. and the .ring.gz files have to be named a certain way. so using the documented .builder names just makes your life easier | 13:44 |
| *** HenryG_ has joined #openstack-swift | 13:46 | |
| *** HenryG has quit IRC | 13:47 | |
| *** tdasilva has joined #openstack-swift | 13:48 | |
| geaaru | notmyname: ok, thank for reply | 13:55 |
| geaaru | *thanks | 13:55 |
| openstackgerrit | paul luse proposed a change to openstack/swift: Fix sporadic false failure in xprofile unit test code (master) https://review.openstack.org/115292 | 13:58 |
| *** HenryG_ is now known as HenryG | 14:07 | |
| *** cebruns_ has quit IRC | 14:12 | |
| *** shakamunyi has joined #openstack-swift | 14:16 | |
| *** chandankumar has quit IRC | 14:17 | |
| *** shakamunyi has quit IRC | 14:32 | |
| *** tsg has joined #openstack-swift | 14:35 | |
| *** shakamunyi has joined #openstack-swift | 14:47 | |
| *** annegentle has joined #openstack-swift | 15:03 | |
| *** annegentle has quit IRC | 15:03 | |
| *** annegentle has joined #openstack-swift | 15:04 | |
| *** tsg has quit IRC | 15:07 | |
| *** kevinc_ has joined #openstack-swift | 15:07 | |
| notmyname | good morning | 15:35 |
| *** nshaikh has quit IRC | 15:35 | |
| *** kevinc_ has quit IRC | 15:41 | |
| *** gyee has joined #openstack-swift | 15:41 | |
| *** nexusz99 has joined #openstack-swift | 15:43 | |
| peluse | good morning | 15:45 |
| notmyname | mattoliverau: will you be able to put a summary of oslo.messaging together by the swift team meeting this week? https://etherpad.openstack.org/p/swift_gap_scratchpad | 15:48 |
| notmyname | hurricanerix: will you be able to put a summary of oslo.logging together by the swift team meeting this week? https://etherpad.openstack.org/p/swift_gap_scratchpad | 15:48 |
| notmyname | cschwede: will you be able to put a summary of oslo.serialization together by the swift team meeting this week? https://etherpad.openstack.org/p/swift_gap_scratchpad | 15:49 |
| hurricanerix | notmyname: yeah, i am working on something. at the least it will be a list of questions that need to be answered, but i will try to fill in as much as i can. | 15:49 |
| notmyname | looks like those are the only 3 oslo libraries left | 15:49 |
| notmyname | hurricanerix: thanks :-) | 15:49 |
| hurricanerix | notmyname: np | 15:50 |
| *** elambert has joined #openstack-swift | 15:53 | |
| *** kevinc_ has joined #openstack-swift | 15:55 | |
| *** wer has quit IRC | 15:55 | |
| *** kevinc_ has quit IRC | 15:57 | |
| *** chandankumar has joined #openstack-swift | 15:57 | |
| *** nexusz99 has quit IRC | 15:57 | |
| *** nexusz99 has joined #openstack-swift | 15:58 | |
| *** wer has joined #openstack-swift | 16:03 | |
| *** tsg has joined #openstack-swift | 16:16 | |
| *** mwstorer has joined #openstack-swift | 16:26 | |
| *** shri has joined #openstack-swift | 17:08 | |
| *** geaaru has quit IRC | 17:08 | |
| *** annegentle has quit IRC | 17:15 | |
| *** dmsimard is now known as dmsimard_away | 17:17 | |
| *** chandankumar has quit IRC | 17:39 | |
| *** shakamunyi has quit IRC | 17:39 | |
| *** nexusz99 has quit IRC | 18:01 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/python-swiftclient: Updated from global requirements https://review.openstack.org/89250 | 18:07 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 18:07 |
| *** annegentle has joined #openstack-swift | 18:10 | |
| *** shakamunyi has joined #openstack-swift | 18:11 | |
| *** aix has quit IRC | 18:15 | |
| *** dmsimard_away is now known as dmsimard | 18:19 | |
| *** dmsimard is now known as dmsimard_away | 18:24 | |
| *** chandankumar has joined #openstack-swift | 18:28 | |
| openstackgerrit | A change was merged to openstack/swift: Update tempurl docstring with methods config option https://review.openstack.org/112559 | 18:28 |
| *** shakamunyi has quit IRC | 18:30 | |
| *** zaitcev has joined #openstack-swift | 18:32 | |
| *** ChanServ sets mode: +v zaitcev | 18:32 | |
| *** shakamunyi has joined #openstack-swift | 18:44 | |
| *** erlon has joined #openstack-swift | 18:55 | |
| notmyname | last night I went through the commit history and planed out the next couple of releases. here's what's tentatively scheduled (reposting during the day for most of you) | 18:59 |
| notmyname | next monday (aug 25) cut 2.1.0 RC. the current WIP changelog is at https://review.openstack.org/#/c/115167/ | 18:59 |
| notmyname | then finalize it on sept 1 | 18:59 |
| notmyname | then do the 2.next release which will be in the juno integrated release on oct 6, finalized on oct 16 (the day of the juno release) | 19:00 |
| *** gyee has quit IRC | 19:08 | |
| *** tsg has quit IRC | 19:30 | |
| *** mahatic has joined #openstack-swift | 19:35 | |
| *** kevinc_ has joined #openstack-swift | 19:37 | |
| *** dmsimard_away is now known as dmsimard | 19:38 | |
| tdasilva | notmyname: wrt 114120, i was just pointing out that current behavior if a client tried to delete an object that did not exist, it would receive a 404 back, with this patch it changes to 204 | 19:39 |
| tdasilva | I think that's the same thing torgomatic was saying... | 19:39 |
| *** dmsimard is now known as dmsimard_away | 19:39 | |
| tdasilva | notmyname: definitely not a big deal, but just seemed like a change in behavior | 19:39 |
| notmyname | tdasilva: yes, we're all saying the same thing :-) | 19:40 |
| *** mahatic_ has joined #openstack-swift | 19:40 | |
| tdasilva | ok | 19:40 |
| tdasilva | :D | 19:40 |
| * notmyname wonders if gerrit can filter out "-1 because can't be merged" vs "-1 because gate failed" | 19:41 | |
| *** joeljwright1 has joined #openstack-swift | 19:42 | |
| *** joeljwright has quit IRC | 19:44 | |
| *** mahatic has quit IRC | 19:47 | |
| *** dmsimard_away is now known as dmsimard | 19:54 | |
| *** tsg has joined #openstack-swift | 19:54 | |
| notmyname | looks like gerrit 2.9 has a "is:mergeable" search query. but we run 2.8 | 19:55 |
| *** dmsimard is now known as dmsimard_away | 19:56 | |
| *** chandankumar has quit IRC | 19:56 | |
| *** dmsimard_away is now known as dmsimard | 19:58 | |
| *** acoles has quit IRC | 20:06 | |
| *** mkollaro has quit IRC | 20:09 | |
| *** acoles_away has joined #openstack-swift | 20:12 | |
| *** acoles_away is now known as acoles | 20:12 | |
| *** ChanServ sets mode: +v acoles | 20:12 | |
| *** Midnightmyth has quit IRC | 20:27 | |
| openstackgerrit | Joel Wright proposed a change to openstack/python-swiftclient: Add importable SwiftService incorporating shell.py logic https://review.openstack.org/85453 | 20:32 |
| *** gyee has joined #openstack-swift | 20:36 | |
| *** dmsimard is now known as dmsimard_away | 20:37 | |
| *** kevinc_ has quit IRC | 20:45 | |
| *** shakamunyi has quit IRC | 20:46 | |
| *** miqui has quit IRC | 20:49 | |
| *** joeljwright1 has quit IRC | 21:22 | |
| *** fifieldt has quit IRC | 21:27 | |
| *** Midnightmyth has joined #openstack-swift | 21:29 | |
| *** fifieldt has joined #openstack-swift | 21:41 | |
| *** annegentle has quit IRC | 22:01 | |
| mattoliverau | Morning all | 22:06 |
| notmyname | mattoliverau: hi | 22:06 |
| mattoliverau | notmyname: I'll make sure I wrote something about oslo.messaging today, at least enough to get a discussion going :) | 22:09 |
| notmyname | mattoliverau: thanks | 22:09 |
| *** HenryG_ has joined #openstack-swift | 22:17 | |
| *** occupant has quit IRC | 22:17 | |
| *** HenryG has quit IRC | 22:19 | |
| *** occupant has joined #openstack-swift | 22:23 | |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Respect device weights when adding replicas https://review.openstack.org/115416 | 22:25 |
| torgomatic | *whew* | 22:25 |
| torgomatic | ringbuilder code is always its own special blend of too-many-moving-parts... of course, I've added some of those parts over the years, so I can't complain too much | 22:26 |
| notmyname | torgomatic: and the best part is that the effective resulting patch is just like that one: +3/-1 lines | 22:27 |
| torgomatic | notmyname: well, it was a simple enough bug once I saw it... the trick was finding it in the first place :) | 22:27 |
| notmyname | torgomatic: that would be good to get into this upcoming release (ie landed by friday) | 22:29 |
| torgomatic | notmyname: well, I'm not gonna +2 it... get to work! ;) | 22:29 |
| *** carolinea has joined #openstack-swift | 22:34 | |
| *** bill_az_ has joined #openstack-swift | 22:34 | |
| carolinea | I have a question about handoff. anyone around to help? | 22:35 |
| torgomatic | carolinea: I can take a shot | 22:37 |
| thurloat | silly question, having trouble finding the docs for it: how do I set the content type of an object using swiftclient ? | 22:38 |
| carolinea | So the replicator will push objects to a handoff zone if it cant write to one of the primary zones is what I see in all of the documentation, and have heard in training; but I tried to produce this in a cluster, and I didn't see handoff taking place. I calculated 1-3 and handoff for an object, wrote the object, saw it in all 3 primary locations. took down the networking on one of the primary locations and didn't see the obje | 22:38 |
| *** occupant has quit IRC | 22:38 | |
| *** occupant has joined #openstack-swift | 22:39 | |
| torgomatic | carolinea: a node-down failure won't cause replication; try unmounting the disk | 22:40 |
| torgomatic | thurloat: I think there's a -H option to set arbitrary headers; try that | 22:40 |
| torgomatic | if it's not -H, it's something like it | 22:41 |
| carolinea | okay, that is what someone told me recently but I wasn't sure. So is the replicator looking for a response from the other object node saying that the drive is down and it needs to go elsewhere? as opposed to the replicator going to handoff if it doesnt get a response? | 22:41 |
| thurloat | swift post -H "Content-Type:text/html" ? or does it need the whole X-Object-Blah ? | 22:41 |
| torgomatic | carolinea: yeah, that... the replicator needs to get a 507 response before it'll push elsewhere. It's to prevent replication storms during a rolling upgrade | 22:42 |
| thurloat | torgomatic: that seemed to do the trick *cheers* | 22:42 |
| torgomatic | if you didn't do this, every time you took a node down for a couple minutes to upgrade its software, the rest of the cluster would start replicating what was on there | 22:42 |
| torgomatic | thurloat: hooray! :) | 22:42 |
| carolinea | torgomatic: thank you! | 22:43 |
| torgomatic | carolinea: you're quite welcome :) | 22:43 |
| carolinea | torgomatic: that makes sense, as to why to control that | 22:43 |
| carolinea | torgomatic: thanks again, have a good night! | 22:43 |
| *** carolinea has quit IRC | 22:51 | |
| *** shakamunyi has joined #openstack-swift | 23:02 | |
| thurloat | Are there performance implications for containers with thousands / tens of thousands / hundreds of thousands of files ? | 23:02 |
| thurloat | rather than having them with paths to define "folders", just a flat container | 23:03 |
| *** tdasilva has quit IRC | 23:05 | |
| *** tsg has quit IRC | 23:14 | |
| *** shakamunyi has quit IRC | 23:18 | |
| notmyname | thurloat: remember that containers aren't nested, so "a/c/o/foo/bar/baz" and "/a/c/o/bar/baz/foo" are in the same container | 23:20 |
| thurloat | notmyname: yeah, sorry if unclear. | 23:20 |
| thurloat | wanting to ensure that having files as /acofoobarbazz and a/c/o/foo/bar/baz aren't any slower | 23:21 |
| notmyname | thurloat: and there is a performance impact for object PUTs as the number of objects in the cluster gets bigger. this is one reason we always recommend to use a lot of containers | 23:21 |
| notmyname | thurloat: correct. slashes in an object name have no impact on object read or write performance | 23:21 |
| thurloat | I just noticed that the swiftclient and glance swift backend behave differently for segmented files | 23:21 |
| thurloat | the swiftclient puts hte segments in their own container, whereas the glance backend puts them all in the same container with .0001, .0002 | 23:22 |
| thurloat | noticed that after about 1tb of glance images had been uploaded to swift, the number of files is growing quickly | 23:22 |
| notmyname | thurloat: hmm..that's interesting | 23:25 |
| thurloat | notmyname: and in the process of migrating some glance images from our ceph backend to swift using the swiftclient, it's laying the files out differently when segmenting | 23:25 |
| notmyname | thurloat: is glance using large object manifests or are they managing the chunks themselves? | 23:27 |
| *** Midnightmyth has quit IRC | 23:28 | |
| thurloat | notmyname: Manifest: glance/05744531-c108-4344-8c19-8f74bf710793- | 23:30 |
| thurloat | on one of the glance images `swift stat` | 23:30 |
| thurloat | and Manifest: glance_segments/b3ac3fd0-fee3-460c-ac4f-b0b71e6d3e6f/1408490408.717103/985661440/209715200 is what i get from when I copy the files in using swiftclient. | 23:31 |
| thurloat | looks manifesty to me | 23:31 |
| notmyname | thurloat: is that 2 different containers? glance and glance_segments? | 23:32 |
| thurloat | uploading the image to glance container, swiftclient automatically created the glance_segments container for the segments | 23:33 |
| *** shakamunyi has joined #openstack-swift | 23:33 | |
| notmyname | oh ok | 23:33 |
| notmyname | thurloat: but when glance puts the chunks into the container, does it use manifests or not? | 23:33 |
| thurloat | yeah it does use manifests | 23:34 |
| notmyname | dynamic or static? | 23:34 |
| thurloat | https://github.com/openstack/glance/blob/master/glance/store/swift.py#L533 | 23:35 |
| notmyname | thurloat: ok, thanks (dynamic, BTW) | 23:36 |
| thurloat | thanks, was just skimming the src to find out. | 23:36 |
| thurloat | I don't think glance is going to be doing anything other than create/retrieve of full object paths since is stores the object path in the DB, doesn't actually do any list calls. | 23:38 |
| notmyname | thurloat: what are you trying to do. you said that glance is putting them in the same container. that works, if you get the prefixes right. | 23:41 |
| thurloat | notmyname: wanted to check in before some client performs a snapshot that fails to upload because of some timeout on a large container. All is working as expected currently, was concerned when I noticed the swiftclient wrote the segments elsewhere. | 23:42 |
| notmyname | thurloat: ah, ok | 23:42 |
| *** HenryG_ is now known as HenryG | 23:42 | |
| thurloat | perhaps at some point, I'll rewrite the glance backend to use multiple containers to keep object count below 10k or something per container. | 23:42 |
| notmyname | thurloat: are you using flash or spinning drives in your container rings? | 23:43 |
| thurloat | notmyname: rusty platters ATM. | 23:43 |
| notmyname | thurloat: ah. then in that case "large containers" would be around 1 million objects. (maybe 2 million, depending on how active they are). if you use flash, large containers become 10-50 million or more | 23:44 |
| notmyname | 100 million even | 23:44 |
| thurloat | beautiful information. thanks. will factor that into our growth costs. | 23:45 |
| notmyname | thurloat: also, remember that this only matters for object writes. object reads are unaffected | 23:45 |
| notmyname | thurloat: so if you have a lot of clients uploading to the same container at the same time, and that container is on spinning drives, you're going to have a bad time. higher IOPS on containers will let you have more write concurrency in a single containers | 23:46 |
| torgomatic | also note that if you're planning to rewrite the Glance backend, making it use static large objects would take container listings out of the equation entirely | 23:46 |
| notmyname | thurloat: ^^ one reason static objects are better | 23:47 |
| notmyname | thurloat: what's you're expected load? are you looking to have 50+ glance backups per second? | 23:47 |
| notmyname | eg with 1 million objects in a container and that container on spinning drives (sharing page cache with object servers), you'll see aroung a 10-20 writes/second limit | 23:48 |
| notmyname | eg with 100 million objects in one container and that container on flash on its own server, you'll see 500+ writes/second limit | 23:49 |
| notmyname | ie, flash is faster | 23:49 |
| openstackgerrit | A change was merged to openstack/swift: Fix sporadic false failure in xprofile unit test code https://review.openstack.org/114781 | 23:49 |
| thurloat | will definitely note static large files when it comes time to optimize the setup, we're currently looking at 1-3 concurrent glance snaps. usually in off hours, since no one likes taking their VMs offline during the day. | 23:51 |
| notmyname | thurloat: so really it all depends on your scale. if you're doing small scale stuff like just a few writes per minute and only a few tens of thousands of snapshots, then you have nothing to worry about | 23:51 |
| thurloat | i'll know what to look out for :) cheers notmyname & torgomatic! | 23:52 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!