| *** pheadron has quit IRC | 00:10 | |
| *** byeager has quit IRC | 00:13 | |
| *** matsuhashi has joined #openstack-swift | 00:19 | |
| *** hurricanerix has quit IRC | 00:28 | |
| *** usvyatsky has quit IRC | 00:49 | |
| *** mkollaro has quit IRC | 01:05 | |
| *** shri has left #openstack-swift | 01:05 | |
| *** tongli has quit IRC | 01:09 | |
| torgomatic | that's interesting... notmyname left a comment on https://review.openstack.org/48331 but didn't say anything about rechecks | 01:25 |
|---|---|---|
| torgomatic | and then Jenkins decided to rerun the check jobs anyway... is this a new thing? | 01:26 |
| notmyname | hmm..that is surprising | 01:26 |
| notmyname | clarkb: ???^ | 01:26 |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Send policy index on async update https://review.openstack.org/71702 | 01:27 |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Send policy index to container on sync update https://review.openstack.org/71703 | 01:27 |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Store policy index in container_stat table https://review.openstack.org/71704 | 01:27 |
| clarkb | yes its a new thing to prevent stale check results | 01:28 |
| *** nosnos has joined #openstack-swift | 01:28 | |
| clarkb | zuul will trigger check tests on changes being actively reviewed with check results >48 hours old | 01:28 |
| notmyname | fun | 01:31 |
| notmyname | thanks for the update :-) | 01:31 |
| openstackgerrit | John Dickinson proposed a change to openstack/python-swiftclient: changed things because reasons https://review.openstack.org/59790 | 01:32 |
| notmyname | ...because the existing commit message was wrong, and I don't really know what else to entitle that | 01:32 |
| * notmyname is out for a while | 01:34 | |
| clarkb | the thought is there will be fewer gate surprises if reviewers have relatively up to date check results | 01:41 |
| *** igor__ has joined #openstack-swift | 01:42 | |
| *** igor has quit IRC | 01:42 | |
| *** jasondotstar has quit IRC | 02:15 | |
| *** jasondotstar has joined #openstack-swift | 02:19 | |
| *** tnewsome has quit IRC | 02:38 | |
| openstackgerrit | A change was merged to openstack/swift: Replace Policy Index with Policy Name in Response Headers https://review.openstack.org/70265 | 02:53 |
| openstackgerrit | A change was merged to openstack/swift: Fix a comment in SLO middleware https://review.openstack.org/71682 | 02:53 |
| openstackgerrit | A change was merged to openstack/python-swiftclient: changed things because reasons https://review.openstack.org/59790 | 02:59 |
| openstackgerrit | A change was merged to openstack/swift: Add tests for swift-ring-builder https://review.openstack.org/71106 | 02:59 |
| *** fandi has joined #openstack-swift | 03:07 | |
| *** basha has joined #openstack-swift | 04:39 | |
| *** saju_m has joined #openstack-swift | 05:08 | |
| *** nshaikh has joined #openstack-swift | 05:22 | |
| *** ppai has joined #openstack-swift | 05:27 | |
| *** basha has quit IRC | 05:27 | |
| *** basha has joined #openstack-swift | 05:28 | |
| *** gyee has quit IRC | 05:36 | |
| openstackgerrit | Shane Wang proposed a change to openstack/python-swiftclient: spellings in python swiftclient https://review.openstack.org/71750 | 05:44 |
| openstackgerrit | Shane Wang proposed a change to openstack/python-swiftclient: Fix misspellings in python swiftclient https://review.openstack.org/71750 | 05:54 |
| *** saju_m has quit IRC | 06:18 | |
| *** psharma has joined #openstack-swift | 06:23 | |
| *** basha has quit IRC | 06:25 | |
| *** jasondotstar has quit IRC | 06:38 | |
| openstackgerrit | A change was merged to openstack/python-swiftclient: Fix misspellings in python swiftclient https://review.openstack.org/71750 | 06:51 |
| *** matsuhashi has quit IRC | 07:05 | |
| *** mmcardle has joined #openstack-swift | 07:20 | |
| *** matsuhashi has joined #openstack-swift | 07:21 | |
| *** thomaschaaf has joined #openstack-swift | 07:23 | |
| thomaschaaf | Hello is there a way to get debug logging enabled for the swift-object-server? I am seeing extremly high io on a single machine. https://gist.github.com/thomaschaaf/8858557 since all files should be in sync I am wondering what is happening | 07:24 |
| *** mmcardle has quit IRC | 07:25 | |
| *** saju_m has joined #openstack-swift | 07:32 | |
| *** bvandenh has joined #openstack-swift | 07:43 | |
| *** foexle has joined #openstack-swift | 07:47 | |
| *** nosnos_ has joined #openstack-swift | 07:51 | |
| *** nosnos has quit IRC | 07:54 | |
| openstackgerrit | Shane Wang proposed a change to openstack/swift: Fix misspellings in swift https://review.openstack.org/71800 | 08:02 |
| *** matsuhashi has quit IRC | 08:10 | |
| *** basha has joined #openstack-swift | 08:11 | |
| *** xga has joined #openstack-swift | 08:25 | |
| *** xga_ has joined #openstack-swift | 08:25 | |
| *** nacim has joined #openstack-swift | 08:36 | |
| *** vvechkanov has joined #openstack-swift | 08:42 | |
| *** acoles_ has joined #openstack-swift | 08:46 | |
| *** acoles_ has quit IRC | 08:47 | |
| *** haomai___ has quit IRC | 08:49 | |
| *** haomaiwang has joined #openstack-swift | 08:50 | |
| *** fbo_away is now known as fbo | 09:05 | |
| *** vvechkanov has quit IRC | 09:10 | |
| *** zanc has joined #openstack-swift | 09:11 | |
| *** mkollaro has joined #openstack-swift | 09:41 | |
| *** nosnos_ has quit IRC | 09:43 | |
| *** basha has quit IRC | 09:52 | |
| *** basha has joined #openstack-swift | 09:52 | |
| *** mkollaro1 has joined #openstack-swift | 09:55 | |
| *** mkollaro has quit IRC | 09:55 | |
| *** saju_m has quit IRC | 09:58 | |
| *** jasondotstar has joined #openstack-swift | 10:00 | |
| *** bada has joined #openstack-swift | 10:18 | |
| *** basha has quit IRC | 10:35 | |
| *** basha has joined #openstack-swift | 10:37 | |
| *** xga__ has joined #openstack-swift | 10:39 | |
| *** xga_ has quit IRC | 10:39 | |
| *** xga has quit IRC | 10:39 | |
| *** xga has joined #openstack-swift | 10:40 | |
| *** jasondotstar has quit IRC | 11:00 | |
| *** xga__ has quit IRC | 11:04 | |
| *** xga__ has joined #openstack-swift | 11:04 | |
| *** xga has quit IRC | 11:05 | |
| *** xga has joined #openstack-swift | 11:05 | |
| openstackgerrit | Sascha Peilicke proposed a change to openstack/python-swiftclient: Support building wheels (PEP-427) https://review.openstack.org/57154 | 11:17 |
| *** xga__ has quit IRC | 11:22 | |
| *** xga has quit IRC | 11:22 | |
| *** mkollaro1 has quit IRC | 11:40 | |
| *** thomaschaaf has quit IRC | 11:47 | |
| openstackgerrit | Christian Schwede proposed a change to openstack/python-swiftclient: Fix swiftclient help https://review.openstack.org/71876 | 12:01 |
| erlon | hi guys, we are working in a feature for swift. We are running to make it ready before the feature freeze. My question is, what is the best approach, to create the blueprint right now and then update it with the code later or, should we wait to create the blueprint when we have some code to show? | 12:02 |
| *** xga has joined #openstack-swift | 12:07 | |
| *** xga__ has joined #openstack-swift | 12:07 | |
| openstackgerrit | Christian Schwede proposed a change to openstack/python-swiftclient: Fix swiftclient help https://review.openstack.org/71878 | 12:08 |
| *** jasondotstar has joined #openstack-swift | 12:09 | |
| *** ppai has quit IRC | 12:13 | |
| *** Longgeek has joined #openstack-swift | 12:23 | |
| *** hurricanerix has joined #openstack-swift | 12:25 | |
| *** hurricanerix has joined #openstack-swift | 12:26 | |
| *** hurricanerix has quit IRC | 12:27 | |
| *** mmcardle has joined #openstack-swift | 12:29 | |
| *** jasondotstar has quit IRC | 12:37 | |
| *** mmcardle has quit IRC | 12:38 | |
| *** fandi has quit IRC | 12:42 | |
| cschwede | erlon: don't know where you're geographically located, but most likely you will get an answer during US daytime since most core devs are located in the US | 13:11 |
| erlon | cschwede: hmmm, Im Brazil, we are 3/4 hours ahead | 13:12 |
| cschwede | well, in that case it should be easy to get some feedback later in the day :) | 13:12 |
| cschwede | may i ask what you're working on? | 13:13 |
| cschwede | erlon: if you want to talk about your feature during the next swift meeting, you should add it to the agenda: https://wiki.openstack.org/wiki/Meetings/Swift | 13:15 |
| erlon | cschwede: sure, we are working in on a s3 driver to the new swift diskfile API | 13:19 |
| cschwede | erlon: you mean using s3 as a backend for swift? | 13:20 |
| erlon | cschwede: how do I edit the wiki, do I need to have special permissions? | 13:20 |
| cschwede | erlon: no, no special permissions needed, just use your launchpad account to login | 13:21 |
| erlon | cschwede: yes, today the code has the diskfile backend and a mem_diskfile, which is an exemple of the implementation | 13:21 |
| erlon | hmmm, ok, find it | 13:22 |
| cschwede | erlon: what use case do you have in mind for using s3 as backend? it's probably easier to just use s3 directly? | 13:24 |
| erlon | cschwede: so, we work with HDS, they have an object store platform (HCP), that can talk a proprietary protocol and S3. The idea is to have swift as a fronted to this storage platform. | 13:26 |
| erlon | I think they could implement swift protocol in HCP as well, I for some reason this must have greater cost to then | 13:27 |
| cschwede | erlon: ah, ok, nice idea! that might be interesting for other object storage vendors too. | 13:28 |
| *** russellb is now known as rustlebee | 13:30 | |
| erlon | cschwede: mhm, that I good point, we can test it in amazon s3 too after making it work with HCP | 13:31 |
| *** haomaiwang has quit IRC | 13:32 | |
| *** haomaiwang has joined #openstack-swift | 13:33 | |
| *** mmcardle has joined #openstack-swift | 13:44 | |
| *** jasondotstar has joined #openstack-swift | 13:50 | |
| *** psharma has quit IRC | 13:51 | |
| openstackgerrit | Christian Schwede proposed a change to openstack/python-swiftclient: Fix swiftclient help https://review.openstack.org/71878 | 13:52 |
| *** Dieterbe has quit IRC | 13:53 | |
| *** Dieterbe has joined #openstack-swift | 13:53 | |
| *** tongli has joined #openstack-swift | 14:10 | |
| *** pberis has quit IRC | 14:14 | |
| *** igor_ has joined #openstack-swift | 14:15 | |
| *** igor__ has quit IRC | 14:15 | |
| *** Trixboxer has joined #openstack-swift | 14:26 | |
| *** nshaikh has quit IRC | 14:33 | |
| *** xga__ has quit IRC | 14:36 | |
| *** xga_ has joined #openstack-swift | 14:36 | |
| *** xga has quit IRC | 14:36 | |
| *** xga has joined #openstack-swift | 14:36 | |
| *** mmcardle has quit IRC | 14:42 | |
| *** haomaiwang has quit IRC | 14:43 | |
| *** haomaiwang has joined #openstack-swift | 14:43 | |
| *** byeager has joined #openstack-swift | 14:58 | |
| *** mmcardle has joined #openstack-swift | 15:05 | |
| *** zul has quit IRC | 15:07 | |
| *** zaitcev has joined #openstack-swift | 15:10 | |
| *** ChanServ sets mode: +v zaitcev | 15:10 | |
| *** mmcardle has quit IRC | 15:10 | |
| *** zul has joined #openstack-swift | 15:10 | |
| *** bvandenh has quit IRC | 15:13 | |
| *** kragniz has quit IRC | 15:18 | |
| *** mkollaro has joined #openstack-swift | 15:19 | |
| *** foexle has quit IRC | 15:34 | |
| *** bsdkurt has quit IRC | 15:38 | |
| *** jergerber has joined #openstack-swift | 15:40 | |
| *** zul has quit IRC | 16:00 | |
| *** zul has joined #openstack-swift | 16:01 | |
| *** xga_ has quit IRC | 16:04 | |
| *** toni__ has joined #openstack-swift | 16:15 | |
| openstackgerrit | Orion Auld proposed a change to openstack/swift: Dynamic pipelines for server configuration https://review.openstack.org/71942 | 16:37 |
| *** tsnider has joined #openstack-swift | 16:38 | |
| *** tsnider has left #openstack-swift | 16:40 | |
| *** mmcardle has joined #openstack-swift | 16:42 | |
| portante | yeah! | 16:43 |
| notmyname | portante: hang on :-) | 16:44 |
| portante | alpha_ori, are you back? | 16:44 |
| notmyname | we're just talking abou tit in the office, and he's going to push it over the old changeset (instead of a new one) | 16:44 |
| alpha_ori | portante: Yep! | 16:44 |
| portante | nice! | 16:44 |
| portante | I am still off other muck here, so I probably won't get to it today | 16:44 |
| alpha_ori | Yeah, I took some time this week and addressed many of the issues to the patch. | 16:45 |
| alpha_ori | I still have a couple of things to do on it, so it prolly won't be ready until Monday anyhow. :) | 16:45 |
| alpha_ori | But it's getting there! | 16:46 |
| notmyname | torgomatic: you're gerrit links are really helpful. Bookmarks->OpenStack->reviews->"Open All in Tabs" | 16:46 |
| torgomatic | notmyname: glad they're helping :) | 16:46 |
| peluse | zaticev: you around or does anyone know what TZ he works in so I know when to ping? | 16:47 |
| notmyname | creiht: thanks for jumping into the eventlet thread | 16:47 |
| notmyname | peluse: he's in NM | 16:48 |
| notmyname | albequerque (or however you spell that) | 16:48 |
| peluse | notmyname: OK, coll. thanks | 16:48 |
| peluse | cool | 16:48 |
| creiht | notmyname: hehe | 16:49 |
| creiht | I was trying not to | 16:49 |
| creiht | :( | 16:49 |
| portante | vacation, huh? | 16:49 |
| notmyname | creiht: ya, I know what you mean | 16:49 |
| creiht | but some of it was just getting a little rediculous | 16:49 |
| notmyname | since we were just talking about gevent instead of eventlet in swift, I kinda hoped someone would mention that. "hey let's move to a lib that's (1) based on eventlet (2) maintained (3) and working on py3k support" instead of something wholly new | 16:50 |
| notmyname | creiht: but can't we just use more apache workers to scale? ;-) | 16:51 |
| creiht | honestly, I'm not sure gevent really gives us that much at the moment :) | 16:52 |
| notmyname | creiht: ya, I wanted your take on that | 16:52 |
| creiht | my preference would be to wait a bit and see how asyncio shakes out and what tools are built on top of that | 16:52 |
| portante | what is the thread title? | 16:53 |
| notmyname | openstack-dev] Asynchrounous programming: replace eventlet with asyncio | 16:53 |
| notmyname | portante: weren't you the first person to respond to it? | 16:53 |
| creiht | http://lists.openstack.org/pipermail/openstack-dev/2014-February/026237.html | 16:53 |
| portante | yes, but I don't see creiht post ... hmmm | 16:53 |
| creiht | http://lists.openstack.org/pipermail/openstack-dev/2014-February/026641.html | 16:54 |
| notmyname | portante: oh, also yesterday I went to the gevent site to take a look at it, and hey! portante is on video linked from gevent's front page! | 16:54 |
| portante | thx | 16:54 |
| creiht | lol | 16:54 |
| portante | oy | 16:54 |
| notmyname | portante: now I know why you want gevent. all those youtube views! | 16:54 |
| portante | fwiw: not attached to gevent, per sae | 16:56 |
| portante | more attached to avoiding throwing the baby out with the bath water | 16:56 |
| notmyname | +1 | 16:56 |
| notmyname | which was creiht's point in his reply | 16:56 |
| portante | it does not seem prudent to rewrite a whole pile a code to asyncio just to help get py3 support | 16:56 |
| *** mmcardle has quit IRC | 16:57 | |
| *** tsnider has joined #openstack-swift | 16:57 | |
| zaitcev | peluse: I'm still reading through policies and still hating the idea of git branch, but I'm getting more comfortable. Also, tab completions of IRC nicks are your friends. | 16:58 |
| *** csd has joined #openstack-swift | 16:58 | |
| *** mmcardle has joined #openstack-swift | 17:00 | |
| *** tsnider has quit IRC | 17:02 | |
| *** tsnider has joined #openstack-swift | 17:02 | |
| *** bsdkurt has joined #openstack-swift | 17:04 | |
| *** mkerrin has quit IRC | 17:05 | |
| openstackgerrit | Orion Auld proposed a change to openstack/swift: Dynamic pipelines for server configuration https://review.openstack.org/43934 | 17:08 |
| alpha_ori | Here's the new jumping-off point, fully rebased and with many of the original comments addressed. ^^ | 17:09 |
| alpha_ori | There are a few more that I'll be working on this weekend. | 17:09 |
| *** jasondotstar has quit IRC | 17:09 | |
| dfg | am i the only one that thinks that dynamic sorting of pipeline is, while neato, really complicated and just kinda scary? when things go in and start doing stuff to the pipeline it freaks me out. i kinda trust that to stay put | 17:09 |
| peluse | zaitcev: ahh, thanks for tab tip :) I started looking at the broker stuff too - wanna have a phone call? We could share some background on both of these things that, at least for me, would accelerate my review of your stuff | 17:10 |
| portante | dfg: not sure it is that dynamic | 17:10 |
| portante | it is defined by the file system layout directory hierarchy, rather than just a single line in a conf file | 17:10 |
| dfg | for example with the dlo/slo moving out to middleware- which was a good change- i'm having to rewrite a bunch of stuff for our deployment to get the next release | 17:11 |
| dfg | portante: thats exactly my point: which is easier to understand: a file system layout directory hierarchy or a single line in a conf file | 17:11 |
| portante | the powerful thing about alpha_ori work is that you can define the relationships for the middleware pieces you need to enforce | 17:11 |
| dfg | like when I'm trying to figure out whats going on and I look at the conf to see what running and surprise thats not what's running. some refactor put in 2 weeks ago caused a bug that caused my pipeline to go haywire and i don't know anything about whats running | 17:13 |
| portante | dfg: so ideally we should have a "what is my final pipeline" in play, if we don't already have that | 17:14 |
| alpha_ori | dfg: You don't /have/ to use dynamic pipelines in your deployment. | 17:14 |
| alpha_ori | dfg: It must be specifically enabled by you. | 17:14 |
| portante | it is a bad thing if one can't determine that | 17:14 |
| alpha_ori | dfg: portante: swift-config -w will show you the result of the dynamic pipeline rendering. | 17:14 |
| portante | I think with alpha_ori's stuff we can get such a tool that will dump it for static or dynamic pipelines | 17:14 |
| dfg | but its an extra 300 lines in the config. if this is a management helper tool why isn't in the management tool? | 17:15 |
| portante | cool! | 17:15 |
| dfg | 300 lines in the wsgi.py | 17:15 |
| portante | wsgi.py might not be the right place for it | 17:15 |
| portante | one of the early reviews was asking that, but since we wanted to vet the concept more in the review we punted on trying to address that | 17:16 |
| alpha_ori | dfg: portante: (in the new patchset it's moved to wsgi_conf.py) | 17:16 |
| portante | cool | 17:16 |
| * portante back in a bit | 17:17 | |
| dfg | like i understand that swiftstack, or any company with a bunch of deployments, could want something like this but it seems only useful to someone who uses it. and is a little freaky to others | 17:17 |
| alpha_ori | dfg: Understood! That's why it's something that is not enabled by default. | 17:17 |
| alpha_ori | dfg: You must specifically enable it. You can even use config.d style configuration without making the pipeline dynamic. | 17:18 |
| *** mmcardle has quit IRC | 17:18 | |
| *** tdasilva has joined #openstack-swift | 17:20 | |
| *** xga has quit IRC | 17:21 | |
| gholt | I'm pretty sure the pipeline is dynamic by default? Or am I misinterpreting what all this auto-inserting stuff has been doing? | 17:22 |
| alpha_ori | gholt: That is correct. What that does in a limited capacity, this change generalizes. But I think that previous change is what dfg noted caught him by surprise, no? | 17:23 |
| dfg | alright- well i guess a separate file is better. idk- i was building pkgs the other day and it was like >100 commits and +3500 lines of code in about a month's time. its just unnerving. | 17:23 |
| gholt | Oh, that's the part I missed. There's a proposal to make things even fancier? | 17:23 |
| dfg | wait- it extends the autoinserrtion stuff? | 17:23 |
| portante | dynamic in the sense that at wsgi start up, the pipeline might be modified from what is in the configuration file | 17:23 |
| gholt | I see the patch now; catching up. | 17:24 |
| portante | not dynamic in that the pipeline will change at any point in time | 17:24 |
| portante | fwiw: orions' work sort-a opened up the gatekeeper work | 17:24 |
| alpha_ori | dfg: no, it doesn't; that stuff has to work whether or not dynamic-pipelines is enabled. | 17:24 |
| alpha_ori | dfg: I meant more conceptually. | 17:25 |
| *** tsnider has quit IRC | 17:25 | |
| *** tsnider has joined #openstack-swift | 17:26 | |
| alpha_ori | dfg: the reason it's not "in the management layer" is that it /enables/ cooperation between management tools and cluster operators. | 17:26 |
| alpha_ori | dfg: I think that includes companies that roll their own. | 17:27 |
| dfg | what scares me is that one day shit's going to break in production and i'm trying to figure out what's going on and I have to look up the topological sorting algorithm in wikipedia to realize that my slo middleware just got sorted after my auth but before sos middleware and right between, etc. etc. Just doing that at all sounds really prone to bad errors. i guess we can turn it off but the general idea seems not ideal to me. | 17:27 |
| *** tsnider has left #openstack-swift | 17:27 | |
| alpha_ori | dfg: All you'll have to do is make the change in your config, do a swift-config -w, and you can see if the pipeline is to your liking. | 17:27 |
| portante | and possible assert that it has not changed from an expected stored value | 17:28 |
| gholt | Yeah, I'm not a fan of adding a packaging dependency system to Swift. But I guess since SwiftStack is for this, I'll not have much sway. | 17:28 |
| gholt | Pretty soon we'll have .swar files or something. Swift Web ARchives... | 17:29 |
| portante | written in java, right? | 17:29 |
| *** gyee has joined #openstack-swift | 17:30 | |
| alpha_ori | Oooh, that's just mean... :) | 17:31 |
| gholt | I'd rather Keep Swift Simple™. If you need to manage a conf, use an external system to populate Swift's basic (it should be more basic, not more complicated) conf files. IMO | 17:31 |
| dfg | +1. we have enough problems as it is. worrying about conf files reordering themselves feels like pulling the rug out beneathe your feet. | 17:33 |
| alpha_ori | gholt: That's what we do, certainly. And it works great cluster management and cluster operation is performed by the same agents on a small team. | 17:33 |
| alpha_ori | gholt: As the swift ecosystem opens up, however, that will increasingly be not the case, we're finding. | 17:34 |
| *** fbo is now known as fbo_away | 17:34 | |
| alpha_ori | gholt: So the ability to make small tweaks to the pipeline independent of how the cluster is generally managed is something that has value -- | 17:34 |
| alpha_ori | gholt: we've heard from our customers. | 17:35 |
| creiht | alpha_ori: I guess I'm a bit confused, why can't these tools just update the .conf as is? | 17:35 |
| creiht | sorry if I missed something previously | 17:35 |
| *** Longgeek has quit IRC | 17:36 | |
| *** jd__ has left #openstack-swift | 17:37 | |
| alpha_ori | creiht: They can, and do; what this patch enables is for cluster operators to insert specific items that they need into the pipeline, whether for diagnostic purposes (I need something on this particular node right now) | 17:37 |
| portante | without change a .conf file and screwing it up | 17:37 |
| alpha_ori | creiht: Or for more permanent purposes (this toolchain doesn't support configuration of a particular middleware and I need this change to persist across updates from a particular management tool) | 17:38 |
| alpha_ori | creiht: It's also important to note that you would probably only enable dynamic pipeline construction if you were using some sort of automated management toolchain. | 17:39 |
| portante | to me, making swift accessible to new folks is an important thing to do | 17:39 |
| portante | for those who are not so sure of things, these kinds of changes (gatekeeper included) help the admin with swift | 17:40 |
| zaitcev | Adding magic like that makes it less accessible. | 17:41 |
| zaitcev | IMHO | 17:41 |
| gholt | I'm not against a lot of this stuff (nice conf systems, migration tools, compatibility layers, cdn integration pieces, etc.) I just wish the everything in the Swift codebase thing would stop. As a developer, it's getting harder and harder to work. And this from someone who's been with it from the beginning. | 17:42 |
| portante | when an admin gets it wrong and things are hard to debug because they get the middleware screwed up, is that more accessible? | 17:42 |
| zaitcev | Frankly the conf.d was already a step too far. | 17:42 |
| creiht | portante: when magic happens and they can't figure out why something isn't working | 17:42 |
| creiht | and I tend to side with gholt | 17:43 |
| creiht | I want things to be explicit | 17:44 |
| portante | creiht: that is why we make swift-config -w available | 17:44 |
| portante | we can't make it magic, we have to make it helpful | 17:44 |
| creiht | portante: but a lot of this auto stuff is magic | 17:44 |
| zaitcev | Why don't you just store the pipeline in MySQL since you're at it. Also has tools to extract the info, you know. | 17:44 |
| portante | creiht: really? | 17:44 |
| portante | can you describe what you see is magic about i? | 17:45 |
| creiht | I explicitly tell it to do something then it goes behind my back to change it | 17:45 |
| zaitcev | The conf.d code is insane and I cannot figure out how it works. | 17:45 |
| alpha_ori | zaitcev: Surely most of that complexity is an artifact of paste. | 17:45 |
| zaitcev | That is an artifact of my small mind, certainly, but by ratcheting this junk up you'll reach the the point when only 1 guy understands it, and then he gets hit by a bus. | 17:46 |
| alpha_ori | zaitcev: That's why the dynamic pipeline code is 300 lines instead of 75. | 17:46 |
| creiht | portante: if it was too difficult for new users, then we failed in documentation | 17:46 |
| portante | really? documentation? | 17:46 |
| portante | that is the answer? | 17:46 |
| zaitcev | Yes | 17:46 |
| portante | okay | 17:47 |
| creiht | our documentation has really fallen behind | 17:47 |
| creiht | that's a different thing | 17:47 |
| creiht | but I think it is a start | 17:47 |
| alpha_ori | creiht: I agree that that's a different thing. | 17:47 |
| creiht | at least a better start | 17:47 |
| gholt | Even so, this could be external. Then: "If you would rather use a conf.d auto-configuration style of managing Swift, you can install swift-conf-magic and..." | 17:47 |
| creiht | I would also think that if we really need conf.d support in paste, perhaps it would be better to contribute that to paste upstream? | 17:48 |
| creiht | rather than hack it into swift? | 17:48 |
| creiht | or what gholt said | 17:48 |
| zaitcev | Well to be sure the OpenStack projects had a conf split-up too, for the purposes of having switcheable auth providers. | 17:49 |
| alpha_ori | creiht: I actually looked at that. | 17:49 |
| gholt | Yeah, I'm not a fan of what a lot of those other groups are doing either. ;) I'm a curmudgeon. | 17:50 |
| creiht | lol | 17:50 |
| alpha_ori | creiht: I found the paste community not to be super-active, and not particularly amenable to change (from a quick survey). | 17:50 |
| creiht | but did you even try? :) | 17:51 |
| notmyname | there. found it. http://github.com/openstack/swift/commit/1c3b75c29140939350807bf0e5faa2d35e7257a8. (we have done the whole "remove everything not explicitly needed" thing once before. and of course just because we reverted that doesn't mean everything must go in today) | 17:51 |
| gholt | Well, at least I'm consistent on something, lol. | 17:52 |
| portante | so the question about pipelines that has always come up for me is: | 17:53 |
| portante | why do I have to read every piece of pipeline middleware to know where it should be placed? | 17:53 |
| portante | if there is a defined sequence, why not just encode that? | 17:53 |
| portante | define the dependencies, run the solver and be done | 17:53 |
| portante | but instead, as an admin I have to see how a new piece works | 17:54 |
| portante | but there are not many alternatives, are ther? | 17:54 |
| portante | I don't get an advantage by moving slo around inthe pipepline, it has to live where it lives | 17:54 |
| gholt | And yeah, I realize (with the old patch notmyname posted) that I've lost this battle many times. I'll still complain every time more stuff gets added until I'm no longer working on Swift or you guys mute me. ;) But I'll give it a rest for today. | 17:54 |
| portante | so why do we bother sticking it into a configuration file when we know ahead of time where it is supposed to be in the pipelinne? | 17:54 |
| alpha_ori | portante: true, and with this code, you could even check a static pipeline to see if it meets declared dependencies. | 17:54 |
| alpha_ori | gholt: I think we all agree that it's a thing that needs to keep being said. | 17:55 |
| portante | if pipelines were pure stylistic preference and order did not matter, I would agree that this kind of change should live outside of swift, but the order does matter | 17:55 |
| creiht | portante: I think it is an aritfact to how paste works, and the mountain of middleware that has been added to swift over time | 17:56 |
| portante | and just saying, read the docs does not seem very helpful | 17:56 |
| creiht | portante: there absolutely is a default order, and it should be what is in the swift/etc/proxy-server.conf-sample | 17:57 |
| creiht | and we should do better to point people to that | 17:57 |
| portante | why put it there/ | 17:57 |
| creiht | including in the saio | 17:57 |
| portante | ? | 17:57 |
| portante | just put it into the code | 17:57 |
| portante | why give somebody a chance to get it wrong? | 17:57 |
| creiht | if you want auto magic configs, then you should just dump paste | 17:57 |
| zaitcev | portante: because (almost) nobody can work on this pile of crud anymore and adding solver makes it worse. It takes years to start on Swift and not everyone has the luxury to "refactor" the heck out of it just for the purposes of education (hello normalize_timestamp). | 17:58 |
| portante | great, let's do that | 17:58 |
| creiht | portante: it is because we currently use paste for that | 17:58 |
| creiht | portante: or use oslo-config | 17:58 |
| creiht | :) | 17:58 |
| creiht | you know the openstack blessed config system | 17:58 |
| portante | I'm with gholt: just mute me when I complain too much, but I'll give it a rest for today. | 17:59 |
| * creiht goes back to vacation mode :) | 17:59 | |
| creiht | I guess we all have a lot to think about | 17:59 |
| * portante portante is writing ctypes classes for sysstat sar data file reading so that we can overcome this binary compatibility issues version to version with sar data files | 18:00 | |
| alpha_ori | gholt: creiht: zaitcev: I'm certainly open to other possibilities. I'd like to have a solution that allows operators to manipulate the cluster out of band of any particular management system. Can you think of an approach that doesn't rely on pushing this upstream into paste or downstream into each management layer? Or monkeypatching? | 18:03 |
| zaitcev | alpha_ori: What is your definiton of "out of band"? | 18:05 |
| alpha_ori | zaitcev: Allowing a management system and an operator to cooperate in making changes to a config. | 18:06 |
| alpha_ori | zaitcev: The way that config.d-style approaches enable systems to work. | 18:06 |
| *** nacim has quit IRC | 18:11 | |
| zaitcev | Maybe you just need your own loader instead of paste, one with more transparent interface. | 18:13 |
| *** shri has joined #openstack-swift | 18:16 | |
| notmyname | stating the obvious: we have a tension between "we can't add everything into swift" and "we can't add nothing". we've got something in the middle today, and we need to stay somewhere in the middle there. but we can't get too close to either side | 18:16 |
| notmyname | and so far, the only rule we have about this is "3rd party APIs need to be implemented outstide of swift" (from an openstack-wide decision a while back) | 18:17 |
| * notmyname thinks the lack of formal guidelines is probably more of a feature than a bug | 18:17 | |
| *** lincolnt has joined #openstack-swift | 18:17 | |
| *** igor_ has quit IRC | 18:17 | |
| torgomatic | I can put on a tuxedo while I -2 some changes if it'll help | 18:17 |
| *** igor_ has joined #openstack-swift | 18:18 | |
| openstackgerrit | Jon Snitow proposed a change to openstack/swift: Add list-containers access level to account ACLs in tempauth. https://review.openstack.org/65596 | 18:19 |
| *** Diddi has quit IRC | 18:19 | |
| torgomatic | notmyname: the other rule we have (the most important one IMO) is that Swift upgrades should be seamless for operators and users | 18:20 |
| notmyname | well, ya :-) | 18:20 |
| torgomatic | if we didn't have that one, we could throw away all the gatekeeper/dlo automatic pipeline shuffling code | 18:20 |
| torgomatic | that's the whole reason we have the modify_wsgi_pipeline method at all: seamless upgrades | 18:21 |
| torgomatic | I'm certainly not suggesting we get rid of it, but it would make my life easier if we did :) | 18:22 |
| tdasilva | hello, I'm wondering if it's possible to test storage policies in the ec branch with a saio deployment? | 18:22 |
| *** jasondotstar has joined #openstack-swift | 18:22 | |
| *** igor_ has quit IRC | 18:24 | |
| dfg | we don't have seemless upgrades at all. the slo/dlo middleware thing is a perfect examples | 18:26 |
| torgomatic | dfg: I'm curious what broke on that one | 18:27 |
| dfg | they got taken out of the proxy- which i fully support. but now they have to be in the pipeline before auth- well my sos middleware has to be after auth but before the slo/dlo- so now IO have to rewrite a bunch of stuff. | 18:27 |
| notmyname | (isn't that why being able to codify that order into the config stanzas is a good thing? it's hard to keep track of what goes where, and you and I have been working on this forever) | 18:29 |
| dfg | which is fine- its not that big of a deal but the dependencies are there and aren't obviuos. and the code writer really has no idea how people are writing their middleware. the pipeline order is importna tand should be explicit. | 18:29 |
| torgomatic | dfg: ah, that's true, I didn't test with sos | 18:29 |
| dfg | as far as the admin not understanding the stuff and we're saving them from making mistakes. we're about 50 to 1 in production problems in developer code bugs vrs admin mistakes. i trust our code a TON less than our admins. it changes everyday | 18:30 |
| *** shri has quit IRC | 18:33 | |
| *** shri has joined #openstack-swift | 18:33 | |
| dfg | i look at that code and think- ummmm- this is complicated. i look at a single line in a conf and i know what's going on. who's to say some refactor doesn't break the whole works. the pipeline is important and reorder problems tend to cause very subtle bugs. couple that with the idea that the stated pipeline that our responsible admins look at can be overridden by a code change and it puts a TON of responsibility on our testers and staging env- which, thou | 18:34 |
| torgomatic | I agree it's complicated, but I couldn't think of a better way to not lose DLO functionality when upgrading Swift :| | 18:37 |
| zaitcev | I thought that kicking DLO/SLO out into middleware was needed for policies | 18:38 |
| zaitcev | somehow | 18:38 |
| dfg | i don't have a problem with that change. I do have a problem with some middleware showing up without my knowing. also the idea that every third party middleware will be handled properly with the reordering | 18:38 |
| torgomatic | zaitcev: yeah, that's the reason it moved; otherwise there was a whole bunch of stuff in the proxy that just didn't work with policies at all | 18:39 |
| torgomatic | like ObjectController's GET worked for normal objects, but then the stuff for large objects didn't work | 18:39 |
| dfg | taking that stuff our of the proxies was really good. on top of that there was a bug with slos/dlos with range requests anyway | 18:41 |
| dfg | my point is that thinking that this reordering everything will make people safer is not right | 18:41 |
| torgomatic | dfg: oh, was there an inadvertent bugfix in there? that's always nice | 18:41 |
| *** mmcardle has joined #openstack-swift | 18:43 | |
| dfg | torgomatic: ya- it caused this odd bug over our cdn that took me a while to figure out. the bug was with 0-0 range requests- the old range_tail code stuff didn't apply them | 18:43 |
| dfg | old meaning like 4 months ago or something. of course a 4 month old line of code is kinda hard to find these days... | 18:44 |
| torgomatic | dfg: huh, wacky. well, I'm glad that works now | 18:44 |
| *** mkollaro has quit IRC | 18:46 | |
| *** shri has quit IRC | 18:47 | |
| *** mmcardle has quit IRC | 18:47 | |
| *** igor_ has joined #openstack-swift | 18:50 | |
| *** igor_ has quit IRC | 18:55 | |
| *** mmcardle has joined #openstack-swift | 18:57 | |
| *** mmcardle has quit IRC | 19:01 | |
| *** Longgeek has joined #openstack-swift | 19:18 | |
| *** Longgeek has quit IRC | 19:23 | |
| *** mmcardle has joined #openstack-swift | 19:24 | |
| *** shri has joined #openstack-swift | 19:32 | |
| tdasilva | zaitcev: I have a saio VM running code from the EC branch, and I was wondering if it's already possible to test storage policies. Do you know if that's possible? Can you point me to any documentation? | 19:33 |
| notmyname | tdasilva: yes, it's possible | 19:34 |
| notmyname | tdasilva: the ec branch is there to be parallel to master, so you can test it just like you'd test master | 19:34 |
| *** gvernik_ has joined #openstack-swift | 19:34 | |
| *** mmcardle has quit IRC | 19:35 | |
| tdasilva | notmyame: yes, i followed all the directions from here: http://docs.openstack.org/developer/swift/development_saio.html, but I'm now trying to figure out how define different policies | 19:36 |
| *** toni__ has quit IRC | 19:41 | |
| notmyname | tdasilva: ah, yes. take a look at etc/swift.conf in the ec branch. I don't think there is a good stand-alone doc yet on them, but the conf is a good place to start | 19:42 |
| notmyname | tdasilva: essentially, you have a separate ring for each policy, and then you enable/configure them in swift.conf. at that point, you've got the ability to set them at container creation | 19:43 |
| tdasilva | notmyname: to set the policy, right? | 19:44 |
| notmyname | tdasilva: ya | 19:44 |
| tdasilva | notmyname: i had heard/read about the multi-ring, but wasn't sure how to create them... | 19:44 |
| tdasilva | notmyname: but i will take a look there | 19:45 |
| notmyname | tdasilva: eg `curl -i -H "X-Storage-Policy: tunafish" -XPUT http://swift.example/v1/AUTH_mine/newcontainer | 19:45 |
| notmyname | assuming you have a policy named "tunafish" | 19:45 |
| portante | after changing swift.conf to have that policy listed | 19:46 |
| notmyname | right | 19:46 |
| portante | and restarting the container servers | 19:46 |
| portante | I think | 19:46 |
| tdasilva | ok | 19:47 |
| notmyname | peluse: torgomatic: what needs restarting? just containers or proxy too? | 19:47 |
| *** igor_ has joined #openstack-swift | 19:49 | |
| notmyname | erlon: still around? I saw your question from this morning, but got into other questions. I'd love to talk about how HDS can integrate with swift | 19:50 |
| *** mmcardle has joined #openstack-swift | 19:52 | |
| *** mmcardle has quit IRC | 19:58 | |
| torgomatic | notmyname: roughly speaking, everything needs a restart | 20:00 |
| peluse | guess I should start on the policy doc.... | 20:02 |
| *** gvernik_ has quit IRC | 20:06 | |
| notmyname | peluse: well get the rollup pieces first :-) | 20:12 |
| peluse | notmyname: yeah, that's probably a bit higher on the prioity list :) | 20:12 |
| *** tdasilva has quit IRC | 20:13 | |
| *** Longgeek has joined #openstack-swift | 20:19 | |
| *** mmcardle has joined #openstack-swift | 20:19 | |
| notmyname | looks like gerrit and zuul are offline right now | 20:20 |
| notmyname | "I was gonna do reviews, but now I can't..." | 20:20 |
| zaitcev | yeah | 20:21 |
| zaitcev | Therefore I watch cat videos | 20:21 |
| *** Longgeek has quit IRC | 20:24 | |
| *** mmcardle has quit IRC | 20:24 | |
| notmyname | gerrit is up. cat videos over | 20:26 |
| *** zackf has joined #openstack-swift | 20:30 | |
| *** openstackgerrit has quit IRC | 20:34 | |
| *** openstackgerrit has joined #openstack-swift | 20:34 | |
| *** igor___ has joined #openstack-swift | 20:34 | |
| *** csd has quit IRC | 20:35 | |
| *** igor_ has quit IRC | 20:36 | |
| *** shri has quit IRC | 20:36 | |
| *** Longgeek has joined #openstack-swift | 20:37 | |
| *** Longgeek has quit IRC | 20:41 | |
| *** jasondotstar has quit IRC | 20:42 | |
| *** mmcardle has joined #openstack-swift | 20:46 | |
| peluse | zaitcev: I'm going to work on getting a policy patch done that adds per policy stats to acct HEAD requests before looking at your pluggable back ends stuff - looks like it will be a good forcing function to get familiar with that part of the code. Will keep you posted... | 20:50 |
| *** mmcardle has quit IRC | 20:52 | |
| *** shri has joined #openstack-swift | 20:53 | |
| *** tdasilva has joined #openstack-swift | 21:10 | |
| tdasilva | notmyname, portante, peluse: thanks for the help...I got storage policy to work on my test environment... | 21:11 |
| portante | nice! | 21:12 |
| *** mmcardle has joined #openstack-swift | 21:13 | |
| notmyname | tdasilva: cool! | 21:16 |
| notmyname | tdasilva: and now you've got the power to make different policies with different replica counts and on different hardware. and that's pretty cool | 21:17 |
| notmyname | (don't run it in prod yet) ;-) | 21:17 |
| portante | or put gluster there | 21:17 |
| portante | :) | 21:17 |
| notmyname | portante: yeah, well that too | 21:17 |
| tdasilva | :D | 21:17 |
| tdasilva | yes, my plan is to try with gluster now | 21:17 |
| tdasilva | on a side note, is there a new flag for specifying the storage policy in the swift client? | 21:19 |
| portante | I think ya gotta create via curl as shown above | 21:19 |
| portante | unless peluse has a set of python-swiftclient changes? | 21:19 |
| *** mmcardle has quit IRC | 21:20 | |
| *** Longgeek has joined #openstack-swift | 21:20 | |
| *** byeager has quit IRC | 21:21 | |
| tdasilva | yeah...using the curl for now..just wondering...:D | 21:21 |
| *** Longgeek has quit IRC | 21:21 | |
| *** byeager has joined #openstack-swift | 21:22 | |
| *** Longgeek has joined #openstack-swift | 21:23 | |
| *** Longgeek has quit IRC | 21:23 | |
| *** Longgeek has joined #openstack-swift | 21:24 | |
| *** Longgeek has quit IRC | 21:25 | |
| *** byeager has quit IRC | 21:25 | |
| *** Longgeek has joined #openstack-swift | 21:25 | |
| *** Longgeek has quit IRC | 21:26 | |
| *** Longgeek has joined #openstack-swift | 21:26 | |
| *** Longgeek has quit IRC | 21:27 | |
| *** Longgeek has joined #openstack-swift | 21:28 | |
| *** Longgeek has quit IRC | 21:29 | |
| *** Longgeek has joined #openstack-swift | 21:29 | |
| *** Longgeek has quit IRC | 21:30 | |
| *** Longgeek has joined #openstack-swift | 21:32 | |
| *** Longgeek has quit IRC | 21:32 | |
| *** Longgeek has joined #openstack-swift | 21:33 | |
| *** Longgeek has quit IRC | 21:33 | |
| *** Longgeek has joined #openstack-swift | 21:36 | |
| *** Longgeek has quit IRC | 21:36 | |
| *** Longgeek has joined #openstack-swift | 21:38 | |
| *** Longgeek has quit IRC | 21:39 | |
| *** Longgeek has joined #openstack-swift | 21:39 | |
| *** Longgeek has quit IRC | 21:40 | |
| *** Longgeek has joined #openstack-swift | 21:40 | |
| *** mmcardle has joined #openstack-swift | 21:41 | |
| *** jasondotstar has joined #openstack-swift | 21:41 | |
| *** Longgeek has quit IRC | 21:41 | |
| *** Longgeek has joined #openstack-swift | 21:42 | |
| *** Longgeek has quit IRC | 21:42 | |
| *** Longgeek has joined #openstack-swift | 21:43 | |
| *** Longgeek has quit IRC | 21:44 | |
| *** Longgeek has joined #openstack-swift | 21:44 | |
| *** Longgeek has quit IRC | 21:45 | |
| *** mmcardle has quit IRC | 21:45 | |
| *** jasondotstar has quit IRC | 21:51 | |
| *** Longgeek has joined #openstack-swift | 21:51 | |
| notmyname | torgomatic: ping re https://review.openstack.org/#/c/63195/ | 21:51 |
| *** Longgeek has quit IRC | 21:51 | |
| *** Longgeek has joined #openstack-swift | 21:52 | |
| notmyname | Longgeek: check your connection. you're spamming the channel with connects and disconnects | 21:52 |
| *** Longgeek has quit IRC | 21:52 | |
| notmyname | torgomatic: how are you proposing that the new code is tested? via tox under py26? 'cause otherwise there are no tests for that code | 21:53 |
| *** ChanServ sets mode: +o notmyname | 21:54 | |
| *** Longgeek has joined #openstack-swift | 21:54 | |
| *** Longgeek was kicked by notmyname (Longgeek) | 21:54 | |
| *** notmyname sets mode: -o notmyname | 21:55 | |
| *** Longgeek has joined #openstack-swift | 21:56 | |
| *** Longgeek has quit IRC | 21:57 | |
| notmyname | really? | 21:57 |
| *** ChanServ sets mode: +o notmyname | 21:57 | |
| *** jasondotstar has joined #openstack-swift | 21:57 | |
| *** Longgeek has joined #openstack-swift | 21:58 | |
| *** Longgeek has quit IRC | 21:59 | |
| *** Longgeek has joined #openstack-swift | 21:59 | |
| *** Longgeek has quit IRC | 22:00 | |
| *** Longgeek has joined #openstack-swift | 22:01 | |
| *** notmyname sets mode: +b Longgeek!*@* | 22:01 | |
| *** Longgeek was kicked by notmyname (msg notmyname when you figure out your connectivity) | 22:02 | |
| *** mmcardle has joined #openstack-swift | 22:08 | |
| *** jasondotstar has quit IRC | 22:09 | |
| peluse | yeah curl for now w/policies - I haven't done anything with the client yet but will put that on the list... | 22:10 |
| *** lincolnt has quit IRC | 22:11 | |
| peluse | tdasilva: note that only the IO path is on the EC branch right now and a few services. Still pending are the replicator, ssync, some middleware, etc. For basic testing it should work for you. Hit us up here if you have issues | 22:11 |
| *** mmcardle has quit IRC | 22:13 | |
| *** fbo_away is now known as fbo | 22:14 | |
| *** Trixboxer has quit IRC | 22:15 | |
| *** csd has joined #openstack-swift | 22:27 | |
| *** mmcardle has joined #openstack-swift | 22:35 | |
| *** mmcardle has quit IRC | 22:40 | |
| *** byeager has joined #openstack-swift | 23:01 | |
| *** mmcardle has joined #openstack-swift | 23:02 | |
| *** mmcardle has quit IRC | 23:09 | |
| *** mmcardle has joined #openstack-swift | 23:10 | |
| *** byeager has quit IRC | 23:21 | |
| *** jasondotstar has joined #openstack-swift | 23:22 | |
| *** mmcardle has quit IRC | 23:25 | |
| *** fbo is now known as fbo_away | 23:25 | |
| torgomatic | notmyname: yeah, tox under py26 I guess. It's sort of hard to test for a resource leak like that. | 23:28 |
| *** mmcardle has joined #openstack-swift | 23:52 | |
| *** mmcardle1 has joined #openstack-swift | 23:56 | |
| *** mmcardle has quit IRC | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!