20:01:02 #startmeeting tc 20:01:03 Meeting started Tue Jun 14 20:01:02 2016 UTC and is due to finish in 60 minutes. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:01:05 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:01:07 The meeting name has been set to 'tc' 20:01:08 o/ -ish 20:01:15 o/ 20:01:20 o/ 20:01:22 Small quorum but we'll make it happen 20:01:28 Our agenda for today is: 20:01:32 #link https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee 20:01:37 * edleafe_ lurks while in his parked car 20:01:41 #topic Tidy of item 5 of the vulnerability:managed tag 20:01:48 #link https://review.openstack.org/300698 20:01:56 edleafe_: please, open the windows or turn the AC on 20:02:05 This one sounds like a good incremental improvement, now that the security folks are happy with it 20:02:13 * flaper87 is happy with it 20:02:14 Still short of a few votes though 20:02:18 vmt members are satisfied with sdake's proposal 20:02:31 hi fungi :) 20:02:32 fungi: ++ 20:02:33 +1 from me 20:02:41 thanks fungi 20:02:43 as both TC and VMT, i think it's good. 20:02:50 flaper87: It's 97F. The AC is running 20:02:58 need two more votes 20:03:07 edleafe_: jeez 20:03:24 * Rockyg suggests edleafe roll down w window to avoid heat stroke 20:03:40 Rockyg: edleafe_ nope just keep the car running with AC! 20:03:44 ttx: oops looks like i forgot to rollcall vote on it 20:03:46 fixed that 20:03:52 looks like we have enough now 20:04:01 ok, approved 20:04:08 #topic Fast-tracking projects.yaml release tags / deliverable changes (dhellmann) 20:04:09 * amrith wanders in ... 20:04:12 dhellmann: want to introduce this one ? 20:04:16 sure 20:04:22 We’ve added some validation rules to the release request process that verify that the thing being released matches the thing as defined for governance. 20:04:28 We’ve started noticing in a few cases that updates to the governance repository to “fix” release-related settings like release models and deliverable definitions have been delaying releases. 20:04:34 I would like to propose a general exception to the review policy for openstack/governance that if the change does not effect governance, and are only related to tags managed by other teams, we not wait the week for “objections”. 20:04:40 For example, changing the release model of an existing deliverable and adding (or removing) the tags for stable or security teams would only need to be reviewed by the relevant team, and then they can be approved by the chair. 20:04:45 Splitting or combining deliverables is a similar sort of change that’s a bit more complex because making new deliverables may require the approval of multiple groups to set the appropriate tags. 20:04:50 However, as long as no new repositories are added I think this sort of change should also be fast-tracked. 20:04:53 * flaper87 notices dhellmann prepared hist intro ahead of the meeting 20:04:55 dhellmann: ++ 20:04:57 Changes that add repositories, and therefore change the voter roles for the TC, should continue to go through the 1 week review period. 20:05:00 * edleafe_ knows that annegentle knows south Texas heat 20:05:17 * dhellmann ends pastebomb 20:05:56 sounds reasonable to me 20:05:57 I'm in favor of this 20:05:59 works for me 20:06:05 +1 from me 20:06:06 sdague: ++ same 20:06:24 I guess we could revert if a TC member ends up objecting two days after the fact 20:06:43 ttx: yes, that's a good point. all of this is ultimately mutable. 20:06:47 no issue from me 20:06:50 dhellmann: ooo my only tough sell point is the "splitting or combining deliverables" part... what's that like? Do you have an example? 20:07:01 dhellmann: gnocci/aodh/ceilometer type stuff? 20:07:02 annegentle : the recent stuff with neutron 20:07:05 dhellmann: so, release: and security: tags changes, + deliverable reorganization ? 20:07:27 ttx: and the stable tag 20:07:45 if I have the correspodning PTL signoff I can fasttrack them 20:07:48 and any other teams to which we've delegated management of tags in the future (like if annegentle's work turns into doc tags) 20:07:49 * edleafe_ has to go - daughter is ready. 20:08:00 ttx: yes 20:08:07 ok, let me capture that 20:08:17 should I write this up more formally as a resolution, or can we agree to a procedural thing without that? 20:08:47 we've made agreements on procedural decisions in-meeting in the past, iirc, like our current fast-track policy on typos 20:08:49 #agreed for projects.yaml chnage affecting stable: security: or release: tags (or deliverable reorganization) the TC chait can fast-track approval if the corresponding PTL approved it 20:08:55 I'm good with meeting agreement 20:08:57 ttx : dhellmann : am generally in favor..don't really see a need for resolution, we could update some instructions/readme somewhere 20:08:59 dhellmann: no, it's a procedural agreement 20:09:08 ok, cool, just making sure 20:09:18 I guess we could document it as house rules 20:09:18 I'm ok with fast-track 20:09:29 * flaper87 pictures dhellmann doing the "I don't need to write a resolution" dance 20:09:34 ttx: +1 to writing things down 20:09:36 I might regret it later :) but seems like trust is best 20:09:40 #action ttx to document house rules 20:09:58 (that's the speech I end up giving at every start of cycle) 20:10:11 lol 20:10:29 an alternative is to move some of those tags into the repo where we're using them and validating them, but we've been focusing on putting them all in the governance repo for consistency so... 20:10:30 ok, anything else on that topic ? 20:10:31 haha 20:10:45 ttx: that's it from me, if we've settled the question 20:11:18 #topic Add project Smaug to OpenStack big-tent 20:11:24 #link https://review.openstack.org/326724 20:11:34 Smaug aims to provide data protection as a service (in a large sense of the word "data") 20:11:47 I could not spot any good reason to deny them / delay their entry, so I'm tentatively +1 on this 20:12:04 saggi: o/ 20:12:12 Questions ? 20:12:17 one concept I read and read but couldn't understand is: 20:12:19 I'm here 20:12:28 what are the plugable pieces? the protections? 20:12:51 There are 3 main pluggable pieces. 20:13:04 1. is what we call protectables 20:13:17 are "Protection Plugins" open source? 20:13:18 It's the resources you want to protect and their relationships 20:13:28 annegentle: yes 20:13:35 sorry saggi, I mispoke, I wondered about the plugins. What are some of the names? 20:13:44 of plugins? 20:14:01 For protection we have cinder\nova\neutron. 20:14:02 annegentle : are you asking about which plugins are implemented/planned or which areas of smaug can have plugins? 20:14:14 https://wiki.openstack.org/wiki/Smaug#Available_Protectables 20:14:34 russellb: ++ 20:14:44 clears up what "protectables" are for me 20:14:57 Things you want to protect. The "what". 20:15:10 dhellmann: what are names of plugins, that's what I'm trying to understand. I understand what you want to protect 20:15:16 right, it's basically calling out to existing services - https://github.com/openstack/smaug/blob/ce2b117a184ade8f376839178fdc34c2d70896b7/smaug/services/protection/protection_plugins/volume/cinder_protection_plugin.py#L41 20:15:37 The user can add external protectables if they are required for the application. They define new types of resources you can protect and how they relate 20:15:40 I understood this to be a thing to put in front of different backup tools that would implement backup for different types of objects in an appropriate way. 20:15:54 They define that a volume is needed by a VM 20:16:13 saggi: as an app developer, do I define a plugin that protects multiple protectables? 20:16:17 it's pluggable in a sense that the user can add entity external to Openstack an they will be included in the tree. 20:16:24 Protection Plugins are the "how" 20:16:35 you can define multiple protection plugins to a single protectable. 20:16:45 so, what would the manila plugin name look like? 20:16:49 it's the admins responsibility to choose what protection plugin to map to what resource 20:17:08 saggi: is this an admin tool or an app developer tool? Are you protecting the service or resources run on the service? 20:17:24 saggi : so I could have a protection plugin for storing things to swift and a protection plugin for writing things to magnetic tape, and then the admin would map the appropriate one to keystone data, cinder, data, etc.? 20:17:49 dhellmann, good question 20:18:04 This will be a bank plugin. 20:18:37 So the protection plugin takes a protectable and puts it in the bank. 20:18:45 We don't enforce all the data being in the bank 20:18:57 but it must put information on where to find the data to the bank 20:19:07 so when we restore we give that information to the plugin 20:19:10 ok, I understand protectables and banks, but I don't understand protections then 20:19:18 Personally I feel like it's a weird mix of very cloudy stuff (advanced service driven by an API) but which would likely be used on very non-cloud-native apps, but then we are no longer judging the usefulness. 20:19:26 saggi: I'm worried this is a really different way to use the term plugin. 20:19:45 ttx: I had that concern as well, that if you use this, you create pet apps even. 20:19:58 ttx: rather than using cloudy architectures 20:20:00 annegentle: pet apps are fine 20:20:05 annegentle: pet apps? 20:20:10 looks like there's just a swift bank plugin today? 20:20:17 yes 20:20:26 saggi: using the "pets vs cattle" analogy, does this mean you create apps that aren't cloudy? 20:20:28 saggi : what changes if any are needed in other projects for smaug to work (or work better). 20:20:28 the value of openstack is that it spans the whole gambit 20:20:36 saggi: and therefore pet apps 20:20:53 protectable: a glance image. protection plugin: glance (it knows how to back up an image). bank: a place to store stuff, like swift 20:20:56 is that a fair summary? 20:21:09 annegentle: pet apps are a possibility, many companies still use them, that's not necessarily a bad thing 20:21:20 trying to make a tl;dr :) 20:21:20 annegentle: yeah, pet apps are fine. It's just the combination of using a cloud API to drive data protection on a pet app that sounds alien. But then I guess with a UI on top... 20:21:22 sdague: +1 20:21:37 russellb: you are correct 20:21:42 saggi: ok thanks 20:21:43 russellb: I believe it is. I asked earlier today in smaug's channel whether it'd make sense for smaug to use glance_Store to be able to talk to more stores 20:21:46 saggi: ^ 20:21:47 russleb: + 20:21:50 russellb : thanks, that's clear 20:21:57 The answer was yes, and it can be implemented as a bank 20:22:17 russellb : thanks for that. makes it clearer 20:22:30 sdague : s/gambit/gamut/ 20:22:41 dhellmann: sure, that too :) 20:22:49 ha 20:22:57 haha 20:22:58 As far as pets vs cattle. Backing up and restoring glance images and network topology is important regardless. 20:23:00 although some may say openstack is a gambit of sorts 20:23:35 Also, since we don't limit the entities you want to protect you can also back up your heat templates. 20:23:40 right, I think we decided a while ago that we were not going to focus on "perceived usefulness" and instead let projects play out and find their community or not 20:23:47 Still short of a few votes. Any other questions you'd like to ask saggi ? 20:23:50 sdague ++ 20:23:54 sdague: yeah, still was curious 20:23:58 sdague +1 20:24:05 sdague : ++ 20:24:08 this doesn't really overlap anything other projects are doing, definitely seems like some users want it 20:24:11 * dims votes 20:24:13 saggi : thanks for clarifying the technical stuff 20:24:15 do I understand correctly that this backs up not just the data but the metadata in openstack as well? 20:24:17 saggi: again, is this service intended for admins or app devs? 20:24:23 i think the dependency graph approach to backup of different resources is interesting 20:24:29 admins 20:24:32 saggi: or maybe I missed it, I'm on a terrible IRC client :) 20:24:33 i.e. would it be used to backup an openstack deployment? 20:24:33 happy to see it explored 20:24:37 but also tenant admins 20:24:46 or to backup, say a database running in the openstack cloud? 20:25:08 * dims sees amrith worry/think about trove's backup capabilities 20:25:09 amrith: The deployment, it might include backing up the DB in an optimized way. 20:25:11 am wondering how this is different from, for example, snapshots of a cinder volume directly. 20:25:17 saggi, ++ 20:25:17 This is where cooperation with freezer comes into play 20:25:21 ttx: o/ 20:25:34 amitry: because it captures more than cinder volumes, and the relationships between them 20:25:36 We now have a majority 20:25:46 amrith : it includes the metadata, and relationships between objects, so you can restore all of that rather than just bits on a volume 20:26:03 hopefully not duplicating the actual work of volume backup necessarily 20:26:10 dhellmann, I'm trying to wrap my head around a practical use case 20:26:13 anyway, that's a detail :) 20:26:15 Classically you back up storage and make scripts. We want to eliminate the need for scripts. 20:26:19 and finding it hard to relate smaug to a database 20:26:21 keeping in mind it's pretty young at this stage 20:26:24 managed by an admin or trove. 20:26:33 so I'm wondering whether maybe databases are not the right target. 20:26:43 for example, in a database there's config information and data 20:26:55 and I'm wondering how one would express that to smaug 20:27:13 and how smaug would, for example, know to quiesce a database and take a transaction consistent backup 20:27:27 in a way that would be better than whatever the database vendor provides 20:27:27 I think that's getting deeper into the tech than we need to. 20:27:31 amrith: sounds like it's more, back up your flavors and such 20:27:32 tbh, I think it's fine to explore all those questions after the meeting 20:27:45 ++ flaper87 20:27:45 amrith: We do it because the DB protectable will tell us it depends on the VM 20:27:49 amrith : as a cloud user, I do not have access to my cloud provider's database backups 20:27:54 this is where the relationships come into play 20:27:56 will take it offline 20:28:05 amrith: you are welcome to contact us on our irc, we'll be happy to discuss 20:28:06 saggi : thanks! 20:28:07 amrith: thx! 20:28:16 saggi, flaper87 dims dhellmann ... wondering how trove can use smaug. thanks! 20:28:20 OK, approving now unless someone screams 20:28:55 ttx: got my vote in 20:29:03 approved 20:29:05 We have docs about how we traverse the resource tree and build the task graph to make dependent tasks between resources happen. 20:29:07 saggi: welcome 20:29:09 Congratz! 20:29:11 horray! 20:29:16 Thanks everyone 20:29:19 thanks saggi ! 20:29:28 thanks guys 20:29:34 and gals 20:29:38 thx saggi will catch you later 20:29:38 #topic Updates projects.yaml to indicate type:service only if a REST API is provided 20:29:44 #link https://review.openstack.org/317094 20:29:55 So... I raised this one since I'm not sure we want to overload the governance projects.yaml in this way 20:30:08 On this one Anne suggests we limit type:service to things which provide a REST service endpoint, so that it could be reused to build API doc links 20:30:18 That results in removing it from Horizon, which I could agree with 20:30:22 But I have three objections 20:30:30 ttx: I originally wanted a handy way to scan for "does this provide an API?" 20:30:33 1/ it implies that type:service is redefined, since Horizon fills the "provides a user-facing long-running service, usually with a REST API" requirement 20:30:44 but fine with abandoning and going the "discover REST API docs" route 20:30:54 2/ it uses a deliverable type tag to specify something which is more of a technical property of a specific component (we have "deliverables" which provide multiple REST API endpoints) 20:31:10 3/ it's a bit of a slippery slope to add extra type of data to projects.yaml, especially data which is not directly consumed by humans. It's not a service catalog imho 20:31:21 ttx: agreed on all 3 20:31:29 annegentle: ok cool :) 20:31:35 :) 20:31:47 not to be pedantic, but whether a long-running service provides an api (rather than just ui) and what protocol is used for that api seem like separate facets too 20:32:01 if anyone has ideas for how to discover what API docs are where for each service please tell me :) 20:32:13 ttx: so add a provides:rest_api tag? 20:32:14 fungi: hence the "usually" 20:32:14 * dims nods to fungi's comment 20:32:30 mtreinish: I think I'll do a docs: -api: http://blah url 20:32:30 mtreinish: I don't think that would be useful 20:32:39 mtreinish: per deliverable 20:32:39 mtreinish's suggestion is what i was thinking of as well 20:33:00 my suggestion here would be to either extend the YAML grammar to make room for a list of APIs provided by the project team / deliverable / repository 20:33:02 +1 20:33:03 yeah, if the point of this is to link to API docs, let's just do the simple thing and add the links. 20:33:03 would that idea work - per deliverable, does it have an API, and where are the docs? 20:33:09 or move that to another repository / YAML file 20:33:26 dhellmann: ++ 20:33:43 annegentle: the trick being some of those deliverable maybe propose multiple APIs (not sure) 20:33:51 dhellmann: one goal is to figure out which services we need to provide nav to on an API docs site 20:33:51 i agree that this isn't something which necessarily needs describing and tracking by the tc (hence a governance tag) 20:33:57 annegentle: I think having a link per deliverable would work fine 20:34:00 +1 to another repository / yaml file (maintained by doc team?) 20:34:00 ttx: eesh. well ok 20:34:15 dims: ugh. hm 20:34:26 ttx: if there are multiple apis they can have a single doc still 20:34:26 dims: if we go down that route, I think realistically we already have a different solution 20:34:36 dims: I mean docs already have to figure this out from non-truth-sources. I see projects.yaml as a source of truth. 20:34:39 annegentle : sure. so maybe a deliverable tag here, and then when it comes to making sure those docs are linked elsewhere we can figure out the details of what "has an api" means for that deliverable? 20:34:50 because we have a repo for the service types list, which went on hold when we actually hit this hard question 20:34:55 dhellmann: I like per-deliverable 20:34:57 that api doc links weren't obvious 20:34:57 * dims thinking 20:34:59 sdague: yeah that too 20:35:23 we need a tag and a url to where the api docs are? 20:35:28 ok, so extend grammar to add collection of API doc links 20:35:31 annegentle: if this is mostly just for api site nav, I think we probably should revive the service types authority and do that there 20:35:43 dims: doesn't have to be a tag, especially if it needs to have a value 20:35:47 sdague: ++ 20:35:49 do we need to keep a tag, or do we just need to review what we have in these repos one time so we can manage the links elsewhere? 20:36:00 sdague: oh, is that the stuff related to the service catalog? 20:36:02 sdague: yeah, that was what I was hinting at 20:36:17 sdague: it's for nav, for outreach to teams... discoverability (how does anyone figure this out?) 20:36:31 johnthetubaguy: yeh, it's been on a hiatus because api-ref was related and more important 20:36:45 so maybe the tag is "has contributed their info to the service catalog working group"? 20:36:48 dhellmann : the "team" concept in releases/ repo ... 20:36:59 dhellmann: in all cases I'd say it would not end up being a tag. Could be some new key on the YAML grammar 20:37:07 sdague: yeah, I see what you mean, what we need is a list of OpenStack APIs and how to find them 20:37:08 since we need it to have a value 20:37:15 I'm definitely abandoning the service:type patch. Would like ideas for docs including APIs 20:37:27 ttx: it sounded like the value is going to be kept in that other repo, though? 20:37:49 annegentle: maybe we can come up with a solution by discussing that offline between you, sdague and me ? 20:38:05 dhellmann: hopefully not, ideally it'll be expanded and agreed upon after https://review.openstack.org/#/c/316396/ 20:38:08 yeh, lets have an offline chat just to figure out if there is another approach here 20:38:09 ttx: yeah that works. 20:38:10 * thingee cuts out for next flight back home 20:38:17 ttx: it's not urgent this week for sure :) 20:38:21 safe travels thingee 20:39:00 annegentle: maybe over emails so that we can think about our responses more :) 20:39:08 ttx: I like email. a lot. too much. 20:39:09 annegentle: so the problem is in creating the motivation for projects to come to you with links? 20:39:20 annegentle: could be a -dev thread 20:39:37 anteaya: problem space is varied :) discoverability, readability, navigation, and source-of-truth 20:39:41 ttx: yeah, ok 20:39:46 annegentle: okay thanks 20:39:49 ok, we have several topics to discuss in open discussion, so I'll move on 20:40:00 annegentle : not a small task for sure. thanks! 20:40:03 annegentle: starting with describing all the goals sounds like a good start 20:40:05 heh 20:40:16 #topic Open discussion 20:40:16 I'm a conflater. sigh. 20:40:18 ttx: ++ 20:40:24 morgan wanted to discuss data files defined in setup.cfg and sdague the API rate limiting 20:40:34 whoever speaks first wins 20:40:38 o/ 20:40:42 omg 20:40:45 lol 20:40:48 notmorgan wins by arguably cheating 20:40:52 actually if sdague wants to go, i'll defer 20:40:52 notmorgan: now you have to speak 20:40:53 * dhellmann pictures confetti raining down on notmorgan 20:40:54 lol 20:40:59 ok so my thing - http://lists.openstack.org/pipermail/openstack-operators/2016-June/010692.html 20:41:00 LOL 20:41:06 * notmorgan lets sdague talk 20:41:11 we used to have this toy rate limitter in nova 20:41:16 let's do sdague first to have one discussion at a time 20:41:18 mine can wait until next week + have a resolution to governance 20:41:23 if there is no time after 20:41:25 which is now gone 20:41:33 sdague: right. 20:41:39 because it was a toy, and really bad. It was disabled in havana by default 20:42:01 however, rate limiting is kind of a fundamentally important unit to API services with unknown users 20:42:33 and it seems like we should try to figure out how to get our community on to the same, or a set of pages to collaborate there 20:42:37 sdague: and i've see a number of bugs, some originally securiyt then made public, some just public around rate limiting. 20:42:41 so are you envisioning building our own version of something like https://pypi.python.org/pypi/turnstile ? 20:42:49 case in point, the vmt fields numerous reports of "denial of service" vulnerabilities which boil down to no way to limit request volume 20:42:50 sdague: the historical answer by the VMT was that this needs to be solved outside of openstack, so we avoided to consider rate-based attacks as vulnerabilities 20:42:50 this has been a real thorn for a number of reasons. 20:42:53 (which is old and apparently unmaintained) 20:42:54 sdague: just thinking out loud, but can't you just do it in apache or whatever you use as a webserver? 20:42:54 fungi: ++ 20:42:58 sdague : mod_ratelimit ? 20:43:03 mtreinish: we can. 20:43:12 and you're on your own if you decide to use eventlet as the server 20:43:24 mtreinish : yep 20:43:25 mtreinish: but regardless a clear message should be sent on how this is expected to work 20:43:25 mtreinish: typically rate limitting for API services isn't just connects 20:43:38 yeah, doesn't it have to be user- or project-aware? 20:43:40 and this should be consistent across openstack - for pure connections, mod_ratelimit is good 20:43:45 at least it wasn't in the toy implementation, or turnstile (which was Vek) 20:43:55 I've heard different opinions on this but I believe the general consensus among those opinions is that we shouldn't do it ourselves but instead let balancers/http servers do it. 20:44:00 but for other things we should have a clear direction: this is what is looks like in openstack 20:44:04 This might be an issue for some projects, of course 20:44:04 sdague, as I said in email, one thought I had a couple of months ago was to have the delimiter project cover this. Looking at the history, Jay didn't like that idea. 20:44:08 yeah, we've (vmt) traditionally punted rate limiter shortcomings to needing documentation/pointers to separate solutions 20:44:08 because not every METHOD /PATH are the same cost 20:44:26 because this is something that should be absolutely consistent in style across openstack (even if some things are more heavily limited than others) 20:44:54 sdague: so the real issue here is going to be IPC. and house keeping. imo 20:45:04 anyone can implement (toy or otherwise a rate limiter) 20:45:14 dhellmann: it would only need to be project aware in having regex paths + METHODS ... which is basically what turnstile does actually 20:45:24 though i think the richness of nginx/apache/etc to specify per-path matching is going to be the right way 20:45:24 ok 20:45:30 pet-path-per-method 20:45:31 it was more or less the memcache cluster version of the toy limitter in nova 20:45:37 less so about "user/project" awareness. 20:45:49 so... I guess here is the set of questions: 20:45:53 since apache, nginx, haproxy all know how to track the conections 20:45:54 sdague : so the question is how to get people to work on it? 20:46:04 1) is it important that we have a consistent thing here? 20:46:16 2) is it acceptable to be documentation 20:46:25 3) is anyone interested in working in this space? 20:46:33 1) I say absolutely to. 20:46:38 1) yes 2) maybe 3) no myself 20:46:43 2) probably. 20:46:47 1) yes 2) sure 3) not me 20:46:49 3) if i have time, I'll contribute 20:46:58 I think it would be good to be consistent about how the API users sees they are rate limited, and that might be documentation(?) 20:46:58 but i can't write / doc / code it all on my own. 20:47:22 I could also see it being helpful to build something that's aware of our API and user definition, though it seems like it wouldn't be too hard to make it generic. 20:47:39 dhellmann: yeh, honestly, I think this could absolutely be pretty generic 20:47:39 because if we build it, then we can include it in our interop testing 20:47:40 1) yes 2) yes 3) no 20:47:41 1. yes, 2. I think so, 3. someone is always willing to work on something 20:47:53 mtreinish: I would argue with your answer on 3 20:48:02 mtreinish : I know a lot of counter examples to #3 20:48:06 dhellmann: ++ 20:48:18 sdague: 1) I don't think it's as important because large cloud providers already solved it in ways that make sense to their ops teams and with the resources they have. 20:48:30 sdague : is this the first example of the TC saying "we want X to exist, someone please make that"? 20:48:37 annegentle: so every new cloud has to solve it for themselves? 20:48:39 annegentle makes an interesting point. 20:48:44 2) documentation can be generic as http://developer.openstack.org/api-guide/compute/limits.html is already 20:49:03 heh, fair enough. I didn't mean to imply we ask and they show up. I meant more people pick weird things to work on and find interesting 20:49:08 someone has done this before, but it's in java: http://www.openrepose.org/ 20:49:12 3) there are many unsolved problems in OpenStack I don't think we would prioritize this one over others? 20:49:28 annegentle : Amen 20:49:32 sdague: we have documentation being proposed in one of the projects that covers a lot of this 20:49:33 mtreinish : that I agree with. 20:49:33 annegentle: especially when solutions already exist 20:49:36 i think keystone maybe? 20:49:40 we can probably adapt it 20:49:43 more globally 20:49:43 so I don't think we should create a rate limiter, but I think we should help our users with rate limiting 20:49:48 sdague: they do now, and docs would be the biggest help since there are myriad considerations 20:49:53 I'm with johnthetubaguy 20:50:01 johnthetubaguy: yeh, that is probably the best path forward 20:50:04 johnthetubaguy: yeah, that's my sentiment as well 20:50:21 Produce doc on how to solve rate limiting consistently 20:50:24 And we should probably start by asking the OPs team how they do it 20:50:30 that might be more effort than building one, but I think our users will be happier afterwards! 20:50:35 flaper87: yep that thread is started 20:50:37 so... the seeds we have is keystone is writing up approaches based on apache, which should be applicable across the board, we start there? 20:50:42 flaper87: see the link I posted 20:50:44 It feels like something that could be driven from the ops side 20:50:55 * flaper87 scrolls back 20:50:58 sdague : sounds like a good plan 20:50:59 sdague: I like the apache base idea 20:51:01 http://lists.openstack.org/pipermail/openstack-operators/2016-June/010692.html 20:51:07 my bad, missed it 20:51:09 thanks 20:51:22 sdague: yes, that's the right start 20:51:26 i think some members of the security project team may also be useful resources for collaboration on protective rate limiting for openstack rest apis 20:51:34 notmorgan: could you get into that thread and speak up about docs being built on the keystone side? 20:51:34 sdague : ++ 20:51:35