20:00:18 #startmeeting trove 20:00:22 Meeting started Wed Jul 24 20:00:18 2013 UTC. The chair is hub_cap. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00:23 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:00:25 The meeting name has been set to 'trove' 20:00:39 \o/ 20:00:46 o/ 20:00:49 o/ 20:00:49 lets give a min 20:01:00 hi 20:01:06 ello 20:01:24 #link https://wiki.openstack.org/wiki/Meetings/TroveMeeting 20:01:25 hello 20:01:27 7o 20:01:33 #link http://eavesdrop.openstack.org/meetings/trove/2013/trove.2013-07-17-20.00.html 20:01:37 #topic action items 20:01:50 heh so AI's will be quick 20:01:56 o/ 20:01:57 vipul: any doc update on the wiki? 20:02:02 o/ 20:02:05 hub_cap: done! 20:02:07 ive moved every page ive looked at 20:02:09 yay! 20:02:09 see https://wiki.openstack.org/wiki/Trove 20:02:09 done? 20:02:12 i moved two today 20:02:18 oh done w/ that page 20:02:20 o/ 20:02:21 cool 20:02:28 are there more pages that got lost? 20:02:33 oh ya vipul tons 20:02:33 moved a bunch of them as well 20:02:35 the next one i think we covered eh? 20:02:35 search reddwarf 20:02:39 or red dwarf 20:03:03 anyhoo we can just update as we see fit 20:03:10 its for older stuff anyway 20:03:21 like i did configuration editing today cuz a mirantis guy asked me about it 20:03:34 so i thikn the next AI is already taken care of eh? 20:03:42 Stop test and Unsuccessful Restart test thing 20:03:49 we decided to move it out of bb testsing 20:03:56 \o 20:04:10 Yep 20:04:22 =o= four arm me 20:04:28 or my top gun pin 20:04:38 ok cool lets move on then 20:04:45 im going ot leave the api discussion to last 20:04:51 so ill skip to imsplitbits topic 20:04:53 \^o 20:04:59 my muscles 20:05:02 lol 20:05:03 lol cp16net 20:05:05 #topic Replication API update 20:05:17 so imsplitbit tell us about replication api 20:05:47 well we made the concensus 20:06:00 that we were going to use /clusters as a helper for cluster ops 20:06:06 instance ops stay in /instances 20:06:09 o/ 20:06:16 I've begun work on the /clustertypes 20:06:25 which is "flavors for clusters" 20:06:33 cool 20:06:35 thats about all I have at this point 20:06:36 thats a good way to put it 20:06:39 ok great 20:06:51 now for the topic at hand 20:06:58 the hp guys dont know much about this yet so 20:07:03 I HATE IT 20:07:04 * hub_cap makes vipul read the Windows ME user guide 20:07:04 lol 20:07:11 wanted to start early :) 20:07:11 * hub_cap pokes SlickNik 20:07:16 windows fun! 20:07:26 3.1 is better 20:07:26 ouch 20:07:26 * SlickNik wakes up 20:07:28 vipul: SlickNik take a minute to read 20:07:30 ME... 20:07:37 #topic capabilities 20:07:40 #link https://wiki.openstack.org/wiki/Trove/extensionsrefactor 20:07:53 i prefer to use the term capabilities to extension earthpiper 20:07:55 fwiw 20:08:08 sho nuff 20:08:11 lets give till 20:10 to finish reading it and talk about the pro/con 20:08:29 brb 20:08:30 then 20:08:39 hub_cap: 20:10 central standard time right? 20:08:45 heh yes kevinconway 20:08:47 not utc 20:08:52 ... 20:08:55 i only take epoch 20:08:59 so for the next five hours? 20:09:04 Wed Jul 24 20:08:27 UTC 2013 20:09:15 u have 1 1/2 minute to keep annoying me kevinconway 20:09:21 ;) 20:09:36 its 15:10 CST 20:09:40 why is time so confusing? 20:09:50 cuz in openstack u only speak in utc 20:09:52 hub_cap: Would it be bad form to add to that wiki article at this time? 20:10:05 grapex: ummm yes 20:10:08 well no 20:10:09 i want integer epoch timestamps, or you do it right! 20:10:11 but no one would read it 20:10:14 :) 20:10:37 but its ok grapex to do that because u wnt the information to persist 20:10:41 so go for it 20:10:48 u can always interject those points when relevant 20:11:09 link us your diff! 20:11:25 LOL just click history 20:11:42 ok vipul SlickNik have a feel for whats going on? 20:11:46 earthpiper: back? 20:11:50 yeah 20:11:51 kinda 20:11:56 kevinconway: happy that we are in utc time? 20:11:58 hub_cap: he is afk 20:12:04 k 20:12:09 so we want to dynamically load a service.py based on service_type? 20:12:16 so yes 20:12:18 i'm never in utc time 20:12:18 but its more than that 20:12:32 its dynamically load the api based on service_type 20:12:37 i think we want to consider the impact to the api over implementation at this moment 20:12:51 as in, /instances/id/users changes payload based on what type your instance is 20:12:51 if i understand correctly 20:12:59 hmm 20:13:03 correct KennethWilke 20:13:22 so not just guestagent, but the trove-api as well. 20:13:25 lets hold off on opinions till earthpiper gets back 20:13:28 correct SlickNik 20:13:47 yeah 20:13:48 the arguments are 1) the user shuldnt care that he has a redis type, he should just pass info to /users (yes bad example) 20:13:50 So .. isn't it kinda odd that the body would change based on a type of database? 20:13:56 you cannot have an opinion unless earthpiper is present??? 20:14:04 no imsplitbit u cannot 20:14:07 lol 20:14:07 he is driving this 20:14:14 vipul: i think so 20:14:14 go earthpiper (pips) 20:14:24 the input/output can differ based on the capability 20:14:26 btw, hello earthpiper! 20:14:36 Hi I am Conrad at Rackspace. 20:14:42 oh that makes sense now 20:14:44 howdy 20:14:55 If you don't know my net handle =) 20:15:18 hello, i am kenneth at rackspace 20:15:22 just... in case 20:15:30 greetings 20:15:33 ok go 20:15:49 he is the only Kenneth at rackspace 20:15:55 lol 20:15:58 what's the frequency 20:16:02 Hello, I am Inigo Montoya… You killed my father. 20:16:03 LOL 20:16:06 earthpiper: go 20:16:09 srsly GO 20:16:10 GO GO GO 20:16:18 weve burned 6 minutes on intros :) 20:16:22 exactly 20:16:23 So what do you guys think? 20:16:26 no 20:16:32 I think that sushi sounds great 20:16:33 hub_cap: trying to understand the motivation for this... 20:16:37 tell us what you think earthpiper 20:16:40 Oh cool. 20:16:42 Thanks 20:16:55 SlickNik: earthpiper will explain 20:17:03 what is the motivation earthpiper 20:17:13 So.. the problem with current extensions is they load everything in that directory. 20:17:21 We want Trove to support more Databases. 20:17:30 and right now the application is rather monolithic. 20:17:42 lets stick to the api earthpiper 20:17:43 not the impl 20:17:49 the why so to speak 20:18:24 i have a high level question 20:18:27 You get a routing conflict with current extensions if you have different requirements for instances functions. 20:18:33 correct 20:18:44 So like Users for postgres users for mysql etc. 20:18:44 so /users doesnt make sense if you host > 1 database type in the same installation 20:19:02 or on my redis boxen 20:19:04 how do u differenciate between postgres' users and mysql users 20:19:13 the one u add users too KennethWilke ;) 20:19:22 are they really different types of users? they are 'database' users 20:19:22 juice: this is irc, ask away 20:19:26 assuming we will allow a redis and mysql install on one instance yes? 20:19:34 yeah I am formulating it :) 20:19:37 one installation imsplitbit 20:19:39 not one instnace 20:19:41 regarding the bit about different POST body's in the API will the POST body's still follow the same structure even though they will be different? 20:19:43 they may have different parameters based on database implementation 20:19:48 or requirements 20:20:00 ok one aggregated infrastructure to manage all dbs 20:20:03 yes for instance postgres requires a default db, from what i think jeffe said 20:20:03 what level of abstraction do we want for trove or what level of abstraction do we think is most suitable for trove 20:20:03 Well we should not make a person only host one type of db system per cluster. 20:20:13 and this is more than just /users its /anything 20:20:25 /db_specific_feature 20:20:28 correct 20:20:33 it could be /flushkeys for redis 20:20:34 is that we plan on configuring trove at deploy time for a specifc implementation and have it build those instance types 20:20:34 ;) 20:20:49 juice: it shoudl be able to handle > 1 service_type 20:20:51 in the same api 20:20:53 or is trove supposed to standup and handle any db implementation type 20:21:02 you configure it juice 20:21:02 so there are two types of things... 20:21:06 to handle what u want to support 20:21:18 avail_types=postgres, mysql, percona, redis 20:21:20 so to speak 20:21:21 additional operations on a db_type.. .which is differnt from the same operation, but different request for db_types 20:21:37 and from the end user do they know which service type they are getting 20:21:42 yes so same uri, different body 20:21:43 I think we can support db_type specific actions 20:21:47 sound familar? 20:21:47 we don't allow them currently to specify the service type on boot 20:21:54 /actions {} 20:22:02 the thing is, if you are managing multiple databases with the same infrastructure, what are you gaining if each one has a completely different command-implementation? 20:22:13 vipul: I don't know- so far it seems like the users call has had to change a lot. We've added things to it and then found out it was ambigious because of how MySQL worked. Maybe I've mistinterpretted what you're saying, but I can't imagine users would "just work" for all db types. 20:22:17 juice: this proposal seeks to change that, I think. 20:22:17 technically juice we do 20:22:18 i mean, why not just clone the 5 or 6 nodes 20:22:33 pass service_type in to the create, and watch it go 20:22:40 right, 20:22:54 if it doesn't work for a set of common types, why force a common infrastructure? 20:22:56 what benefit does having a bunch of duplicate api nodes w/ a single config diffence buy you jcooley? 20:23:03 its gonna be the same codebase 20:23:14 i dunno, i have different api nodes for nova, swift, savanna, ... 20:23:16 so the difference between 6 nodes is gonna be service_type=blah 20:23:21 grapex: Right, so I agree there may be differences, and I think instead of a body that's compeletely different for different databases, we come up with either a genericized API that works across, or we make things optional 20:23:22 ok so the user specified the service type and they are now wanting to interact with a redis or postgres instance 20:23:38 jcooley: but you should be able to spin up ubuntu and centos or whatever on one nova deployment 20:23:42 This is just to make the deployment of new db types easier. 20:23:53 right, with the same api 20:24:00 on the same boxes if you would like 20:24:05 or not like 20:24:10 well, that's where i'm getting lost 20:24:10 you can do what you want with it 20:24:26 vipul: I think having a generic user call won't be satisfying for power users, unless we offer it in addition to a type specific call 20:24:29 so even if not every db has a concept of users, ideally they'll all have a concept of connections. is that a common ground we can build off of? 20:24:40 not really 20:24:41 take redis 20:24:43 So there could be a user create call specific for MySQL, but then we could make a generic one for any DB. 20:24:43 no users at all 20:24:44 sure, flexibility is nice but i'm not seeing the code sharing. 20:24:44 nil 20:24:48 only infrastructure sharing. 20:24:48 users? 20:24:51 It returns 404 20:24:55 rather than a 200 20:25:04 because that route is not defined for redis 20:25:05 so do we, since redis can support resizes, resets, all the things trove /instances support 20:25:14 do we have to stand up a diff service to host it? 20:25:32 and do we need a different docbook altogether? 20:25:37 just so we can have separate services? 20:25:39 hub_cap: No, i think it's fine to return not-implemented if things don't match 20:25:48 but that not impelemtned should come from the Guest Agent 20:25:53 i dont agree 20:25:54 which determines whether that operation can be done or not 20:25:59 id prefer to return "there is no route for this" 20:26:01 i think not implemented should be returned by the api 20:26:17 more informative than a 404 20:26:22 the same thing when you pass /instances/id/honeybooboo 20:26:22 exactly 20:26:25 Yeah, we don't want the guest agent to have to be that intelligent about what the API can do. 20:26:28 yeah not implemented doesnt sound like something that should be returned 20:26:37 i like vipul's idea of placing the differences in the guest agent... 20:26:38 why is it that they can invoke the same route at the same endpoint in oen case but not the other 20:26:43 jumping to solutions here but what comes to mind is that each database would have it's own discrete api command structure and TROVE simply becomes a router to the different api implementations 20:27:02 juice: that is one of the approaches 20:27:02 grapex: but the guest agent needs to be intelligent about what the API can do since the guestagent will be the one doing it! 20:27:03 hub_cap: 501 Not Implemented 20:27:15 you might even extend that and have different instances of task manager out there for the various implementations 20:27:17 juice: That's my favorite idea, with the addition that you'd have a generic instance resource that could do actions that were performable by any db type 20:27:23 naw taskmgr shoudl be generi 20:27:24 c 20:27:44 this approach allows you to keep a generic front end yet have documentable and implementation specific apis 20:27:48 the big differences are the api+guest impl 20:27:54 SlickNik: Of course, but the guest agent shouldn't have to understand someone made a request for a Redis operation and the server is MySQL 20:28:00 juice: that means Trove has many APIs 20:28:03 stop typing 20:28:03 The API should know that and not even ask the guest agent 20:28:12 do we agree that trove can host > 1 capability 20:28:17 yes 20:28:19 yes 20:28:20 yes 20:28:21 yes 20:28:24 #vote yes 20:28:25 good 20:28:26 yes 20:28:28 yes 20:28:31 #voted 20:28:34 i can start a vote kevinconway ;) 20:28:35 o/ 20:28:41 ok good 20:28:42 #agreed 20:28:43 lets move on 20:28:47 #done 20:28:49 now its a matter of the proposal 20:28:57 do we 1) dynamically change the route 20:29:02 or 2) define namespaces for the routes 20:29:23 i see 3 things here... 20:29:23 2 20:29:26 1 is potentially more confusing for the user (imho) and 2 is potentially more annoying for the user (since there is 1 type) 20:29:31 grapex: disagree. 20:29:33 whats 3? 20:29:35 1. choosing which extensions should be loaded instead of all 20:29:41 2. dynamic route / dynamic body 20:30:04 3. fixed route w/namespaced route 20:30:04 api should not have all the intelligence or we have to bake the polymorphism into it 20:30:09 hold up vipul 20:30:16 1 is a installation procedure 20:30:23 and 2/3 are the api differnces 20:30:27 I think we agreed 1 wasn't an option if there's >1 capability 20:30:37 1 will happen 20:30:37 Correct grapex 20:30:44 hub_cap: is 2) above basically defining new routes for each db type? 20:30:47 ok.. makes sense 20:30:54 esp: yes 20:30:56 u will choose the types you want to run based on your needs 20:31:09 avail_types=cassandra,oracle,couchdb 20:31:15 KennethWilke: thx. yeah I like that better. 20:31:17 we should revise that original vote to say "trove should dynamically support > 1 impl at runtime" 20:31:25 esp: yw :) 20:31:27 juice: Good idea 20:31:33 huh!?!? 20:31:34 no 20:31:35 i'm ok with dyanmically loading a route in the backend, but i am not ok with those supporting differnt bodies 20:31:38 2: means to me that you dynamically look up the instance type and do the work 20:31:42 it was jcooley that brought up to not support > 1 20:31:46 i can see that we may have different validation rules for a db_type 20:31:47 3 means that its defined in the uri 20:31:54 we are getting out of hand 20:31:56 but the request body shoudl be the same/similar 20:31:56 let me rephrase 20:31:59 cp16net: in any other case do you not need to know the instance type if trove supports multiple dbs? 20:32:04 * hub_cap bangs the gavel 20:32:05 hub_cap: only meant that if they're really different and you can't share implementation, you're not gaining anything. 20:32:21 we will support > 1 impl in the same codebase/install/api server 20:32:22 period 20:32:33 if you want cassandra and redis, you can get them 20:32:33 thats not being argued 20:32:34 vipul: I decided I couldn't stand the totally dynamic, every type uses the same request path idea when I realized different db_types would eventually have different bodies for the same request paths. 20:32:45 if u have to restart the api server to install a new type, thats fine 20:32:50 * hub_cap bangs the gavel and points at grapex 20:32:57 grapex: i agree 20:33:12 lol netsplit 20:33:16 hub_cap: Am I in contempt? 20:33:19 yes 20:33:20 * KennethWilke shame 20:33:21 :P 20:33:26 you have the floor hub_cap 20:33:33 so the question is this 20:33:47 NO 20:33:49 do we have an api that supports dynamic bodies, because we cannot guarantee that every call to /users will be the same 20:34:00 I just wanted to be in contempt too :) 20:34:03 or do we support namespaced urls, where the payload is the same 20:34:19 hub_cap: May I approach the bench? 20:34:21 but you have to type the type in 20:34:24 grapex: aye 20:34:32 I got dibs next. 20:34:39 yes you do earthpiper 20:34:42 then me 20:34:46 then i! 20:34:48 no imsplitbit 20:34:50 never 20:34:52 Another thing to consider is in addition to namespaced requests for specific types, we could also make requests to the existing instances path w/o the db_type to do things which *ANY* instance could do 20:34:54 damnit 20:35:06 Think of it like calling a super class's interface 20:35:18 it will, resizes will still go thru /instance/id 20:35:23 /instance/id/resize 20:35:25 NO ACTIONS 20:35:36 but we all *love* actions 20:35:46 Those are our favorite!! 20:35:48 disagree 20:35:57 * juice notes sarcasm 20:35:59 /instance/id/resize, /instances/id/user vs /instances/id/mysql/user, /instances/id/user vs /instances/id/redis/user 20:35:59 But they're so hard to use... :( 20:36:07 j/k 20:36:11 Put me back in contempt 20:36:19 So can I interject really quick. 20:36:22 * hub_cap urges grapex to use Windows Millennium Edition 20:36:27 I think it would be less redundancy (in your DTO or value objects) if request payloads and response payloads are the same across db_types. 20:36:28 earthpiper: plz do 20:36:31 Ok 20:36:37 take a gander at client flows 20:36:41 at the bottom of the propsal 20:36:44 https://wiki.openstack.org/wiki/Trove/extensionsrefactor 20:36:44 esp: we cant guarantee that 20:36:48 hub_cap: While I love the idea, I'm worried that this opens up the opportunity for the APIs of the different implementations to diverge to a point where they look like completely different APIs. This means that as a consumer, I have to relearn / re-work the API every time I need a different type of instance. This seems like a serious disadvantage for a managed db solution, which should make my life easier by providing somewhat of a consiste 20:37:04 hub_cap: nope but it's a good place to start. 20:37:05 esp: You could of course reuse classes that were similar as base objects between the request bodies of various types. 20:37:18 SlickNik: +1 20:37:21 SlickNik: +1 20:37:28 im not sure what SlickNik is saying 20:37:32 voting for #1 or #2? 20:37:36 1 i believe 20:37:37 neither 20:37:42 or voting against to ever have /usrs 20:37:45 so i have a half-baked idea wrt to marrying namespaces and dynamic 20:37:47 SlickNik: Do you mean a user switching between MySQL and Percona or something similar like that? 20:37:58 not taking a side of the #1 or #2 fence yet. 20:38:08 may i? 20:38:16 i dont think earthpiper got to interject 20:38:20 he just said go read some shit 20:38:22 earthpiper: gogogog 20:38:23 So the problem is 20:38:27 If we support other databases 20:38:29 stuff changes 20:38:31 grapex: agreed but I guess I'd like to see how different each imp will be to say for sure 20:38:31 peroid 20:38:36 If we have a single endpoint, and a single API... why would someone be required to figure out what the body or URI shoudl be based on a type 20:38:59