18:00:03 #startmeeting trove-bp-review 18:00:03 Meeting started Mon May 19 18:00:03 2014 UTC and is due to finish in 60 minutes. The chair is SlickNik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:04 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:00:06 The meeting name has been set to 'trove_bp_review' 18:00:55 o/ 18:01:02 Giving folks a few minutes to trickle in. 18:01:10 ........ o/ 18:01:19 \o/ 18:01:21 o/ 18:01:23 Agenda at: 18:01:23 o/ 18:01:26 #link https://wiki.openstack.org/wiki/Meetings/TroveBPMeeting 18:01:29 o/ 18:01:33 o/ 18:02:02 o/ 18:02:16 #topic Trove client for cross-region-backup 18:02:51 this bp is marked as approved 18:02:59 esmute: around? 18:03:27 amrith: Yes, this was approved earlier. 18:03:47 I was talking to esmute about why he added it to the bp queue. 18:04:12 SlickNik: i had asked him to add it the queue prior to our clustering agreement at the summit; it was mostly pertaining to how region-awareness would work 18:04:43 to make sure we are using the catalog to look up the endpoints? 18:04:44 amcrn: Yup, that's what he mentioned. 18:04:47 the concern was around whether "region" as a field would be the strategy we'd be moving forward with (vs. say namespacing) 18:05:23 but we're all aligned on that now, so it should be good to go 18:05:26 o/ 18:05:34 https://wiki.openstack.org/wiki/Trove/Replication-And-Clustering-With-Nodes-5 18:05:39 seems consistent with what esmute is doing 18:05:59 okay, amcrn. I don't see him around atm. I'll fill him in when he gets back. 18:06:00 vipul: yep, so i think we can move onto the next agenda item 18:06:05 Let's move on. 18:06:15 #topic Add visibility filter to datastores 18:06:32 seems very logical, as for me 18:06:37 #link https://blueprints.launchpad.net/trove/+spec/datastore-visibility 18:06:38 Currently there is no way to hide datastores while keeping them active 18:07:15 one question 18:07:25 yes 18:07:37 how do you plan to hide one datastore from user A and show it to user B 18:07:49 denis_makogon: it's binary: admin can see it, but user can't 18:07:55 what amcrn saif 18:07:59 *said 18:08:14 do we plan to at some point do what glance does with images and make it visible to tenants? 18:08:24 vipul, ++ 18:08:26 does the admin view show the extra details if the datastore is visible or not? 18:08:39 yes the admin view shows everything 18:08:43 and admin can change visibility 18:08:43 thats actually i as asking for 18:09:02 so this model can be later extednded to have members 18:09:04 is the intent to make the datastore only usable by the admin and not by other users? 18:09:14 im here... reading 18:09:46 here the datastore has no concept of ownership 18:09:48 no its just to make the datastore "hidden" but usable if you know the uuid of the version 18:10:11 to be used in a sort of "beta" stage 18:10:11 the get call will still shwo the store 18:10:16 i guess I'm trying to understand why this is being implemented. 18:10:25 what problem are we solving, for whom? 18:10:28 that way if u want only certain users to be able to access the datastore you can provide them with the uuid 18:10:35 To have a "secret menu" of datastore types. 18:10:39 amrith: one example why i'd really like this feature: i can deploy a new datastore (or version), and use my admin role to do a production smoke-test 18:10:45 yea, doesn't it seem better to introduce datastore visibility per tenant instead of simply hiding it? 18:10:46 Like ordering the skinless fried chicken at PopEyes. 18:10:53 amrith: for exampkle you have a new datastore u want to test in production, but dont really want all the users to be able to access it yet 18:11:00 vipul, ++ 18:11:21 i'd vote for tenant as well 18:11:23 so is there a reason we are implementing this particular solution instead of a proper acl mechanism for tenants? 18:11:32 is expediency the reason? 18:11:40 yes 18:11:41 with the longer term intent being tenant based acls? 18:11:49 it seems that we can extend admin role to use passive datastores for provisioning 18:11:57 amrith: I guess we could do it either way. In our experience if there's a feature that's merely hidden but not acl'd, no one will get made at you if they use it and it isn't perfect. 18:11:59 no need to add another flag 18:12:02 ok. how much harder would the cadilac be? 18:12:21 grapex: the problem with this approach is if that ID got 'leaked' then anyone could potentially use it 18:12:25 It depends on the use case. 18:12:40 we could prevent usage with tenant based acl approach 18:12:49 vipul: +1 18:12:53 ae70b2b0-1af3-4ad6-8a0d-112e2cb15a37 18:12:57 SlickNik: Right. For Rax as you can all tell we'd be ok, but sounds like the other parties here would run into problems 18:12:58 oops sorry guys 18:13:00 kevinconway: DEAR GOD NO! 18:13:02 dont use that uuid 18:13:06 lol 18:13:09 hahahaha 18:13:10 why can't we just extend current implementation, that based upon passive/active datastores for admin tenant ? 18:13:51 so we dont want to add tenant id for every datastore 18:13:58 it becomes harder to add new customers 18:14:19 in images, we had image members because the owner was resposible for the sharing 18:14:39 yeah the datastores aren't user content though. they don't "own" datastores 18:14:52 iccha1, it seems that same approach is applicable here 18:15:06 aren't we also missing a "role" concept here that would actually make ACL sane? 18:15:07 we could take the galnce approach by implementing is_public, by default it is public. Then we also add membership that manages non-public datastore ACL 18:15:10 kevinconway iccha1: good point 18:15:14 denis_makogon: noone owns the datastore, then the admin has to add every sinfle tenant 18:15:15 we can do what nova flavor does, if a particular datastore is public it will be visible to everybody but if its private then you need to add tenant access explicitly 18:15:47 yeah that makes sense 18:15:50 saurabhs: Sounds like a good compromise 18:15:54 saurabhs, ++ 18:16:02 some sort of the black list 18:16:11 white list :) 18:16:15 iccha1: +1 18:16:17 and until you want to run that datastore as private keep it that way and once you are ready to go make it public and then nothing needs to be done more for every tenant 18:16:30 +=1 18:16:34 +1 18:16:36 +1 18:16:44 cp16net: Are you now called cp17net? 18:16:51 lol 18:16:51 lol 18:16:54 I like that approach as well. 18:17:04 Or would that be cp16neu? 18:17:23 iccha1: Can you update the bp with details using this approach? 18:17:31 will do SlickNik 18:17:41 ok, if it gonna be easier to maintain at production, then i'm up for the suggestion 18:18:23 iccha1: Once that's done, ping me / another core and we can go over it and approve. 18:18:23 so what was the decision? to use the approach of is_public and private? 18:18:36 like glance? 18:18:50 sure SlickNik . the concensus is public is vosible to all. for private u need to explicitly add tenant 18:19:07 thanks for the clarification. 18:19:10 amrith: Yes, if it's private, you need to specify a tenant white list for access. If it's public, it's visible to all. 18:19:17 +1 18:19:19 thx 18:19:40 Okay, let's move on. 18:19:50 #topic Database log files manipulations 18:20:03 #link https://blueprints.launchpad.net/trove/+spec/dbinstance-log 18:20:12 stage one was proposed, wiki was cleaned-up 18:21:32 denis_makogon: I see that. 18:21:42 denis_makogon: should "allowed" be replaced with "enabled" 18:21:44 Stage 1 will cover pushing specific log file from a guest to Swift 18:22:10 denis_makogon: I still have a couple of questions around some rough edges in the spec. 18:22:21 SlickNik, which? 18:22:25 denis_makogon: "datastore_log_files": "general_log, log_slow_queries, log-error" 18:22:45 Why are the first two underscore, and the last one hyphen? 18:22:50 denis_makogon: never mind that is configuration is for something else 18:23:26 SlickNik, it is a misspell (re-mark) 18:23:47 denis_makogon: will the logs in swift be automatic deleted when instance gets deleted? 18:23:50 Also can we have some better association with the log names, (use the default log filenames, perhaps?) rather than using arbitrary strings? 18:24:05 SlickNik, everything should be underscore 18:24:16 denis_makogon, I would like to know how this feature you are implementing works with logrotate? 18:24:32 amrith, log rotate is out of the stage 1 18:24:56 ok, is there a bp that lists the various stages? 18:25:02 amrith, yes 18:25:09 denis_makogon: so at the time when POST is received.. we upload just the current log file ? 18:25:17 amrith, current BP describes Stage 1 18:25:24 vipul, yes 18:25:33 the last time this blueprint was brought up, there was quite a bit of concern surrounding the fact that the user will have no idea what timerange the current log contains 18:25:36 SlickNik, what do you mean? 18:26:39 i echo the comment by amcrn (though I wasn't at the last review) 18:26:55 when you turn on general query logging, you could get a very short interval on a busy system 18:27:00 and that's not of much use 18:27:12 similar issues will exist with high freq logging on other data stores 18:27:12 amcrn, its pretty hard to find out to which time range does the log responsive 18:27:18 i don't know what we could do about it though 18:27:47 I understand multiple iterations but do others feel that this limited v1 is useful enough to implement by itself 18:27:50 do you want to inspect the first line.. and the last line and add it to the filename scheme or something? 18:28:06 it sure would be nice to have some way to view the logs and this seems like a good start 18:28:14 robertmyers: +1 18:28:17 vipul, not all files list date in the message, sometimes continuation lines won't have timestamps 18:28:27 our service is a black-box.. and this gives some visibility to users 18:28:32 robertmyers, +1 18:28:50 robertmyers, thx 18:29:03 vipul: true, but it just seems odd that as a user you'll execute a request to get the log(s), hoping that they contain a certain time interval. not only that, the user will have no idea when it's been rotated. 18:29:06 amrith: good point.. another reason why we can't determin time range 18:29:22 amrith, unfortunately with huge logs it's pretty hard to discover the actual time stamp 18:29:25 in my opinion, there needs to be some sort of indication that the file has rotated 18:29:42 denis_makogon, vipul ... not true 18:29:49 should the backup include all rolls of the same log file? 18:29:52 logrotate can give you that information if you ask it nicely ;) 18:29:53 yeah otherwise you will have a file with 0 lines in the worst case 18:30:03 I think in a phase 2 or something.. we could look at something like incremental log file uploads.. you provide the timestamp.. and we get you everything from that to now 18:30:22 so out of curiosity 18:30:28 why aren't we doing like RDS? 18:30:34 and exposing data in a table? 18:30:35 droppin' A bombs 18:30:41 vipul, agredd 18:30:44 *agreed 18:30:59 and for nosql stores exposing as a collection 18:31:15 amrith: can all datastores do that? write lines to a table instead? 18:31:17 is the idea to display the log in a horizon interface, or just a simple file download from swift? 18:31:19 http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.MySQL.html 18:31:40 vipul, the link abov eis for mysql 18:31:46 similar for other RDBMS' 18:31:59 esmute, simple download, since we're not going to use them inside Trove 18:32:01 for nosql stores the data is exposed using a collection of some sort 18:32:10 me looking for a link 18:32:17 * amrith looking for a link 18:32:35 amrith: that doesn't sound like they are just writing to a table.. 18:32:39 denis_makogon: how would you access them if the instance is deleted? 18:32:53 amrith: looks like you can do both ways 18:33:05 download file and view in a table 18:33:08 esmute, you can go to swift panel and download them 18:33:09 well, NoSQL stores have no tables. didn't you hear about the Oracle guy who went into the NoSQL bar in Atlanta and walked out because there were no tables? 18:33:14 how about we don't push anything to swift we keep it on the machine locally and let user read that with API and let user deal with taking the backup of the log if needed 18:33:24 so what was wrong with just scooping up everything in the log dir included rolls? 18:33:37 we provide the API which allows user to read number of lines specified from the file 18:33:43 saurabhs: that's a lot of crud to pump through our api 18:33:50 kevinconway: I think that's exactly what RDS does. 18:33:52 speaking of, since these are truly configuration parameters, why are parameters like long_query_time being handled outside of a configuration-group? 18:33:55 the op maybe too huge 18:33:57 vipul, ++ 18:34:09 vipul: you can control how many lines you want to allow to be read at one time 18:34:10 kevinconway: example http://docs.aws.amazon.com/AmazonRDS/latest/APIReference//API_DescribeDBLogFiles.html 18:34:50 saurabhs: sure.. but the first person to say give me 1 million.. will probably be worse than if they were to download the same object using swift 18:35:20 and having a marker to tell where in the file you want to grab would be nice 18:35:28 vipul, agreed, log pulling will be possible only through Swift storage 18:35:28 but not nessesary for first pass 18:36:06 So here's my 2c on this. 18:36:07 denis_makogon, I think I got the answers to my questions. and as robertmyers and vipul say this may be a good start. I do however agree with amcrn that this may be a little hard to use and should use configuration groups. 18:36:16 I can definitely see this as valuable. 18:36:16 i think we should avoid anything that requires us to write a log parsing agent and stream results through rabbit 18:36:43 kevinconway, totally agree 18:36:45 kevinconway: +1 18:36:49 so does the first pass cover only the active log file or all log rolls in the directory as well? 18:37:13 However, the current API details in the spec seem rather simplistic and don't address the logrotate issue. 18:37:18 kevinconway: we might as well zip everything up and upload all 18:37:30 So do we want v1 to address the logrotated artifacts, or not? 18:37:51 SlickNik, no, log rotation is for Stage 2 18:37:51 SlickNik: why not just have a file_created timestamp, that will at least hint towards a rotate, and it's good enough for a v1 18:37:51 SlickNik: if we do the whole folder wont we get that? 18:37:54 do all instances have log rotate set up? 18:38:02 is this available to trove users? 18:38:29 iccha1, for now - no 18:38:37 some datastores do it automatically.. otherwise its up to the deployer 18:38:54 amcrn: +1 on a file created / last modified timestamp. 18:39:07 iccha1, https://wiki.openstack.org/wiki/Trove/DBInstanceLogOperationV2 18:39:19 if we add the timestamps, then my only remaining concern is: why are we not using configuration-groups for configuration parameters 18:39:27 SlickNik, seems valid to have those timestamps 18:39:45 amcrn: i don't get it 18:40:11 amcrn: which config params? 18:40:11 vipul, i guess, amcrn meant: static configuration VS. configuration-groups 18:40:12 amcrn: oh i think i just got it.. to enable disable logs 18:40:15 i'm slow 18:40:30 vipul: thanks, that makes two of us. 18:40:46 ex: http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysvar_long_query_time 18:40:59 this could technically be handled using configuration-groups 18:41:06 which is sadly one second minimum… 18:41:09 SlickNik, ... is this review for v1 or v2? I just read the v2 spec and there are problems with the way logrotate is being proposed to be used. 18:41:34 amrith: This is for v1. 18:41:37 amrith, this is V1 18:41:40 yeah lets just stick to the first draft 18:41:43 thx, I'll hold off 18:42:21 i think we are in agreement now? 18:42:25 Quick tangent: This is kind of a minute issue with the API get call for the list of logging files: seems like right now the datastore_log_files argument is a single string containing multiple things with commas, maybe that should be an array? 18:42:55 "datastore_log_files": "general_log, log_slow_queries, log-error" s/b "datastore_log_files": ["general_log", "log_slow_queries", "log-error"] 18:43:25 grapex, it can be, it's still up to us to decide actual types 18:43:26 grapex: +1 probably just an oversight 18:43:40 it should be an array denis_makogon 18:43:46 tru 18:43:47 vipul, get it 18:43:55 bah, just add .split(',') on the guest. what could go wrong? 18:44:20 kevinconway: forgot the .trim() 18:44:22 DOH... 18:44:25 for the first iteration i would suggest to use static configuration against conf. groups 18:44:26 doh! 18:44:33 So here's the summary I have. This BP needs to: 18:44:33 1. be updated withcreated/modified timestamps 18:44:33 2. use existing config groups 18:44:33 3. include slight API modifications (underscore consistency, array of params) 18:45:00 denis_makogon: Can you update the bp with this feedback? 18:45:26 SlickNik, would you elaborate on #1 please 18:45:29 for #1 is that an API param? 18:45:46 vipul, no, the part of the response 18:45:57 vipul / amrith: The actual created / modified time of the log as part of the response. 18:46:06 not a param to the request. 18:46:09 SlickNik, that may be very hard to do. 18:46:17 it's already in the filename 18:46:31 "locationURL" : "http://somewhere.com:PORT/dblogcontainer/{instance_id}/filename.timestamp" 18:46:32 what about configuration files rendering ? 18:46:44 you mean the posix create/touch times? 18:46:52 kevinconway, yes 18:46:57 kevinconway: when i brought it up, that's what i meant, yes 18:46:57 amrith / vipul: something like the LastWritten here http://docs.aws.amazon.com/AmazonRDS/latest/APIReference//API_DescribeDBLogFiles.html 18:46:59 Oh, not the timestamps on messages or the file on the trove system 18:47:00 ok 18:47:02 kevinconway: yes 18:47:30 so, last words on that 18:47:31 well, lastwritten will be hard but if he can do it, that would be nice 18:47:33 #2 18:47:47 do we need all those timestamps if we just grab the entire log dir? 18:48:00 amrith, SlickNik that's a request param.. 18:48:06 you're saying that we should look over assigned conf group and look for the log entries in that ? 18:48:14 oh never mind.. it's in the example 18:48:32 yes, I meant the example. 18:48:41 and then response to a user that there are some (or none) of the logs available for user ? 18:49:37 why we can't just exclude log attributes from configuration groups ? 18:49:38 knock-knock.. this thing still on? 18:49:40 denis_makogon: Sorry, I don't understand the question. 18:49:55 SlickNik, could you please elaborate #2 ? 18:50:50 denis_makogon: Let's chat about #2 immediately after the meeting. 18:50:55 ok 18:51:01 let's move on 18:51:04 denis_makogon: I'd like to get another bp looked at here if possible. 18:51:06 Thanks 18:51:19 #topic Pluggable conductor manager 18:51:28 boden, around? 18:51:57 This one seemed so simple too... 18:52:00 I'll email him and let him know about this meeting 18:52:17 grapex: lol 18:52:32 I thought it already was pluggable? 18:52:38 SlickNik: Seriously we should just +1. :p 18:53:00 you run a separate daemon 18:53:06 robertmyers, nope, manager is hardcoded inside the trove-conductor script 18:53:07 robertmyers: I don't think that it is. 18:53:14 it can be anything 18:53:20 yes it seems liek what's in the BP should be min-bar for all future 'managers' 18:53:22 robermyers: i kind of agree 18:53:39 is there a way to _not_ run conductor to make room for a custom daemon? 18:53:41 But I also see vipul's point. Its inconsistent to offer a Trove daemon without a pluggable manager 18:53:50 all the other managers in Trove and Nova seem to be pluggable 18:53:54 though it is kind of overkill 18:53:55 kevinconway: yes, just write your own 18:54:05 and run it in place of the trove onw 18:54:07 no, i mean is there a way to disable it 18:54:09 yep 18:55:00 kevinconway: make a noop conductor 18:55:05 and run it 18:55:07 boden needs to show up for these and follow better blueprint conventions 18:55:07 done 18:55:09 i think we should provide the plumbing in Trove to allow the actual thing to be swapped 18:55:12 I don't think there's a wiki for this 18:55:22 yes you can do it without even needing Trove.. but that's beyond the point 18:55:22 it says i dont have permission to view it 18:55:24 robertmyers: the BP is because you can't do that 18:55:25 * amrith takes action to send email to boden 18:55:25 though it is incredibly simple and was easy to read. 18:55:34 where is the link? 18:55:45 robertmyers: Yes, but then you have to write a whole new thing. As opposed to being able to write an impl that overrides only some of the functionality. 18:55:56 vipul SlickNik: +1 18:56:01 +1 18:56:04 #link https://blueprints.launchpad.net/trove/+spec/pluggable-conductor-manager 18:56:05 SlickNik: -1 18:56:06 :) 18:56:11 lolz 18:56:13 I'm not saying we should do it. 18:56:47 i think we should just approve this one.. others disagree? 18:56:52 This is an interesting point of view as well http://lists.openstack.org/pipermail/openstack-dev/2014-April/033898.html 18:57:23 vipul: I'd like to only do it once it has a sponsor (i.e. someone to commit on doing the work). 18:57:44 SlickNik: fair enough 18:57:45 So I'm inclined to table it until when boden is actually around. 18:57:57 it's a pretty simple task-to-do 18:58:00 SlickNik, ... email sent ... 18:58:01 like 10 mins 18:58:19 denis_makogon: although simple, it set's a precedent that we might not want to 18:58:23 denis_makogon: nothing takes just 10 minutes :P 18:58:32 Okay, I think that's all for today. 18:58:37 Thanks all! 18:58:42 #endmeeting