18:00:04 #startmeeting trove-bp-review 18:00:05 Meeting started Mon Jul 21 18:00:04 2014 UTC and is due to finish in 60 minutes. The chair is SlickNik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:06 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:00:08 The meeting name has been set to 'trove_bp_review' 18:00:35 o/ 18:00:50 giving folks a few minutes to trickle in 18:01:04 Agenda at: 18:01:06 #link https://wiki.openstack.org/wiki/Meetings/TroveBPMeeting 18:02:05 o/ 18:02:10 o/ 18:02:19 o/ 18:02:20 o/ 18:02:56 o/ 18:03:00 denis_makogon: You have 3 BPs scheduled. In the interest of fairness - and time, we'll discuss the first one, move on to boblebauce's bp, and then come back to the other two. 18:03:13 SlickNik, of course 18:03:55 o/ 18:04:08 o/ 18:04:22 #topic Datastore log files operations 18:04:30 #link https://blueprints.launchpad.net/trove/+spec/dbinstance-log 18:04:30 so, it's mine 18:04:42 we've did reviewing of it, twice already 18:04:59 the last concerns were related to HTTP routes, payloads 18:05:14 i've did alot with vipul to flesh it out 18:05:29 o/ 18:05:45 Yes. Can you point out where you have addressed SlickNik's concerns as listed on the whiteboard? I couldn't find where they were addressed. 18:05:46 so, and this review request was raised to kill this beast finally, since it should be in Juno release 18:06:28 sorry, I don't get it. is the review to accept the proposal or kill the proposal? 18:06:41 amrith, review request =) 18:07:02 I didn't know we needed formal meetings to stop the progress on features in Trove. 18:07:05 vgnbkr +1 18:07:11 vgnbkr, cases are described in Justification section 18:07:16 j/k 18:08:02 So I wasn't clear how audit fit into all this. Are you planning to enable auditing somehow? 18:08:05 This review is to get the proposal accepted. 18:08:18 vgnbkr, all concerns are not fitting into first proposal, but itss a good start to accomplish goals 18:09:00 I thinkt he audit stuff just needs to be taken out.. to avoid confusion on what is being propsed 18:09:08 vgnbkr, audit will be accomplished by 3d party tools, our task to pull logs out of instance, since we cannot access instance through ssh or whatever 18:09:35 o/ 18:09:41 denis_makogon: I don't think this is a very good proposal for Audit requirements — I can see this as a good convenience to developers (i.e. I want to find out the contents of my slow-query logs, for example). 18:09:50 this proposal is pretty simple.. provide a mechanism to ship datastore logs out of Trove instances 18:10:04 vipul, yes, that's what i was trying to say 18:10:21 i think the extra wording confuses that message denis_makogon 18:10:49 vipul, i see 18:11:35 denis_makogon: Yes, in that case, please remove the requirements section about "audit", since it doesn't look like this is aiming to tackle that. 18:11:40 so, as vipul said, initial proposal is simple - provide a mechanism to ship logs from instance to storage 18:12:16 done 18:12:42 one minor nit, my browser has a plugin that detects plagarism (vestiges of being a teacher) and it flagged this page. Turns out there's some chunks that have been copied without attribution. Justification: Database Audit was copied from an article http://www.comwise.com.my/learn-about-database-security-auditing-tools/. If that's the case, we should attribute the content. 18:13:33 let's just delete that section too -- not necessary to talk about auditing why/how/at all 18:13:41 done and done 18:13:43 Does it include operations to enable logging? i.e., to create the log files that the user wants to see? 18:14:02 vgnbkr, its up to configurations API 18:14:46 vgnbkr, by use of configurations API, user will be able to enable/disable logging configuration 18:15:11 vgnbkr, spec states about it 18:15:20 vgnbkr, API section 18:15:37 vgnbkr, https://wiki.openstack.org/wiki/Trove/DBInstanceLogOperationV1#How_does_it_works 18:16:01 denis_makogon: So what's the behavior if the configuration group has logging disabled? 18:16:30 I did read that, but it really wasn't very clear what you were getting at there. 18:16:41 SlickNik, user would not be able to pull logs, because they are not enabled by the applied configuration 18:17:02 denis_makogon: There likely needs to be a new error response to the client, I take it? That's something that's part of the API contract and should be called out. 18:18:04 denis_makogon: Also the "Public API" section confuses me. It states: 18:18:04 Three new resources, log-create, log-show will be exposed as part of the Trove API. 18:18:08 The log-show is used to provide an ability to list all available(availability defined by Trove) database logging filenames per instance. 18:18:08 The log-create is used to provide an ability to save database logging file into the Swift container, required attribute - instance. 18:18:23 SlickNik, user will see an empty log ist 18:18:25 *list 18:19:19 denis_makogon: That is not good UX — for there is no indication that this feature is tied to config groups at all! They would never guess how to enable it. 18:19:36 denis_makogon: That section says "three" but lists only "two". 18:20:13 SlickNik, fixed, actually two 18:21:24 denis_makogon: What's the API response on error — (eg. when swift is not accessible?) 18:21:57 Suthan Venkataramanaiah proposed a change to openstack/python-troveclient: Prevent extra API call when hostname not available https://review.openstack.org/108459 18:21:58 SlickNik, design was taken from backups API, user will receive 403 Forbidden 18:22:33 denis_makogon: The spec does not call that out. 18:22:40 denis_makogon, SlickNik -- we shouldn't be editing this spec on the fly as we find issues.. i think it's better for everyone if Denis goes back.. and addresses all these points 18:22:41 SlickNik, i see 18:22:50 denis_makogon: In general, the spec needs to cover these issues. 18:23:20 skipping 18:23:47 denis_makogon: I find it concerning that after going through multiple iterations for multiple specs, you still come back with specs that are missing key parts of the API information. 18:24:34 SlickNik, each time new concern are being raised, which weren't asked ever before 18:24:49 let's just skip it 18:26:11 denis_makogon: not every concern should have to be called out. Let's talk offline about how we can achieve writing better specs from the get-go, so as to reduce churn at these meetings. 18:26:46 #topic Use oslo-rootwrap 18:27:01 #link https://blueprints.launchpad.net/trove/+spec/rootwrap-for-guest 18:27:09 boblebauce: around? 18:28:41 hmm, the review linked to that BP has noting to do with root wrap 18:28:57 I'm here (I work with boblebauce) 18:29:21 Ramashri Umale proposed a change to openstack/trove: Adds backup/restore support for mongodb https://review.openstack.org/78339 18:29:22 robertmyers: yes, the review linked is more a code cleanup 18:29:52 SlickNik, I'd agreed to work with boblebauce about this. I think I'll have to do some more. 18:30:13 Didn't realize this was coming up today 18:30:19 else I'd have chatted with him about it 18:30:43 yeah, root wrap will be a major change 18:30:53 Suthan Venkataramanaiah proposed a change to openstack/python-troveclient: Prevent extra API call when hostname not available https://review.openstack.org/108459 18:31:13 yes 18:31:22 I think what boblebauce is proposing is much smaller 18:31:38 yes, that is just bug fix 18:31:50 I'll work with him about this. that was supposed to be just a 'sample'. 18:31:58 the BP is a major overhaul 18:31:58 but I think there's more work to be done before this can be reviewed. 18:31:59 amrith / sbadia: So I think we all agree that the BP is worth tackling. While coming up with the spec, we need to look at a couple of things: 18:31:59 - What other OpenStack projects do? There is root-wrap code in oslo, how do we leverage this 18:31:59 - What would the impact of these changes be? 18:32:07 amrith: a sample? is the first one free? 18:32:16 SlickNik, a couple of things 18:32:25 (and I'll get to kevinconway's comment in a second) 18:32:42 one thing I'd like is to hold the bp in vipul's name till we reach agreement on what is to be done 18:32:59 and I'll get you the answers to the questions you pose above. 18:33:07 kevinconway, the second is half-price. 18:33:09 ; 18:33:11 ;) 18:33:37 I've been looking at other projects over the weekend 18:33:41 sounds like half a product tho :-P 18:33:44 and not all have done this migration, I think. 18:34:21 so, I'll follow-up on my action item from the meeting on 07-19 18:34:24 and get back to y'all 18:34:36 unless sbadia has other thoughts, I'd call time on this one. 18:34:40 yeah i think it was rejected in glance 18:34:43 dont rem why 18:35:00 amrith: nop, it's ok 18:35:01 iccha, some projects rejected it because of scope pre-icehouse 18:35:09 indeed 18:35:11 amrith: I think nova is the one that has it done. Depending on the scope / impact, I'm okay with punting this based on other priorities. 18:35:14 I'm not sure if root wrap will work nicely with our guest 18:35:24 specifically backups and restores 18:35:38 we call subprocess directly 18:35:47 can we agree that this is not targeted for juno (grin) 18:35:59 any +1's on that? 18:36:09 amrith: It isn't targeted for juno, already 18:36:22 Hello 18:36:29 amrith: Milestone target: ongoing 18:36:45 Series: Accepted for future 18:37:06 ok. boblebauce sorry I dropped the ball on this one. we should chat; maybe after this meeting. I know we chatted a bit after the last meeting but, my apologies. 18:37:43 amrith: no problem ! 18:38:21 Okay, let's keep moving. 18:38:39 #topic Disk space validation coefficient 18:38:48 #link https://blueprints.launchpad.net/trove/+spec/restore-disk-space-coefficient 18:39:15 this sounds like a fun math equation 18:39:19 So, there's a bug report related to validation of disk space 18:39:28 kevinconway, it is 18:39:31 So this is a feature to detect when an operation would fail, and fail more gracefully. Shouldn't we instead be looking to make the functionality work? 18:40:01 vgnbkr, there's a problem with backuping/restoring tools 18:40:14 vgnbkr, most of them can't do streaming 18:40:32 vgnbkr, but mysqldump, xtrabackup can 18:40:35 denis_makogon: is that the problem? 18:40:44 it's more than streaming isn't it? you can't stream 20 gigs into a 1 gig disk 18:40:56 kevinconway: +1 18:40:57 Right. So shouldn't we make a non-streaming backup/restore work in all cases? Can't we address the diskspace issue somehow? Perhaps mounting a temp volume? 18:41:47 what do you mean by streaming? 18:42:09 Backup/restore by default streams the backup directly to swift, without storing it on the local drive. 18:42:11 you get a file, you have to restore it. what does streaming mean? 18:42:29 amrith, writing data directly into stdout without any specific preparations 18:42:38 so streaming only make backups take less memory 18:42:49 robertmyers: +1 18:42:49 not sure how this applies to restore 18:43:10 amrith, robertmyers, the problem is in the size of provided disk, you can't restore 20G to 1G disk, but you can deployers can configure Trove to accept size of disk that can be calculated with specific coefficient 18:43:11 yeah seems like the issue is with all backups that exceed the disk space of the instance 18:43:24 amrith: streaming = writing the backup to swift directly so that the backup doesn't take up space on local disk. 18:43:35 denis_makogon: so we should require at least a 20g in that case 18:43:36 so denis_makogon is looking for a way to prevent failed builds due to a backup being too big. is that correct? 18:43:41 amrith: at least that's the implication here. 18:44:11 I hear two very different problems being described. 18:44:16 one is a restore problem. 18:44:16 I thought there was already a review for this 18:44:21 one is a backup problem. 18:44:38 so q to denis_makogon ... you appear to be solving the restore problem. yes? 18:44:44 kevinconway, not at all, its all about quota on size of disk 18:44:56 amrith, yes 18:45:32 ok, so if this is a restore problem, then ... 18:45:38 this is an extension of the bug you are fixing 18:45:44 so, if you want to restore X Gb, you would need to use X * per_datastore_coefficient disk size 18:45:47 where we said a backup is ~ size of on disk storage 18:45:56 amrith: +1 18:45:57 but you are looking to do some multiplication 18:46:04 to deal with the fact that the backup may be compressed 18:46:07 or some such 18:46:23 amrith, that's why it's configurable per datastore 18:46:50 amrith, deployer/dev knows how much additional space will be used by one or another strategy 18:47:19 there's a backup on swift 18:47:22 denis_makogon: so a person backing up a 10g volume would need a 20g volume if the coefficient is 2? 18:47:23 it gets mounted and accessible 18:47:34 seems a little hight 18:47:38 amrith, for mysql coefficient is 1 of course, because it can accept data as stream directly 18:47:38 high 18:47:43 so someone has to guess how much disk space is needed in total for this backup inclusive of any intermedite files that may be required 18:47:48 robertmyers, yes 18:48:05 that doesn't seem like a solution 18:48:09 I don't get it. 18:48:29 how do I know that a compressed backup of 10GB can fit in a database of 10GB with indexes and stuff like that? 18:48:45 as robertmyers says, this doesn't sound like a solution to me. 18:48:50 amrith, you can't know 18:49:02 denis_makogon: This seems to me to be a bad idea. Seems to me like this propagates the idea of having "voodoo numbers" in our conf settings that doesn't really solve anything concrete. 18:49:03 so, let it fail I say 18:49:20 I'd rather we do something like tag the size of the instance volume that generated the abckup and let the user base the size of the new instance on that as a recommendation. 18:50:06 if it's bad - lets skip it and find more appropriate solution 18:50:07 magic numbers: -1 18:50:10 amrith: it's like a box delivered to you in the desert. you don't know what's in it so you have to guess and ask "What's in the booox?" 18:50:15 amrith: so then the user would be stuck paying indefinitely for storage that was used only to do the restore, no? 18:50:25 no, it's a hint 18:50:43 a user can say restore this for me, pick a size you think is appropriate 18:50:53 or a user can say restore on size X GB, and it may fail 18:51:16 or we can do the obvious 'du' before we do the dump and record that size, rounded up etc., 18:51:22 and use that as the starting point 18:51:27 IIRC, we do store the size of the data on disk, as part of the backup. 18:51:43 SlickNik: yes 18:51:45 SlickNik, I think we store the size of the backup file 18:51:51 amrith: no 18:51:55 ok 18:52:02 volume used 18:52:09 amrith: No - we also store the size of the data on the volume. 18:52:12 volume size (total) or space used? 18:52:19 so we should just require at least that much 18:52:32 how user can know how much space he needs to restore a backup ? 18:52:35 and that should be a good enough hint of what volume size should be used. 18:52:39 amrith: space used 18:52:49 SlickNik: +1 18:52:53 user is not able to know that strategy X takes up to two times of backup size 18:52:55 OK, then that's a start 18:53:05 let's now get to the issue of streaming vs. non-streaming 18:53:09 denis_makogon: add documentation 18:53:16 for the users 18:53:17 for mysql; the total space used is sufficient as a starting point 18:53:32 for others you may have to stage an intermediate hunk of stuff in (say) /tmp 18:53:45 is that what denis_makogon is referring to as 'streaming' and 'non-streaming'? 18:54:10 I question whether this is an issue for other datastores 18:54:18 I think this would apply to mysql too 18:54:33 robertmyers, I'll defer to the author of the proposal on that. 18:54:35 mysqldump does not stream 18:54:40 amrith, i was referring to the tools that cannot accept data as income stream 18:54:42 I'm just trying to figure out what's being requested here 18:55:03 denis_makogon: look at xtrabackup 18:55:15 it doesn't stream it directly in 18:55:17 so in effect, denis_makogon is making a proposal for the case of a solution that cannot do what mysql does 18:55:23 it loads it on disk first 18:55:25 robertmyers, xtrabackup can do data streaming 18:55:28 or that needs an intermediate space 18:55:35 denis_makogon: look again 18:55:42 stream out 18:56:12 the stream in is just a temp space that is not the actual restore 18:56:17 so robertmyers I think we agreed that we're talking about sreaming 'in' only 18:56:19 i.e. restore 18:56:42 yeah and I'm saying it doesn't even apply to mysql 18:56:51 in our current form 18:57:21 Okay, time check here. 18:57:29 SlickNik +1 18:57:37 i'll mark it as obsolete 18:57:38 denis_makogon: Once again, I think the lack of clarity in the actual spec is causing a lot of confusion as to what's actually being proposed here. 18:58:12 ok 18:58:34 Suthan Venkataramanaiah proposed a change to openstack/python-troveclient: Prevent extra API call when hostname not available https://review.openstack.org/106843 18:58:41 denis_makogon: and what the problem that we are trying to address really _is_ 18:58:46 Okay, thanks. 18:59:19 #endmeeting