20:59:23 #startmeeting reddwarf 20:59:24 Meeting started Tue Mar 26 20:59:23 2013 UTC. The chair is hub_cap. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:59:25 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:59:27 The meeting name has been set to 'reddwarf' 20:59:40 hello 20:59:51 hello hello 21:00:00 * hub_cap waits 2 min for stragglers 21:00:10 we're still in a retro 21:00:16 >>> time.sleep(120) 21:01:07 Hi 21:01:14 hey 21:02:19 #link https://wiki.openstack.org/wiki/Meetings/RedDwarfMeeting 21:02:22 hey guys 21:02:23 time to start! 21:02:25 yo 21:02:38 #link http://eavesdrop.openstack.org/meetings/reddwarf/2013/reddwarf.2013-03-19-21.02.html 21:02:46 #topic Action Items 21:03:24 Howdy 21:03:27 vipul: yer up (thought u were in a meeting) 21:03:37 Vipul just getting back from a meeting. 21:03:49 ahhhh getting back 21:03:52 Skip him to the end of action items... 21:04:05 kk 21:04:25 im asking grapex about reddwarf lint integration 21:04:27 hes on the phone w/ me 21:04:40 he has not done that yet 21:04:49 #action grapex to patch reddwarf with xml lint integration 21:05:13 Sounds good. 21:05:17 next is me 21:05:36 #action hub_cap to talk to everyone about the actions stuff at the summit 21:05:41 yes, instance_actions. 21:05:44 we need to brainstorm on it, since its not fleshed out 21:05:48 no longer instance_actions 21:05:48 I'm looking forward to that one. 21:05:53 DEFFFFFFF 21:05:57 ah, name change gotcha. 21:06:03 bakc 21:06:06 back sorry 21:06:13 vipul: you have a few action items 21:06:18 wb, just going through action items. 21:06:38 Mine are: 21:06:50 SlickNik: go head w/ yours now, the regex one 21:06:57 SlickNik to update the release wiki page with regexps for release/pre-release 21:07:00 I updated the python-reddwarfclient release wiki page with the reg-exps. 21:07:10 SWEET 21:07:17 nice 21:07:34 Also, got the redstack refactoring patch checked in (thanks for the reviews all). 21:07:39 woot! 21:08:00 vipul and robertmyers: action items about the notifications blueprint 21:08:05 Ok so i updated the Backups spec to call everything Backups instaead of snapshots 21:08:13 so #1 is done 21:08:22 #2, I still need to submit a patch to database-api 21:08:22 nice 21:08:32 #action vipul to submit patch for Backups API to database-api 21:08:51 hub_cap: the notifications one.. I have not filed the blueprint 21:08:58 okey, want me to file vipul? 21:09:07 yes, please or get me a dummy 21:09:22 this is for usage/billing notifications? 21:09:23 ya dummy only 21:09:32 SlickNik: yes 21:09:43 vipul: https://blueprints.launchpad.net/reddwarf/+spec/dummydummy 21:09:48 (heart) 21:09:52 damn that didn't work! 21:09:56 hhaahah <3 21:09:58 < 3 21:09:59 <3 21:10:22 cool, i'll file the BP 21:10:30 will need robertmeyers input 21:10:37 okey. lets go on 21:10:37 sure 21:10:45 i htink the first actual item is notifications 21:11:00 #topic Reddwarf Notifications 21:11:11 So i left that in the meeting agenda since we didnt make progress 21:11:14 so we have a blueprint now, but its not really filed 21:11:34 ok vipul, lets make sure robertmyers gives u templates 21:11:34 I'd like to get clarity on the this one this week 21:11:36 ;) 21:11:40 vipul: def! 21:11:41 so let's file the bp and fill out details 21:11:43 cool 21:11:50 just ping robertmyers on irc 21:11:55 will do 21:11:57 thanks! 21:12:22 so movign on? 21:12:28 yep 21:12:28 Yes, lets 21:12:34 #topic Backups Discussion 21:12:40 Sounds like it. More discussion on that to come later. 21:12:43 lets start w/ restore API decision 21:12:51 do we have a decision :) 21:12:55 drum roll 21:13:00 give me context...? 21:13:04 im a bit out of the loop 21:13:16 restorePoint: { instanceID | backupID } 21:13:20 AHHHHHH 21:13:25 OR 21:13:25 well you and mr simps where going to work it out 21:13:28 type or no type, I think... 21:13:33 right 21:13:33 that is the question 21:13:47 we will _not_ be using instanceID for teh initial iteration 21:13:51 we had a long convo about it 21:14:11 it will _only_ be restorePoint {backupUUID, and eventually a Date} 21:14:30 so the use case where the backupUUID is really old and the date is really new 21:14:34 and there are other backups in between 21:14:52 how will that be implemented 21:15:13 well.... seems like that would be a failure case 21:15:23 we arent going to roll w/ a date from the first iteration 21:15:30 i think weve decided to go w/ the manual backup case first 21:15:33 fair enough. 21:15:37 ah ok 21:15:38 and our impl will _not_ have a date 21:15:49 Very similar to what we are planning then. 21:15:54 and we will let you guys solve the xtra+binlogs+date thing ;) 21:15:57 so the first iteration of this thing is similar to ours.. you restore to the backup and not point in time 21:16:02 correct 21:16:26 ya we will put both impls (xtra/mysqldump) in the public for reddwarf installations to decide 21:16:31 thanks for the vote of confidence hub_cap :) 21:16:32 so we will need that stratregy pattern 21:16:40 juice: :D 21:16:45 we liked your approach in hindsight :( 21:16:58 but i guess we roll with this first 21:17:01 vipul: haha ya i know... 21:17:02 I was looking at stevedore actually. 21:17:04 i did too ;) 21:17:16 robertmeyers is still working on the stream to swift approach though 21:17:18 but hey, simple is better 21:17:20 They have a good driver model that we can use for the strategy/plugin approach. 21:17:31 SlickNik: ya? hmmm that might be a good use for it 21:17:37 assuming u can config it 21:17:47 and that its not something thats compiled into the python egg 21:18:08 stevedore is good 21:18:21 ok so how about 21:18:37 backup unique name constraints? 21:18:46 restorePoint: { backupRef: http://dddd/backups/backupID } 21:19:02 im ok w/ that, i dont think it matters too much 21:19:12 we have a precedense (if that's how you spell it) for refs 21:19:22 as long as u can just do { backupRef: backupID } shorthand 21:19:35 precedence I think 21:19:36 yea we can write the parsing logic to accept either 21:19:36 w/ the flavor/image calls u can just pass the ids 21:19:44 well let me ask this 21:19:45 thanks juice! 21:19:52 do we _need_ the ref? 21:19:57 what is the reason for it 21:20:22 well, we want to eventually add the date 21:20:35 robertmyers: i dont think that affects the imageref 21:20:42 i mean ref vs uuid 21:20:47 consistency 21:20:51 refs are for things that could be non local 21:20:53 flavorRef 21:21:09 sure i get that 21:21:19 precedents actually 21:21:22 but those are from a legacy system 21:21:25 lol juice 21:21:41 and technically flavorRef is just a uuid now 21:21:44 and imageRef _is_ a url 21:21:49 cuz it comes from a different system 21:21:59 ah ok 21:22:03 if backups were their own system (like glance) then id see the need for urls 21:22:07 example http://docs.openstack.org/api/openstack-compute/2/content/CreateServers.html 21:22:16 sorry for multi post 21:22:17 "name" : "new-server-test", 21:22:17 "imageRef" : "http://servers.api.openstack.org/1234/images/52415800-8b69-11e0-9b19-734f6f006e54", 21:22:17 "flavorRef" : "52415800-8b69-11e0-9b19-734f1195ff37", 21:22:34 k cool, let's go with ID then 21:22:50 do we want to name it Ref but accept UUID? 21:22:55 There's some value in being agnostic of which system resources came from and using a ref across the board, though. 21:22:57 or just name it 'backupID' 21:23:22 hmmmmmmmmm 21:23:35 let me touch base w/ our resident rest guru 21:23:57 ok i'm going with BackupRef and allow the value to be either URI or UUID 21:24:06 and fi that doesn't fly with him we can change easily 21:24:14 vipul: fine by me for now 21:24:21 Yeah, this is not clearcut…fine by me for now as well. 21:24:37 i'll update the wiki 21:24:38 #action hub_cap to bring findings for backupRef vs backupUUID 21:24:56 so robertmyers whats the latest on streaming backups / failure handling 21:25:23 I got it mostly working... just trying to deal with large files 21:25:46 over 5GB need to be split up 21:26:06 do you keep a counter as you go? 21:26:08 oh ya swift file size limitations 21:26:13 so it is a little tricky to do that while you are streaming 21:26:14 @robertmyers: eagerly awaiting your initial patch set upload. :) 21:26:23 :) 21:26:26 http://docs.openstack.org/developer/swift/overview_large_objects.html 21:26:44 well, I think the first I'll put a gist up for all to see 21:26:58 Yes, would love to see what the integration points are. Patch set upload can come later. 21:27:05 robertmyers: perfect tahtd be great 21:27:13 ill also find imsplitbit's original code 21:27:22 thanks for sharing and the effort robertmyers 21:27:39 That would be awesome! Thx robertmyers and hub_cap. 21:27:41 you welcome 21:27:50 your welcome 21:27:51 the download use case.. is that something we're also doing directly? 21:27:51 :) 21:27:59 or is that something we should consider swiftclient for 21:28:02 you're 21:28:03 https://github.com/imsplitbit/cf_stream_uploader imsplitbit ? 21:28:16 https://github.com/imsplitbit/cf_stream_uploader 21:28:17 yep 21:28:23 damn im good 21:28:26 DEF poc code 21:28:29 robertmyers: have a look-see at this too https://github.com/imsplitbit/cf_stream_uploader/blob/master/cfstream.py 21:28:33 but should give you an idea 21:28:55 thatd be like 3 lines in ruby imsplitbit 21:29:04 nice 21:29:09 nice, thanks! 21:29:11 oh don't start that fight 21:29:16 ok on to "backup unique name constraints" 21:29:21 #link https://github.com/imsplitbit/cf_stream_uploader 21:29:23 #link https://github.com/imsplitbit/cf_stream_uploader/blob/master/cfstream.py 21:29:26 LOL 21:29:32 no answer :( 21:29:34 datsun180b, it would... 21:29:41 :-) 21:29:47 so annashen and i talked about the name stuff, and she has changed the code to be driven by uuid 21:29:55 robermeyers: are you addressing the download use case as well? 21:29:57 im not sure we have a answer on the uniequeness tho right vipul? 21:30:09 vipul: thats a restore, totally different beast ;) 21:30:41 vipul: download is handled by the swift manifest file 21:31:01 ok, so swiftclient would work for that 21:31:10 assuming that we'll write the manifest 21:31:10 probably 21:31:12 stream download shoudl be HELLA easy 21:31:16 yes 21:31:20 cool 21:31:32 so do we have any reason for constraining the name of a backup? 21:31:41 we dont currently constrain a instance name 21:31:43 yes 21:31:49 why robertmyers? 21:31:54 * hub_cap is curious 21:32:01 if we use if for the swift name 21:32:01 this is the user supplied name robertmyers 21:32:03 actually why don't we constrain the instance name 21:32:05 shelling out to do md5sums, for shame 21:32:09 vipul: cuz nova doesnt 21:32:17 hmm i thought it did 21:32:17 datsun180b: POC 21:32:28 nope cuz all my instances are name foo ;) 21:32:34 You can have 2 instances with the same name in nova. 21:32:36 yes of course, i'm only ribbing him because it's required 21:32:36 maybe that's nova client behavior 21:32:46 lol 21:33:21 actaully it's a good point.. what shoudl the object be named in swift 21:33:23 perhaps, vipul, I think there _is_ a client check... 21:33:24 vipul: its display_name 21:33:33 vipul: likely something date based 21:33:47 datsun180b, there was actually a reason for that. the code had to run in minimal memory space and for some reason it was hitting a ceiling for memory so I shelled out and that solved that 21:33:50 since when u start a snap, i assume we will be in a "backup" state 21:33:53 so why have a name at all 21:34:01 vipul: context 21:34:11 same reason why give a name at all for instances 21:34:13 a uuid is ugly 21:34:21 gives a human something readable 21:34:31 but by no means should we name them off that 21:34:38 i figure md5 is probably compiled and being purpose-build for that is going to blow four lines of python out of the water 21:34:49 i don't mean to distract 21:35:00 swift object name = date + backup name? something like that? 21:35:13 when they are browsing swift, is it iimportant to give the same context i guess 21:35:21 Yes, I agree.. 21:35:21 why not backup uuid???? 21:35:30 since a uuid is uu 21:35:58 sure, that would be easy to restore too 21:36:00 last ting i want to see tho is naming it something like this, 祈票禁神祝-backup 21:36:07 ha 21:36:08 lol 21:36:09 lol 21:36:14 lolly pop 21:36:32 fair nuff - the name should be optional then 21:36:38 welllllll..... 21:36:46 name is not optional 21:36:47 if we are doing automated backups... 21:36:49 its just not unieque 21:36:56 who cares what the anme is if it's date based 21:36:59 not optional but unique. 21:37:08 lol SlickNik 1/2 right 21:37:13 not optional, not unieuq 21:37:16 man i cant type 21:37:32 it will be _exactly_ the same as the way our /instances name works 21:37:32 yeah, that's what I meant.. typed it out wrong :P 21:37:35 hehe 21:37:36 what if we want to add the ability to do a single table backup or single database 21:37:37 what are we going to name the snapshots when we make it automated then 21:37:49 vipul: we can figure that out when we go automated 21:37:57 blahdb-backupdate? or something liek that? 21:38:21 I still like putting the name in the backup file though (instead of the UUID) 21:38:29 agree with SlickNik 21:38:39 if we're using it for context in Reddwarf.. why not the same context in swift 21:38:49 because these are exposed to the user. 21:38:54 well, could that be part of the backup extention? 21:38:58 well lets not assume that a user should be having context wrt that 21:39:09 we have not defined that a user can/shoudl remove them 21:39:15 so why make it easy for them to ;) 21:39:16 because even 祈票禁神祝:01/29/2013:00:00:00 has some meaning to the user, whereas a UUID does not. 21:39:16 seems easy enough to factor out 21:39:31 well they'll see it anyway.. (assuming you're putting it in public swift) 21:39:36 sure we are 21:39:45 bye vipul :P 21:39:50 he mustve hit the wrong key combo 21:40:00 don't hit command + delete 21:40:00 cmd-Q'ing us vipul? 21:40:05 lol 21:40:07 The name is swift must be unique 21:40:17 yup juice 21:40:34 And unless its some meta data the name is mostly what they'll see in swift 21:40:37 well, mostly true, you just over write the existing file 21:40:39 ok we need a 'naming_strategy.py' :) 21:40:45 juice: thats a good idea actually 21:40:53 put the name / description in the metadata 21:41:15 lol vipul 21:41:21 heh 21:41:43 actually, you got a second shot for the name that appearing in swift container for an object 21:41:54 what do yall htink of that? uuid for the filename, and name/desc in the metadata for the file 21:42:32 Sounds good 21:42:32 i will just sit and watch for the finals 21:42:44 hmm not too thrilled but i can live with it 21:42:52 I'm okay with that as long as the metadata is readily available/apparent in the UI. 21:43:00 The only restriction on object names is that they must be less than 1024 bytes in length after URL encoding. For example, an object name of C++final(v2).txt should be URL encoded as C%2B%2Bfinal%28v2%29.txt and therefore be 24 bytes in length rather than the expected 16. 21:43:22 do we really want to ahve to take swift naming considerations into account? 21:43:50 no.. 21:43:58 :D 21:43:58 we can make that a configurable thing.. we're not using a private swift.. it's going into their public cloud swift.. 21:44:07 so it's visible.. which is why i bring it up 21:44:18 sure, neither are we... 21:44:43 so if it is visible and they will see it.. the context argument should apply to swift object name as well me thinks 21:44:49 id rather make it less understandable for now 21:44:59 which may mean they're more likely to delete it :( 21:44:59 i dont want to deal w/ constancy between the systems 21:45:21 ok crap a user named it incorrectly in our system, so they go rename it cuz they dont like the missspell in swift 21:45:44 well yea i guess that could happen as well 21:46:09 let's go with UUID.. and if we need to.. we can make that a configurable/pluggable thing 21:46:11 simplicity = uuid / metadata for things that dont matter and arent unique (names...) 21:46:15 for sure vipul 21:46:25 at the end of the day the name doesn't matter to reddwarf.. but it may to the user 21:46:29 #agreed 21:46:42 ya its all code, we can figure it out 21:46:45 I can live with UUID and name/desc in the metadata for now. 21:46:52 sounds good. 21:46:58 #agreed even 21:46:59 ok the failure handling thing 21:47:16 i put that in the agenda.. to see how we want to handle the case where streamed upload fails 21:47:18 all for siliently failing and sayign nothing to customers? 21:47:33 sure 21:47:34 :P 21:47:34 do we want to retry? or just fail 21:47:39 retry for sure 21:47:40 aye :P 21:47:47 we shoudl ahve retry logic for X (configurable) retries 21:47:57 honestly each chunk is gonna potentially fail 21:48:02 so we need to add retry there 21:48:16 ideally we'll do MD5 checksuming durging the backup too 21:48:46 does swift give you a md5 for every segment? 21:49:05 Where do we store the checksum(s)? 21:49:05 md5 at the end i think 21:49:07 we can calcualte md5 on the stream .. for every segment.. and comapre? 21:49:14 I think so 21:49:46 also u can give swift the md5 and itll compare it to make sure its clean 21:49:48 SlickNik: just in memory calculate on th efly 21:49:51 the fly 21:49:52 the api takes an etag 21:50:17 no, I mean, we want to verify on restore right? 21:50:42 yeah the download process coudl do that 21:50:43 def on restore too 21:51:05 probably another case we should consider.. swift probably has the md5 for that object as metadata or something? 21:51:05 we will cross that bridge when we do restores :D 21:51:24 vipul: it is the ETAG header 21:51:26 we're working on that portion as well hub_cap 21:51:59 oh nice as one shabang vipul? 21:52:16 robermeyers: ok so it's going to be there for us 21:52:28 hub_cap: yea in parallel i suppose 21:52:31 SWEET 21:53:04 gotcha robermyers. 21:53:06 ok so do we feel comfortable w/ backup/restore now? should we move on? 21:53:11 robermeyers: so there may be one use case.. where the user could potentially replace the segments 21:53:22 and the md5 would be changed 21:53:45 i think they would have to change the manifest too right robertmyers? 21:53:45 well then that is their fault :) 21:53:52 Well, we need to figure what are we using the md5 for. 21:54:03 do we even want to consider that? i guess the option would be to manually store the md5 of the entire thing somewhere 21:54:07 yes, you can upload a new manifest 21:54:16 If it's 1. to check data integrity, then it's fine to use the ETAG header. 21:54:54 2. If we're worried about security where someone can swap out the backup and change the MD5 if they have access to Swift. Then: 21:55:18 a. We need to not use MD5 and use something more cryptographically secure 21:55:42 b. We need to have a place to persist said hash 21:55:56 SlickNik: Yea was just thinking in our db table 21:56:00 but maybe that's an enhancement 21:56:02 or throw up hands for now :) 21:56:13 yah the poc code used md5 just because it was cheap and there. def need to use something better 21:56:13 vipul: +1 to enhancement 21:56:14 I like this guy ^^^ <3 21:56:36 Lets make it work then we can make it better then 21:56:38 I'm all for staying simple now and enhancing later. 21:56:40 lets make it more secure when a user actually goes and does that 21:56:42 yup 21:56:43 well in order to do anything more fancy, i don't think you can stream it 21:57:04 you could encrypt as you stream so it is still an option 21:57:11 yup 21:57:22 we good? moving on? 21:57:24 ye 21:57:25 s 21:57:27 yes 21:57:27