16:05:26 <daemontool> #startmeeting thu-04-02-2016
16:05:27 <openstack> Meeting started Thu Feb  4 16:05:26 2016 UTC and is due to finish in 60 minutes.  The chair is daemontool. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:05:28 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:05:30 <openstack> The meeting name has been set to 'thu_04_02_2016'
16:05:38 <daemontool> Hi Team
16:05:42 <ddieterly> hi
16:05:45 <reldan_> hi
16:05:51 <daemontool> today topics are avaialble here https://etherpad.openstack.org/p/freezer_meetings
16:06:26 <daemontool> we have today a specific etherpad for the tenant backup topic here https://etherpad.openstack.org/p/tenant-backup
16:06:45 <daemontool> #topic Define common metadata for all backups
16:07:12 <daemontool> so we need to define a metadata that will be used by all the backup methods
16:07:31 <daemontool> ie. file based, incremental, cindernative, novanative and so on
16:07:58 <daemontool> we have so far a schema example available here https://etherpad.openstack.org/p/tenant-backup
16:08:43 <daemontool> reldan,  we do not use anymore the old medata, right?
16:08:59 <reldan_> Nope, it was only swift based
16:09:18 <reldan_> so it is obsoleted and deleted
16:09:41 <daemontool> ok
16:09:58 <reldan_> Actually about tenant backup
16:10:14 <reldan_> Who is the driver? Who defines requireents?
16:10:24 <reldan_> requirements
16:11:08 <daemontool> reldan,  Igood question
16:11:31 <reldan_> Because otherwise we can create something, that is very good but not very practical
16:11:33 <daemontool> I think, one way of defining basic requirements
16:11:49 <daemontool> is to get them from our Companies
16:12:00 <daemontool> so from HP I think someone should talk with Arun
16:12:17 <daemontool> from 99Cloud we can get some from EinstCrazy and zhangjn
16:12:36 <reldan_> Yes, exactly - we need a guy who wants “tenant backup” and ask - how do you see this feature? how are you going to use it?
16:12:37 <daemontool> I'll take the requirements from Ericsson
16:12:46 <daemontool> yes
16:12:54 <daemontool> I'd say that as a quick win
16:12:59 <daemontool> for volumes backups
16:13:19 <daemontool> we can integrate with the cinder volumes incrementals feature
16:13:28 <daemontool> available from Liberty
16:13:30 <zhangjn> volumes backups online
16:14:01 <daemontool> zhangjn, by online you mean, without downtime right?
16:14:03 <reldan_> For example I have question - if we have many volumes, we cannot implement backup immidiatele. All backups will have different time. How we specify the time of tenant backup then?
16:14:04 <daemontool> ok
16:14:08 <daemontool> I'm adding the requirements
16:14:11 <daemontool> on etherpad
16:14:34 <daemontool> reldan, probably we can manage that from the scheduler?
16:16:05 <reldan_> Yes, let’s say we started our backup at 2:00 and we have 4 volumes - 1G, 2G, 100G, 1K G - so backups very completed at 2:00:30, 2:00:50, 2:04:00, 2:14:00
16:16:31 <reldan_> But for tenant backup we need to specify time - should it be 2:00?
16:17:12 <zhangjn> Maybe define every volume backup time
16:17:39 <ddieterly> how about start time and end time for each volume?
16:17:39 <reldan_> Or what to do if by some reason we can do backup for 3 of 4 our volumes. And 4-th returns error
16:17:46 <zhangjn> Use can config every volume or vm backup time
16:17:49 <zhangjn> user
16:18:11 <reldan_> Yes, exactly - here a lot of questions and it will be great to some product owner - who knows how it should be from practical point of view
16:18:12 <daemontool> ddieterly, we do not know the end time :(
16:18:18 <reldan_> to have
16:18:55 <dmellado> daemontool: would it be possible to have some kind of estimate for that?
16:19:03 <reldan_> If we have no such person, I can imagine some metadata format - but I doupt it will be very good in real life
16:19:22 <daemontool> we use the cinder api, so it's cinder dependant
16:19:30 <daemontool> it depends on the volume size
16:19:47 <daemontool> but I'm not sure we can do an estimate
16:19:55 <daemontool> it depends on the current load of the cinder storage nodes also
16:20:13 <daemontool> on how many backup are happening etc
16:20:16 <daemontool> so
16:20:18 <dmellado> I see
16:20:20 <daemontool> on which case
16:20:30 <daemontool> the volumes that belongs to a tenant
16:20:36 <daemontool> should be taken at the same time?
16:20:55 <zhangjn> backup_start_time and backup_end_time in our schema
16:21:01 <daemontool> ok
16:21:06 <dmellado> even if they do start at a given time we won't be sure if they end at the same time
16:21:12 <dmellado> so if they are thought as ' a block '
16:21:23 <dmellado> maybe we'd have to wait for the slowest one to end
16:21:58 <daemontool> I think we need to define  probably a timeout, that would be backup_end_time?
16:23:57 <daemontool> dmellado,  ++
16:24:09 <daemontool> reldan,  your question is about what we do if one volume fail?
16:24:25 <daemontool> so here we have the same challenge of parallel backups right? :)
16:24:27 <reldan_> Yes
16:24:34 <reldan_> Yes, exaclty
16:24:40 <daemontool> we solved that by defining a policy
16:24:41 <daemontool> like
16:25:06 <daemontool> strict: if one fail, the job session fail, and all the volumes backups needs to be retaken
16:25:25 <reldan_> Yes, but we need it in metadata
16:25:36 <daemontool> reldan,  yes
16:25:39 <daemontool> ok
16:25:58 <reldan_> So probably we will have corrupted tenant backups
16:26:03 <daemontool> so we can have a key called
16:26:13 <openstackgerrit> Pierre Mathieu proposed openstack/freezer: Fixing database config file parsing  https://review.openstack.org/276333
16:26:18 <daemontool> failure_policy: {strict, flexible} ?
16:26:24 <reldan_> Yes
16:26:44 <reldan_> Let’s say we want to restore tenant backup now, and the latest one contains only 3 volumes of 4
16:26:56 <reldan_> And we have previous backup with 4 of 4
16:27:02 <reldan_> What I should do for restore?
16:27:44 <daemontool> so that backup
16:27:48 <daemontool> to have 3 of 4
16:28:07 <daemontool> it mean the user set failure_polic: 'flexible'
16:28:12 <reldan_> yes
16:28:15 <daemontool> policy
16:28:22 <reldan_> should we restore 3 of 4?
16:28:32 <reldan_> or should we restore previous one with 4 of 4?
16:28:32 <daemontool> yes
16:28:38 <daemontool> 3 of 4
16:28:41 <reldan_> or should ve restore one from old and 3 from new?
16:28:55 <daemontool> I think 3 of 4
16:29:11 <daemontool> because the backup session if OK according to the policy
16:29:27 <daemontool> and is just the previous backups available from the restore date set by the user
16:29:42 <daemontool> we have failure_policy
16:29:47 <daemontool> it semplify
16:29:51 <daemontool> the whole thing
16:30:03 <daemontool> because if you have 'strict'
16:30:10 <reldan_> But what I should do if I wnat to restore all 4?
16:30:11 <daemontool> 3 of 4 is not valid
16:30:36 <reldan_> Let’s say I have 1 web node, 2 sql node (master - slave), 1 processing node
16:30:38 <dmellado> wouldn't that lead to data issues? that would happen if any data from changes in between
16:30:44 <daemontool> in that case the user shoudl have the the 'failure_policy': 'strict'
16:30:49 <reldan_> If I restore 3 of 4 - it will be not very good for me
16:31:10 <daemontool> reldan,  yes I agree
16:31:14 <daemontool> so in that case you set
16:31:21 <daemontool> 'failure_policy': 'strict'
16:31:24 <reldan_> I suppose we need to have a mechanism to choose backup for restore not only by datetime
16:31:34 <daemontool> yes
16:31:49 <reldan_> like in my sql: select last full backup to this date
16:31:50 <zhangjn> yes
16:32:04 <daemontool> that's a requirement
16:32:10 <daemontool> ++
16:32:29 <openstackgerrit> Pierre Mathieu proposed openstack/freezer: Fixing database config file parsing  https://review.openstack.org/276333
16:32:32 <reldan_> How we are going to store our metadata? In mysql?
16:32:42 <daemontool> reldan,  one sec
16:32:47 <daemontool> let me add the requirement
16:33:21 <daemontool> ok
16:33:23 <zhangjn> mysql backup we can use xtrabackup tools.
16:33:57 <daemontool> zhangjn, we have been analyzing that, we can talk about that off line if you want
16:34:05 <daemontool> I'll tell you the whole story
16:34:14 <daemontool> :)
16:34:28 <zhangjn> OK
16:34:36 <daemontool> reldan,  I think,
16:34:41 <daemontool> we need to store the metadata
16:34:59 <daemontool> in the API and also in the media storage we set
16:35:09 <daemontool> for cindernative we do not have a media storage to set
16:35:29 <daemontool> but I think, we should store it on the freezer-api
16:35:34 <daemontool> cause from that we can compute metrics
16:35:38 <daemontool> and show data from the web ui
16:36:08 <daemontool> for now, I wouldn't solve which db are we going to use as api backup other than elastic search
16:36:11 <reldan_> Should freezer-agent know about api?
16:36:14 <daemontool> cause that's story itself
16:36:21 <daemontool> the scheduler knows
16:36:45 <daemontool> the agent return the metadata
16:36:47 <daemontool> to the scheduler
16:36:51 <reldan_> Ok )
16:36:55 <daemontool> and the scheduler upload it to the api
16:36:59 <daemontool> as we are doing now
16:37:04 <daemontool> we have that already
16:37:04 <reldan_> So who is the guy responsible for metadata format?
16:37:19 <openstackgerrit> Pierre Mathieu proposed openstack/freezer: Fixing database config file parsing  https://review.openstack.org/276333
16:37:28 <daemontool> I think we need 2 dudes for that
16:37:37 <daemontool> 1 backup the other :)
16:37:47 <daemontool> well, it's more ha
16:37:55 <zhangjn> ha
16:38:00 <dmellado> daemontool: gotta run, would you put the 'proposed schedule' for the source code overview in etherpad?
16:38:07 <daemontool> dmellado,  yes
16:38:11 <daemontool> thanks a lot for your inputs
16:38:20 <dmellado> np, thanks! ;)
16:38:32 <daemontool> reldan,  zhangjn are you ok doing that?
16:38:35 <reldan_> I believe it should be someone who knows requirements.
16:38:38 <daemontool> vannif, needs to do that also
16:38:42 <daemontool> zhangjn, and vannif ?
16:38:46 <daemontool> me?
16:39:04 <daemontool> it should be someone that works on the agent
16:39:10 <daemontool> and someone that works on the api
16:39:23 <daemontool> the requierments we can define that together
16:39:30 <daemontool> as we need to collect them from our companies also
16:39:33 <reldan_> I can create metadata - but it may be just useless for guy who actually is going to use tenant backup
16:40:16 <daemontool> is vannif  around there?
16:40:27 <daemontool> he did quite a good job last time with scheduler metadata
16:40:38 <reldan_> Nope he is not here
16:40:44 <daemontool> ok
16:40:54 <daemontool> so I can do it for now
16:41:08 <daemontool> reldan, and zhangjn  if you wants to do it too
16:41:12 <daemontool> you are welcoem
16:41:22 <zhangjn> we have a holiday(Spring Festival).
16:41:26 <reldan_> Thank you :)
16:41:35 <daemontool> ok reldan we'll do it
16:41:42 <reldan_> Just because now we have only that
16:41:46 <reldan_> As a tenant, I need to use Freezer to backup all my data and metadata from an OS Cloud and restore it at my convenience.
16:41:52 <reldan_> And it is good
16:42:09 <reldan_> But it would be great a real person, who wants this because he is really running something
16:42:25 <reldan_> and it is really important for him to have tenant backup
16:42:35 <reldan_> Like guy who runs e-commerce application
16:43:18 <daemontool> reldan, can you ask to Arun there?
16:43:20 <reldan_> Otherwise it is more like - we need a tenant backup just because it is cool
16:43:21 <daemontool> that's his job
16:43:50 <reldan_> Oh, I don’t have lync on my comp. Probably ddieterly can
16:44:12 <daemontool> ddieterly, can you have a conversation with Arun
16:44:12 <zhangjn> I have some customer want this function.
16:44:27 <daemontool> and get the requirements for the backup as a service in general?
16:44:29 <reldan_> zhangjn: Great. Could you gather requirements?
16:44:34 <daemontool> I'll do the same
16:44:57 <ddieterly> yes
16:45:12 <daemontool> also Pierre can have some requirement
16:45:15 <daemontool> Slashme
16:45:19 <daemontool> ddieterly,  thanks a lot
16:45:54 <zhangjn> Yes, I will gather requirements.
16:46:00 <reldan_> Yes, sure - let’s gather requirements first. If we can, otherwise we will implement something that may not very usefull for our customers. I’m afraid this situation
16:46:03 <daemontool> we have a product owner here
16:46:05 <reldan_> Thank you
16:46:14 <openstackgerrit> Pierre Mathieu proposed openstack/freezer: Fixing database config file parsing  https://review.openstack.org/276333
16:46:16 <daemontool> ok
16:46:20 <zhangjn> I will push our guys to join in this project.
16:46:34 <daemontool> zhangjn, lol thank you :)
16:46:46 <daemontool> now
16:46:51 <reldan_> Currently I can start to add —force and —increment to cindernative
16:46:58 <daemontool> ok let's move forward
16:47:01 <daemontool> reldan,  that's good
16:47:09 <zhangjn> Some POC will start after Spring Festival
16:47:11 <daemontool> I think zhangjn  can do that
16:47:19 <daemontool> reldan,  with your guidance?
16:47:31 <daemontool> so he'll take familiarity with the base code?
16:47:34 <reldan_> Yes, sure - it shouldn’t be very difficult
16:47:48 <reldan_> zhangjn: please contact me anytime
16:47:53 <daemontool> ok, please arrange a meeting
16:48:07 <zhangjn> ok, I will
16:48:10 <daemontool> ty
16:48:12 <daemontool> zhangjn,  when the Spring Festival
16:48:16 <daemontool> will ends up?
16:49:01 <daemontool> ok let's move forward
16:49:04 <daemontool> to the next topic
16:49:29 <daemontool> #topic Source code walkthru definition
16:49:34 <zhangjn> Feb 6 ~ Feb 15
16:49:55 <daemontool> zhangjn,  ok so we'll arrange the source code walkthough session
16:50:02 <daemontool> after the 15th of Feb?
16:50:14 <zhangjn> yes
16:50:17 <daemontool> ok
16:51:01 <daemontool> let's move forward
16:51:03 <daemontool> to
16:51:15 <zhangjn> I have a beginer in freezer. learn and learn, :)
16:51:16 <daemontool> #topic  Mitaka-3 release
16:51:24 <daemontool> ok :)
16:51:28 <daemontool> so
16:51:33 <daemontool> I'm doing that currently
16:51:46 <daemontool> there are few issues, like adding modules to global-requirements.txt
16:52:02 <daemontool> fixing devstack api plugin (it's required by something I can't remember) and the tagging
16:52:53 <daemontool> we should be good for M3 timeline at
16:53:26 <daemontool> 13th of Feb
16:54:06 <daemontool> 8-12th Feb
16:54:09 <daemontool> need to verify
16:54:14 <daemontool> anyway that is important
16:54:24 <openstackgerrit> Pierre Mathieu proposed openstack/freezer-web-ui: Using a smarter way to get freezer-api URL  https://review.openstack.org/275804
16:54:37 <daemontool> moving forward
16:54:50 <daemontool> #topic Change meeting to 11am GMT every Thu to facilitate people from Asia to participate
16:55:12 <daemontool> anyone wants to submit a patch for that to the openstac-infra/meetings repo?
16:55:28 <daemontool> I can do that
16:55:43 <reldan_> great
16:55:45 <daemontool> so the topic are over
16:55:58 <daemontool> there's anything else you want to discuss?
16:56:03 <zhangjn> great
16:56:08 <daemontool> anyone wanted to say somethign and didn't had the chance?
16:56:22 <daemontool> something more on the volumes meta?
16:56:32 <daemontool> reldan,  we need to set a meeting
16:56:41 <ddieterly> 11 AM GMT is like 4AM for me
16:56:41 <ddieterly> yikes
16:56:46 <daemontool> with frescof_
16:56:48 <daemontool> mmmhhh
16:56:49 <daemontool> ok
16:56:55 <daemontool> that's the issue
16:57:02 <daemontool> how can we do this at a time
16:57:08 <daemontool> that is suitable for US and Asia?
16:57:25 <ddieterly> 2 PM GMT?
16:57:26 <daemontool> 1PM ?
16:57:41 <daemontool> zhangjn, ?
16:57:42 <zhangjn> 4 AM ?
16:57:50 <zhangjn> or PM?
16:57:50 <daemontool> lol
16:57:53 <daemontool> am
16:58:09 <daemontool> in us 11AM GMT is 4AM
16:58:29 <daemontool> ddieterly, 1 PM sound?
16:58:35 <zhangjn> GMT 4 AM?
16:58:35 <ddieterly> early
16:58:53 <ddieterly> that's 6AM my MST
16:58:58 <daemontool> how is this problem solved
16:59:06 <daemontool> in the other teams?
16:59:17 <ddieterly> i could do 2 PM GMT which is 7 AM MTD
16:59:27 <daemontool> zhangjn,  is that ok?
16:59:31 <ddieterly> what is that in Japan?
16:59:33 <daemontool> it will be 10pm in china
16:59:57 <ddieterly> who is in china?
17:00:02 <daemontool> zhangjn, and EinstCrazy
17:00:16 <ddieterly> how is 10pm for them?
17:00:25 <daemontool> they will work on baas
17:00:30 <daemontool> mainly cinder and nova
17:01:11 <ddieterly> if i have to, i can do 1PM GMT
17:01:49 <daemontool> 1.30 pm that would be 9.30 pm ?
17:01:51 <daemontool> 6.30?
17:02:39 <zhangjn> I think 2 PM is ok
17:03:15 <daemontool> zhangjn,  ddieterly  thanks for your flexibility
17:03:28 <daemontool> so 2PM GMT is the new time
17:03:37 <daemontool> I'll send a patch
17:04:33 <zhangjn> ddieterly ,thank you :)
17:04:35 <daemontool> #endmeeting thu-04-02-2016