13:00:50 <HenryG> #startmeeting neutron_db
13:00:51 <openstack> Meeting started Mon Jul  7 13:00:50 2014 UTC and is due to finish in 60 minutes.  The chair is HenryG. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:52 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:55 <openstack> The meeting name has been set to 'neutron_db'
13:01:06 <HenryG> salv-orlando will be joining late today. We may finish before he shows up!
13:01:15 <HenryG> I don't have much to talk about today, except get a status on the code.
13:01:24 <HenryG> After that we can have open discussion for any other items.
13:01:32 <HenryG> #topic Code status
13:02:00 <HenryG> I have some review links in the agenda.
13:02:11 <HenryG> Agenda #link https://wiki.openstack.org/wiki/Meetings/NeutronDB#Agenda
13:02:42 <HenryG> So it seems the FK dependency error is solved?
13:03:17 <akamyshnikova> yes :)
13:03:26 <HenryG> Very nice. Thanks!
13:03:46 <HenryG> The global requirements is approved and merging now!
13:04:39 <HenryG> I am a little unclear on the model freeze ...
13:04:46 <akamyshnikova> I check the script and in fact it is ready for reviewers.
13:05:26 <akamyshnikova> jlibosva did the great job with models I want to thank him for that!
13:05:55 <HenryG> Yes, the freeze looks good, except ...
13:06:16 <HenryG> I would prefer not to call the freeze module 'icehouse.py'
13:06:27 <jlibosva> HenryG: I think icehouse should go away
13:06:37 <jlibosva> I mean the module :)
13:07:03 <akamyshnikova> There is a frozen.py that I used in script and head that I usedin testing
13:07:15 <HenryG> jlibosva: ah, that makes more sense now, thanks
13:07:15 <akamyshnikova> used in testing
13:08:10 <jlibosva> I also got echo from marun that markmcclain had some suggestions. I wrote him an email this morning. I don't know anything specific.
13:09:15 <markmcclain> jlibosva: right… I kicked around some ideas with marun
13:09:19 <HenryG> jlibosva: I can corner markmcclain later this week at the mid-cycle sprint. I will interrogate him. :)
13:09:53 <jlibosva> markmcclain: you can present ideas  here if you have a spare minute :)
13:10:36 <markmcclain> HenryG: haha
13:11:22 <markmcclain> jlibosva: we had kicked around the idea of how to avoid the need to duplicate so much code
13:12:13 <markmcclain> also the healing should be able to be run by operators multiple times right?
13:14:02 <HenryG> markmcclain: you mean the healing should not be constrained to a specific point in the migration timeline?
13:15:09 <markmcclain> for Juno was wonder if we should always heal last
13:15:40 <markmcclain> I was planning to ping mike bayer and get his thoughts on this too
13:18:37 <salv-orlando> I think we always said healing should be idempotent so that it could in theory be executed multiple times in the migration timeline. But I might be wrong, I tend of dream about things and then think they’re real.
13:19:18 * HenryG looks at the spec ...
13:19:46 <jlibosva> I think you always need metadata object that db will be migrated to
13:20:16 <akamyshnikova> For this moment it is running through db_healing migration once.
13:21:04 <salv-orlando> ok so I was dreaming.
13:21:22 <salv-orlando> or to be more precise,I was thinking we were doing something different
13:21:47 <salv-orlando> so healing will happen at a given point in the migration timeline, and that’s the only point where migrations will be healed.
13:21:49 <salv-orlando> is that right?
13:22:10 <salv-orlando> sorry “migrations will be healed” should have been “schema will be healed”
13:22:19 <HenryG> Currently that is the way the code is done, I think
13:24:05 <salv-orlando> how many of you will be at the sprint this week?
13:24:07 <salv-orlando> I won’t
13:24:13 <HenryG> If it heals to the current models, whatever 'current' is, then it will break any migrations that happen between the previous schema and the current models.
13:24:38 <HenryG> At the sprint: HenryG markmcclain marun
13:24:38 <akamyshnikova> At this moment it is done in this way. What do mean about multiple times? like running db-manager --heal or what?
13:25:32 <akamyshnikova> HenryG, is right we need some frosen models to compare
13:26:38 <salv-orlando> what markmcclain was suggesting is that operators should have been able to run the healing multiple times. The uses cases are mostly corner cases. I think the rationale for having a healing migration which would do the delta with current models was to avoid the duplication which lead to writing a “frozen” snapshot of icehouse models.
13:26:44 <markmcclain> HenryG: you're correct about identifying ensure we run the correct set of migrations relative to the timeline
13:27:31 <salv-orlando> Still, I’m talking without knowning what I’m talking about. If you’ve already investigated this situation and concluded one must have a frozen version of models, than I will keep my mouth shut.
13:28:20 <markmcclain> salv-orlando: from my standpoint the frozen models feel odd
13:28:53 <markmcclain> the duplicated code sets off a warning, but I don't have a technical solution that is better :)
13:29:13 <jlibosva> salv-orlando: sorry if I don't understand - but what scheme is the destination? One containing all models in current code?
13:29:27 <salv-orlando> markmcclain: smae here. And just like you I don’t have a better technical solution at hand (so that’s why I’m saying I’m talking without knowing what I’m talking about)
13:31:04 <salv-orlando> I was thinking od comparing the current DB schema against the current DB models at the instant where the healing migration is ran. Can you help me understnad why that’s not possible?
13:32:19 <jlibosva> I think the schema containing all tables at custom migration timeline is unknown.
13:32:32 <jlibosva> at custom point in migration timeline*
13:32:33 <akamyshnikova> it is possible but it will run create tables or change it. and then will run the migration that will fails trying to do the same
13:33:07 <akamyshnikova> as this tables or changes will be in models
13:35:10 <salv-orlando> jlibosva: unknown under the assumption of being able to run the migration offline, or even with online migrations?
13:35:35 <jlibosva> salv-orlando: online
13:35:55 <jlibosva> I think offline is not supported, right?
13:36:40 <HenryG> One advantage of having the healing migration at a specific point in the timeline is that we can test it and ensure it works. If it is made more generic for any point in the timeline, some weird model or type usage might break it since we never tested against that. We would have to have some kind of gating test against all previous models every time a model change is made?
13:37:20 <salv-orlando> HenryG: ah you’re playing the testing wildcard!
13:38:28 <salv-orlando> I don’t have any argument on this side, so I guess I have to give up
13:38:30 <salv-orlando> ;)
13:38:48 <jlibosva> salv-orlando: markmcclain btw there is an option to migrate to the latest https://review.openstack.org/#/c/102857/10/neutron/db/migration/models/head.py that excludes models copying
13:38:58 <HenryG> I think we need to hear what ideas markmcclain and marun are thinking of.
13:39:46 <markmcclain> HenryG: it's mainly me and not marun
13:39:53 <HenryG> jlibosva: what if someone adds a new module with a model?
13:40:19 <jlibosva> HenryG: he must add the path to this module :]
13:40:26 <markmcclain> and basically that all kinds of warnings signs seem to be present that this approach may not be right
13:41:01 <markmcclain> I went on holiday right after talking with him, so don't really have alt tech proposal
13:42:44 <HenryG> My thought thus far has been one healing migration at a specific point in the timeline. So I may have been rather myopic about what we needed to do. But I am open to ideas.
13:43:34 <salv-orlando> let’s put aside the ability of running the migration multiple times.
13:43:51 <salv-orlando> this was a side-effect rather than the main reason for what markmcclain is suggesting
13:43:56 <markmcclain> agreed starting with once will be a huge step :)
13:44:07 <akamyshnikova> there is a change for review https://review.openstack.org/96438 You can suggest everything  here in comments
13:44:19 <salv-orlando> I think when he say “warning signs” he’s talking about having two module with frozen db models state
13:44:39 <salv-orlando> one for the ‘head’ snapshot and one for the icehouse snapshot
13:45:09 <salv-orlando> but since I’m only assuming what markmcmclain is suggesting, I will defer to him…
13:45:21 <markmcclain> salv-orlando: right
13:46:09 <jlibosva> oh yeah, that's a WIP. I left it there just in case. Once we agree where the healing will be performed, unnecessary models will be removed.
13:47:15 <markmcclain> ok so left move forward with healing at a single point in the timeline
13:47:21 <markmcclain> it removes lots of complications
13:47:34 <akamyshnikova> I've got one question for open discussion
13:47:43 <markmcclain> that leaves the question of the proper way to freeze the models
13:47:50 <HenryG> jlibosva: So we will remove icehouse.py and probably head.py then. Leaving just frozen.py
13:48:18 <akamyshnikova> head.py could be used in testing :)
13:48:45 <HenryG> akamyshnikova: true. Let's discuss that in the review comments.
13:49:21 <HenryG> Thanks markmcclain and salv-orlando
13:49:36 <HenryG> #topic Open discussion
13:49:52 <akamyshnikova> I've got one question.
13:49:55 <akamyshnikova> Just sync models with migrations shows the difference: one model uses �default� parameter, but in migrations it is used �server_default�.  https://review.openstack.org/82073 But there are a lot of usages of �default� is used in both models and migrations.
13:49:55 <akamyshnikova> The thing is: if we use �default�, not �server_default� we don�t have any default value in database. Do we need default value in db or not?
13:50:36 <akamyshnikova> markmcclain, salv-orlando I want to know your opinion about this
13:51:49 <HenryG> Based on problems encountered by colleagues when testing other Neutron features, I would say yes.
13:52:23 <salv-orlando> akamyshnikova: I’m trying to dig deep in my memory
13:52:42 <salv-orlando> there was discussion a lot of time ago about default and server_default
13:53:17 <salv-orlando> do I remember correctly that in order to ensure the default is set in the db schema for all backends supported by openstack you _must_ use server_default?
13:54:17 <akamyshnikova> if you want to have default value in db you should use server_default
13:54:47 <salv-orlando> ok then it’s a no brainer. Using default is just a bug
13:55:23 <akamyshnikova> So we need to replace all default with server_default?
13:55:23 <HenryG> Do we need to fix all the 'default' bugs before healing?
13:56:41 <akamyshnikova> I think that may be we should add correct values in frozen and then after heal migration change all models
13:57:08 <akamyshnikova> as if we change now we should write also migrations for this
13:58:26 <akamyshnikova> salv-orlando, what do think about my suggestion? does it sound ok?
13:58:33 <HenryG> akamyshnikova: hmmm. I think it would actually be better to fix before healing. Yes, a migration is needed, but that should not be a problem.
13:58:38 * markmcclain steps out for another meeting
13:59:05 <akamyshnikova> HenryG, ok, I'll do that :)
13:59:26 <salv-orlando> I’m not sure relying on the healing process is what we want to do. It is true that it’s always a problem of models diverging with schema
14:00:11 <HenryG> salv-orlando: yes, the healing's primary function should be to add missing tables.
14:00:23 <salv-orlando> but here we have a situation were the divergence will become evident if you rely on schema autogeneration, which I wish we’ll remove as soon a as possible too.
14:01:29 <akamyshnikova> salv-orlando, ok, thanks! I will do this. I just want to be sure that is better to do
14:02:16 <HenryG> salv-orlando: This week I plan to rebase the removal of auto-gen on the healing change, so it can go in soon after.
14:02:22 <salv-orlando> cool
14:03:07 <HenryG> We are out of time. Any other questions?
14:03:38 <akamyshnikova> no, I don't have any
14:03:41 <HenryG> Quick note on trivial change for devstack: https://review.openstack.org/105059
14:04:00 <HenryG> Not sure if it is actually needed. Please take a look.
14:04:28 <HenryG> Place your comments, if any, in the review.
14:04:35 <HenryG> Thanks everyone!
14:04:40 <HenryG> #endmeeting