17:00:10 <alaski> #startmeeting nova_cells
17:00:10 <openstack> Meeting started Wed Jun 17 17:00:10 2015 UTC and is due to finish in 60 minutes.  The chair is alaski. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:11 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:14 <openstack> The meeting name has been set to 'nova_cells'
17:00:21 <alaski> hi
17:00:25 <melwitt> o/
17:00:46 * bauzas waves
17:00:49 <belmoreira> o/
17:01:00 <alaski> #topic Tempest testing
17:01:10 <alaski> melwitt: you're up
17:01:37 <melwitt> since the last patch mentioned last meeting merged, the cells job is looking good: http://goo.gl/b7R8wq
17:01:59 <melwitt> it's been only failing on patches that are really failing tests
17:02:00 <bauzas> \o/
17:02:15 <alaski> awesome
17:02:16 * bauzas prepares to wave again
17:02:33 <melwitt> the patch to change it to voting is up at https://review.openstack.org/#/c/190894/ but we're waiting on some advice from mtreinish about whether we should add it to run on tempest changes as well
17:03:29 <melwitt> there's another question in the review on whether to run the job against devstack changes too
17:03:54 <melwitt> once we get those answers I'll update the patch and then it should be good to go
17:04:01 <alaski> excellent
17:04:03 <melwitt> I think that's it on the tempest testing status
17:04:08 <bauzas> \o/
17:04:12 <bauzas> (I said that)
17:04:16 <alaski> fwiw I think we should run on tempest, not sure about devstack though
17:04:32 <alaski> thanks for the update
17:04:45 <alaski> #topic Specs
17:05:00 <alaski> there are still a couple of specs up
17:05:08 <alaski> reviews always appreciated
17:05:15 <alaski> https://review.openstack.org/#/c/169901/ https://review.openstack.org/#/c/190147/
17:05:31 <dansmith> oh, I forgot to wave, but I'm here
17:05:53 <alaski> cool
17:06:03 <belmoreira> about #190147 and the migration
17:06:57 <belmoreira> to start use cellV2 we need to "signal" that someway and the operators will need to migrate the data...
17:07:22 <belmoreira> preferably with a nova-manage command
17:07:25 <bauzas> +1
17:08:18 <alaski> a strategy we could use would be to look up flavors in the new table, and fall back to the old one if they're not there
17:08:19 <belmoreira> should we have a global spec for data migration? since we need to create a common nova-manage
17:08:54 <belmoreira> alaski: but we should be in cellsV1 or cellsV2
17:09:08 <belmoreira> otherwise I think we add confusion
17:09:48 <johnthetubaguy> alaski: could we get those spec reviews into the etherpad, so they get higher priority, maybe they are already there? https://etherpad.openstack.org/p/liberty-nova-priorities-tracking
17:09:53 <alaski> I've been assuming we would have both for a period of time.  cellsv1 would have a single v2 cell as the parent cell
17:09:57 <alaski> johnthetubaguy: should already be there
17:10:04 <johnthetubaguy> oh wait, I saw them! my bad
17:10:27 <bauzas> yeah, that's the migration path, right
17:11:03 <bauzas> so, a tool for migrating flavors to cellsv2 API DB sounds reasonable, and doesn't confuse
17:11:10 <bauzas> even if people run cells V1
17:11:26 <bauzas> because both can interlace
17:11:57 <alaski> belmoreira: that's a good question on a global migration.  I could see a command to handle all of the individual migrations, but I think they should be separately runnable as well
17:12:13 <belmoreira> alaski: sorry, can we be more verbose about "cellsv1 would have a single v2 cell as the parent cell" :)
17:13:02 <alaski> your current top level cell would become a v2 cell, with api and cell.  so it would be a v1 parent cell and also a v2 api and cell
17:13:38 <alaski> the top level v2 cell would run v1 code to route message properly
17:13:54 <alaski> I should probably draw that out somehow
17:14:04 <bauzas> agreed
17:14:13 <belmoreira> alaski: ok, thanks
17:14:40 <alaski> #action alaski draw out how cellsv1 and v2 will interact during transition period
17:15:43 <bauzas> alaski: just for the records, I think the transition period can last a certain time :)
17:16:09 <alaski> belmoreira: back to your original question about a signal, I think if at all possible the code should work with migrated or unmigrated data
17:16:12 <belmoreira> Running the migrations separately could mean that I only migrate flavors and nothing else
17:16:31 <alaski> belmoreira: then the migration can be run at any time
17:16:55 <bauzas> shouldn't we consider to migrate the flavor information on the fly with the object ?
17:17:04 <bauzas> ie. when calling Flavor.save()
17:17:15 <bauzas> and leave the nova-manage tool for instant upload
17:17:55 <alaski> that's a possibility, and if we need to do that we can
17:18:07 <bauzas> yeah, would be far better
17:18:28 <alaski> but there's typically not much flavor data, and rarely saved, so doing it 'online' with a nova-manage command would be my preference
17:19:05 <bauzas> alaski: well, I care about the extra specs, but your point is fair
17:20:30 <belmoreira> bauzas: just for my understanding... calling Flavor.save() will be data migration? in that case only new/updates flavors will be added in the new table
17:21:17 <alaski> belmoreira: let me think about a global migration a bit more.  I was thinking we would migrate one table at a time, but that could be tedious for a deployer
17:21:31 <bauzas> belmoreira: that's a pattern for allowing objects to care about the migration by themselves, ie. we load either from source A or B, but at the end, we write on B
17:21:34 <alaski> belmoreira: but I also don't want to have to wait for all migrations to be done before we can run any for testing
17:21:53 <belmoreira> alaski: fair
17:22:50 <belmoreira> bauzas: I understand that... but for the particular flavor case it can took some time. no?
17:23:48 <bauzas> belmoreira: well, you hydrate your object from the Cells V2 API first, and if not found, from the regular nova DB, but you only save to the Cell V2 DB
17:24:01 <bauzas> cell V2 API DB I mean
17:24:12 <johnthetubaguy> alaski: so the bit I like here is being able to stagger the migration into small chunks so you can limit the DB load on production, a single command to do all the migrations would make life easier
17:24:13 <belmoreira> bauzas: ok
17:25:09 <alaski> johnthetubaguy: yeah, I envision a command to run all the subcommands
17:25:37 <alaski> and some things will be probably be handled by object saves, helping to spread the load
17:25:50 <johnthetubaguy> alaski: makes sense
17:26:23 <alaski> we can continue this on the flavor review, but it seems like we could also use something to discuss the high level migration plan on as well
17:26:58 <johnthetubaguy> alaski: the cells devref maybe?
17:27:19 <alaski> could be, similar to https://review.openstack.org/#/c/190209/
17:27:21 <johnthetubaguy> for a "once its all done" overview
17:27:39 <johnthetubaguy> alaski: +1
17:27:46 <johnthetubaguy> thats some good stuff there :)
17:27:53 <bauzas> moving on then ? :)
17:28:02 <belmoreira> for the flavor spec we will update it asap
17:28:09 <alaski> belmoreira: great
17:28:14 <alaski> bauzas: moving on
17:28:17 <johnthetubaguy> alaski: honestly, I am good with merging that, if it makes it easier to find
17:28:23 <alaski> #topic Open Discussion
17:28:32 <alaski> johnthetubaguy: I like it as a living document right now
17:28:39 <johnthetubaguy> alaski: cool
17:28:57 <belmoreira> https://review.openstack.org/#/c/192098/
17:29:07 <dheeraj-gupta-4> alaski: The second part of cell-ost-mapping spec impl. is at https://review.openstack.org/#/c/192098/
17:29:40 <alaski> cool, I started to go through that but didn't quite finish
17:30:16 <alaski> bauzas: what's the status on having a request_spec object?
17:30:32 <bauzas> alaski: on its way, good progress
17:30:52 <bauzas> alaski: the main discussion we currently have is how to manage the transition period
17:30:58 <alaski> great.  can I start working on persistence yet?
17:31:13 <bauzas> alaski: technically, you can take a look at the object
17:31:25 <alaski> okay
17:31:34 <bauzas> alaski: but I'm going to provide a new change, since I recently discovered that hints are API validated \o/
17:31:48 <alaski> ahh, cool
17:31:52 <bauzas> I was thinking we were waiting for strings and dict of strings, but that's not the case
17:31:57 <bauzas> https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/schemas/v3/scheduler_hints.py
17:32:24 <bauzas> so I can remove that ugly DictofStrings field which was making me uncomfortable and use a real Hints object
17:32:42 <alaski> nice
17:33:08 <alaski> I want to get that work going so I can play with a buildrequest object as well, though that spec hasn't merged yet
17:33:16 <bauzas> alaski: just be aware that I'm iterating over the changes very often
17:33:36 <bauzas> alaski: just because I'm discovering craps while I'm using that in the FilterScheduler
17:33:58 <alaski> bauzas: no problem, as long as there's some stability in at least the object name I can work with that
17:33:59 <bauzas> alaski: but at least, I think you can take the fields as example
17:34:16 <bauzas> the fields shouldn't really change, rather the hydrating methods
17:34:25 <alaski> okay, even better
17:34:30 <bauzas> to care all the corner cases
17:35:03 <bauzas> alaski: that's cool you're planning to work on persisting it
17:35:12 <bauzas> alaski: I was thinking it was a due to me
17:35:44 <alaski> I wrote the spec, so I figured I had volunteered
17:35:49 <bauzas> alaski: just one note, I'm planning to add a new field https://review.openstack.org/#/c/179224/11/specs/liberty/approved/check-destination-on-migrations.rst,cm
17:35:57 <bauzas> alaski: sure thing, that's cool
17:36:11 <bauzas> alaski: that will free up some time for me to make decent reviews then
17:36:14 <alaski> but I need it for the buildrequest stuff too so I'll work on it so I'm not blocked, and you can focus on the object itself
17:36:35 <bauzas> cool, that works
17:37:02 <alaski> anything else to discuss?
17:37:59 <alaski> thanks all!
17:38:04 <alaski> #endmeeting