09:00:35 <anteaya> #startmeeting nova-net-to-neutron-migration
09:00:35 <openstack> Meeting started Tue Jan 27 09:00:35 2015 UTC and is due to finish in 60 minutes.  The chair is anteaya. Information about MeetBot at http://wiki.debian.org/MeetBot.
09:00:36 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
09:00:39 <openstack> The meeting name has been set to 'nova_net_to_neutron_migration'
09:00:47 <anteaya> jlibosva: hello
09:00:48 <spandhe> hello!
09:00:50 <anteaya> hi obondarev
09:00:56 <anteaya> this is spandhe
09:01:01 <obondarev> anteaya: hi
09:01:11 <anteaya> I met her today and she works with yahoo's deployment
09:01:22 <anteaya> so she is attending today's meeting
09:01:29 <anteaya> anyone else here for the migration meeting?
09:01:34 <obondarev> great!
09:01:36 <spandhe> Hi obondarev !
09:01:47 <anteaya> gus: you here?
09:01:51 <obondarev> spandhe: nice to meet you :)
09:01:59 <anteaya> belmoreira: ping
09:02:19 <anteaya> let's look at today's agenda
09:02:23 <anteaya> #link https://wiki.openstack.org/wiki/Meetings/Nova-nettoNeutronMigration
09:02:34 <spandhe> obondarev: I have tried your earlier patch in Yahoo (with few changes) and have tested out successful live migration. I am trying to see thr new approach now.
09:02:35 <belmoreira> hi, I'm late...
09:02:42 <anteaya> belmoreira: welcome
09:02:50 <anteaya> spandhe: let's follow our agenda
09:03:00 <anteaya> spandhe: then we can have a chance to discuss
09:03:00 <obondarev> spandhe: cool!
09:03:03 <spandhe> anteaya: ok..
09:03:06 <anteaya> #topic he state of the Neutron spec (obondarev)
09:03:09 <anteaya> spandhe: thanks
09:03:18 <anteaya> so the specs first
09:03:22 <anteaya> obondarev: take it away
09:03:23 <obondarev> so for the specs
09:03:33 <obondarev> just saw an activity on the specs this morning
09:03:39 <anteaya> \o/
09:03:42 <obondarev> most probably related to the discussion at nova midcycle
09:03:51 <anteaya> it was yes
09:03:53 <obondarev> gus has uploaded new revisions on the specs so he probably knows more than me :)
09:04:12 <anteaya> gus: pingity ping ping
09:04:22 <obondarev> I'm going to go through them and left comments, if any
09:04:31 <anteaya> obondarev: wonderful thank you
09:04:43 <obondarev> anteaya: are you at the midcycle?
09:05:07 <anteaya> obondarev: I am, I am only there tomorrow morning and then I have to leave for the cinder mid-cycle
09:05:16 <anteaya> the two overlap
09:05:28 <obondarev> anteaya: ok, great!
09:05:28 <anteaya> so we discussed the specs today
09:05:50 <anteaya> and there is agreement to solve the simplest use case, as we have been working to do
09:06:12 <anteaya> and I would really like to get at least the first part of the two part spec merged this week
09:06:23 <anteaya> so that is this patch
09:06:27 <anteaya> #link https://review.openstack.org/#/c/147723/4
09:06:34 <anteaya> I think we are close
09:06:57 <anteaya> so if folks can review that then hopefully we can get that polished and finished this week
09:07:01 <anteaya> that is my goal
09:07:14 <obondarev> anteaya: is Salvatore at the midcycle as well?
09:07:24 <anteaya> obondarev: he was there today
09:07:31 <anteaya> and then had to go to vmware meetings
09:07:35 <obondarev> I'm asking because one of the comments from Salvatore was his concern on whether we need a proxy in nova at all.
09:07:45 <anteaya> hopefully he will be there tomorrow before I leave
09:07:52 <obondarev> As I started to work on it I'd really like this concerns to be sorted asap :)
09:07:53 <anteaya> obondarev: yes, I saw that
09:07:59 <anteaya> obondarev: yes
09:08:15 <anteaya> obondarev: can you submit your code to gerrit, so folks can look at it?
09:08:16 <obondarev> good to know he's there
09:08:22 <anteaya> he is, yes
09:08:27 <anteaya> I was happy to see him
09:08:28 <obondarev> yeah. I'll try to put a WIP on gerrit this week
09:08:35 <obondarev> hopefully this will be something that makes sense and not a complete waste of reviewers time
09:08:36 <anteaya> can you do that today please?
09:08:44 <anteaya> anything is a start
09:08:59 <anteaya> it is frustrating for me to tell people there will be code and there is no code
09:09:00 <obondarev> not sure for today
09:09:13 <anteaya> just submit what you have
09:09:19 <anteaya> it doesn't have to be perfect
09:09:26 <anteaya> but anything is better than nothing
09:09:36 <obondarev> yeah, I know, I'll try
09:09:42 <anteaya> it really holds up discussions when folks have nothing to look at
09:09:44 <anteaya> thank you
09:10:07 <anteaya> and progress on the second patch of the spec hinges on folks being able to see code
09:10:36 <anteaya> so we need to have some code to make any progress on https://review.openstack.org/#/c/142456/ in a meaningful way
09:10:40 <anteaya> #link https://review.openstack.org/#/c/142456/
09:11:01 <anteaya> obondarev: seeing what you have written may answer some of salv-orlando's concerns
09:11:16 <obondarev> anteaya: I see
09:11:30 <anteaya> obondarev: thanks, so anything you have is great
09:11:48 <anteaya> if folks make you feel unhappy for putting something up in wip state, then let me know
09:12:05 <anteaya> since you should be supported by showing unfinished coce
09:12:07 <anteaya> code
09:12:15 <anteaya> and if you don't, I will address it
09:12:27 <obondarev> thanks anteaya
09:12:31 <anteaya> thank you
09:12:45 <anteaya> obondarev: and yes, I want your efforts to be valued, not wasted
09:13:05 <obondarev> anteaya: appreciate that :)
09:13:11 <anteaya> so without gus, I'm not sure what else we need to talk about regarding the specs
09:13:15 <anteaya> obondarev: :)
09:13:24 <anteaya> does anyone else have anything for specs?
09:13:28 <jlibosva> nope
09:13:29 <anteaya> or shall we move on?
09:13:45 <anteaya> okay great, please review them as soon as you are able
09:13:57 <anteaya> moving on
09:14:07 <anteaya> #topic the state of implementation (obondarev)
09:14:19 <obondarev> so we already started to talk a bit about implementation I guess
09:14:57 <obondarev> will try to put a wip on proxy asap
09:15:03 <anteaya> thank you
09:15:18 <obondarev> I see jlibosva has progress on his DB migration patch
09:15:24 <obondarev> jlibosva: can you please update?
09:15:34 <anteaya> #link https://review.openstack.org/#/c/148260/
09:15:41 <jlibosva> I'm gonna send new PS soon. I'm rewriting the code structure to be more declarative
09:15:51 <anteaya> jlibosva: great thank you
09:15:53 <obondarev> jlibosva: thanks
09:16:06 <jlibosva> every neutron table has its own definition how the table should be populated
09:16:27 <jlibosva> currently I'm in state of getting ports and fixed/floating ips populated and that's the last part
09:16:36 <jlibosva> so ETA this week I think prototype should be ready
09:16:43 <obondarev> jlibosva: perfect!
09:16:48 <anteaya> jlibosva: wonderful work, thank you
09:17:04 <belmoreira> jlibosva: great
09:17:09 <anteaya> spandhe: did you have any more comments for obondarev?
09:17:55 <spandhe> anteaya: I have questions about the previous spec.. mostly because I wasnt involed earlier.. I can talk to him later too...
09:18:15 <anteaya> spandhe: doens't hurt to ask now
09:18:27 <anteaya> since this is logged it would be helpful to ask
09:18:32 <obondarev> spandhe: you mean juo spec, right?
09:18:37 <obondarev> juno*
09:18:50 <spandhe> anteaya: ok cool.. obondarev, have questions on kilo specs
09:19:08 <obondarev> spandhe: ah, ok
09:19:33 <spandhe> are we not planning to do a live migration? I thought you kind of achieved that in Juno spec.. or at least we did using that code
09:19:59 <spandhe> seems like both the APIs are down during the db migration period.
09:20:22 <spandhe> Is my understanding correct?
09:20:41 <obondarev> spandhe: right, the first iteration does not include live migration
09:20:56 <obondarev> spandhe: but API are partially down
09:21:18 <obondarev> spandhe: Nova API is down during only initial db migration
09:21:43 <obondarev> spandhe: then it's up again
09:21:58 <spandhe> obondarev: ok.. whats initial db migration? Also, once the db is migrated, where do new VMs come up? in nova-net land or neutron land?
09:22:29 <obondarev> spandhe: so after the db migration the source of thruth will be in neutron db
09:22:43 <obondarev> truth*
09:23:00 <spandhe> obondarev: ok.. and nova is the active API and the calls are proxied to neutron API, am I right?
09:24:01 <obondarev> spandhe: nova is the active API, calls are operated by nova-network which will go for data to neutron DB
09:24:10 <obondarev> spandhe: using special proxy in nova
09:25:09 <spandhe> obondarev: ok thanks.. I am not sure what this special proxy is.. does it help nova in accessing neutron DB? because usually nova user wont have acess to neutron db..
09:26:05 <obondarev> yep, it does, it should go to neutron to get data
09:26:35 <obondarev> after that we can start The big migration loop
09:26:58 <spandhe> obondarev: ok.. thanks! I got the gist now.. I can review the specs and the db migration script..
09:27:14 <obondarev> spandhe: great! thank you!
09:27:24 <anteaya> spandhe: thanks for asking
09:27:33 <anteaya> so implementation
09:27:47 <anteaya> does anyone have anything more on implementation?
09:27:52 <jlibosva> nope
09:27:58 <obondarev> not from me
09:28:00 <spandhe> anteaya: thanks for giving me the opportunity :)
09:28:13 <anteaya> spandhe: glad to have your participation
09:28:17 <anteaya> okay moving on
09:28:23 <anteaya> #topic documentation
09:28:37 <anteaya> obondarev: you did your part here, thank you
09:28:43 * gus apologises for arriving late.
09:28:44 <obondarev> not much to update on the docs side
09:28:51 <anteaya> yes
09:28:56 <anteaya> actually let's go back
09:28:59 <anteaya> #undo
09:29:00 <openstack> Removing item from minutes: <ircmeeting.items.Topic object at 0x1ffd090>
09:29:06 <anteaya> gus: hello
09:29:21 <anteaya> gus: anything to share in the specs/implementation topics?
09:30:14 <gus> I put up a new revision a few hours ago that addresses all the comments so far.
09:30:38 <anteaya> gus: yes thank you
09:30:42 <gus> In particular, it now says that we won't deal with cells in this spec, but we expect that to be a followup.
09:31:10 <jlibosva> gus: i.e. in L cycle?
09:31:16 <anteaya> belmoreira: we talked this morning about that
09:31:36 <anteaya> belmoreira: and understanding that to move forward we need to focus on the simplest use case
09:31:55 <anteaya> but I am talking with alaski who is one of the authors of cells
09:32:13 <anteaya> and working on figuring out what we can do for your situation, belmoreira
09:32:19 <gus> jlibosva: It's possible, but I doubt it.  It will need some more discussion/involvement from interested parties to make sure we do it in a way that is most useful, and we'll need some much better testing environments than we typically have available (devstack won't cut it).
09:32:23 <belmoreira> anteaya: thank you
09:32:31 <anteaya> whether it is within this initiative or outside it
09:32:41 <anteaya> belmoreira: we are thinking of you, by all means
09:33:07 <anteaya> we just need to figure out how to keep the migration path moving forward as well, as it has stalled in teh past
09:33:28 <jlibosva> gus: makes sense. thanks
09:33:28 <anteaya> so in short, the spec is saying simplest use case
09:33:48 <anteaya> we are providing some tools which we invite deployers to take and expand to fit their needs
09:34:16 <anteaya> folks who work with us and have specific needs, such as cern and cells, we will work with to find a path forward
09:34:26 <anteaya> but that won't make everyone happy
09:34:44 <anteaya> so we are telling folks that we know that
09:34:50 <anteaya> to set reasonable expectations
09:35:16 <anteaya> make sense?
09:35:34 <belmoreira> anteaya: I think is very reasonable
09:35:39 <spandhe> totally
09:35:40 <anteaya> belmoreira: thank you
09:35:45 <anteaya> spandhe: great thank you
09:35:54 <anteaya> gus: thanks for your input
09:36:00 <anteaya> any more now that gus is here?
09:36:05 <anteaya> or back to documentation?
09:36:23 <anteaya> moving on
09:36:32 <anteaya> #topic documentation
09:36:40 <anteaya> back again
09:36:55 <anteaya> so obondarev thank you here, emagma has said he will lead this
09:37:09 <anteaya> the difficulty is the meeting time, which I don't expect him ever to make
09:37:11 <obondarev> anteaya: awesome!
09:37:24 <jlibosva> anteaya: where is he located?
09:37:28 <anteaya> I get up in the middle of the night but I don't expect others from north america to do so
09:37:31 <obondarev> anteaya: indeed, we talked about it with him too
09:37:36 <jlibosva> ah
09:37:37 <anteaya> jlibosva: in texas I think
09:37:58 <anteaya> and I got an email from somone named chris cannon who also said he wants to help
09:38:09 <anteaya> which is awesome but he is in north america as well
09:38:11 <obondarev> anteaya: so if we can have an alternate time for this meeting that would be good I think
09:38:20 <obondarev> just like we have for neutron team meeting
09:38:25 <anteaya> so I'm sill looking for somone to attend meetings
09:38:32 <anteaya> obondarev: I'd like to avoid that acutally
09:38:53 <anteaya> as disruption in the meeting time reduces effectiveness for the transition period
09:39:04 <obondarev> anteaya: I see
09:39:06 <anteaya> this time works for russia, australia and eu
09:39:11 <anteaya> which is why I picked it
09:39:27 <anteaya> let's just proceed and hope someone from docs is able to attend meetings
09:39:28 <obondarev> anteaya: should we find a compromise then?
09:39:55 <anteaya> obondarev: do you want to ping emagma and ask him what he needs to get started on the docs process?
09:40:12 <anteaya> I think just ensuring somone talks to emagma is our best way forward
09:40:21 <anteaya> obondarev: if you would do that, that would be great
09:40:26 <obondarev> anteaya: agree
09:40:30 <obondarev> anteaya: sure I can
09:40:35 <anteaya> great, thank you
09:40:47 <anteaya> anything more for docs?
09:40:58 <anteaya> #topic testing (belmoreira)
09:41:09 <anteaya> belmoreira: anything to say here?
09:41:29 <belmoreira> As ops point of view for now I don't have nothing
09:41:34 <anteaya> okay great
09:41:37 <anteaya> I'll proceed
09:41:40 <jlibosva> I have something to discuss/think about
09:41:51 <anteaya> oh okay, jlibosva go ahead
09:42:00 <jlibosva> Do we plan to test particular tools?
09:42:06 <anteaya> good question
09:42:10 <anteaya> let me share something
09:42:15 <jlibosva> as I was working with db things, I think some real big bunch of data would be helpful
09:42:27 <anteaya> jlibosva: ah I see
09:42:43 <anteaya> jlibosva: well mikal and jhesketh do test data
09:42:55 <anteaya> with their db datasets ci
09:43:21 <anteaya> jlibosva: so it might be possible to chat with jhesketh to see how they work with real data
09:43:30 <jlibosva> anteaya: is it the turbo-hipster or something else?
09:43:33 <anteaya> and utilize their approach if possible
09:43:40 <anteaya> turbo-hipster
09:43:43 <jlibosva> ok, I'll try to reach out to him
09:43:50 <anteaya> which has the name db datasets ci
09:44:03 <anteaya> jlibosva: thank you, I think that would be a good first step
09:44:15 <anteaya> and at the nova midcycle we discussed testing
09:44:19 <jlibosva> anteaya: ok, thank you too for the info
09:44:28 <anteaya> jlibosva: sure
09:44:48 <gus> mikal is at nova midcycle, jhesketh is still here in .au if you want me to ask him tomorrow.
09:44:48 <anteaya> and one suggestion is to package up the tools and let deployers test them
09:44:55 <anteaya> and incorporate feedback
09:45:07 <gus> (I feel bad I have no idea how my colleagues' tool works)
09:45:08 <anteaya> gus: yes thank you that would be awesome
09:45:15 <anteaya> gus: ha ha ha
09:45:26 <gus> will do.
09:45:28 <anteaya> working in -infra I had to get over that long ago
09:45:31 <anteaya> :)
09:46:15 <anteaya> so belmoreira you mentioned this morning/yesterday to me that you could test the db migration and proxy once you see some code
09:46:19 <anteaya> belmoreira: yes?
09:46:52 <anteaya> spandhe: would you be able to do the same?
09:46:53 <belmoreira> the proxy it will be easier to test
09:47:04 <anteaya> belmoreira: great, I'll take what I can get
09:47:04 <gus> testing is going to be awkward generally for this project, given that the usual devstack/grenade tests aren't so useful (multi-node devstack testing isn't quite ready yet).
09:47:07 <spandhe> anteaya: yup..
09:47:16 <anteaya> spandhe: great thank you
09:47:32 <belmoreira> data migration, I'm still checking how can we do it
09:47:41 <anteaya> gus: yes, I think we are going to have to stumble through testing until we reach a happy place amoungst ourselves
09:47:54 <anteaya> belmoreira: great, well don't keep your findings a secret
09:48:15 <anteaya> belmoreira: once you learn anything, do share so we can figure out how to get the support to make it happen
09:48:38 <anteaya> gus: and yes multi-node did come up today
09:48:42 <belmoreira> anteaya: sure
09:48:57 <anteaya> gus: which we are working on but yes, isn't there yet
09:49:00 <anteaya> belmoreira: thank you
09:49:38 <anteaya> so for testing, basically we have to have some code and then try some things and share what we tried and keep going until we decide it is good enough
09:49:58 <anteaya> which isn't great for satisfying the spec, but I don't know what else to suggest at this point
09:50:16 * jhesketh is here
09:50:23 <anteaya> hello jhesketh
09:50:27 <jhesketh> (trying to catchup on context)
09:50:41 <anteaya> jhesketh: the tl;dr is we would like your input on testing data migrations
09:51:04 <anteaya> jhesketh: you being the turbo-hipster dude, we were hoping you might have some wisdom to share
09:51:28 <jhesketh> tl;dr answer would be it's worth doing
09:51:37 <anteaya> jhesketh: that is good
09:51:45 <anteaya> and worth knowing thank you
09:51:48 <jhesketh> not necessarily as a 3rd party CI, but at the very least before you upgrade so you know how long it'll take
09:51:51 <gus> want to be able to test a db migration tool on largish amounts of data.  wondering what your experience is re getting access to that, privacy, etc.
09:52:25 <jhesketh> turbo-hipster is designed in ways to make it difficult to get data out, I did a talk on that at the summit
09:52:27 <jhesketh> let me find a link
09:52:46 * jlibosva remembers the talk from Atlanta
09:52:49 <jhesketh> https://www.youtube.com/watch?v=VeoUpmPim8c
09:52:55 <jhesketh> oh, well, not much has changed
09:53:04 <anteaya> #link https://www.youtube.com/watch?v=VeoUpmPim8c
09:54:08 <anteaya> so jlibosva perhaps you can watch/re-watch that video and follow up with jhesketh if you have questions?
09:54:24 <anteaya> and of course, anyone else as well
09:54:44 <anteaya> jhesketh: thank you
09:54:49 <gus> if we have large nova-net volunteers, then we could set up a similar 3rd party CI migration test.  We have the additional complication that verifying the neutron version matches the nova-net version is itself an interesting exercise.
09:55:06 <jlibosva> jhesketh: I reserve a slot in your time schedule :)
09:55:09 <anteaya> true
09:55:32 <anteaya> so we have more to discuss on the testing front
09:55:54 <anteaya> let's all keep that in mind as we move forward and bring questions and comments to next week's meeting
09:55:56 <anteaya> fair?
09:55:59 <gus> we should be able to do a round trip: migration tool -> nova-neutron-proxy -> back to something that matches the same nova API.
09:56:01 <jlibosva> I have still question, how to validate neutron db is correct
09:56:17 <anteaya> jlibosva: ah, another good question
09:56:26 <anteaya> gus: that sound reasonable
09:56:27 <jlibosva> specifically vif migrations
09:56:27 <gus> we'll need additional testing from the neutron pov however.
09:57:07 <anteaya> hmmmm, more thought needed here
09:57:29 <anteaya> can we capture these thoughts, questions in the spec comments at all?
09:57:37 <jlibosva> ack
09:57:45 <anteaya> #link https://review.openstack.org/#/c/142456/
09:57:46 <gus> jlibosva: I can't think of any magic answers here - I think we'll just need to add the usual unittests to cover cases we foresee while writing the code as best we can.
09:58:15 <anteaya> gus: yes, let's start with what we can do and expand from there
09:58:37 <jlibosva> gus: I don't think UT would be useful here as it validates implementation. We're more interesting in making sure the actual mapping between two databases is correct. The implementation is not that crucial
09:59:15 <anteaya> jlibosva: is there anyone else doing what you think needs to be done?
09:59:25 <anteaya> can we copy a current effort at all?
09:59:42 <anteaya> time to wrap up
09:59:42 <gus> jlibosva: Agreed.  I don't know how to do that without writing some other new code that will be based on the same possibly incorrect assumptions however :/
10:00:04 <anteaya> let's comment in the testing section of https://review.openstack.org/#/c/142456/8/specs/kilo/migration-from-nova-net.rst
10:00:08 <jlibosva> yeah
10:00:17 <anteaya> and talk more about testing next week
10:00:21 <jlibosva> anteaya: I was just thinking loud, I'll try to bring it to specs
10:00:28 <anteaya> please review the latest spec
10:00:35 <anteaya> jlibosva: of course, and thank you
10:00:38 <anteaya> :)
10:00:47 <anteaya> great meeting everyone, thank you!
10:00:52 <anteaya> see you next week
10:00:54 <gus> yep, suggest things for the spec and I'll attempt to merge them.
10:00:58 <anteaya> #endmeeting