09:00:42 <gsagie> #startmeeting dragonflow
09:00:43 <openstack> Meeting started Mon Feb  1 09:00:42 2016 UTC and is due to finish in 60 minutes.  The chair is gsagie. Information about MeetBot at http://wiki.debian.org/MeetBot.
09:00:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
09:00:46 <openstack> The meeting name has been set to 'dragonflow'
09:00:55 <gsagie> Hello everyone, who is here for the dragonflow meeting?
09:01:02 <dingboopt> o/
09:01:02 <gampel> Hi gal
09:01:04 <diga_> o/
09:01:05 <steve_ruan> hi gal
09:01:05 <oanson> Hi. I am.
09:01:08 <nick-ma_> Hi gal
09:01:20 <raofei> i'm here
09:01:34 <Shlomo_N> Hi
09:01:57 <gsagie> #info dingboopt, gampel, diga, steve_ruan_oanson, nick-ma, raofei, Shlomo_N in meeting
09:02:31 <gsagie> #info dingboopt, gampel, diga, steve_ruan, oanson, nick-ma, raofei, Shlomo_N in meeting
09:03:03 <gsagie> gampel: DuanKebo is coming? maybe contact him
09:03:43 <gampel> Ok i will
09:03:47 <raofei> His espace is offline
09:04:08 <gsagie> okie, hopefully he will join soon
09:04:10 <gsagie> hi yuli_s
09:04:15 <gsagie> #topic security groups
09:04:20 <yuli_s> Hello !
09:04:30 <gsagie> Raofei: when do you think we will be ready to start implementing this?
09:04:34 <yuli_s> da jia hao
09:04:41 <gsagie> after all the constraints are removed
09:04:49 <gsagie> from the controller side
09:05:01 <gsagie> dingboopt: here?
09:05:05 <dingboopt> yes
09:05:23 <gsagie> would you like to update what you have so far, i saw you made some very good progress
09:05:27 <raofei> We are startting implenttng controller app in local setup.
09:05:30 <gsagie> and even fixed some bugs on the way for router :)
09:05:52 <dingboopt> I think the next step is to implement the SG app
09:06:30 <gsagie> okie, so we make sure to merge and test any patches that are open and hope we can see the sg app in the upstream code soon
09:06:33 <gsagie> so we can all review it
09:06:46 <dingboopt> yes
09:06:52 <gsagie> #topic port security
09:06:59 <raofei> yes, we think so.
09:07:04 <gsagie> ok great
09:07:26 <gsagie> dingboopt, raofei: any update on this? i think we solved most problems in the spec
09:07:42 <gsagie> so everything should be clear to start implementing this, any open issues?
09:08:51 <gsagie> #action gsagie get port security review merged
09:08:53 <gsagie> #info https://review.openstack.org/#/c/263019/
09:08:55 <dingboopt> yes
09:09:07 <gsagie> ok, so no open issues
09:09:13 <raofei> yes. port security will be started after security group.
09:09:17 <gsagie> okie good
09:09:33 <gsagie> #topic DB consistency
09:09:49 <gsagie> nick-ma: want to update on this?
09:10:13 <gsagie> I think gampel would like us to have some discussion on this, mostly the spec you sent vs the journal implementation
09:10:14 <nick-ma_> i proposed the spec and also researched ODL journal. I think we do the same thing. the only difference is the implementation of distributed lock.
09:10:39 <gsagie> #info https://review.openstack.org/#/c/268005/    DB consistency spec
09:11:03 <gampel> In ODL they use the SQL DB as the lock ?
09:11:05 <nick-ma_> in ODL, distributed lock is implemented by a local update lock (select for update). in our spec, we try it in different way.
09:11:09 <nick-ma_> yes
09:11:44 <gampel> nick-ma_: Ok I will review it today
09:11:46 <gsagie> so if i understand right, the only difference is that in your spec we do full sync in case of a problem and in the journal its more ordered
09:12:21 <nick-ma_> yes. we have a full-sync mechanism.
09:12:28 <gsagie> of course we need to align some code for the journal to work, but its doable
09:13:06 <nick-ma_> we can reuse the journal db implementation as the distributed lock and incorporate it with our sync mechanism.
09:13:08 <gsagie> nick-ma: so you believe the approach you wrote in the spec is better then the journal?
09:13:40 <diga_> hey
09:13:46 <gsagie> hi diga!
09:14:01 <nick-ma_> we can stick to our spec.
09:14:12 <diga_> I have started on writting test for API, will send 1st patch today EOD
09:14:34 <gsagie> diga: ok, we arent in testing yet, but we will get there soon :)
09:14:41 <diga_> okay
09:15:00 <diga_> do you want me to start on other stuff ?
09:15:03 <gsagie> nick-ma: to be honest i am unclear what is the better approach
09:15:38 <nick-ma_> the journal is working there.
09:15:42 <gampel> I think that in the phase we should keep it simple
09:17:14 <nick-ma_> the problem is that we need to fix all the inconsistency problems at the mean time. it is not that simple.
09:17:27 <gampel> so maybe in the first step do the  journal ?  I am not sure i fully understand the differences  lets take it to our channel latter
09:17:52 <gsagie> nick-ma: how much time you think each solution should take
09:18:13 <gsagie> if you have time constraints maybe i can take some or help
09:18:51 <nick-ma_> you are working on the db sync, right?
09:18:53 <nick-ma_> gal
09:19:04 <gsagie> nick-ma: either me or oanson will finish it soon
09:19:31 <nick-ma_> i'll update you guys with my thoughts in email or review. we can start other topics then. :-)
09:19:48 <gsagie> okie
09:20:12 <gsagie> so lets put an action item just to pick one of the two options and start working on it
09:20:48 <gsagie> #action gsagie, nick-ma, gampel decide which is the better db consistency solution and start working on it
09:20:55 <nick-ma_> ok
09:21:01 <gsagie> #topic publish-subscribe
09:21:07 <gsagie> gampel: would like to update?
09:21:23 <gampel> yes
09:21:36 <gampel> I just updated the spec
09:21:41 <gampel> #info https://review.openstack.org/#/c/263322/
09:21:53 <gsagie> #info https://review.openstack.org/#/c/263322/   pub-sub spec
09:22:14 <gampel> it include reliability and plugability
09:22:17 <gsagie> So this spec talks about our pub-sub abstraction and i believe also the reliability mechanism between the pub-sub module and the local controllers
09:22:50 <gampel> yes it is missing the TOPIC support for the selective distribution
09:23:06 <gsagie> okie, so everyone please review it, we are going to merge it soon probably so if anyone notice any problem
09:23:18 <gampel> code is submitted without the reliability
09:23:19 <gsagie> i think we going to merge it with default True, so its working by default
09:23:21 <raofei> OK
09:23:33 <nick-ma_> ok
09:23:35 <gampel> #info https://review.openstack.org/#/c/263733/
09:23:37 <gsagie> so if you notice anything strange that might be relate to it, open a bug please on gampel ;)
09:23:48 <dingboopt> ok
09:23:58 <gsagie> j/k, you can assign it to me too :)
09:24:03 <gsagie> ok
09:24:24 <gsagie> gampel: are there any open issues there?
09:24:24 <gampel> please review the patches
09:24:47 <gampel> as we discussed today the pubsub use client side filtering
09:24:48 <gsagie> We need to update Duan also to point the right people into this patch
09:25:05 <gampel> currently in zmq
09:25:39 <gampel> so when used with the selective proactive messages will still reach the clients
09:25:40 <gsagie> we need to make sure to merge this with https://review.openstack.org/#/c/274336/
09:26:37 <gsagie> btw nick-ma, saw the migration patch, looks good to me, i added some reviewers from Neutron which knows these areas good and after their review we can merge it, but thanks for working on this
09:27:02 <gsagie> gampel: need to make sure to document this in the spec
09:27:08 <nick-ma_> ok.
09:27:12 <gampel> yes it is there
09:27:17 <gsagie> ok great
09:27:24 <gampel> in the pub-sub spec
09:27:33 <gsagie> anything else on this topic?
09:28:04 <gampel> No i hope to add the TOPIC registration support to the spec this week and finish the reliability part
09:28:04 <gsagie> okie, lets move on
09:28:10 <raofei> about the DNAT, although it's merged.
09:28:26 <gsagie> okie sounds good, lets just put action to manage it
09:28:29 <raofei> but for external network gateway MAC, your solution need statistic configuration
09:28:41 <gsagie> #action gampel finish pub-sub spec in regards to TOPIC registeration and reliability
09:28:53 <gsagie> raofei: lets talk about this now
09:28:55 <gsagie> #topic DNAT
09:29:06 <raofei> I have a suggestion/solution to get/update external gw mac dynamically
09:29:26 <gsagie> raofei: okie great, we had some ourselfs as well, can you send us in email and we can iterate on this?
09:29:28 <raofei> I added some commit to the merged patch last week
09:29:33 <raofei> OK
09:29:40 <gsagie> ohhh, didnt see it, i will take a look
09:29:52 <gsagie> #action gsagie look at distributed dnat spec comments
09:30:05 <raofei> So, for DNAT, do you think to raise a more detailed spec or just coding?
09:30:07 <gsagie> btw raofei, you can propose new patch with your ideas on top of the merged spec
09:30:32 <gampel> i didn't see it either sorry , please ping us next time
09:30:37 <raofei> OK
09:30:40 <gsagie> raofei: i think we can start coding, there is alot of things that are unrelated to this problem and we can start working on them, like
09:30:51 <gsagie> starting to bring all the floating ip data from the plugin to DF DB
09:30:59 <gsagie> and reading it in the controller
09:31:06 <gsagie> and then writing the application
09:31:31 <raofei> yes, I see
09:31:57 <gsagie> But its good that we also solve the case you mention, we do need to support dynamic change of MACs in the egress flows in case the MAC for external gateway changes, which might happen in active/standby cases
09:32:18 <gsagie> But we can continue discussing in parallel i know gampel had some idea to use learning flows
09:32:31 <gsagie> so we can discuss it
09:32:39 <gsagie> raofei: so you also start working on dnat?
09:32:41 <raofei> OK, agree
09:33:10 <raofei> yes. security group will be done mainly by dingbo
09:33:22 <gsagie> #action raofei start implementing distributed dnat
09:33:42 <gsagie> #action gsagie, gampel review raofei ideas and together solve the external MAC changes problem in the design
09:33:46 <gsagie> okie great
09:33:48 <gampel> Ok
09:33:56 <raofei> I hope we can complete some feature indenpendent in coding
09:34:36 <gsagie> raofei: yeah, hoping to see the DNAT patches soon, if you need help at any stage please let us know
09:34:36 <gampel> raofei: I am not sure I understand ?
09:35:06 <raofei> gampel, what's your doubt?
09:35:12 <raofei> sure.
09:35:28 <gsagie> #topic redis db
09:35:40 <gsagie> DuanKebo is not here, but there is a spec patch please help review
09:35:50 <gsagie> #info https://review.openstack.org/#/c/274340/
09:36:03 <gsagie> #topic selective proactive
09:36:34 <gsagie> For this we also need DuanKebo, but lets put an action item to speak about it next meeting, i will start doing some work for this
09:36:41 <gsagie> and we can all iterate on the spec
09:36:57 <gsagie> #info selective proactive spec https://review.openstack.org/#/c/273877/
09:37:22 <gsagie> We need to make sure to add "topics" registeration both for the DB and for the pub-sub APIs
09:37:39 <gampel> yes i will add it in another patch
09:37:43 <gsagie> but basically, its going to be the DB drivers responsbility to implement this
09:37:57 <gsagie> #action gsagie start working on OVSDB monitor thread
09:38:10 <gsagie> #action gsagie start working on adding tenant information from plugin
09:38:29 <gsagie> anyone has any question regarding this? i prefer we talk about it next week when Duan is here unless anything important
09:39:23 <gsagie> okie, if anyone has any question feel free to raise it in the channel
09:39:25 <gsagie> #topic testing
09:39:34 <gsagie> oanson: might sharing what are you working on ?
09:39:55 <steve_ruan> is "service injection " started?
09:39:59 <oanson> I'm looking into testing the ovs flows without setting up VMs
09:40:20 <oanson> The idea is to create a fast lightweight testing I/S for DF applications.
09:40:31 <gampel> steve_ruan: this is not targeted to mitaka
09:40:35 <gsagie> steve_ruan: hi, this is currently not in our priorities for this release
09:40:53 <oanson> Using e.g. ovs-appctl ofproto/trace, we can trace virtual (and generate real) packets through the flow.
09:41:01 <steve_ruan> @gampel @gsagie, thanks
09:41:06 <gsagie> steve_ruan: but certainly its the next topic we want to tackle, and i think provide alot of extra value, so we might start this in parallel
09:41:27 <gsagie> steve_ruan: if you guys are actually planning to use it, it might be an important feedback for us and we can give it more priority
09:41:42 <gsagie> oanson: cool
09:42:00 <steve_ruan> thanks
09:42:03 <gsagie> so everyone can build a simulated tests based on oanson framework to test applications
09:42:15 <oanson> That's the plan.
09:42:22 <gampel> oanson:  I think this will improve our CI testing  and allow application level testing Ping/DHCP etc
09:42:23 <gsagie> which can be good for the security groups and the dnat apps
09:42:31 <oanson> I will create a (hopefully) simple API for others to rely upon.
09:42:41 <gsagie> yuli_s: want to share your update regarding OVS flows tests and fullstack tests?
09:42:53 <Shlomo_N> Sound great. Good job oanson.
09:43:15 <yuli_s> gsagie: sure,
09:43:16 <gsagie> I think in general we want everyone that adds a new feature/code from now on to also add tests unit/fullstack/functional
09:43:37 <yuli_s> I commited the DHCP tests
09:44:03 <gsagie> #info DHCP tests done by yuli_s https://review.openstack.org/#/c/273548/
09:44:03 <yuli_s> fixing a number of issues found during review
09:44:23 <yuli_s> so, hope it will go in master code today
09:44:42 <gsagie> yuli_s: ok good, and next you addressing the destructor problem right?
09:44:52 <yuli_s> I also commited class I used to create VMs
09:45:18 <yuli_s> so, I will be able to commit more tests soon
09:45:21 <gsagie> we also have these APIs tests that we want to do scale/performance testing for many API calls on the Neutron server
09:45:26 <gampel> #info https://review.openstack.org/#/c/273595/
09:45:59 <gsagie> #action yuli_s fixing the current fullstack tests destructor problem
09:46:18 <gsagie> #action yuli_s perform API performance testing plan
09:46:38 <gsagie> ok good job yuli_s, we do need you to start in parallel on the API/DB scale thinking as well
09:46:51 <yuli_s> ok
09:47:05 <gsagie> because we are testing different solutions and its important we have something that can verify we dont hurt performance at each patch
09:47:07 <gsagie> okie
09:47:20 <gsagie> Shlomo_N: would like to update regarding scale tests for data plane?
09:47:32 <Shlomo_N> sure
09:47:45 <gsagie> diga: you going to submit some unit tests soon?
09:47:56 <gsagie> think you mentioned that in the start of the meeting :)
09:48:26 <Shlomo_N> I am currently trying to setup a testing env. with DVR with Liberty. Have few problems in setup.
09:49:24 <Shlomo_N> I have uploaded a performance testing spec https://review.openstack.org/#/c/272559/.
09:49:26 <gampel> Shlomo_N: you have initial  result for DF
09:49:53 <Shlomo_N> Yes, already shared with you the results.
09:49:53 <gsagie> Shlomo_N: okie, lets take this offline then, i saw DuanKebo sent you the details of the person in Beijing that does
09:50:02 <gsagie> scale tests
09:50:09 <Shlomo_N> ok
09:50:46 <gsagie> okie, anyone else would like to discuss about the tests?
09:50:48 <gampel> All please try not to avoid  trailing white spaces in the spec and avoid more then 80 char line length
09:51:01 <gsagie> #topic open discussion
09:51:13 <gsagie> steve_ruan: i would like to talk with you about the use case
09:51:57 <gsagie> steve_ruan: the question is how seriously you guys need it :) because we also have some internal use cases that needs this and
09:53:48 <gampel> We need to stabilize our CI it fails some times randomly (the none voting)
09:54:19 <gsagie> okie
09:54:26 <gsagie> We need to think when its time to make it voting
09:54:30 <gsagie> and which one :)
09:54:52 <gsagie> anyway steve_ruan: we can discuss this after this meeting and see what we can do to help with this use case
09:55:05 <gsagie> #topic bugs
09:55:11 <gsagie> anything special here?
09:55:19 <oanson> About the non-voting CIs, I think we should wait for yuli_s's destructor fix. I see some strange race issues there.
09:55:35 <gampel> Ok
09:56:06 <gsagie> oanson: yes i agree, but not all tests are related to these, we can think about making tempest voting
09:56:40 <gsagie> but lets postpone this call to the next meeting, as you said its not stable enough
09:57:02 <gsagie> #action oanson, yuli_s help make the fullstack tests stable
09:57:20 <gsagie> Thanks everyone for joining :)
09:57:26 <Shlomo_N> Thank you
09:57:31 <nick-ma_> thank you.
09:57:33 <gampel> Thank you
09:57:37 <oanson> Thank you.
09:58:07 <gsagie> #endmeeting