14:03:30 <ajo> #startmeeting neutron_qos
14:03:31 <openstack> Meeting started Wed Jul 15 14:03:30 2015 UTC and is due to finish in 60 minutes.  The chair is ajo. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:03:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:03:36 <openstack> The meeting name has been set to 'neutron_qos'
14:03:40 <ajo> #topic status update from the trenches
14:04:35 <ajo> So, we're advancing a bit slow, but it's nice to see more testing coming from all directions
14:04:39 <ajo> and more patched merged.
14:04:54 <ajo> Still, the important ones are taking more time than expected,
14:05:08 <ihrachyshka> yeah, I shout "service plugin!!"
14:05:18 <ihrachyshka> but hopefully, we'll get it today. I'll do everything for that
14:05:28 <ajo> and we just found that using the callbacks to extend introduced a DBDeadlock, we're partly lucky that it was triggered twice in a row
14:06:05 <ajo> digging into mixinless core-resource extension
14:06:19 <ajo> we also found that relying directly on callbacks is a messy mechanism
14:06:43 <moshele> ajo: this is the etherpad for qos sync https://etherpad.openstack.org/p/qos-sync I think echo one should add his tasks so we can keep track on everything
14:06:59 <ajo> and we must provide a better interface, similar to ml2 extension drivers, but more for the general neutron, I talked with mkolesni about it... not to be fully done for neutron now, that needs broader discussion,
14:07:19 <ajo> but he will do something in the middle, which can be consumed by the ML2 extension drivers (with an adaptor) and by other plugins
14:07:26 <irenab> ajo: can you clarify this one?
14:07:29 <ihrachyshka> moshele, ok, I'll update the etherpad. I didn't know about that one
14:07:42 <ajo> #link https://etherpad.openstack.org/p/qos-sync
14:07:46 <ajo> sure irenab ,
14:07:50 <ajo> let me explain the thing
14:08:19 <moshele> ihrachyshka: thanks
14:08:22 <ajo> extending a core resource, doing it with louse coupling, requires "something" to handle a few things
14:08:40 <ajo> 1) extending the core resource attributes (what we already do with AFTER_READ)
14:09:02 <irenab> someting  = plugin that handles the esource extenision, right
14:09:22 <irenab> resource
14:09:22 <ajo> 2) tracking core resource updates to keep track of such extended atrtibutes in DB
14:09:37 <ajo> in this case , the bindings to policies of networks and ports
14:09:41 <ajo> yes, in this case
14:09:45 <ajo> resource = networks / ports
14:09:59 <ajo> and something (right now) is the service plugin
14:10:08 <ajo> after looking at...
14:10:14 <ajo> the ml2 extension drivers, let me link to int
14:10:16 <ajo> int->it
14:11:09 <ajo> #link https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/driver_api.py#L893
14:11:15 <ajo> after looking to that,
14:11:18 <ajo> an example is this:
14:11:31 <ajo> https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/extensions/port_security.py
14:11:38 <ajo> they're basically doing that, but only within ml2
14:11:38 <gcossu> nice link
14:12:03 <irenab> ajo: what I am missing is what is not already handled by current implementation
14:12:11 <ajo> basically, they came to almost the same interface we're finally exposing in the service, via callbacks
14:12:26 <irenab> We skip ML2 extension, since resource is extended by qos plugin
14:13:03 <irenab> ajo: so what you want is to add Resource extension interface?
14:13:04 <irenab> to be implemented by QoS plugin?
14:13:11 <ajo> mkolesni is thinking about generalizing the resource extension, and decouple that from the plugin
14:13:22 <ajo> for now, he's going to provide that in the same file,
14:13:29 <ajo> and make it available to all plugins
14:13:41 <ajo> so you have a common class to call from your plugin
14:13:48 <ajo> to update_network/port create_network/port
14:13:53 <ajo> and get the bindings done/undone
14:14:04 <ajo> and also, put the dict extension in that class
14:14:26 <irenab> ajo: the rational to eanble ML2 to call QoS plugin via this interface?
14:14:49 <ajo> irenab: create a ml2 extension which will call this class
14:14:59 <ajo> and other plugins, can call this class
14:15:04 <ajo> at a later time, in neutron
14:15:12 <ajo> we will need to find the right way for plugins not needing to do such thing
14:15:16 <ajo> (manually)
14:15:37 <ajo> just say "I support this resource extension", yey...
14:15:37 <irenab> ajo: I think you jump into technical details before explaining what you want to solve :-)
14:16:15 <ajo> irenab, it's mike idea's actually, I was refusing to listen this morning, but he's so convincing...
14:16:20 <ajo> and I believe he's right
14:16:38 <moshele> I feel we need another code sprint :)
14:16:39 <ihrachyshka> ajo, why isn't he on the meeting?
14:16:42 <irenab> What is missing with current callback AFTER_READ?
14:16:47 <ihrachyshka> he should have convinced all of us!
14:16:52 <ajo> he just wants to move all the resource extension/extension tracking out of the class
14:16:58 <ajo> into a separate one
14:17:06 <ajo> I think that's neat, and ends in a cleaner plugin
14:17:17 <ajo> and it's something we can reuse neutron wide later
14:17:26 <irenab> ajo: the change that was done for ML2Plugin is absolutly no-go
14:17:28 <ajo> but one step at a time
14:17:35 <ajo> irenab: correct
14:17:43 <irenab> so I assumed we trying to resolve this
14:17:49 <ajo> correct
14:17:55 <ajo> this, and more :)
14:18:07 <ajo> as we solve this, we want to clean up the way we extend resources
14:18:10 <ihrachyshka> ajo, can we take more of this and less of more?
14:18:15 <irenab> lso to resolve this (without more) what is missing?
14:18:35 <ihrachyshka> irenab, I like your 'result first' approach
14:18:44 <irenab> ihrachyshka: :-)
14:18:58 <irenab> more of: lets keep focused
14:19:13 <ihrachyshka> I'm fine with good design as long as we stick to schedule (that we don't have, and I blame our Lord Miguel for not guiding us!)
14:19:15 <ajo> I agree, I couldn't convince mkolesni to do less of more, but I convinced him to do something very thin
14:19:57 <irenab> ajo: ca you share the plan, I still do not understand what was missing so it required to “polute” ML2 plugin
14:20:10 <ajo> irenab, you will see it in next patch.
14:20:24 <ihrachyshka> yes, please make communication explicit. you talk via phone, you have chats in redhat irc...
14:20:26 <ajo> basically it's putting the relevant parts of the ml2 extension driver available in qos_plugin
14:20:31 <ihrachyshka> and then it's not clear what's going on
14:20:37 <ajo> he's just going to move functions, not reinvent the world
14:20:38 <irenab> ajo: can you still explain the problem,  will see the solution in the patch :-)
14:21:17 <ajo> the problem is, we planned to use callbacks for all this,
14:21:27 <ajo> and callbacks are not suitable for this as they are today
14:21:34 <ajo> only the AFTER_READ part is ok
14:21:41 <ajo> and still we will move that in a separate interface
14:21:52 <ajo> tracking the resources via AFTER_UPDATE AFTER_CREATE for ports
14:22:08 <ajo> does not work, because it's too tied to ml2 (those callbacks are only called from ml2)
14:22:11 <ajo> well
14:22:19 <ajo> some of them, some other don't exist
14:22:21 <ajo> since
14:22:39 <ajo> the plan was to use callbacks temporarily, until we had the definitive solution from armax to "resource extension"
14:22:55 <irenab> ajo: initially we discussed that qos extension has two parts : policy management and port/net association.
14:23:07 <ajo> correct
14:23:12 <ajo> is the port/net association
14:23:21 <ajo> and dissociation
14:23:24 <ajo> the pain point
14:23:38 <irenab> if we split this in two extensions and make ML2 extension for second, this will be the ML2 way
14:24:04 <ajo> irenab: I don't follow
14:24:20 <ajo> we want something that works for all , not ml2 only
14:25:11 <gsagie> so the call backs are called from the db layer?
14:25:14 <irenab> ajo: I agree, but this looks like complicating the flow
14:25:33 <ajo> it's harder to discuss than to implement
14:25:42 <ajo> basically it's the same we have scinded to a separate class
14:26:04 <ihrachyshka> irenab, ok, the problem seems to be clear; let's now give them one day to see the solution
14:26:04 <ajo> just a few methods moved out to another class, not inside the plugin itself all together
14:26:20 <ihrachyshka> ajo, will it be up for review tomorrow?
14:26:24 <ajo> yeah, I will push mkolesni to show us his work by tomorrow
14:26:33 <ihrachyshka> good, push is the right word
14:26:54 <irenab> ihrachyshka: +1, we can always fallback for ML2 Extension of net/port qos association to call into Qo plugin
14:27:13 <ihrachyshka> irenab, yeah, as long as we don't start to fallback one day before merge :)
14:27:26 <ajo> ihrachyshka, that's my point
14:27:29 <ihrachyshka> that's why we need to move quickly and get solution up tomorrow
14:27:34 <ajo> I don't want to change directions
14:27:38 <ihrachyshka> ajo, when do you leave for PTO?
14:27:54 <ajo> next week I'm out, but I'll dedicate 30m-1h daily
14:28:02 <ajo> for mail, and reviews
14:28:04 <irenab> ajo: sometimes better change than keep digging :-)
14:28:07 <ajo> and probably irc
14:28:34 <ihrachyshka> ajo, good. we'll try to limit pings
14:28:36 <ajo> irenab, I have limited the dig depth, if I left mkolesni alone he'd rewrite the whole callbacks :D :P :)
14:28:50 <ihrachyshka> ajo++ for staying online on PTO for emergencies
14:29:05 <ihrachyshka> ajo, yeah, those Java guys...
14:29:08 <ajo> irenab, if we find it getting too complicated for some other reasons, I agree, falling back to a simple ml2 extension would be good
14:29:27 <ajo> and then next cycle can be used for doing something more general
14:29:36 <gsagie> its important to see how a plugin adds this support
14:29:47 <irenab> ajo: fine, this mwill require spliiting qos extension into two, but generally simple
14:30:05 <ajo> irenab, splitting in two?
14:30:22 <ajo> gsagie, yes we will need to document that
14:30:29 <irenab> extensions/qos.py to split
14:30:42 <ajo> irenab: I don't follow :)
14:30:58 <irenab> ajo: we can take it offline in case will be needed
14:31:06 <ihrachyshka> can we move forward with the topic? I think we are stuck on that Mike's patch, and we have ETA and plan for it already. And yeah, the way we will rewrite stuff when in master.
14:31:22 <irenab> I just mentioned the code change that will be required to make ML2 extension fallback
14:31:47 <ihrachyshka> irenab, you can try to describe the idea in email maybe. I also don't follow.
14:32:02 <irenab> ihrachyshka: fine, will do it
14:32:12 <ihrachyshka> any other high prio patches?
14:32:28 <ihrachyshka> if not, I vote for rules inside policies one from me: https://review.openstack.org/200608
14:32:41 <ihrachyshka> ajo, eagarly waiting for review ;)
14:32:55 <irenab> ajo: want to update on cli apprach?
14:33:07 <ihrachyshka> ajo, once we will have +2 from you, it should be easier to get another one
14:33:23 <ajo> yes, that patch is important
14:33:25 <ajo> please review
14:33:34 * ajo <- please, review!
14:33:41 <ajo> about CLI
14:34:07 <ajo> we found a stopper for using a common qos-create-rule / qos-update-rule
14:34:18 <ajo> if we provide <type> after that,
14:34:34 <ajo> we have not found a sane way to provide different arguments per rule in neutron-client
14:34:35 <ajo> so
14:34:42 <ajo> the call is to have a separate create/update per rule
14:34:46 <ajo> so we may have
14:34:52 <ajo> qos-create-bandwidth-limit-rule
14:34:57 <ajo> qos-update-bandwidth-limit-rule
14:35:17 <ihrachyshka> ajo, is it just for help message sake?
14:35:24 <ajo> qos-{create, update}-<type>-rule for every rule type
14:35:28 <ajo> ihrachyshka, also for argument parsing
14:35:35 <ihrachyshka> ajo, because afaik passing random --options should still work
14:35:48 <ajo> we don't want to accept a ton of optional arguments, where some are not really optional for some rule types
14:35:59 <gcossu> ihrachyshka: I'll try to review https://review.openstack.org/200608
14:36:09 <ihrachyshka> gcossu, thanks
14:36:27 <ihrachyshka> for cli, I'm fine and don't mind. any new rule type will require a patch for neutronclient anyway.
14:36:34 <ajo> yes
14:36:36 <ajo> that's correct
14:36:44 <ihrachyshka> and if the approach simplifies things, I'm good
14:36:48 <ajo> and every rule type has a different URL path
14:37:03 <ajo> it will be weird, btw, that...
14:37:21 <ajo> qos-rule-delete, qos-rule-show, will need to specify rule type
14:37:36 <Vichoward> if it simplifies i'm okay with it also
14:37:46 <ihrachyshka> ajo, ouch, that's a bit weird indeed
14:37:49 <irenab> ajo: no, no need for rule type
14:37:53 <ihrachyshka> ideally, id is enough
14:38:04 <ajo> irenab, how then do you point it to the right api url?
14:38:23 <ajo> qos/policy/<policy-uid>/<rule-type>/<rule-uuid> ?
14:38:36 <irenab> ajo: hope vikram can clarify
14:38:54 <irenab> maybe you right, lets see
14:39:18 <ajo> I've seen that other services (lbaasv2) solves that by exposing "rules" at a lower level
14:39:28 <ajo> they have that in monitor or heartbeats or something like that
14:39:31 <ajo> so you can just do
14:39:44 <ajo> qos/rule/<uuid> for some operations
14:40:11 <ajo> but...
14:40:20 <ihrachyshka> yeah, that simplifies cli but makes API inconsistent a bit
14:40:25 <ajo> correct
14:40:34 <ajo> we can just do a first version, and refine over time
14:40:37 <ajo> we're experimental
14:40:39 <ajo> so it should be ok
14:40:52 <ajo> once done, we may make the odds exposed and discussed IMHO
14:41:21 <ihrachyshka> ok, let's do the simple form even if it requires --type
14:41:30 <ajo> if having a less beautiful API makes CLI people happier, I'm ok with that
14:41:37 <ajo> other option, could be ... being able to discover the rule type
14:41:44 <ajo> by looking at the policy
14:42:06 <ajo> so we , in two calls, read the policy, ... find what's the category of the rule, then... kill the rule
14:42:17 <ajo> or then show the rule
14:42:25 <ajo> and then we don't need to provide type
14:42:39 <ajo> we already do a lookup when somebody passes the policy name, instead of the policy id
14:42:51 <ajo> irenab, vikram , thoughts?
14:42:58 <ajo> d
14:43:07 <ajo> devil is in the details..
14:43:21 <gcossu> :)
14:43:23 <ihrachyshka> let's have simple thing first. then go into UX
14:43:30 <ajo> +1
14:43:42 <irenab> for me it looks doable, but I have very limited knowledge on cli implementation
14:44:12 <ajo> I'd then just stick to providing the type for now, may be later we can make it optional by auto-discovert
14:44:16 <ajo> discovery
14:44:21 <ihrachyshka> there is another thing that is overdue: devref that we said before will be somehow ready for Wed: https://review.openstack.org/201536 and followups
14:44:33 <ihrachyshka> ajo, what's the plan here?
14:45:18 * ajo plans to duplicate himself :)
14:45:24 <ihrachyshka> I know it seems like it's not a big deal since it's not code, but I believe it's a must for merge-back, and the earlier we start to document stuff, the better
14:45:34 <ajo> yes
14:45:37 <ihrachyshka> ajo2 = copy.copy(ajo)
14:45:45 <ajo> it's a very expensive operation
14:45:49 <ihrachyshka> ajo, do you need any help with that?
14:45:58 <ajo> last two copies still taking too many resources, instead of producing
14:46:01 <ihrachyshka> ajo, you're not that huge
14:46:17 <ajo> ':D
14:46:22 <ihrachyshka> so, any help?
14:46:40 <ihrachyshka> I think I'm better in writing than coding, so
14:46:55 <ajo> if you can address current comments, and join to your other patch
14:47:03 <ajo> I think gsagie was doing something too
14:47:04 <ajo> I'd really thank you
14:47:13 <ajo> while I keep tackling the plugin towards merge...
14:47:16 <ihrachyshka> ok
14:47:23 <ajo> thanks -as usual-
14:47:46 <ihrachyshka> any more critical pieces?
14:47:54 <ajo> gsagie, did you have time to look into the devref of your pieces?
14:48:13 <ajo> ihrachyshka, I will respond to comments
14:48:22 <ajo> so you can update the parts I'm more aware correctly
14:48:51 * ajo looks again for the list of patches..
14:49:09 <moshele> ajo: for full flow with ovs agent we need this 2 to merge https://review.openstack.org/#/c/201975/ https://review.openstack.org/#/c/201583/
14:49:09 <gsagie> ajo: not yet
14:49:50 <ajo> ahm nice, I wil check your update moshele , thanks
14:49:55 <ihrachyshka> moshele, I see the 1st one -1's on jenkins
14:50:05 <moshele> ajo: my patch is failing becuase a bug in the ovs driver which gsagie fixed https://review.openstack.org/#/c/201583/
14:50:20 <ihrachyshka> ah, there is order involved
14:50:31 <ajo> ah, ok
14:50:39 <ajo> let's make a comment on the other
14:50:55 <ihrachyshka> moshele, can you rebase on gsagie's?
14:51:07 <ihrachyshka> moshele, I've sent it to gate, but it will buy us several hours
14:51:26 <ajo> moshele, I can rebase it if you want
14:51:38 <moshele> ajo: yes please
14:51:43 <ajo> ack, I will
14:51:45 <ihrachyshka> ajo, make sure not to rebase the Gal's one :)
14:51:56 <ajo> yep
14:52:22 <ajo> ok, any more critical pieces ?
14:52:39 <ihrachyshka> ajo, we may also +A https://review.openstack.org/#/c/202119/
14:52:48 <ihrachyshka> but that's not critical :)
14:53:05 <ihrachyshka> though I will be more happy to see less patches in the queue :)
14:53:20 <ajo> done
14:53:30 <ajo> ihrachyshka, should we wait for jenkins before +A ?
14:53:32 <ihrachyshka> ok, I think we can go back to trenches?
14:53:36 <ihrachyshka> ajo, I think no
14:53:42 <ajo> ok, I never got the logic of that
14:53:50 <ihrachyshka> ajo, at least that's what I see from what zuul does
14:53:52 <moshele> ajo: get_info rpc will return version object or dict in  the current implementation (I know it should be object at the end)?
14:54:01 <ajo> ihrachyshka, may be it was necessary back in time
14:54:18 <ajo> moshele, it should be object in the end, but the current dict usage may work
14:54:37 <ajo> ihrachyshka, we made them dict-compatible for now, right?
14:55:22 <ihrachyshka> ajo, well... they are similar, but not identical to dict. you can call .to_dict() to get the real one
14:55:35 <ihrachyshka> though I'm not sure where we are in terms of serialization
14:55:53 <moshele> ajo: for know for the server-> agent flow to work we need dict
14:56:15 <moshele> know -.> flow
14:57:21 <moshele> ihrachyshka: I think we need to convert to dict in registry.get_info
14:57:38 <moshele> ihrachyshka: just for now to make it work
14:58:00 <ihrachyshka> moshele, I'm fine either way. we'll get objects later.
14:58:12 <moshele> ok
14:58:13 <ihrachyshka> do whatever you need to integrate agent side :)
14:58:22 <ajo> moshele, you can access the objects as dicts
14:58:24 <ajo> they will work
14:58:42 <ajo> just for sending over wire you may need to use the functions I pointed this morning
14:59:28 <ajo> ok https://review.openstack.org/#/c/201975/ rebased
14:59:30 <ajo> #endmeeting