20:00:11 <johnsom> #startmeeting Octavia
20:00:12 <openstack> Meeting started Wed Oct 24 20:00:11 2018 UTC and is due to finish in 60 minutes.  The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:13 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:00:15 <openstack> The meeting name has been set to 'octavia'
20:00:33 <johnsom> Hi folks. Welcome to another episode of Octavia this week.
20:00:54 <nmagnezi> o/
20:01:37 <johnsom> #topic Announcements
20:01:44 <johnsom> Stein Milestone 1 is this week
20:02:25 <johnsom> The release team is changing how milestones are done. We still have the milestone dates, but we are not cutting a milestone release. That is unless we want one.
20:02:55 <johnsom> Other than that, I don't have any more announcements. Anyone else?
20:03:10 <nmagnezi> Nothing on my end
20:03:45 <johnsom> Ok then, moving on
20:03:46 <johnsom> #topic Brief progress reports / bugs needing review
20:04:03 <johnsom> I have been focused on testing and octavia-lib work.
20:04:28 <johnsom> I am done adding gates that test IPv6 with actual traffic. It's just waiting on patches to merge.
20:04:39 <johnsom> Specifically I would really like to see this merge:
20:04:44 <xgerman_> o/
20:04:47 <johnsom> #link https://review.openstack.org/#/c/589292/
20:05:19 <johnsom> and
20:05:21 <johnsom> #link https://review.openstack.org/611460
20:05:23 <xgerman_> #link https://review.openstack.org/#/c/604479/ <- needs eyes
20:05:29 <johnsom> So we can get started on backporting those
20:05:52 <xgerman_> and #link https://review.openstack.org/#/c/585864/
20:06:45 <johnsom> The octavia-lib work is coming along, but I realized we need to move forward on the driver lib using an endpoint, so I'm working on that today.  More on that discussion later on the agenda
20:07:25 <johnsom> I also plan to spend a bit of time on the certificate setup stuff folks are running into trouble with. I will probably start a doc for it that will eventually become part of the install guide.
20:07:58 <johnsom> Any other updates today?
20:08:42 <johnsom> #topic RFEs
20:09:07 <johnsom> I wanted to bring to the team's attention an RFE that came in for "Add notifications about object's status changes"
20:09:16 <johnsom> #link https://storyboard.openstack.org/#!/story/2004122
20:09:56 <johnsom> We have had some discussion in the channel about this, but wanted to give the team a heads up to comment on the proposal before we approve it.
20:10:15 <johnsom> Work has already started it appears.
20:10:35 <johnsom> Any discussion now on the RFE?
20:11:15 <johnsom> Ok
20:11:24 <johnsom> #topic Octavia-lib and the need for a new Octavia controller process
20:11:57 <johnsom> As I have moved the code over for the provider drivers to use octavia-lib I worked on removing the dependency on octavia.
20:12:28 <johnsom> Since we don't want to make the Octavia project into a library, we don't really want to add it to global requirements.
20:12:50 <johnsom> Which gets into issues with the driver_lib callbacks for the status/stats.
20:13:19 <johnsom> We have wanted to make this use an endpoint for the drivers to send the stats/status data back.
20:13:30 <johnsom> I realized now is the time.
20:13:54 <johnsom> The problem comes in that with a provider driver, really the only part of Octavia you need is the API service.
20:14:05 <johnsom> Which is implemented as a WSGI application.
20:14:51 <johnsom> Sadly, this means forking out a process that handles these stats/status updates is not practical. Most notably because we can't get shutdown events.
20:15:25 <johnsom> So, the path I am going down is creating another process that will run along side the API service, that will handle the stats/status updates.
20:15:42 <johnsom> The updates will pass from octavia-lib to the new process via unix domain sockets.
20:16:00 <johnsom> This makes the issues around securing the endpoint easier.
20:16:09 <johnsom> Any thoughts/comments on that approach?
20:16:25 <nmagnezi> hmm
20:16:32 <nmagnezi> Just to get my head around it
20:16:40 <nmagnezi> What are the alternatives?
20:16:54 <nmagnezi> Not saying I disagree, just for thinking about this more
20:17:20 <johnsom> Well, we declare Octavia is a library and change it's release cycle, then add it to global requirements.
20:17:20 <xgerman_> yeah, can’t we just send it to an API endpoint?
20:18:02 <xgerman_> aka API server gets a stats endpoint and we call it a day?
20:18:11 <johnsom> We can setup a UDP/TCP endpoint and figure out how to secure it. This would either need the new process or would require the operator to install the HM and make it reachable from the API process.
20:18:54 <johnsom> xgerman_ So the driver would have to get a keystone token?
20:19:13 <nmagnezi> johnsom, good point
20:19:18 <xgerman_> yeah, why not
20:19:36 <xgerman_> or shared secret or…
20:20:33 <xgerman_> so the proviser driver would run in said new process or would it run somehwere else and communicate witj Ocatvia?
20:20:54 <johnsom> Seems a bit messier, in that it's more setup people installing would have to do. I.e. setup the account in keystone and make sure it has the right RBAC.
20:21:37 <johnsom> It still runs in the context of the API process as the API server loads it.
20:21:48 <xgerman_> well, most people install with installers (OSA, TripleO, …) so don’t see that as a problem
20:22:05 <johnsom> Lol, we seem to get a lot of questions about that.....
20:22:26 <nmagnezi> johnsom, say with go with the suggestion you proposed, unix domain socket, how would that look for drivers who run on a different node such as F5?
20:22:30 <xgerman_> ok, if it runs on the api server we can just restrict access to thet endpoint to lcoalhost?
20:23:11 <johnsom> nmagenzi This part of the code has to run on the API server anyway as it's a loaded driver.
20:23:58 <johnsom> xgerman_ Yeah, that is basically what I an doing with the unix domain socket.
20:24:32 <nmagnezi> johnsom, true. Just want to know what if the actual provider software needs to pull status of some object and it lives on an external node
20:25:10 <xgerman_> yeah,  think having a web server restricted to ips (hello loadbalancer) is more scalable
20:25:33 <johnsom> If a part of their driver lives on a different node, they will need to figure out how to request that of either their driver in the API node, or via the main Octavia API
20:26:29 <xgerman_> Adding this to the Ocatvia API makes sense to me… and then haveoctavia-lib access it (maybe keystone-auth, maybe ip-lock)
20:26:58 <johnsom> I think it needs to be stronger than ip-lock really.
20:27:16 <xgerman_> well, we cna leve that to the individual operator ;-)
20:27:21 <xgerman_> just make it pluggable
20:27:32 <johnsom> I lean towards the process as it separates the stats/status load from the API infrastructure uWSGI, etc.
20:28:04 <johnsom> I worry about putting that much data down the same path as the API calls.
20:29:29 <xgerman_> I believe we cna scale api pretty well — and adding some other process complicates things
20:29:51 <johnsom> What are the downsides to adding a process?
20:30:18 <xgerman_> will it be another service like an o-*
20:30:23 <johnsom> Right
20:31:16 <johnsom> o-da for driver_agent?  hahaha
20:31:56 <nmagnezi> Works for me :D
20:32:21 <xgerman_> yeah, probably minimal downside since people install with OSA, TRIPLE-O… etc. but I am also not sure if that needs to scale independently from the api server since you plan to hitch it to that container/host anyway
20:32:36 <xgerman_> so could also be an endpoint
20:33:00 <xgerman_> and all we “save” is some authentication and net traffic
20:33:08 <johnsom> In reality it is an endpoint either way, just implementation is different.
20:33:48 <xgerman_> o-hm can scale independently — with the provider driver tied to o-api…
20:34:23 <johnsom> So you would add the requirement of running the o-hm always?
20:34:50 <xgerman_> nah, I am saying we could just add that to o-api and could be good
20:35:03 <johnsom> And that the API needs to be able to reach an endpoint on the o-hm (not a requirement today)
20:35:32 <johnsom> I guess you have more faith in the abilities of uWSGI than I do
20:35:38 <xgerman_> ok, let’s back off
20:36:15 <xgerman_> our event pipeline is n o-hm… so if we do something else for processing of status we need to replicate that
20:36:57 <johnsom> It is similar but simpler than the o-hm code.
20:37:00 <johnsom> #link https://github.com/openstack/octavia/blob/master/octavia/api/drivers/driver_lib.py
20:37:36 <xgerman_> ok, so o-hm would need to be co-located with o-da for the amphora-driver?
20:37:42 <johnsom> That is the code that would move into the new "thing"
20:38:03 <emccormick> Hey, is there a way to specify which version of amphora agent gets installed in the diskimage build?
20:38:13 <emccormick> I see this here: amphora-agent git /opt/amphora-agent https://git.openstack.org/openstack/octavia
20:38:21 <xgerman_> johnsom: this codes lacks all the event stuff 9
20:38:22 <emccormick> but wouldn't that fetch master by default?
20:38:31 <xgerman_> e.g. throwing it on the n-lbaas queu
20:38:39 <johnsom> If/when we move the amphora-driver to use this path, o-hm would need to send that data over to the amphora-driver, yes.
20:38:48 <xgerman_> yes
20:39:06 <xgerman_> emccormick: can youw aint until our Open Discussion section in our meeting
20:39:38 <emccormick> Oh I'm sorry, I popped in from another window and didn't see the meeting start. My apologies for interrupting
20:39:53 <xgerman_> no worries :-0
20:40:23 <xgerman_> johnsom: still wrapping my head around that. Yes, we would not want o-hm access the o-api
20:41:29 <xgerman_> so if we have amphora-driver and venfdor Y — this o-da would run on the api host and the hm host?
20:42:05 <johnsom> No, just the API host. Currently o-hm does all of this on its own (plus some)
20:42:33 <xgerman_> yeah, but we should move that part out of o-hm and into o-da — or not?
20:43:02 <johnsom> Should at some point to be "clean"-er-ish
20:43:52 <johnsom> More than I plan to do in Stein however
20:43:55 <xgerman_> ok, this is a strike against an o-api endpoint since we then would need to create connectiovity o-hm<->o-api
20:44:35 <xgerman_> but we could also make that similr to o-cw which get integrated into o-hm as a “library”
20:44:39 <johnsom> Which isn't a huge leap given o-hm talks to the other service APIs, but yes
20:46:00 <xgerman_> mmh, I still  (1) think an extra service complicates our offerings (2) endpoints should be RESY and o-api
20:46:01 <johnsom> that is true.
20:46:48 <johnsom> that is true was about the library part
20:47:00 <xgerman_> yeah, got that :-)
20:47:11 <xgerman_> I know my 2nd part is controversial
20:47:41 <xgerman_> and we lost nmagnezi
20:48:11 <nmagnezi> Sorry I try to multi-task here O_o
20:48:17 <xgerman_> ok
20:48:38 <johnsom> Well, I don't want to burn all of our time on this. I can post the octavia-lib side today and you can look at it.
20:48:44 <xgerman_> ok
20:48:53 <johnsom> This would give some time to think about it.
20:49:07 <johnsom> Though I do want to move fast on this as we have people waiting on octavia-lib.
20:49:52 <johnsom> #topic Open Discussion
20:50:03 <johnsom> Other topics for today before we run out of time?
20:50:44 <johnsom> lol
20:51:03 <nmagnezi> Maybe emccormick :)
20:51:19 <johnsom> I side messaged him the info he needed.
20:52:03 <johnsom> Which for the log is these two variables:
20:52:06 <johnsom> #link https://github.com/openstack/octavia/blob/master/devstack/plugin.sh#L63
20:52:21 <nmagnezi> that's very.. efficient
20:52:30 <nmagnezi> (The side msg)
20:52:47 <xgerman_> +1
20:52:48 <johnsom> Ha, easy when I know the answer and where to point in the code
20:53:25 <johnsom> Ok, let's wrap up then.
20:53:25 <emccormick> o/
20:53:53 <johnsom> Please give me your thoughts on the octavia-lib discussion in the next day or two.
20:54:10 <emccormick> thanks for hte info
20:54:11 <johnsom> Also, please try to find some time for reviews. We have a lot of open stuff.
20:54:52 <johnsom> #endmeeting