20:00:31 <xgerman> #startmeeting Octavia
20:00:32 <openstack> Meeting started Wed Jun  3 20:00:31 2015 UTC and is due to finish in 60 minutes.  The chair is xgerman. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:00:35 <openstack> The meeting name has been set to 'octavia'
20:00:39 <blogan> hello!
20:00:45 <xgerman> #chair blogan
20:00:46 <openstack> Current chairs: blogan xgerman
20:01:01 <blogan> just you and me xgerman
20:01:05 <blogan> lets talk about our feelings
20:01:07 <dougwig> o/
20:01:08 <xgerman> #link https://wiki.openstack.org/wiki/Octavia/Weekly_Meeting_Agenda#Agenda
20:01:17 <madhu_ak> hi
20:01:32 <xgerman> brogan that became a crowd pretty quick
20:01:57 <xgerman> #topic Announcements
20:02:12 <xgerman> sbalukoff has to wear a blue suite or something
20:02:14 <johnsom> o/
20:02:23 <sbalukoff> Haha!
20:02:29 <blogan> sbalukoff no more crazy hats
20:02:51 <sbalukoff> I'm wearing one today, and specifically did so for the group photo. XD
20:02:57 <blogan> lol
20:03:00 <xgerman> I bet it says IBM
20:03:25 <johnsom> Did you get your copy of the "Songs of IBM" yet?
20:04:13 <xgerman> moving on
20:04:15 <xgerman> Hotels are getting tight for Midcycle: https://etherpad.openstack.org/p/LBaaS-FWaaS-VPNaaS_Summer_Midcycle_meetup - make sure to book and mark yourself booked in the ether pad
20:04:28 <dougwig> getting tight is an understatement.
20:04:54 <sballe> o/
20:04:55 <blogan> id idnt think the midcycle would be such a big draw to take up all the hotels in seattle
20:05:17 <xgerman> we even checked other weeks and things are bad throughout the summer
20:05:27 <dougwig> i'm in the hampton, 1.4 miles away.
20:05:28 <ajmiller> o/
20:05:32 <Aish> o/
20:05:43 <xgerman> we got the Renaissance
20:05:51 <xgerman> weirdly it got cheaper when we added days
20:06:39 <xgerman> we have a few backyards people can camp at :-)
20:06:47 * TrevorV_ late... RIP
20:07:01 <xgerman> blogan requested a dog house.. working on that :-)
20:07:28 <blogan> a portable dog house
20:07:32 <blogan> on wheels
20:07:42 <xgerman> so it can be towed?
20:07:49 <blogan> obviously
20:08:06 <xgerman> Protected instances (service VM) proposal https://review.openstack.org/#/c/186357/ and there are plans for a nova blueprint
20:08:23 <blogan> what is the tldr version of that?
20:08:34 <blogan> protected instances so nova won't clean them up if they do some kind of cleanup?
20:08:42 <xgerman> no
20:08:53 <xgerman> the trove and cue people don’t like the idea of a service tenant
20:09:04 <xgerman> so they like to put their service vas into each tenants space
20:09:14 <blogan> oh really?
20:09:14 <xgerman> but the tenant shall not be able to access them
20:09:21 <dougwig> oh my, that has some crazy network implications.
20:09:28 <blogan> i don't like the diea of a user seeing the load balancer amphora
20:09:33 <blogan> they shouldn't know it exists
20:09:43 <xgerman> well, that’s where shadow tenants come in
20:09:43 <ajmiller> How are quotas handled?
20:10:04 <dougwig> how is a shadow tenant different than a service tenant?
20:10:04 <xgerman> ajmiller that stuff is still in early discussion
20:10:09 <ajmiller> ok
20:10:12 <xgerman> each user gets one
20:10:14 <blogan> shadow tenant for every tenant
20:10:17 <xgerman> yep
20:10:19 <blogan> which sounds yuck
20:10:22 <dougwig> hmm, why?
20:11:06 <xgerman> they like the isolation nova provides between different tenants
20:11:24 <xgerman> and don’t like to put all their vas under one service tenant since they loose that isolation
20:11:28 <dougwig> "they" ?
20:11:31 <xgerman> vas=vms
20:11:33 <blogan> trove
20:11:38 <dougwig> trove devs or users?
20:11:41 <xgerman> they=cue, trove, etc.
20:11:55 <xgerman> not sure how much they consulted users
20:12:29 <dougwig> hmm, interesting, thanks for bringing it up.
20:12:33 <blogan> octavia would still need a mgmt network connected to all the amphorae anyway, so that isolation is kind of moot no?
20:12:48 <blogan> unless we started spinning up mgmt networks for each tenant
20:12:54 <blogan> or load balancer
20:13:47 <rm_work> <_<
20:13:48 <xgerman> yeah, I can see some use from the sharing over tenants
20:14:06 <xgerman> since there probably are limits how many vas a nova tenant can realistically control
20:14:20 <xgerman> sharding
20:14:49 <xgerman> anyhow, it’s early in their thinking
20:14:56 <blogan> we should probably talk with amrith a bit
20:14:58 <rm_work> ah yeah, that's interesting -- if we have 5000 VMs on one nova tennant, is that a problem? O_o
20:15:09 <xgerman> rm_work yep
20:15:13 <blogan> large numbers are never a problem
20:15:16 <rm_work> heh
20:15:19 <rm_work> such webscale
20:15:27 <dougwig> it's a problem if you hate money.  oh wait, this is open-source, so you do.
20:15:47 <blogan> if money = dougwig, yes
20:16:08 <johnsom> The iptables issues start showing up after ~200-250 from what I have seen
20:16:44 <blogan> iptables bc of security groups?
20:16:47 <johnsom> Maybe kilo is better
20:16:57 <sballe> johnsom: can you recap the issue in one line?
20:17:50 <johnsom> The way the iptables are updated when a tenant adds/removes an instance causes delays as all of the hosts update their iptables
20:18:09 <blogan> is that only with security groups enabled though?
20:18:22 <blogan> or is iptables being used for something else
20:18:27 <blogan> i'm sure it is
20:18:35 <blogan> but just curious
20:19:00 <johnsom> There was a proposal on the table to us ip sets to reduce the number of rules required to update, but I don't know if it made it in
20:19:34 <xgerman> yeah, there probably are quite a few scaling issues and sharing over tons of tenants might alleviate them
20:19:40 <xgerman> sharding
20:19:52 <xgerman> anyhow, moving on
20:19:59 <xgerman> Global Load Balancing
20:20:13 <xgerman> dougwig + I went to some inaugural meeting on that
20:20:31 <blogan> i missed it, day off yesterday
20:20:34 <rm_work> ah, i wanted to go to that, but i guess i didn't see the time
20:20:39 <rm_work> didn't know it happened already
20:20:47 <dougwig> it's the lbaas meeting slot, happening again next week
20:20:56 <dougwig> minutes were sent to ML
20:21:05 <xgerman> yep, right now they are looking for use cases
20:21:19 <xgerman> just wanted to raise awareness
20:21:23 <rm_work> i feel like we went through a lot of this about a year ago or so
20:21:38 <johnsom> Yeah, didn't see that either.  Have they picked an approach yet?
20:21:40 <rm_work> I wonder if any of the work we did would be useful
20:21:47 <dougwig> johnsom: no, not yet.
20:21:58 <xgerman> yeah, still collecting use cases
20:22:13 <xgerman> but we know it will involve LBaaS and Desihgnate
20:22:31 <blogan> would lbaas's responsibility be spinning up the reginal load balancers?
20:22:49 <xgerman> yeah, likely
20:23:01 <blogan> and GLBaaS would be just orchestrating Designate and LBaaS
20:23:04 <dougwig> blogan: not really, gslb is more about picking the right A record, which involves a lot of health monitoring like LB's.
20:23:06 <xgerman> yep
20:23:11 <rm_work> yeah
20:23:18 <dougwig> now that we contradicted each other.  :)
20:23:20 <blogan> okay so then what would LBaaS do?
20:23:20 <rm_work> we had started working on some stuff involving PowerDNS and a custom backend
20:23:29 <xgerman> dougwig is right
20:23:30 <blogan> i always thought this was more a DNS product
20:23:34 <rm_work> yeah
20:23:43 <rm_work> it is essentially DNS
20:23:53 <xgerman> well, you need orchestration
20:23:53 <rm_work> but IIRC designate's model didn't work well for it
20:23:58 <johnsom> My guess is they think we have some health monitoring they can use
20:23:59 <dougwig> not really just DNS, no.
20:24:19 <johnsom> Not sure I agree, but that would be my guess
20:25:19 <dougwig> well, it's just DNS in as much as load-balancing is just echoing tcp.  :)
20:25:21 <blogan> so a nameserver would give otu A records to load balancers right (using some algorithm)?
20:25:40 <xgerman> well, they also want geo
20:25:47 <blogan> hence the algorithm
20:25:59 <xgerman> anyway, there is an extra meeting so let’s defer to that
20:25:59 <dougwig> gives out A records to anything.  it's half the puzzle, which can be combined with regional LB's, but it's not required.  IMO.
20:26:26 <blogan> xgerman: good idea
20:26:38 <xgerman> #topic Brief progress reports
20:27:35 <dougwig> the octavia neutron driver is so awesome that it's now modifying itself.  it's AI self is named ptoohil, and it added TLS support.
20:27:46 <blogan> lol
20:27:46 <xgerman> yep
20:27:47 <johnsom> I have mostly been doing maintenance activities, i.e. fixing the broken unit tests due to wsme changes, etc.
20:27:53 <blogan> i hope ptoohill does not become self aware
20:27:58 <blogan> he will destroy all of us
20:28:17 * xgerman hides
20:28:21 <ptoohill> :D
20:28:23 <ptoohill> MUwhahaha
20:28:31 <TrevorV_> PUT updates are WiP again, testing thoroughly this time.  Also, the bug fix for lb-deletion is almost ready for review, though I think some more bugs were found during its testin.
20:28:45 <TrevorV_> shut up, pothole
20:28:46 <TrevorV_> :P
20:28:57 <ptoohill> :(
20:30:10 <ptoohill> On that note, I have been heads down on some internal stuffs. I may be on this for a couple more weeks. So, besides some small updates here and there i've not done much. I was in the process of updating neutron-lbaas to handle the consumer registration. and i forgot to update tests in the hook tls patch, otherwise that patch is 'ready'.
20:30:30 <xgerman> cool
20:31:05 <johnsom> I will also be slightly distracted setting up the sonar trial we talked about last week.
20:31:53 <blogan> i have an update on the next topic
20:32:28 <xgerman> #topic How to solve tightly coupled drivers (compute, network, amphora)
20:32:28 <xgerman> brogan the floor is yours
20:32:37 <blogan> okay so first question, do we want any combination of compute driver, network driver, and amphora driver to work with each other no matter which one is actually used?
20:33:12 <blogan> that was always my assumption, though now i don't think its easy to do
20:33:25 <blogan> by this i mean, any amphora driver can be used with any network driver
20:33:40 <xgerman> yep, mix&match is pretty
20:33:48 <blogan> i agree
20:34:51 <blogan> but problem is when a network is hot plugged to an amphora, the interface may or may not need to be raised on the amphora
20:35:14 <blogan> i dont know i guess the way it is right now the network driver and amphora driver are tightly coupled with the plug_vip and post_plug_vip methods
20:35:22 <dougwig> doesn't that imply a nic flap interface needs to be in the amphora driver?
20:35:44 <blogan> dougwig: would that automatically detect a new itnerface and raise it?
20:36:04 <blogan> and vice versa when unplugging
20:36:19 <dougwig> no, it'd be a method in the amphora driver that the network driver would call on changes.
20:36:31 <blogan> oh thats the post_network_plug method
20:36:40 <xgerman> yeah
20:36:41 <blogan> and post_vip_plug methods
20:36:48 <xgerman> yeah, they are speced
20:37:07 <johnsom> Yep, have those.  We don't call directly from amp driver, we call from the controller worker
20:37:11 <blogan> but what is done in those method has to know what the network driver's plug_network and plug_vip method do
20:37:52 <blogan> which makes them tightly coupled
20:38:07 <dougwig> from a high-level it sounds to me like we don't have the right interfaces defined.
20:38:16 <blogan> i agree
20:38:17 <xgerman> +1
20:38:52 <blogan> we could move the vip plugging and network plugging into the amp driver, which would solve it but that doesn't make sense to have thsoe methods in the amp driver
20:39:27 <blogan> i think this is going to require more brainstorming so table that
20:39:50 <blogan> the other thing i explored was should we allow drivers to call other drivers with their methods
20:40:27 <blogan> and xgerman brought this up last week that this would invite recursive driver calling because since we wan't to mix and match, we can't guarantee taht one driver will not call the other driver backa nd forth
20:41:14 <dougwig> the recursion would only happen if the driver methods are too large, so that goes back to the interface maybe not being right.
20:41:25 <sballe> +1
20:41:35 <blogan> too specific and you lose flexibility though
20:42:00 <blogan> bc the controller worker calls these methods in a set order
20:42:30 <blogan> the whole poitn of the plug_vip method was to solve for different ways that all of us have to actually set up a VIP
20:42:40 <xgerman> we can change that - conditional flow is pretty close...
20:43:17 <dougwig> plug_vip is in the network driver, right?  so maybe the amphora driver needs a plug_nic, and i'm fine with it being called by the network driver.
20:43:20 <blogan> i hate to say it, we may need to have some kind of pluggable flow runner
20:43:58 <blogan> so the amphora driver with that method would have to call neutron to plug a nic
20:44:01 <blogan> right?
20:44:06 <johnsom> I thought we had a pluggable flow runner, called task flow in the controller worker.   We may just not be using it in the best way
20:44:18 <dougwig> we need a whiteboard.  write down the interfaces in three columns . draw some arrows.  break it down as needed.
20:44:34 <xgerman> that means we need ti wait for the mid cycle?
20:44:44 <blogan> maybe
20:45:02 <dougwig> you're welcome to solve it sooner, but it's beyond IRC for me at this point  :)
20:45:05 <blogan> probably best, but we all kind of think of the problem and bring solutions
20:45:49 <xgerman> so if we throw mix and match out of the window and make something which works and the refactor?
20:46:05 <blogan> im all for getting it working
20:46:32 <xgerman> same here
20:46:57 <xgerman> and if we need to refactor the interface so be it
20:46:58 <johnsom> I don't want to walk away from mix and match yet.  I think we have limited use cases at the moment that makes the abstraction hard
20:47:18 <blogan> johnsom: i think he's saying just get it working now and get mix and match working at the midcycle
20:47:31 <blogan> or at least a path forward on it
20:47:38 <johnsom> blogan: +1 to that
20:48:01 <johnsom> Easy is to pass in the subnet info via taskflow storage
20:48:52 <blogan> yeah i think thats what we'll ahve to do, but then we'll have to pass in a collection of subnet info as well but thats solveable
20:49:14 <johnsom> Yep
20:49:28 <xgerman> yeah, we need to revisit our flows and the input/output
20:49:38 <xgerman> and hopefully we can streamline
20:49:48 <johnsom> Yes!
20:50:18 <blogan> question, were yall still planning on using a spare pool of amphorae?
20:50:47 <xgerman> up in the air
20:50:56 <johnsom> Yes, I think so.  You aren't?
20:51:07 <johnsom> Just going to boot as needed?
20:51:09 <blogan> not right way since we're spinning up containers
20:51:44 <blogan> and talking to alex at the summit he was intrigued by doing something similar (lxd i think)
20:51:56 <xgerman> yep, we need to reassess
20:52:09 <xgerman> especially with VRRP hitting earlier
20:52:11 <johnsom> Yeah, there is interest in containers
20:52:11 <blogan> but having a spare pool that works fun woudl still save a few seconds
20:52:22 <blogan> fun = fine
20:52:28 <xgerman> yep
20:52:33 <blogan> but containers can't hot plug
20:52:46 <blogan> at least magnum can't yet and neither can our internal container solution
20:53:08 <blogan> which makes the spare pool harder to do
20:53:15 <blogan> and really just the entire workflow
20:53:21 <johnsom> Yep
20:53:35 <xgerman> we can probably drop the spare pool for now
20:53:43 <blogan> well put it low priority
20:54:01 <johnsom> I still think there is value
20:54:13 <blogan> enough value to make it a high priority?
20:54:23 <blogan> like put it before health manager?
20:54:52 <johnsom> No
20:55:01 <johnsom> Health manager is higher for sure
20:55:03 <blogan> okay, i think we're on the same page then
20:55:13 <xgerman> yep
20:56:03 <blogan> ok im done for now
20:56:39 <xgerman> #topic Open Discussion
20:58:00 <blogan> jurassic world?
20:58:02 <xgerman> anything?
20:58:10 <johnsom> blogan no the delete member fix, I lean towards doing refactoring in a patchset so we don't end up with partial refactoring in the repo.  It's just a different style thing.
20:58:23 <blogan> johnsom: i just updated my comments :)
20:59:11 <johnsom> Yeah, you and xgerman are advocating doing this part now, so I will work on that for the delete member flow.
20:59:14 <blogan> basically, its not a big deal to me and getting it working is mroe important
20:59:41 <xgerman> I am just saying if we want to do it we need to start somewhere :-)
21:00:07 <blogan> it will bother me like crazy so dont worry i will end up doing it one night
21:00:16 <blogan> but there is value it just getting it working
21:00:19 <johnsom> I will also open two bugs, one to fix the constants and one to streamline the storage
21:00:29 <xgerman> open bugs is good
21:00:38 <blogan> no bugs is good too :)
21:00:45 <xgerman> oh, we are out of tinme
21:00:52 <blogan> talk to yall later
21:00:54 <xgerman> #endmeeting