16:00:01 <johnsom> #startmeeting Octavia
16:00:02 <openstack> Meeting started Wed Jul 29 16:00:01 2020 UTC and is due to finish in 60 minutes.  The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:03 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:05 <openstack> The meeting name has been set to 'octavia'
16:00:09 <johnsom> Hi everyone
16:00:09 <cgoncalves> o/
16:00:21 * xgerman lurking
16:00:28 <ataraday_> hi
16:00:32 <gthiemonge> Hi
16:00:55 <aannuusshhkkaa> hi
16:00:59 <johnsom> #topic Announcements
16:01:09 <johnsom> Vote for PTG week
16:01:23 <johnsom> There is a poll open for which week we should have the PTG
16:01:31 <johnsom> #link http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016098.html
16:01:53 <rm_work> o/
16:02:10 <johnsom> You should have got an e-mail about it, but if not the link in that e-mail should work
16:02:42 <johnsom> Also, call for presentations for the virtual summit
16:02:51 <johnsom> #link https://cfp.openstack.org/
16:03:32 <johnsom> Registration for the summit is also open.
16:03:35 <johnsom> #link https://www.openstack.org/summit/2020/
16:04:32 <johnsom> Finally, I wanted to mention that our gates are broken due to a pyroute2 bump the neutron folks needed. 0.5.13 included another change that broke looking up interfaces.
16:04:39 <johnsom> I have opened the following bug:
16:04:50 <johnsom> #link https://github.com/svinota/pyroute2/issues/724
16:05:04 <johnsom> There is a proposed patch, but unfortunately they didn't merge it and release last night.
16:05:43 <johnsom> A bit later in the meeting we should talk about our timeline and if we should revert the 0.5.13 version bump in upper-constraints.
16:05:51 <johnsom> Any other announcements today?
16:06:02 <cgoncalves> one other announcement: Queens and Rocky releases are officially EOL
16:06:34 <rm_work> \o/
16:06:36 <rm_work> woooooo
16:07:04 <cgoncalves> all open stable/queens and stable/rocky patches were abandoned
16:07:30 <johnsom> Yeah, that was great! Supporting those old OS releases and versions was a pain.
16:07:49 <johnsom> cgoncalves Thank you, I had the window open to do that, but you beat me to it!
16:09:01 <johnsom> Any thing else?
16:09:29 <johnsom> #topic Brief progress reports / bugs needing review
16:10:05 <johnsom> I have wrapped up the first pass on backporting the failover flow update to the amphora v2 driver. It is passing for me.
16:10:07 <johnsom> #link
16:10:19 <rm_work> Sweet
16:10:22 <rm_work> link? :D
16:10:28 <johnsom> #link https://review.opendev.org/738292
16:10:46 <johnsom> Sorry, pebkac
16:11:00 <rm_work> delta-based metrics seems like it's about to merge, once gates are fixed! thanks for the reviews
16:11:33 <johnsom> I have pivoted back to working on cleaning up the tempest tests.
16:11:34 <rm_work> the next thing in that chain is a refactor to make the drivers actually workable for more than just databases, and also more sane for third-party stuff that uses the agent to report stats
16:11:38 <rm_work> #link https://review.opendev.org/#/c/737111
16:11:46 <ataraday_> On top of my list review of update failover v2, I rebased experimental jobs change on top of it - as soon as gates fixed - we can check jobs there as well.
16:12:00 <rm_work> That is ready for review, and I'd love to at least know in general whether there's major objections to the overall plan
16:12:32 <johnsom> Excellent on both accounts
16:13:20 <johnsom> Ha, that got big. I will add it to my review list. I have a few things on there already
16:13:45 <rm_work> we have an additional stats driver on top of that as an example -- but it is only for an internal stats service we use... though I think could be easily modified to work with something else like prometheus or influx/carbon
16:14:11 <rm_work> ah like 75% of that is just things moving from one file to another (over half is just a test file that moved, really)
16:14:15 <cgoncalves> friendly reminder to add patches to the priority review list for better visibility/tracking
16:14:17 <cgoncalves> #link https://etherpad.opendev.org/p/octavia-priority-reviews
16:14:27 <ataraday_> Also cgoncalves change is ready https://review.opendev.org/#/c/739053/ - we may consider have jobboard disabled by default if we still want to drop v1/v2 code paths and do not feel that jobboard is ready to be default in next release
16:14:30 <johnsom> Yeah, I really need to spend some quality time there.
16:14:37 <rm_work> yeah i'll for sure get to that at some point soon T_T
16:15:01 <johnsom> ataraday_ That is the next topic on the agenda!
16:15:10 <cgoncalves> ataraday_, thank you for the reviews!
16:15:20 <ataraday_> oops!
16:15:24 <rm_work> yeah I think v2 without jobboard (as much as we do want jobboard) makes me more comfortable, to allow a slow transition
16:16:14 <johnsom> No worries, just highlighting the topic. It is an important one today.
16:17:22 <johnsom> Ok, any other updates today, or are we ready for the main event?
16:17:35 <cgoncalves> hmm, now that I'm thinking about this again I guess we need to make some code in the devstack plugin conditional to jobboard being enabled. for example, skip redis install and octavia_persistence mysql database
16:18:09 <johnsom> #topic Amphora V2 driver status for Victoria
16:18:12 <rm_work> yeah make a devstack var that sets that AND the config i guess
16:18:42 <ataraday_> cgoncalves, good point - this should be skipped
16:18:52 <johnsom> Ok, at the PTG we set Victoria milestone 2 as the checkpoint on if we are ready to transition the default driver to the amphora v2 code.
16:19:12 <johnsom> #link https://etherpad.opendev.org/p/octavia-virtual-V-ptg
16:19:53 <johnsom> MS 2 is this week, so we need to make a call if we are ready for that transition
16:20:25 <johnsom> I have been working with the amphora v2 driver a bunch over the last few weeks for the failover backport and I have some concerns.
16:20:38 <johnsom> We put together an etherpad:
16:20:46 <johnsom> #link https://etherpad.opendev.org/p/octavia-amphora-v2
16:21:11 <rm_work> Personally, as much as I LOVE to deprecate things as fast as possible, and we'd need to do this to start the deprecation clock on v1 i think... I am not super comfortable opting to make it the default driver when we're only just now getting it to feature parity with v1. :/
16:21:26 <johnsom> I think for the most part, the major code work is done, but I think we have some bugs/learning to do.
16:22:14 <rm_work> Yeah I'd much rather have a cycle to work out the bugs a bit and get some real world testing in, before we just go shove this in people's faces. We've worked very very hard to try to gain some amount of trust, I don't want to throw that away by releasing something prematurely
16:22:15 <johnsom> My biggest concern is I have had some endless loops fire off where jobs continuously get re-dispatched.
16:22:45 <johnsom> This burned my cloud down in one case by allocating all of the IPs on the lb-mgmt-net
16:22:55 <rm_work> I think I would feel this way even if you didn't have concrete bugs to point to -- it's just my gut talking
16:23:31 <johnsom> So, personally I think we should hold on making V2 with jobboard the default one more cycle to work out these issues.
16:23:32 <ataraday_> I can say if set v2 as default WITHOUT jobboard there will be still 2 changes to be merged - failover update and healthmotinor fixes
16:23:59 <johnsom> Yeah, so I kind of like this idea, switching to v2 with jobboard disabled.
16:24:10 <cgoncalves> +1
16:24:15 <johnsom> I don't see any real issues (maybe a few minor things) that would stop that
16:24:49 <johnsom> In theory we could start the v1 deprecation as well, which really the sooner the better
16:25:09 <rm_work> hmmm
16:25:15 <rm_work> if you feel more comfortable with that
16:25:41 <rm_work> so we'd switch at the end of this cycle (on release) which is... in a couple months?
16:25:47 <rm_work> need to look up the date again
16:26:04 <johnsom> Yeah, the issues I see don't happen with jobboard disabled. It's only when we are using the jobboard that things get redispatched and we need to update status.
16:26:17 <rm_work> yeah i think i could be ok with that
16:26:22 <cgoncalves> +1. starting deprecation signals on the v1 means not adding new features to it and that is a good thing
16:26:24 <johnsom> #link https://releases.openstack.org/victoria/schedule.html
16:26:37 <rm_work> the flows need to get tested more... but it's maybe less complexity without jobboard in the loop
16:27:15 <rm_work> so yes, 2 more months
16:27:20 <johnsom> Yeah, I would say we land cgoncalves changes, then switch to v2 default and deprecate v1 soon there after. We will be running the new code paths and transitioning to a more "provider driver" like path.
16:27:26 <rm_work> I think that'll be acceptable
16:28:18 <johnsom> I will say, currently v2 is *slightly* slower than v1 in that it is making more round trips to the DB. This is some optimization work we still need to complete, but I don't see it as a show stopper.
16:28:43 <johnsom> Ok, so any concerns with that plan?
16:28:49 <cgoncalves> should we alias amphora -> amphorav2 too? and create "amphorav1"
16:29:01 <johnsom> Next question, are we doing this as an alias change?
16:29:05 <rm_work> that would be a sane way to do that i think
16:29:11 <johnsom> lol, yeah, that question
16:29:15 <rm_work> swaps the default with a path back to explicit v
16:29:21 <rm_work> *explicit v1
16:30:39 <johnsom> Yeah, I'm thinking amphora becomes amphorav1, amphorav2 stays, amphora points to amphorav2.
16:31:06 <cgoncalves> +1. ship it!
16:31:10 <rm_work> +1
16:31:17 <rm_work> and now i need to afk for like 10m, br
16:31:19 <rm_work> *brb
16:31:27 <ataraday_> +1
16:31:44 <johnsom> Ok, I think we have a plan
16:32:00 <cgoncalves> you're no fun, folks. everyone's in agreement
16:32:16 <johnsom> Any other questions or comments on the amphora v2 driver?
16:33:17 <johnsom> Ok. If anyone has some cycles to help develop solutions to the issues on the etherpad, help is welcome. I think we just need to learn a bit more and create a few fixes.
16:33:48 <johnsom> I wish Josh was still around, he could probably fix it in a matter of minutes. lol
16:34:10 <johnsom> #topic Open Discussion
16:34:35 <johnsom> Any other topics today?
16:34:44 <johnsom> Is anyone planning a summit presentation?
16:36:23 <johnsom> I am considering doing a "new features" presentation. We have a bunch of stuff we can introduce and demonstrate
16:36:53 <cgoncalves> I think a deep-dive on the new features would be nice. I remember it has been presented in past releases and it was well received
16:37:13 <xgerman> sounds like a workshop :-)
16:37:14 <cgoncalves> lol, we are in sync :)
16:37:57 <johnsom> Ok, I may submit something. Let me know if you are interested in co-presenting.
16:38:47 <cgoncalves> sure, I can join you as co-presenter if no one else is interested
16:39:29 <johnsom> I have an internal presentation I need to create as well, so I guess I will be engineering-by-slide here soon. lol
16:39:40 <aannuusshhkkaa> will the metrics be a part of the "new features" if we are able to complete it before the summit?
16:40:41 <johnsom> Certainly could be. I was planning on things that we have released but not preseneted, but if there is high confidence in there being work complete to present, we should do that!
16:41:43 <aannuusshhkkaa> yay! thanks!!
16:42:09 <johnsom> I can help with preparing slides too, so we would collaborate on the content. I usually put them up on google drive for the presenters to work on it together.
16:43:13 <johnsom> Ok, so think about topics/content and I will bring the topic up again next week. If you want to submit your own presentation that is great as well, just remember I think it is down to a week before the deadline.
16:43:25 <johnsom> presentation proposal deadline
16:43:39 <johnsom> #link https://cfp.openstack.org/
16:44:30 <johnsom> Ok, one more thing, xgerman would like to have an informal "happy hour" type video chat after the meeting. If you are interested, the link is here:
16:44:41 <johnsom> #link http://meet.google.com/ita-bmmu-uyw
16:44:53 * xgerman approves this message
16:44:59 <johnsom> lol
16:45:11 <johnsom> Since we haven't had an in person chat in a while...
16:45:37 <johnsom> No worries if you can't make it, no Octavia decisions will be made, just a hang out and chat.
16:45:49 <xgerman> +1
16:46:07 <johnsom> Any other topics today?
16:46:52 <johnsom> Ok, thank you all!
16:46:54 <johnsom> #endmeeting