20:00:02 #startmeeting Octavia 20:00:03 Meeting started Wed May 25 20:00:02 2016 UTC and is due to finish in 60 minutes. The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00:04 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:00:06 o/ 20:00:07 The meeting name has been set to 'octavia' 20:00:09 Hi folks 20:00:11 o/ 20:00:12 hi 20:00:21 Howdy folks! 20:00:24 hi 20:00:38 hola 20:00:38 #topic Announcements 20:00:40 hey guys 20:00:56 I don't have any exciting announcements today. Anyone else? 20:00:57 o/ 20:01:07 announcement; welcome back Johnsom! ;-) 20:01:15 Ha, yes, thanks! 20:01:21 yep, we survived the thunderstorm 20:01:27 Um... not exciting, but I'm back for a while, and should have more time to devote to upstream work, which means I will be re-commencing reviews on stuff. 20:01:33 Was dougwig that bad that you are excited I have returned? grin 20:01:35 Get ready for a bunch of -1's. 20:01:48 sbalukoff Excellent! 20:01:52 sbalukoff: who are you again? 20:01:56 yes, i was, i forgot until 20s before. 20:02:21 \o 20:02:30 #topic Brief progress reports / bugs needing review 20:02:40 #link https://review.openstack.org/#/c/257201/ 20:02:46 Ok, what has been going on? I saw some merges and patches 20:02:47 dougwig likely while boarding a plane ;-) 20:02:48 #link https://review.openstack.org/305525 20:02:48 #link https://review.openstack.org/310326 20:02:48 #link https://review.openstack.org/317692 20:02:49 #link https://review.openstack.org/#/c/306084 20:02:59 #link https://review.openstack.org/306669 20:02:59 #link https://review.openstack.org/308091 20:03:01 #link https://review.openstack.org/309635 20:03:03 #link https://review.openstack.org/310069 20:03:06 * Frito braces for the floodgates 20:03:08 geez 20:03:09 there's more but those can go frist 20:03:11 Nice! 20:03:21 sbalukoff There is a list to start with.... Grin 20:03:32 =D 20:03:39 I did get time to review the L7 doc spec. Others please have a look too 20:03:55 Yay! 20:04:19 Trying as best I can to live up to my commitment to support our docs work. 20:04:39 #topic Merging the Cores 20:04:54 TrevorV you have the floor as I think you added this 20:04:56 vote? 20:05:09 * xgerman ducks 20:05:14 Yeah, no big deal. 20:05:28 A couple weeks ago we had a conversation about potentially merging the core groups 20:05:48 Yep, don't remember the final outcome 20:05:59 I don't recall seeing anyone against the idea. 20:06:12 At the time we didn't really say this was good or bad to do, but lately there has been a drop in reviews from contributors (including myself), and some of our patches have sat idly with one +2 20:06:18 Yeah, I almost want to say that dougwig was going to go make it happen 20:06:34 So... yes? Let's make it happen. 20:06:44 dougwig comments? 20:06:44 If we can merge the core groups, we can sort-of mitigate the lack of +2/+A we've been seeing, with the same 1 week cadence, and hopefully be alright to progress 20:06:45 +1 20:06:51 yep, this was me. i need to checkin with our ptl, then i'll make the change. i spaced it. *writes note*. 20:07:09 dougwig, all good homie, everyone's plates have been full with something or another lately 20:07:19 Yep. 20:07:25 #action dougwig to contact overlord for core reviewer merge between lbaas and octavia projects 20:07:32 0/ 20:08:04 #topic Revisiting Status visibility 20:08:10 #link https://bugs.launchpad.net/neutron/+bug/1585250 20:08:12 Launchpad bug 1585250 in neutron "Statuses not shown for non-"loadbalancer" LBaaS objects on CLI" [Undecided,In progress] - Assigned to Elena Ezhova (eezhova) 20:08:13 i added this and the next one 20:08:25 blogan You have to floor... 20:08:36 floor/channel/tow ticket 20:08:44 all the above 20:08:48 so that bug basically is askign for an easier way to show statuses, they want it on the objects themselves 20:08:58 im explaining in the bug report why pools can't be done that way 20:09:10 I certainly would like to see it at LEAST on the resource itself. 20:09:13 Maybe not in the list. 20:09:15 but listeners probably could bc i think we have given up on the idea of eventually gonig teh shared listeners route 20:09:16 Idk, arguable. 20:09:57 well shareable pools has issues, until at least we switch to octavia's api which allows seeing resource info in scope of a listener/loadbalancer 20:10:03 would an attached tree on a load balancer response resolve this ask? 20:10:18 we already have a statuses call 20:10:24 a2hill hahaha 20:10:34 yea, but as part of the reponse? 20:10:46 maybe 20:10:52 is the fundamental issue having to make multiple calls to see the statues? 20:10:53 It wouldn't solve it for me 20:10:54 but they only get that if they use single create 20:11:01 once that merges 20:11:04 hmm 20:11:23 a2hill, for me its more about the status concerning a singular object 20:11:34 I'd rather not have to check the whole tree just to see OPstatus of a member 20:11:46 and this particualr case is for creating a pool individually, wouldn't make sense to return back lb info, well i guess we could do something like that but it doesn't feel right either 20:11:50 well yea, its just thats sorta difficult 20:12:05 Agreed, a2hill 20:12:09 what about a cut up tree for each resource 20:12:17 well first thing's first, do we ever intend on having shared listeners? 20:12:22 We are async, so status returned on create calls is "suspect" anyway 20:12:30 Or a dictionary of resources, "pool: status" etc 20:12:36 tree is calculate, but we can parse and return related part of tree as a new status field for each resource 20:12:46 johnsom, no no not on create, I mean on "get /pools/id/members/id" 20:12:52 I haven't heard a need for shared listeners. 20:12:53 that should give me a op-status 20:12:59 Or at least, nobody clamoring for them. 20:13:05 but that op status will be different per listener 20:13:28 well it can be different 20:13:31 Right 20:13:44 Status on member is really helpful for end users as well 20:13:54 Which means the object could have a list of statuses per different listener. I know we talked about that before and said "no", but I'm still not against it 20:14:10 and something like that would solve our ui asks too 20:14:21 or however this is solved would alleviate thier concerns 20:14:57 right. 20:15:05 they're going to have to make antoher GET request on the member to get the status, why is doing a GET on /loadbalancer/{lb_id}/statuses any different? they just have to traverse the tree that returns 20:15:20 tell that to them 20:15:21 I do worry about column sprawl in the CLI, but status is fairly useful IMHO 20:15:31 but it is additional queries and logic that has to be built 20:15:36 blogan, literally the traversal is the issue. 20:15:38 and in reach's case, thats a scale issue 20:15:43 Not that its "problematic" its just "cumbersome" 20:16:27 well, TrevorV|Home in some cases it is also problematic 20:16:55 scaling status details for thousands of loadbalancers is proving difficult were finding 20:17:09 Right, or if an LB has a brazillion listeners, right? 20:17:18 right now its not a big deal, and were moving forward, but once things fill up its going to get really heavy 20:17:21 well if adding the statuses in the single GET request to a member and/or pool is wanted go for it, the CLI may start to look ugly 20:17:35 yeah, we don’t have paging 20:17:41 Most load balancers will have 1 or 2 listeners. 20:17:45 which we should add when we overhaul that 20:18:13 sbalukoff, that's true, but I like to keep in mind the edge cases 20:18:35 nah the problem is our UI team wants to give color coordinated alerts based on status on a page taht shows a list of a customers load balancers 20:18:37 TrevorV|Home: I don't like the regular use case to suffer because of outlandish edge cases. :/ 20:19:02 blogan: that still translates to additional logic that if isnt done right could cause issues for them 20:19:08 primarly because of the seperate statuses 20:19:31 so its a separate statuses issue? provisiong_status and operating_status or just that its separate api calls? 20:19:44 separate api calls 20:19:50 blogan So what you guys are looking for is a call that gets the status of all of the lbs in a tenant? 20:20:00 which is why i kinda suggested tagging the related part of tree to the response 20:20:08 okay so basically they want the statuses show in a list of pools, members, lbs, etc 20:20:14 sbalukoff, agreed, but with fewer listeners the "ugly" problem is moot. 20:20:24 someone else reported this bug, im just speaking from our side 20:20:29 TrevorV|Home: Agreed. 20:20:35 shouldn't the question of showing status on indivudual GET/cli be separate from a batch call for UI speed? 20:21:11 dougwig +1 20:21:16 dougwig: you mean CLI doesn't need to implement it? 20:21:39 I'm not entirely sure what you're getting at dougwig 20:21:39 i would think step 1 would be the basic fix. step 2 would be optimize. imo. 20:21:50 I'd like to get the op-status on the cli "show" for an object 20:21:58 Eh... I'm of the opinion that having a couple extra status fields in the CLI is pretty useful. :) 20:22:08 That might just be me, though. 20:22:12 I agree sbalukoff 20:22:15 sbalukoff: +1 20:22:17 Yeah, me too. Status in the show output is good 20:22:21 sbalukoff: +1 20:22:29 idk about the "list" output, but def the "show" output 20:22:33 Yep! 20:22:46 +1 20:22:54 TrevorV|Home: well the list would probably show what the show shows...thats redudnant 20:22:58 To be clear, though, I'm not against the current status tree existing, its totally useful. 20:23:00 fnaval: plus one'd himself again 20:23:02 :P 20:23:03 lol 20:23:10 blogan, that's not true, there are omitted fields in the list compared to the show 20:23:11 (for visibility) 20:23:14 lol 20:23:35 Right. 20:24:04 +1 for fnaval so he's not the only one ;-) 20:24:12 =D 20:24:12 Alright, I haven't read blogan's novel yet on this bug. Should we all comment on the bug? 20:24:18 okay so should this go into nlbaas api? even though its nto exactly the future 20:24:21 johnsom: Yes. 20:24:27 I think we are coming to agreement on the CLI issue. 20:24:38 well its raising more questions :) 20:24:42 List status should be a different RFE 20:24:48 Commenting on the bug is a good idea, even if we just say "This will be provided in octavia" 20:24:50 you know? 20:24:52 not the cli, but the fact that we're okay with it giong into the api 20:25:00 i'd expect a list lb for the vip to be more important than the status. 20:25:21 dougwig: one call must have all the things in it 20:25:25 we should just make a db dump api call 20:25:48 in json 20:25:51 hahaha, careful blogan, it will be next... 20:25:59 Heh! 20:25:59 blogan: connect your UI to your sql db 20:26:02 make any query you want 20:26:12 :) 20:26:32 Alright, so "comment on bug" is the result of the convo? 20:26:35 So, now, ODBC, JDBC, ...? 20:26:37 thats actually something theyve considered i think :P 20:26:39 j/k 20:26:41 i think 20:26:46 Yes please. Comment away! 20:26:49 i wouldn't be surprised 20:26:52 lol 20:27:07 rax should just rewrite it all as quarkaas 20:27:14 lol 20:27:19 #action Everyone comment on the status/CLI RFE 20:27:19 sounds to me like everyone is okay with putting the statuses in the GET and LIST of resources, not just LB 20:27:19 Haha 20:27:41 #topic Error Message as LB API Attribute 20:27:46 i never said i was ok with LIST. look at nova list... you get id, name, ip. 20:27:57 #link https://bugs.launchpad.net/neutron/+bug/1585266 20:27:58 Launchpad bug 1585266 in neutron "Can't specify an error type on LBaaS objects that fail to provision" [Wishlist,New] - Assigned to Brandon Logan (brandon-logan) 20:27:58 dougwig: isn't there a details list? 20:28:06 that nova supports 20:28:17 blogan: im only in agreement so it gets reach of our back. Otherwise i think a few additional calls are too bad, even at the scale. was just playing devils advocate since i know they wont stop on this one 20:28:26 still think we need paging but that’s for N-API 20:28:27 guys guys guys, topic changed. 20:28:32 arent* 20:28:48 okay error description, i tagged it as rfe but wantedt o see what yall thought 20:28:51 i was in middle of typing >< 20:28:57 Yeah, sorry, was trying to keep things moving. 20:29:02 i've been reading octavia and nlbaas bug reports today, so thats why i've been bringing these two up 20:29:10 I'm not sure what this means. 20:29:14 Plus I want comments captured for future generations.... 20:29:18 We want to include an error response in the "show" or something? 20:29:24 I thought that already existed 20:29:47 so v1 had status_descritpion on all the objects and if the object ever ERRORed it'd give reason why, similar to hwo nova will show a stack trace in its show server command if it ERRORs 20:29:48 http://developer.openstack.org/api-ref-compute-v2.1.html#showServer 20:29:57 I actually had this thought recently. Nova has a "reason" field for the status. This is a decent idea, but can be hard to implement well. 20:30:02 yeah 20:30:22 sounds like there could be some scrubbing of the error message needed 20:30:30 yea, this would be good to have. would it bubble up nova errors too i assume? 20:30:36 Would we worry about this in nlbaas or in octavia or both? 20:30:39 depends on how its implemented 20:30:58 well thats antoher topic, do we just do a hard stop on all nlbaas features or do we do both? 20:31:07 +1 on this being a good thing to have. 20:31:07 TrevorV|Home Remember we are on a path where nlbaas and octavia become one, so..... 20:31:29 johnsom, right, but we're on that path to *not* duplicate work, but this is an nlbaas v2 request, amirite? 20:31:47 it'd carry over to octavia api 20:31:50 yea, but until we can say thats actively being done, and we have something deployable thats not including nlbaas we kinda need features in both places 20:31:51 Yeah, so until we are merged, both is the right answer 20:31:51 lbaas api 20:31:55 sorry.. i type slow.. 20:32:27 Think of it as motivation to merge.... 20:32:34 +1 to a2hill typing too slow. 20:32:37 ha ha ha 20:32:38 >< 20:32:49 so.. 20:32:50 johnsom: Yep, that it is! 20:32:53 Yeah, alright, so I agree it'd be nice 20:33:04 It'd help "newbies" understand problems that they're experiencing too 20:33:14 blogan: ooh, is there? 20:33:27 Yeah, we get a lot of "It's in error or pending_*" why? questions now 20:33:50 the other side of this is logging and alerts ? 20:34:04 dougwig: http://developer.openstack.org/api-ref-compute-v2.1.html#listServersDetailed 20:34:31 We need to get better logging across octavia, unless I missed that patch already ha ha 20:34:41 blogan: so, it's a db dump 20:34:44 i mean customers logs 20:35:02 So, this one with the "Reason". I'm in favor of it. I think it should follow a cleanup of our revert/rollback and be implemented as such. 20:35:05 Oh, like logs from haproxy 20:35:05 got it 20:35:17 if they want to see whats going on with things, this may be a bit different, but could probably be coupled with alerting, though i dont know where or if those things are .. things 20:35:18 johnsom: +1 20:35:21 ok, dougwig tell the neutron drivers we approve ;) 20:35:27 johnsom, agreed, for octavia. Can that be done in neutron lbaas in a similar fashion though? 20:35:29 TrevorV|Home: not just haproxy 20:35:30 Customer logs from haproxy, that is a WHOLE different issue... 20:35:39 * TrevorV|Home is unfamiliar with neutron lbaas's rollback system 20:35:51 not what i was getting at i guess 20:36:01 neutron labs would need to get error info from the driver 20:36:09 Well, I think in n-lbaas we implement the field/db. Leave it up to the driver to populate. 20:36:16 +1 20:36:21 Wouldn't it mostly be a pass through from the driver? 20:36:32 our current rollback system involves singing, "rollback rollback, rawhide!" to yourself, and... that's it. 20:36:33 yeah driver can populate it or give it to the plugin to populate 20:36:33 or repopulate with sucks 20:36:41 Unless the request rolls back before hitting the driver 20:36:46 johnsom, ^^ 20:36:57 lol 20:37:18 TrevorV|Home well, then it's neutron's problem (just kidding) 20:37:41 if the requests rolls back the lb would not be in ERROR state i don't tink 20:37:52 before it gets to teh driver i mean 20:38:05 That's a fair point... 20:38:11 Whatever we do, we need to make sure the user cries. 20:38:17 Just sayin'. 20:38:20 I should be perfectly reasonable for n-lbaas exception handling (yes, I'm laughing too) to populate that field if it makes sense. I.e. driver not found or something 20:38:20 continues to cry 20:38:22 Load balancing is serious business. 20:38:52 no kidding 20:39:14 I won't comment on that in light of recent events in my life... 20:39:41 Seriously though-- let's not get too much into the weeds on this right now, eh. I'm hearing general agreement that having error messages that are more helpful than a Microsoft error code is a good thing. 20:39:42 Ok, so I'm thinking this is a "Please comment" as well. Generally a good idea 20:39:53 Yep! 20:40:09 now, who's going to do the work? 20:40:10 Ahh, I was totally firing up the uuid generator for the error codes.... 20:40:19 Haha! 20:40:46 blogan, didn’t you step forward? 20:40:48 #action Everyone comment on the "error message" in API RFE 20:40:54 Right. 20:40:58 * johnsom steps to the right 20:41:12 * xgerman steps far back 20:41:23 * blogan leaps into the back 20:41:32 * Frito slips into the shadows 20:41:44 * fnaval +1's himself. 20:41:48 lmao 20:41:50 We can say it's a good thing and get to it when we can.... 20:41:51 Huzzah! 20:41:55 well played sir 20:41:59 johnsom +! 20:42:08 be sure to comment on the statuses one 20:42:09 #topic Neutron-LBaaS Tempest Plugin 20:42:19 #link https://review.openstack.org/#/c/321087 20:42:31 ok, so i got this ask from the neutron folks - they want to run the neutron-lbaas tests but we dont have a tempest plugin 20:42:31 Ok, who added this and can talk to it? 20:42:36 me 20:42:55 i haven't done a plugin before so comments is appreciated 20:43:16 i recall that madhu had done a few for the other projects - was trying to ping him but he hasn't been online of late 20:43:37 Aren't these plugins typically in a separate repo? 20:43:50 looks like assaf is asking for nonvoting jobs for these first to verify they work as intended and then switch them to voting once validated 20:44:00 is there a reason why we don't have a tempest plugin hooked up to our gates? could it be because it can't selectively choose tests to run? 20:44:24 fnaval: tempest will run these tests? 20:44:41 Well, the whole tempest thing has been evolving/changing for the last year, so.... 20:44:54 if so, i think their runner can run tests based on a pattern option 20:45:01 the gate jobs will run those tests with the plugin, but some changes need to happen with post_test_hook.sh or some scripts like that 20:45:25 yeah but is it tempest running them, i think so 20:45:40 ok, well - if anyone has any comments or experience on setting up a gate job with TempestPlugin, please make some comments or reach out to me later. 20:45:56 fnaval: you're best bet is probably amuller in neutron 20:45:59 your 20:46:04 I wanted to see whether it would be a good idea for the rest of the team; if so, then i'll continue working on it 20:46:18 blogan: cool - i'll continue speaking with him 20:46:50 he directed me to this PR: https://review.openstack.org/#/c/304324/ 20:46:52 fnaval Thanks for following up on this! 20:46:56 which I'll probably use as a template 20:47:03 thanks for the floor johnsom! =) 20:47:41 blogan: i'll also look into regex name test matching 20:47:49 fnaval If you want me to get madhu online for some questions, I can do that. He just won't be working on this much in the near term. 20:48:12 yes that would be great too johnsom! i'll send an email as well 20:48:20 Ok, sounds good 20:48:28 just want to pick his brain on it 20:48:36 #topic Open Discussion 20:48:36 thanks 20:48:53 I have been told https://github.com/rackspace/gophercloud is getting LBaaS V2 support - so look and comment... 20:49:12 Huh! 20:49:46 Hmmm, not sure I would want to have a project called Gopher in Texas.... 20:50:08 yeah, RAX what’s up with the naming? 20:50:26 isn't it go based? 20:50:27 Go as in go language 20:50:40 Running short on 'go' puns. 20:50:40 pher as in uh 20:50:47 gopher 20:50:48 phertastic 20:50:56 Haha! 20:50:58 yep blogan knows 20:50:59 Yeah, I know. Just seems like it could be dangerous given the guns and all 20:51:34 guns are no more dangerous to gophers than lawn mowers 20:51:39 oh yea guns - they like guns here 20:51:48 gun thread! 20:52:03 as long as you don't bring them in or sell them in the parking lot 20:52:32 the feedback I got from the team doing that is that our docs suck are not up-to-date 20:52:52 and OpenStack in general doesn’t follow a common API designe 20:52:55 xgerman You are welcome to pick a docs bug and help fix that! 20:53:19 #link https://bugs.launchpad.net/octavia/+bugs?field.tag=docs 20:53:26 well, I think that feedback was expected 20:53:29 Yes! 20:54:17 Any other open discussion items today? 20:54:26 guys, can i steal 2 mins for a question? 20:54:29 reminder: please look at my commits 20:54:30 Yep! 20:54:41 aloha you have 6 20:54:46 alhu 20:54:49 alhu, please, this is the time to ask! 20:55:03 o-wk can be deployed with multiple instances, what about o-hm and o-hk? 20:55:19 they can be deployed as well 20:55:32 in multiple instances as long as they share the same db 20:55:40 Yes, all of them should be deployable in multiple instances 20:55:45 from the impl, looks like it accesses db directly, will that cause any race conditions? 20:55:57 DB’s lock 20:56:08 so don’t think so 20:56:24 those are queries, how the db gets locked? 20:56:46 The sqlalchemy transactions should be handling that 20:56:57 Its transactional .. so should be fine 20:57:14 alhu or are you wondering what happens when one message overtakes the other? 20:57:16 what if it's deployed on multiple hosts? 20:57:29 sqlalchemy can handle that as well? 20:57:39 yeah, DB side is fine 20:57:46 yes, xgerman 20:57:47 Yes, the transactions are held in the database 20:58:00 i c, thanks guys 20:58:04 but if we are running 2 instances of say h-hk 20:58:05 alhu we send enough messages that it should get back to normal pretty quick 20:58:32 who does both mange to keep the spare count ? 20:58:37 how* 20:58:38 i'm not too confident in a multi write galera cluster 20:58:47 manage* 20:58:55 hk does 20:59:12 and that’s all in DB 20:59:12 yea what if we have 2 instances of them running 20:59:32 we have at least 3 running and haven’t seen issues 20:59:49 not sure if anyone has the room after us but were basically at time. 20:59:49 bana_k 's question si valid 21:00:02 We haven't seen any issues in our multi-node env 21:00:07 but it can cause issues 21:00:07 bana_k yes. we use a transaction when setting the "busy" flag. As for spares, they are also allocated inside a transaction 21:00:26 let's move discussion to lbaas then 21:00:32 We can continue in the lbaas channel 21:00:34 #endmeeting