20:03:16 #startmeeting Octavia 20:03:17 Meeting started Wed Jan 7 20:03:16 2015 UTC and is due to finish in 60 minutes. The chair is sbalukoff. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:03:18 lol 20:03:18 WOOO 20:03:18 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:03:20 The meeting name has been set to 'octavia' 20:03:23 #topic Roll CAll 20:03:23 o/ 20:03:24 #help 20:03:24 o/ 20:03:25 \o/ 20:03:27 o/ 20:03:30 o/ 20:03:32 success 20:03:33 o/ 20:03:33 hi 20:03:44 hello 20:03:56 Ok, this is today's agenda, such as it is: 20:03:59 #link https://wiki.openstack.org/wiki/Octavia/Weekly_Meeting_Agenda#Agenda 20:04:33 o/ 20:04:41 o/ 20:04:55 o/ 20:04:55 #topic Brief progress reports 20:05:06 I'll go first on this: 20:05:34 Trevor and I collaborated over the holiday to get a skeleton of some code started for the amphora API. 20:05:55 It's totally incomplete, with no testing, and certainly doesn't work yet... and probably puts the code in the wrong place. 20:06:04 * TrevorV did something without blogan around (mostly)... feels accomplished mildly 20:06:07 But my plan is to get a WIP review going on this after this meeting. 20:06:24 * dougwig sharpens his -1 finger. 20:06:47 Should be enough to get people started on what we're thinking, and yes, I imagine we're going to go through a lot of patch sets before I see any +1's or +2's. 20:06:54 sbalukoff, can you ensure to respond to some of my comments in the spec? It makes difference for some of the driver decisions 20:07:23 a2hill: Yes, I had been holding off on updating the spec because I wanted to get some actual python coded. 20:07:29 Prepare to be horrified y'all. 20:07:36 Gotcha ;) 20:07:37 Hahaha 20:07:38 Anyway, who would like to go next? 20:07:43 a2hill: wait didn't you take ETO? :P 20:07:50 yes o.O 20:08:02 rm_work: I took some. 20:08:17 sbalukoff: rule #1) python files should end with ".rb" 20:08:22 I am mostly getting caught up from vacation time. We have been working on the amphora driver API 20:08:24 * blogan chokes dougwig 20:08:28 sbalukoff: yes but a2hill is on ETO *now* T_T 20:08:34 dougwig: Thanks for the help, yo. XD 20:08:39 ruby 4ever! 20:08:44 wanted to attend meeting 20:08:52 * dougwig metaprograms his way out of blogan's chokehold. 20:08:54 * rm_work puts xgerman and dougwig in the corner 20:08:57 * sbalukoff stares at a2hill incredulously. 20:09:02 >< 20:09:07 Ok! 20:09:28 I'm not sure what all people were able to get done over the break, so I'm reluctant to call people out individually. 20:09:44 For my update I was catching myself up on things and starting on the haproxy driver 20:09:44 I did some more work on the taskflow/api server which starts a nova vm... it now writes to the DB 20:09:58 i started the api to queue code, carlos is taking that over though 20:09:58 excellent! 20:10:09 I literally just got back today from vacation <_< 20:10:16 Any particular reviews y'all would like to point out that need eyes at this time? 20:10:48 well i have some that are pendign the taskflow discussion 20:10:59 * dougwig sharpens his pitchfork. 20:11:02 HAHA 20:11:02 ^^ 20:11:27 Anyone have reviews that are not pending the taskflow discussion you'd like to get eyes on this week? 20:11:32 johnsom sorry to interject, i have no idea what you mean by "amphora driver API" 20:11:34 maybe we could move to the taskflow discussion and then come back based on the conclusion on 20:11:51 the API you are building client for TrevorV 20:11:52 TrevorV: amphora-driver-interface blueprint code 20:12:01 or that 20:12:17 Oooh oh oh okay, I was concerned there was duplication 20:12:18 Sorry sorry 20:12:19 "amphora driver interface" 20:12:25 this review needs some reviews https://review.openstack.org/#/c/136499/ 20:12:33 Yay! 20:12:37 TrevorV: needs to fix my one comment 20:12:42 in min going to be updating that interface to the new spec xgerman? 20:12:46 is* 20:12:52 yes 20:12:55 kk 20:13:08 Ok. 20:13:18 blogan I'll get on that later this evening 20:13:55 Ok, let's go onto the taskflow discussion, as I see this taking a while. 20:14:00 #topic To tightly couple interfaces with taskflow or to not (german & blogan) 20:14:08 duhdundun 20:14:10 I yield the floor to y'all. 20:14:22 i rephrased your original topic 20:14:32 "tightly copule" infers to not 20:14:35 lol 20:14:42 Ill start off by saying it would be nice to not have to build tasks in the driver and instead wrap them so its less coupled 20:14:45 oh haha that makes sense then 20:14:55 +1 a2hill 20:14:55 firstly, we're all agreed to use taskflow, so this is NOT a taskflow vs not discussion, right? 20:15:01 bc I am not advocating not using taskflow, im just saying that the interfaces should not be tightly coupled with it, they should be dumb about hwo they are executed 20:15:06 dougwig +1 20:15:10 dougwig: +1 20:15:10 dougwig: I believe so 20:15:21 Correct dougwig 20:15:30 We're discussing where the task-flow related code will exist. 20:15:44 this is, are our driver interfaces flow aware, or are they "create_foo, update_foo, delete_foo" 20:15:59 and i dont think teh drivers should return flows, I know ML2 is currently doing that but that is a WIP, and I personally don't think it is the best design 20:15:59 xgerman: Could you share your thoughts on this with us? 20:15:59 and it's the controller's job to wrap a flow around those. 20:16:15 correct dougwig 20:16:20 ^^ +100 dougwig 20:16:42 fwiw, i'm in the yes taskflow, drivers are not aware, camp. 20:17:23 ok, I don't think drivers returning flows is a big burden -- it's just soem boilerplate code 20:17:39 and ensures that we can have an revert method we can call for each method 20:17:52 not that its a burden, its that its tightly coupled 20:17:56 but we can have the interface force a definition of a revert method 20:17:57 but if we go away from flows, then the drivers all have to be rewritten. 20:18:01 xgerman: Is this harder to achieve with "dumb" methods? 20:18:08 and it'd seem trivial to have the higher layer wrap the flow 20:18:35 It IS trivial to have them at a higher level 20:18:42 dougwig: Do you think we will ever move away from flows? 20:18:44 thing with that is the driver implementor has to worry about controller code also, not that its a big issue 20:18:53 well, it's only trivial if the driver doesn't need any advanced flows 20:19:25 when we looked atour discussion I didn't want to preclude the drivers from having the abilkity to define flows if they want to 20:19:28 a driver can do its own flows without needing to be part of the main flow, right? aren't those orthogonal issues? 20:19:34 the driver can still choose to use taskflow underneath if it wants 20:19:40 If we wrap things in the controller, the controller needs to know the order of the method and which methods to actually call 20:19:42 a2hill: I don't think there's any way to avoid the drivers being aware of what the controller is doing... 20:19:43 thats a downside 20:20:14 yeah, we want the controller to be in control 20:20:27 then in that case, wraping it makes sense 20:20:31 and not have the driver call out to other drivers 20:20:32 it will be, it'll call interface methods 20:20:39 For what it's worth, I'm currently on neither side of this discussion. :) 20:20:49 the driver should be a "performer" 20:20:53 drivers should be, "here's some info, do what you need to do, tell me the result". flows are an interface for abstracting async, concurrency, failure rollback, etc. those aren't "driver" functions. 20:21:21 well, historically drivers did stuff async, etc. 20:21:29 and drviers can still do that 20:21:52 well, then they should be flow aware to do that in a sane way 20:22:11 to me, it's like embedding the thread scheduler in a filesystem driver. let the filesystem driver read/write blocks, let the higher level worry about how to organize it. 20:22:41 +1 with dougwig :) 20:22:57 dougwig: You mentioned in previous discussions (and above) that you'd like to see another layer of abstraction for this. I thought you were joking when you first mentioned it. How would you envision that working exactly in this case? 20:23:38 wrapping of the driver methods in a flow in the controller code, not in the driver code 20:23:52 drivers have tight uncoupled entry points "do_x", and the controller has a layer that encodes how to put those primitives into a sane flow. if the primitive is so large the controller can't do that, it's an indication that the driver is being asked for too large a piece. 20:24:02 what blogan said, yeah. 20:24:27 and if its too large a piece, the interface needs to be decomposed into further methods 20:24:47 I really want a solution that has clear and complete error handling. 20:24:53 You do lose the specific abiljity to revert as part of the task when you just wrap a 'dumb' method though and not sure its doable otherwise 20:24:57 blogan: That seems to imply that all end-devices need to use roughly the same flows. Or are you thinking that for some drivers, some of the methods called would just be noops if they don't need them (and then the controller potentially is calling a ridiculous number of methods to accommodate every strange device?) 20:25:20 johnsom: if the driver is doing little bits, then the encoding for error handling and rollback is actually only done once (in the controller), instead of left to every driver author. 20:25:23 sbalukoof +1 - what's easy for one driver might be difficult for the next 20:25:53 (Again, just playing devil's advocate here. Because I'm kind of a jerk.) 20:25:54 sbalukoff: i'm not assuming a 1:1 mapping on the controller flows and the driver interfaces. 20:26:21 dougwig: I'm not quite sure I follow you. 20:26:39 yeah, I am confused, too 20:26:46 same here 20:26:58 if I call say update_lb -- that can be done async, sycn, and wahtever 20:27:14 e.g. the driver interfaces might be create_vip, create_fip, add_member x3, create and assoc hm. the flow might be to create the lb, and have a tree for all of that, plus it's rollback scenarios. 20:27:14 and consist of figuring out the state on the lb, not doing that... 20:27:23 What if the next step in the flow needs to be done serially? 20:28:09 thing is, from what i can tell thus far, is unless you are subclassing taskflow you cant add the specific rollback scenarios, which makes it difficult to de-couple this 20:28:19 That is to say-- assume one driver does it sync, another async... do we need another method there for the controller to check on the status of the last command in case it was done async and pause before the next serial command? 20:28:43 a2hill we can always write an abstraction for that 20:28:45 i'm assuming you can simplify drivers and just have them all be sync, and let taskflow async them. 20:28:53 if taskflow is used in the controller to call these dumb methods, and one method is async hte other is sync, it shouldn't matter, its all going to look async to the controller 20:28:54 then the serialization is up to the flow. 20:29:03 not if youre wrapping with functortask 20:29:35 so we are making assunptions: 20:29:48 1) driver tasks a simple and synchronous 20:29:49 you can provide it params but i dont see a way to tell the flow to rollback if 'this particular task' fails 20:30:13 xgerman: We could explicity state that in the contract... 20:30:34 im not assuming they're synchronous 20:30:34 Anything running a taskflow is going to be doing it in its own thread anyway-- that thread can wait indefinitely. 20:31:01 well, waiting is another "task" 20:31:02 the rollback could be a revert to a "snapshot" 20:31:05 (Which... well, might be a problem in some cases. Obviously we'll want to be able to interrupt hung tasks.) 20:31:15 I suppose we could have tasks in the controller that has defined revert policies that calls the dumb methods, which wouldnt be using functortask in that case 20:31:22 bedis: It's not always that simple. 20:31:23 indefinite waiting threads is a bad thing 20:31:30 yep 20:31:53 so all we are saying is we don't want to wrap driver functions into classes 20:31:55 johnsom: that's what timeouts are for. 20:32:44 sbalukoff: is the configuration stored in the DB ? 20:33:09 xgerman: i think we're saying we dont want to have driver implementations return flows 20:33:16 +1 blogan 20:33:24 that said, if an error occured, a rollaback may also generate an error :/ 20:33:37 s/occured/occurs/ 20:33:38 and let the code that is calling the driver wrap those methods in tasks/flows 20:33:42 bedis: In theory yes-- but there's the problem of resolving what's in the DB with what is actually deployed. That's the whole problem, really. :) 20:33:52 just to point out, it will be easier to test if the interface is decoupled from flows. 20:33:58 bedis: Yep, that's my point! 20:34:30 Ok... 20:34:57 For example: I need to build haproxy configs, call api client, so my update() method would do just that. Then the controller would have a Task that calls the drivers update() method and have revert policies defined. This adds a lot of logic to the controller, but sounds like thats what we want and it de couples taskflow from the driver 20:35:10 I see this more a question of "where should the intelligence be"? If it's all in the controller, drivers are probably simpler to write, so long as they can do things in a way that's very similar to the reference implementation. 20:35:29 If it's in the driver, that leaves a lot more work for the driver writer, but also a lot more flexibility 20:35:45 a2hill: would that revert be to call update, and then call update again with the old config, and if that also fails, start a new amphora with the old config? that's driver neutral rollback. 20:35:55 could be 20:35:56 It also means that users and operators may not have a unified experience of how Octavia works, depending on which drivers / amphorae they're using. 20:36:08 depends on what/how we want to revert i suppose 20:36:19 Then I'd say the logic shouldn't be in the driver layer sbalukoff 20:36:30 Which i've said from the beginning 20:36:33 TrevorV: Why? 20:36:51 revert might look different dpending on the driver 20:36:55 Why would we want a different experience per user? 20:36:55 speaking as a driver writer, the guys writing new drivers do *NOT* have the deep knowledge of the overall system and failure cases that the guys writing the controller have. that "increased flexibility" becomes "worse quality", IME. 20:37:15 +1 20:37:20 they don;t need that 20:37:27 xgerman, exactly, and thats why using functorTask isnt really a solution either because you cant define revert policies for it 20:37:40 dougwig: Very good point. Also echoes my experience as an operator. 20:37:40 yes, don't use FunctorTasks 20:37:44 xgerman who doesn't need what? 20:37:46 so, we cant decouple if the revert is driver specific 20:38:06 a2hill i dont know what you mean 20:38:24 so whatever the driver is doing might be more complicated than just a call 20:38:27 you have to define the tasks in the driver that has the revert etc.. policies 20:38:42 which can easily be reverted 20:39:01 the controller should update configuration through a "transaction", driving the driver through small "atomic" operations 20:39:22 xgerman a2hill I think we've missed a step here. You're saying the roll-back operation should happen in the driver? I'm arguing against that. 20:39:29 why can't teh driver interface define do_this() and undo_that() methods? 20:39:52 xgerman: nothing prevents more complicated drivers from hijacking their runtime entry point and doing *whatever* they want. being more complicated is supported in both scenarios, i think. 20:39:58 like undo_update? 20:40:13 a2hill: yes if that is what the real issue is here 20:40:25 that would solve the revert policies i suppose 20:40:27 dougwig I just wanted to give the driver writer a framework to do this in an organized way 20:40:35 actually yea, thats kinda nice 20:40:51 dougwig: And this continues to work with taskflow especially if the driver interfaces are all sync by contract. 20:40:53 so in my above example, the revert policy would just call the 'undo' methods 20:41:04 sbalukoff: right 20:41:06 so its still defined by the driver, but the task built in controller 20:41:08 a2hill: yes 20:41:15 I like that 20:41:33 blogan: +1 20:42:27 not sure how that is different from wrapping those two functions ina class 20:43:04 IT would be the same thing, just not coupled to the interface. If im understanding your confusion correctly :P 20:43:09 er to taskflow 20:43:54 xgerman: the difference is driver writers dont have to worry about taskflow and flows, and they just ahve to worry about writing the methods to do exactly what they say they are supposed to do 20:44:02 its clearer 20:44:10 For the driver writers... 20:44:11 and easier to test 20:44:22 Who are likely not to have intimate knowledge about everything the controller has to do. 20:44:34 no and they shouldn't 20:44:39 they don't need that knowledge - 20:44:40 but they wouldnt need to if we do it this way 20:44:42 Not to mention, in my opinion, we shouldn't couple an interface (which is meant to be generic) to a technology. 20:44:42 Also, the controller taskflows are likely to get a lot more complicated with Octavia v2.0.... 20:45:02 Leaving that kind of logic in the driver at that point seems like a really bad idea. 20:45:10 sbalukoff: what logic? 20:45:10 which logic? 20:45:13 lol 20:45:41 yeah, all I wanted to give driver writers a framework so if assign_vip is ten steps they could make that more orgznized 20:46:05 if its ten steps then they can organize that themselves 20:46:23 Well, maybe I'm thinking about this wrong, but I'm talking about roll-back logic when, say, you're trying to bind to a specific IP on 100 amphorae and it doesn't work on one of them... 20:46:24 yeah, hence tey would return a flow containing those ten steps 20:46:31 But... again, I'm probably not thinking about this correctly. 20:46:41 xgerman: or they use taskflow in their own driver 20:46:55 xgerman: or they solve it in another way that they prefer 20:47:51 i wish this had come up at the hackathon :( 20:47:58 bc whiteboarding would be great! 20:48:02 (or provide a parent method to request a flow handle and populate it, then return success.) 20:48:05 yep 20:48:13 ((optionally)) 20:48:59 I felt the flow stuff was lightweight enough o warrant just saying get_flow 20:49:02 Not to be a stick in the mud, but from what I can collect popular opinion on the topic at hand is decoupling driver from task-flow. 20:49:10 blogan: It did come up at the end of the week at the hackathon, I believe.... but parties had already left at that point. 20:49:20 sbalukoff: ouch 20:49:22 I think this is what im thinking about, is this wrong? https://gist.github.com/the2hill/0a40cbe6963d8a1097a4 20:49:28 maybe the driver can implement a method such as "snapshot_point" and should be able to revert to that point 20:49:48 bedis: maybe, but to keep it simple right now i'd leave that to do in future versions 20:49:49 a2hill that seems accurate to me 20:49:56 a2hill: +1 20:49:59 a2hill and that is exactly a taslflow.task 20:50:05 i'm not sure we've got any new information to offer here, nor consensus. so... vote, ML, let this brew, other? 20:50:08 yea, just not coupled in the driver 20:50:10 bedis: I'm not sure we want to hand that requirement to each driver writer. Some states are hard to revert to. 20:50:11 bedis that's definitely a possibility here. 20:50:18 xgerman: it is, but to the driver writer its not 20:50:41 dougwig: I think consensus on this is blocking some work here. 20:50:41 driver doesn't need to know taskflow, or how to pass parameters around, how to load parameters, get return values 20:50:45 oh, ok, I just did an class OctaviaRask(task.Task): 20:50:46 Or lack thereof, I mean. 20:50:59 blogan, they don.t 20:51:02 sbalukoff: well, not for HAProxy, it's a "cp haproxy.cfg.snaptshot haproxy.cfg" :) 20:51:24 bedis: Yes, but launching amphora, plumbing them, etc. are much more complicated tasks. ;) 20:51:33 heh :) 20:51:41 Ok, so! 20:51:52 I really want to not be blocked on this. 20:51:57 I'm not deep enough :/ 20:51:59 Ivote? 20:52:04 sbalukoff vote sounds good to me 20:52:15 I think blogan and a2hill and I are pretty much aligned 20:52:16 xgerman: would you be okay with the interfaces defining do and undo methods and the layer above just wraps them in tasks/flows? 20:52:29 i dislike blogan, im against anything he says 20:52:30 #vote To couple or not to couple, that is the question. Couple, Not Couple 20:52:31 xgerman: Ok! That's great to hear. 20:52:32 :) 20:52:32 could somebody summarize the outcome? it looks like we are in agreement. 20:52:32 :) 20:52:44 violent agreement 20:52:59 all agreement in this group is violent 20:53:01 by transitive association, that means xgerman and i agree as well. 20:53:03 #vote Shoud we couple drivers with task flow? Yes No 20:53:07 I think the outcome is NOT coupling taskflow in drivers/interfaces, but definitely to use taskflow in the controller 20:53:12 rm_work +1, but i hate you 20:53:16 Dammit... been a while since I ran a vote. 20:53:17 startvote 20:53:34 #startvote Should we couple drivers with taskflow? Yes No Sandwiches 20:53:35 Begin voting on: Should we couple drivers with taskflow? Valid vote options are Yes, No, Sandwiches. 20:53:36 Vote using '#vote OPTION'. Only your last vote counts. 20:53:38 (I'm lost...) 20:53:44 #vote No 20:53:46 #vote Yes 20:53:46 #vote No 20:53:49 a2hill: i violently agree, I hate me too :P 20:53:53 #No 20:53:54 #vote No 20:53:55 #vote No 20:53:56 #vote No 20:53:56 #vote Yes 20:53:56 #vote No 20:53:56 #vote No 20:53:57 #vote no 20:54:02 #vote Yes 20:54:03 :P 20:54:04 that was an oddly worded vote 20:54:05 #vote Sandwiches 20:54:05 yes 20:54:14 #vote yes 20:54:15 #vote yes 20:54:28 #vote yes 20:54:39 30 seconds to get your votes in... 20:54:53 that does not look like violent agreement nor consensus. 20:54:54 #vote No 20:55:11 im so confused, i thought we were aligned 20:55:16 yeah weird I thought we were agreeing 20:55:16 Me too. 20:55:22 lol 20:55:26 >< 20:55:28 #endvote 20:55:29 Voted on "Should we couple drivers with taskflow?" Results are 20:55:30 Yes (5): xgerman, mwang2, johnsom, sballe, ajmiller 20:55:31 No (9): rm_work, a2hill, sbalukoff, dougwig, jorgem, TrevorV, crc32, bedis, blogan 20:55:45 The lines seem to be: HP, and everyone else. :P 20:55:46 the question was confusinf 20:55:49 i should have left myself on Sandwiches 20:55:57 rm_work: +1 20:56:08 xgerman: How would you have rather seen the question worded? 20:56:23 Maybe we should have quickly summarize the solution we were voting for 20:56:28 should driver methods return flows? 20:56:34 it seems that blogan and a2hill siuggested an Octavia verison of the task class to use 20:56:40 xgerman: If you were basically in agreement with dougwig and blogan, could I ask the three of you to work together on what that 'agreement' is/was and send it to the ML? 20:57:09 the example that phil gave us 20:57:18 xgerman: https://gist.github.com/the2hill/0a40cbe6963d8a1097a4' 20:57:21 https://gist.github.com/the2hill/0a40cbe6963d8a1097a4 20:57:26 yeah 20:57:29 yea, thats what i was thinking we were going for, that decouples it from the driver 20:57:31 yeah that ^^ 20:57:31 you're fine with that? 20:58:16 Ok, shall we try to vote on that instead in the last minute here? 20:58:17 testing the controller class in that case is also easy to test out with mocking. All the flow logic gets tested easily. 20:58:39 if we run out of time, let's continue this in channel, as people are blocked. 20:58:42 you can test somehting which has undo and execute ina class easy, too 20:58:46 sbalukoff: make sense now that we know what we are voting on 20:59:00 #startvote Should we follow this general model for defining driver interfaces https://gist.github.com/the2hill/0a40cbe6963d8a1097a4 ? Yes No Sandwiches 20:59:01 Begin voting on: Should we follow this general model for defining driver interfaces https://gist.github.com/the2hill/0a40cbe6963d8a1097a4 ? Valid vote options are Yes, No, Sandwiches. 20:59:02 Vote using '#vote OPTION'. Only your last vote counts. 20:59:11 #vote Yes 20:59:15 #vote Yes 20:59:16 #vote Yes 20:59:17 #vote Yes 20:59:20 lol 20:59:31 #vote Yes 20:59:46 folks have about 20 seconds to vote, given the clock. 20:59:47 #vote Yes 21:00:01 HP folks? Please vote? 21:00:05 or #vote abstain? 21:00:13 they're on strike :) 21:00:14 i think we can discuss this in the main channel 21:00:14 Sandwiches 21:00:17 well, we will refactor to taskflow like Ml2 in a year anyway 21:00:20 #vote abstain 21:00:21 crc32: abstain is not a valid option. Valid options are Yes, No, Sandwiches. 21:00:28 #vote Yes 21:00:36 #vote Sandwiches 21:00:46 #vote Sandwiches (I am concerned about the fact that this implies drivers will have undo methods in them, but this may be irrelevant for this discussion) 21:00:47 Dang. 21:00:47 rm_work: Sandwiches (I am concerned about the fact that this implies drivers will have undo methods in them, but this may be irrelevant for this discussion) is not a valid option. Valid options are Yes, No, Sandwiches. 21:00:56 #vote Sandwiches 21:01:01 #endvote 21:01:02 Voted on "Should we follow this general model for defining driver interfaces https://gist.github.com/the2hill/0a40cbe6963d8a1097a4 ?" Results are 21:01:03 Yes (7): a2hill, sbalukoff, dougwig, jorgem, TrevorV, bedis, blogan 21:01:04 Sandwiches (2): rm_work, sballe 21:01:08 #endmeeting