16:00:21 <rakhmerov> #startmeeting Mistral
16:00:25 <openstack> Meeting started Mon Jun  6 16:00:21 2016 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:26 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:28 <openstack> The meeting name has been set to 'mistral'
16:00:37 <rakhmerov> hi all
16:02:00 <rakhmerov> still waiting for people to join
16:02:07 <hparekh_> Hi
16:02:10 <rakhmerov> hparekh_: hey
16:02:16 <d0ugal> Hey
16:02:25 <jpeeler> hello (first time here)
16:03:33 <rakhmerov> hi guys
16:04:12 <rakhmerov> ok, let's skip our traditional "Review Action Items" topic since last time there was a very few people
16:04:34 <rakhmerov> #topic Current Status by team members (current activities, roadblocks etc.)
16:04:59 <mgershen> Hi
16:05:01 <rakhmerov> my status: still finishing up refactoring of Mistral engine components
16:05:04 <rakhmerov> mgershen: hi )
16:05:40 <hparekh_> My status: Just reviewed some patches and worked on policy related blueprint.
16:05:49 <rakhmerov> in parallel I started writing a spec for Custom Actions API: https://review.openstack.org/#/c/325769/
16:05:53 <mgershen> Hi rakhmerov, I'm working on some internal stuff. I hope to finish by next week.
16:06:04 <rakhmerov> hparekh_: how is that BP going? policy
16:06:16 <rakhmerov> mgershen: yes, I know :) It's ok
16:06:33 <rakhmerov> mgershen: ping me when you finish, there's a number of things to work on
16:07:15 <hparekh_> its almost finished will post patch soon working. Actually I am working on tests
16:07:27 <rakhmerov> hparekh_: very cool
16:07:33 <rakhmerov> looking forward to see it
16:07:34 <rakhmerov> d0ugal, jpeeler: can you guys say a couple of words about your involvement with Mistral and how you would like to participate?
16:08:08 <rakhmerov> if you want )
16:08:16 <d0ugal> I have been working on TripleO for a while and want to help improve Mistral to aid the TripleO use-case.
16:08:37 <rakhmerov> d0ugal: do you work together with Ryan?
16:08:50 <d0ugal> rakhmerov: Yeah, me rbrady and jpeeler work together :)
16:08:51 <jpeeler> i believe i will also be supporting the tripleo use case as well
16:08:55 <rakhmerov> ok
16:09:01 <rakhmerov> I see
16:09:28 <d0ugal> but hopefully we can just help out in general too!
16:09:30 <jpeeler> but i hope to fix generic mistral bugs and such too
16:09:32 <jpeeler> right
16:09:47 <rakhmerov> I know that rbrady has been interested in that Custom Actions API since we found that this part is not really designed well enough in Mistral
16:10:05 <rakhmerov> ooh, I see
16:10:34 <rakhmerov> well, ok. Then I'd like to ask you to participate in that spec that I pointed to when it's finished
16:10:55 <rakhmerov> https://review.openstack.org/#/c/325769/
16:11:09 <rakhmerov> because I guess that should be pretty important for you
16:11:34 <d0ugal> rakhmerov: +1, I'll take a look
16:11:47 <rakhmerov> and as far as other stuff, would you like me to assign something relatively simple to you or you think you know Mistral well enough to be able to approach something more serious?
16:11:59 <rakhmerov> like new feature or serious improvement
16:12:07 <d0ugal> rakhmerov: Sure, I would be happy to take something simple.
16:12:14 <rakhmerov> d0ugal: ok
16:12:21 <d0ugal> I would like to do something more serious, but I am not sure yet how much time I can dedicate to Mistral
16:12:23 <rakhmerov> then I'll do it offline
16:12:27 <d0ugal> Great
16:12:48 <rakhmerov> yeah, and feel free to communicate on whatever is needed to complete that
16:13:01 <rakhmerov> ok, thanks guys
16:14:13 <rakhmerov> ok, I actually also planned to talk a little bit about Custom Actions API
16:14:21 <mgershen> rakhmerov: OK np, I'll ping you when I'll be free.
16:14:26 <rakhmerov> ok
16:14:34 <rakhmerov> #topic Custom Actions API
16:15:30 <rakhmerov> so, maybe it's a little too early to talk about it but anyway, I'll just tell some basic info for now, maybe you'll have something to say at this early stage too
16:16:10 <rakhmerov> generally, the background comes from the need to write new actions which essentially Python classes inherited from mistral.actions.base.Action
16:16:27 <rakhmerov> it has essentially method run() that needs to be implemented
16:17:13 <rakhmerov> the problem is that so far most of our users and customers have been writing custom actions that weren't related with OpenStack
16:17:27 <rakhmerov> they didn't require any serious integration with OpenStack
16:17:41 <rakhmerov> in this case implementing them is pretty straightforward thing to do
16:18:48 <rakhmerov> but, in TripleO team wants to write new actions that could leverage other OpenStack services and hence use things like security context that an action is running in
16:19:39 <rakhmerov> we already provide actions for accessing OpenStack services the attempt was made to do something similar for other actions
16:20:12 <rakhmerov> by using modules like mistral.context that provides basically with auth token, project, user
16:20:34 <rakhmerov> however, we never considered this module as a stable thing that all actions developers could use
16:21:01 <rakhmerov> long story short, we need to establish a Python API for developing new Mistral actions
16:21:44 <rakhmerov> as I see it, it should provide access to security context (user, project, token), workflow context (workflow execution id, task id, action id etc.)
16:21:47 <rakhmerov> at minimum
16:22:15 <rakhmerov> and other useful things that may simplify actions development
16:22:29 <rakhmerov> like, for example, ability to reuse other actions that already exist
16:22:36 <rakhmerov> some kind of composition
16:23:12 <rakhmerov> so that we can do something like: action_utils.get_action("nova.servers_get").params(1,2,3).run()
16:24:15 <rakhmerov> there's also need to explicitly define some data types that are serializable and can be used on executor side
16:24:24 <rakhmerov> like mistral.worfklow.utils.Result
16:24:31 <rakhmerov> that actions are allowed to return
16:24:53 <d0ugal> dprince: Have you seen this Mistral WIP spec? https://review.openstack.org/#/c/325769
16:25:06 <d0ugal> dprince: We are just talking about it now.
16:25:11 <rakhmerov> d0ugal: unlikely, I just started it today
16:25:12 <rakhmerov> :)
16:25:15 <d0ugal> oh :)
16:25:28 <rakhmerov> so, other things that we probably need to do as well is to move existing OpenStack actions into a separate repo
16:25:51 <d0ugal> rakhmerov: I know that rbrady would like to write a spec for moving the actions
16:26:00 <rakhmerov> to be able to work on them separately and implement a script to be able to install only subset of them
16:26:10 <rakhmerov> d0ugal: ok, yes
16:26:32 <rakhmerov> d0ugal: I'm just wondering whether it should be a new spec or we can do it as part of this one?
16:26:46 <rakhmerov> at first glance, it should be a new spec
16:26:54 <d0ugal> rakhmerov: Good question.
16:27:11 <d0ugal> I am not sure how complex it is, but I would agree that a new spec makes sense
16:27:16 <d0ugal> It is related, but different enough.
16:27:18 <rakhmerov> and btw, that Custom Actions API, in my opinion, should be moved to a different subproject
16:27:29 <d0ugal> subproject?
16:27:30 <rakhmerov> d0ugal: yep
16:27:38 <d0ugal> Like it's own repo?
16:27:38 <rakhmerov> yes, that is why..
16:27:42 <rakhmerov> right
16:27:46 <d0ugal> +1
16:27:51 <rakhmerov> the reason is the following:
16:28:29 <d0ugal> Having a standalone Python library is much clearer than making part of a large codebase a public Python API.
16:28:35 <rakhmerov> these interfaces(sorry, my old Java habit to think in interfaces for key abstractions) and classes can be used by both engine and executor
16:28:46 <rakhmerov> d0ugal: yes, exactly
16:29:00 <hparekh_> +1 agree
16:29:10 <rakhmerov> ideally, we should be even able to package Mistral executor separately
16:29:20 <rakhmerov> with as few dependencies as possible
16:29:34 <rakhmerov> because it's supposed to be lightweight and very simple
16:30:20 <rakhmerov> and one of those dependencies should be something like mistral-common (need to think about naming) where all that Custom Actions API should live
16:30:49 <rakhmerov> and if Mistral engine (server side) needs those things too it can also have that dependency
16:31:11 <rakhmerov> thoughts?
16:31:58 <rakhmerov> :) hopefully it all makes sense
16:32:05 <d0ugal> I think it sounds great.
16:32:12 <d0ugal> I don't like the name mistral-common
16:32:20 <rakhmerov> yeah
16:32:29 <d0ugal> I don't want to bikeshed that now, but we have tripleo-common and IMO it was a mistake
16:32:48 <rakhmerov> what was it for?
16:32:59 <d0ugal> That is the problem :) The purpose isn't clear
16:33:04 <rakhmerov> what was your initial idea of it?
16:33:08 <d0ugal> so it has everything in it
16:33:10 <rakhmerov> aah
16:33:12 <rakhmerov> yeah
16:33:21 <rakhmerov> I see, yes
16:33:24 <d0ugal> The initial idea was common code between the CLI and API. However, plans changed
16:33:39 <rakhmerov> maybe we need to make it serve a certain purpose, yes, for clarity
16:33:44 <d0ugal> +1
16:33:53 <rakhmerov> like mistral-actions-lib, dunno
16:34:05 <d0ugal> mistral-actions might be enough
16:34:18 <dprince> d0ugal: looking
16:34:24 <rakhmerov> yep, may be
16:34:32 <rakhmerov> dprince: hi Dan
16:34:38 <d0ugal> Our Mistral actions (and workflows) are in tripleo-common, I would like to move them to tripleo-workflows (a new repo) eventually.
16:34:48 <rakhmerov> ok
16:35:01 <mgershen> mistral-actions mistral-actions-lib are both good IMO
16:35:35 <d0ugal> mgershen: +1
16:35:46 <rakhmerov> mistral-actions may be slightly confusing if we want to make it a lib for developing actions but not for actions themselves
16:35:54 <dprince> d0ugal: I like the idea. I think it is based on feedback that Ryan and I gave to Renat at the summit
16:36:10 <rakhmerov> as far as actions themselves, like OpenStack actions, we could use mistral-extra that already exists
16:36:17 <d0ugal> dprince: Yes, I think so. That is why I thought you might have an opinion :) but I guess you need time to read it
16:36:42 <dprince> d0ugal: yeah. I'll read it more carefully :)
16:36:45 <rakhmerov> dprince: yes, I'm now trying to aggregate all thoughts on this and write up an initial spec
16:37:01 <dprince> d0ugal: I like the idea of tripleo-workflows too
16:37:21 <dprince> d0ugal: FWIW I initially organized it like this last year but people weren't buying it at the time: https://github.com/dprince/tripleo-mistral-workflows
16:37:58 <rakhmerov> what were they saying on that?
16:38:01 <rakhmerov> why not?
16:38:15 <dprince> rakhmerov: I think it was just to early to debate where things lived at the time
16:38:22 <rakhmerov> ok
16:38:37 <d0ugal> Yeah, I don't even remember the location being in the discussion :)
16:38:40 <dprince> d0ugal: that said, I would like to see us make it work ASAP... and actually wire it in soon
16:38:46 <dprince> d0ugal: regardless of where it lives
16:39:05 <rakhmerov> ok, so early feedback would be appreciated (I mean that spec)
16:39:10 <d0ugal> dprince: +1
16:39:16 <rakhmerov> I'll try to finish it asap
16:39:21 <rakhmerov> ideally tomorrow
16:39:22 <dprince> rakhmerov: absolutely, thanks for that
16:39:33 <rakhmerov> with as many details as I can come up with at this stage
16:39:41 <d0ugal> rakhmerov: I'll review tomorrow morning, I don't have much time left today.
16:39:44 <rakhmerov> dprince: no problem, I know that it's a pain
16:39:59 <rakhmerov> d0ugal: no problem, thanks
16:40:36 <rakhmerov> d0ugal, jpeeler: you guys are very welcome
16:40:48 <rakhmerov> I hope you'll enjoy working with us )
16:41:08 <d0ugal> Thanks, I am looking forward to it. Mistral is a neat project!
16:41:09 <jpeeler> thanks /me is still getting started
16:41:21 <rakhmerov> I'll try to assign something to you tomorrow (it's almost midnight here in Siberia :) )
16:41:40 <rakhmerov> that's all from my side on that topic
16:41:57 <rakhmerov> if you'd like to add something pls go ahead
16:42:01 <d0ugal> rakhmerov: wow, that is late :)
16:42:07 <rakhmerov> for me it seems like we're on the same page
16:42:15 <d0ugal> Yes, I think so.
16:42:18 <rakhmerov> d0ugal: yeah, but it's ok, I'm used to it
16:42:31 <d0ugal> I'm sure rbrady will also review the spec
16:42:41 <rbrady> I plan to :)
16:42:45 <rakhmerov> can afford being a little late for work tomorrow morning :)
16:42:58 <rakhmerov> rbrady: hi Ryan )
16:43:05 <rakhmerov> yes, please
16:43:12 <rakhmerov> it's not finished yet though
16:43:19 <rakhmerov> ok
16:43:24 <rakhmerov> #topic Open Discussion
16:44:16 <rakhmerov> one more thing that I'd like you to have a look at is the scope for Newton-2
16:44:17 <rakhmerov> https://launchpad.net/mistral/+milestone/newton-2
16:44:44 <rakhmerov> if you want to add something to it, it's a high time you did that )
16:44:56 <rakhmerov> for example, if something is very much needed
16:45:04 <rakhmerov> bugs and blueprints
16:45:51 <dprince> rakhmerov: hi, so has anybody every proposed an idea around on_fail error handling
16:46:12 <rakhmerov> dprince: what's that?
16:46:13 <dprince> rakhmerov: like we have on_error... but it doesn't handle the case where an unhandled exception is thrown
16:46:39 <rakhmerov> dprince: ooh, you mean errors not related to action execution?
16:46:44 <dprince> rakhmerov: I think it is a corner case rbrady and I hit in the past few weeks
16:47:04 <dprince> rakhmerov: perhaps, this would be like an unhandled stack trace from any action
16:47:10 <dprince> rakhmerov: on_error doesn't get it
16:47:25 <d0ugal> It's a bit like in unittests, they can pass, fail or error.
16:47:30 <rakhmerov> dprince: yes, right. It only handles "expected" errors provided by actions
16:47:43 <dprince> rakhmerov: normally how I debug this is by looking at the Execution result itself
16:47:45 <rakhmerov> hm.. interesting
16:47:51 <rakhmerov> yes
16:47:58 <dprince> rakhmerov: it is an idea and I think we are perhaps missing something here
16:48:10 <rakhmerov> may be, yes
16:48:17 <rakhmerov> need to think about it
16:48:30 <dprince> rakhmerov: our use case is we'd like to make sure that all errors (or failures) get Log a zaqar message accordingly
16:48:31 <rakhmerov> share your thoughts on this
16:49:34 <dprince> rakhmerov: I think we could draft up a spec around the problem and perhaps better approach it from that angle
16:49:56 <rakhmerov> dprince: well, at the first glance, from engine perspective it's not a big deal to handle such failures
16:50:01 <dprince> rakhmerov: but the general idea was encountered while rbrady and I were pair pramming to write these https://review.openstack.org/#/c/300200/10/workbooks/baremetal.yaml
16:50:08 <rakhmerov> dprince: yes, that would be helpful
16:50:48 <dprince> rakhmerov: cool, Yeah when I looked at the engine and how exceptions are already caught I don't think this is a big change
16:51:03 <dprince> rakhmerov: we'd just want to get the interface around it correct
16:51:09 <dprince> rakhmerov: another idea for you
16:51:11 <rakhmerov> dprince: I'm now actively making it much much simpler
16:51:40 <rakhmerov> dprince: honestly, we had this part slightly messy bit it's not getting improved
16:51:41 <dprince> rakhmerov: has anyone ever proposed the idea around passing input parameters via on_error/on_complete/on_success?
16:51:46 <rakhmerov> I mean how we handle errors
16:52:14 <dprince> rakhmerov: like it would be super cool to be able to re-use a generic error handler by passing parameters to it
16:52:23 <rakhmerov> dprince: yes, I guess you mean something like https://blueprints.launchpad.net/mistral/+spec/mistral-explicit-task-parameters
16:52:31 <rakhmerov> have a look
16:52:36 <rakhmerov> I guess this is it
16:53:11 <dprince> rakhmerov: okay, so this would be acceptable it seems like
16:53:19 <dprince> rakhmerov: just hasn't been implemented yet :)
16:53:22 * d0ugal needs to run, thanks for the meeting. I'll catch up with the end in the notes.
16:53:23 <rakhmerov> hparekh_ has already made some preparation steps towards that BP as part of another BP
16:53:25 <dprince> rakhmerov: opportunity
16:53:32 <rakhmerov> dprince: right
16:54:07 <rakhmerov> dprince: although, this thing is slightly ambiguous
16:54:21 <rakhmerov> if we look into details
16:54:48 <rakhmerov> not even ambiguous, we basically need to define its semantics well
16:55:10 <rakhmerov> like what do those parameters mean exactly?
16:55:31 <rakhmerov> I see it as just a shortcut for publishing variables into context
16:55:59 <rakhmerov> other opinions were different
16:56:25 <rakhmerov> like we should run such tasks in some isolated data context if we explicitly pass parameters
16:57:05 <rakhmerov> or, another option: if we pass params explicitly then we just override values for of task action parameters
16:57:12 <rakhmerov> e.g.:
16:57:14 <dprince> rakhmerov: there are some parameters in the context that aren't easy to access unless you pass them explicitly I think
16:57:15 <rakhmerov> task1:
16:57:34 <rakhmerov> action: my_param= <% $.some_yaql%>
16:57:43 <dprince> rakhmerov: like the error status/message from the calling task... in a generic fashion
16:57:51 <rakhmerov> if we do task1("val") we would just ignore that yaql
16:58:09 <rakhmerov> dprince: yeah
16:58:26 <rakhmerov> so, I mean there's something to discuss on that
16:58:53 <rakhmerov> I think we need a spec actually for that, too
16:59:06 <rakhmerov> to list out those options and choose the best one
16:59:15 <dprince> rakhmerov: yes, probably
16:59:32 <rakhmerov> #action rakhmerov: have someone to write a spec for explicit task params BP
16:59:40 <rakhmerov> ok, time's up
17:00:00 <rakhmerov> it was good to have you all on the meeting!
17:00:05 <rakhmerov> thanks for joining )
17:00:08 <rakhmerov> bye
17:00:13 <hparekh_> Thanks bye !!!
17:00:21 <rakhmerov> #endmeeting