16:00:16 <rakhmerov> #startmeeting Mistral
16:00:16 <openstack> Meeting started Mon Dec  1 16:00:16 2014 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:18 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:21 <openstack> The meeting name has been set to 'mistral'
16:00:31 <rakhmerov> hi
16:01:04 <NikolayM> hi !
16:02:30 <rakhmerov> NikolayM, is Jenkins now working fine?
16:02:44 <NikolayM> yes
16:02:55 <NikolayM> two patches is already merged
16:03:04 <rakhmerov> ok
16:03:55 <dzimine> hi all
16:04:03 <rakhmerov> I wonder why devstack gate failed for https://review.openstack.org/#/c/137763/
16:04:09 <rakhmerov> Nikolay, please take a look
16:04:14 <rakhmerov> hi dzimine
16:04:21 <rakhmerov> ok, let's start
16:04:38 <NikolayM> rakhmerov, it is just ok
16:04:45 <rakhmerov> why?
16:05:10 <rakhmerov> was it before Nastya's patch merged in or after that?
16:05:23 <NikolayM> before
16:05:35 <rakhmerov> ok, then it should be fine
16:05:37 <rakhmerov> agenda: https://wiki.openstack.org/wiki/Meetings/MistralAgenda
16:05:39 <NikolayM> after Nastya's patch devstack works fine
16:05:44 <rakhmerov> ok
16:05:55 <rakhmerov> #topic Review Action Items
16:06:07 <rakhmerov> we had no previous action items
16:06:28 <rakhmerov> #topic Current status (progress, issues, roadblocks, further plans)
16:08:25 <rakhmerov> my status: I took PTO last Friday. Almost finished fixing that race condition, today I changed the design a little bit and just want to do more testing. There's still something that I need to think about it though but I may want to postpone it for a while and switch to "join"
16:09:15 <dzimine> nothing on my side or winson's : long break last week in the US
16:09:53 <rakhmerov> I'm a little confused. Was it really a whole week break?
16:09:55 <NikolayM> added advanced tests on workflow-resume, fixed resume algorithm, fixed "Application context not found" error in tests, fixed bug when we didn't see error at action class init
16:10:09 <rakhmerov> I thought thanksgiving is just one day
16:10:50 <dzimine> yes, but as it's Thu, everyone is taking Fri off.
16:11:11 <rakhmerov> NikolayM, very good ) I'm so glad we fixed that "Application context not found" error
16:11:17 <rakhmerov> it was a pain in the butgt
16:11:27 <rakhmerov> dzimine, ok
16:11:48 <rakhmerov> NikolayM, I think that's why nobody answered us in ML about WSME
16:12:12 <rakhmerov> #topic Release "Kilo-1" progress
16:12:23 <NikolayM> I also think so :)
16:13:10 <NikolayM> rakhmerov, what is left for Kilo-1?
16:13:26 <rakhmerov> yep, just a sec
16:13:47 <rakhmerov> ok, as far as "kilo-1" release we now have effectively 2 BPs that didn't start yet
16:13:54 <rakhmerov> "pause-before" and HA
16:14:21 <rakhmerov> Bryan was going to start "pause-before" this week as far as I remember
16:14:29 <rakhmerov> and it's pretty simple to do I think
16:14:42 <rakhmerov> if he has some problems we'll help him
16:14:52 <dzimine> is "join" in Kilo-1?
16:15:08 <rakhmerov> #action rakhmerov, get in touch with Bryan to see how his work's going
16:15:13 <rakhmerov> dzimine, yes
16:15:29 <rakhmerov> I'll continue to work on it tomorrow
16:16:06 <rakhmerov> by the end of this week most likely I'll get it done
16:17:08 <rakhmerov> Nastya is also about to start doing HA testing, she's now figuring out what's needed: where and how we can do it
16:17:39 <rakhmerov> I don't think this is something that could be done just till the end of this cycle
16:17:41 <rakhmerov> though
16:17:49 <NikolayM> we have mistral 0.2 planning doc at https://docs.google.com/a/mirantis.com/document/d/1ycCOBS4gj5-3vbHjBEThTJ9gSNOOWGpX0kgdcfVw9IY/edit#
16:17:52 <dzimine> before or after rakhmerov's fix?
16:17:57 <rakhmerov> IMO, this is an ongoing piece of work
16:18:03 <NikolayM> don't dorget about it :)
16:18:20 <rakhmerov> we won't
16:18:25 <rakhmerov> dzimine, what do you mean?
16:19:14 <rakhmerov> regarding HA testing I mean that we'll do some basic testing for now to make sure our main functionality works at scale
16:19:28 <rakhmerov> but we'll keep working on that moving on
16:19:45 <dzimine> I mean any load may be problematic without the fix that you have on a review.
16:19:56 <dzimine> Anyways it'll take care of itself :)
16:20:02 <dzimine> by exposing the bugs.
16:20:10 <rakhmerov> ooh, I hope it will be in in 1 day
16:20:16 <rakhmerov> in the worst case in 2 days
16:20:30 <rakhmerov> yes :))
16:20:46 <dzimine> Nastya do you have a plan on what/how to test?
16:21:26 <dzimine> we had a weird bug when workflow locked on 2 cpu VM. I'll ask Winson to report it.
16:21:35 <rakhmerov> btw, dzimine, if you have your own requirements on what needs to be tested first (I mean your interest) please share
16:21:56 <rakhmerov> yes, you told me but it's really weird
16:22:12 <rakhmerov> would like to see some details (may be logs?)
16:22:25 <rakhmerov> I think Nastya is not here today
16:23:04 <dzimine> yes, I think he managed to repro it on his dev env, I have just asked him to file it.
16:23:24 <rakhmerov> we already discussed the general plan and she definitely has a high-level understanding
16:23:34 <rakhmerov> ok
16:23:42 <rakhmerov> would be helpful
16:24:24 <rakhmerov> btw, dzimine, did you try Mistral in multiprocess configuration on some of your tasks?
16:24:52 <rakhmerov> it's when you have separate engines and executors (may be even api's)
16:25:11 <dzimine> no.
16:25:17 <rakhmerov> ok
16:28:04 <rakhmerov> ok, I think we're mostly on target. Productivity after summits is always not the best but I feel it's not increasing :)
16:28:14 <rakhmerov> #topic Open Discussion
16:28:25 <rakhmerov> any questions to discuss from your side?
16:28:30 <dzimine> ok, little thing:
16:28:38 <rakhmerov> some concerns, roadblocks?
16:28:38 <rakhmerov> yes
16:28:49 <dzimine> task defaults with reverse workflows, obviously don't work, right?
16:29:02 <rakhmerov> ....it's now increasing....
16:29:10 <dzimine> the question is how to set up a default failure policy with reverse workflow.
16:29:32 <rakhmerov> dzimine, we need to check it
16:29:35 <dzimine> with "join" implemented it may not be pressing, but at least need t odocument.
16:29:38 <rakhmerov> I'm not sure it doesn't work
16:30:04 <rakhmerov> I guess policies should work with task defaults in case of reverse workflows
16:30:07 <dzimine> honestly I didn't check if it can work, was just curious
16:30:12 <rakhmerov> let us check that pls
16:30:14 <dzimine> let's check.
16:30:18 <rakhmerov> yep
16:30:56 <dzimine> Next: I am thinking we (I?) should do a bit more in writing specs.
16:30:58 <rakhmerov> #action akuznetsova: check if task-defaults work for reverse workflows (policies)
16:31:03 <dzimine> We do write blueprints,
16:31:16 <dzimine> but not in hi level of details.
16:31:22 <rakhmerov> agree
16:31:36 <rakhmerov> it's all about priorities, you know
16:31:39 <dzimine> and than, when feature is implemented 1) not all corner cases are there,
16:31:50 <dzimine> and 2) we dont know if we are done or not.
16:31:59 <rakhmerov> yes
16:32:01 <dzimine> Example: for-each.
16:32:13 <dzimine> It's so good to have it in,
16:32:20 <rakhmerov> strictly speaking, "for-each" is not 100% ready
16:32:27 <dzimine> now I have a bag of feedback on this.
16:32:40 <rakhmerov> result ordering, sequential processing are not done
16:32:46 <NikolayM> for-each for now has a set of limitations
16:33:26 <rakhmerov> dzimine, can you  pls share that feedback in some form? BPs, specs?
16:33:50 <rakhmerov> NikolayM, please create new BPs for what's left on "for-each"
16:33:55 <dzimine> well that's my point, "when"? is it ready? what is left? where do we track this? all this project management routines that keep it safe
16:34:15 <rakhmerov> #action: NikolayM, create new BPs for what's left on "for-each"
16:34:30 <NikolayM> ok
16:34:43 <dzimine> So the smallest step I think is to put more details into the blueprint, that describes in more details on what needs to be done, how it is expected to work, etc.
16:35:06 <rakhmerov> https://blueprints.launchpad.net/mistral/+spec/mistral-dataflow-collections
16:35:26 <dzimine> rakhmerov and I have discussed pretty deep corner cases of for-each, I can help capture them in a blueprint.
16:35:49 <rakhmerov> let's just have a discussion in the BP itself (I know it's not very convenient but there isn't a better way now I think)
16:36:01 <rakhmerov> ok
16:36:35 <dzimine> And a new one that we thought of and people just asked to do is to introduce a "parallel" count to "throttle" the execution. parallel=1 makes it sequential.
16:36:41 <rakhmerov> I am ok with creating new BPs if you find it convenient
16:37:11 <rakhmerov> you still mean "for-each" thing?
16:37:16 <dzimine> yes
16:37:40 <rakhmerov> ooh, you're saying it's not just a binary choice "parallel" or "sequential"
16:37:43 <rakhmerov> ok
16:37:57 <rakhmerov> I see
16:38:20 <rakhmerov> NikolayM, it's going to be even much more interesting than we thought )
16:38:49 <rakhmerov> I think it's a nice feature
16:39:48 <dzimine> I'll type it up in the etherpad referenced from the blueprint.
16:39:51 <rakhmerov> because if we, for example, have an array of a thousand elements and need to create a thousand parallel iterations then it may be an overage for just one task
16:39:58 <rakhmerov> ok
16:40:11 <rakhmerov> this is very good
16:40:29 <dzimine> But back to a general comment, let's do a write-up on what we implment, and if there are any cut corners call them out after implementation.
16:40:51 <rakhmerov> ok
16:41:31 <dzimine> the use case for parallel throttling is also concern on the target system. In our example it's VMWare vCenter and the user wants to keep it at no more than 5 new VM requests at a time.
16:41:55 <rakhmerov> yes, I'm actually thrilled to get "join" and "for-each" asap
16:42:19 <rakhmerov> yeah, you told me
16:43:03 <rakhmerov> but when we were talking about it I was thinking about a system-wide throttling rather than a "for-each" parameter
16:43:11 <rakhmerov> it should be both eventually though
16:44:17 <rakhmerov> #action dzimine, put information about additional features/requirements for "for-each" in the etherpad
16:44:26 <dzimine> ^^ done.
16:44:32 <rakhmerov> ok
16:44:44 <rakhmerov> one question that I have
16:45:01 <rakhmerov> do we need to support Keystone API v2 in Mistral?
16:45:06 <rakhmerov> any input on that?
16:45:09 <dzimine> Nikolay, how do you track what's not done in for-each? E.g., ordering of results, handling multiple arrays, etc?
16:45:31 <dzimine> Keystone v2 - anyone asked for it?
16:45:39 <rakhmerov> yes, that's the point
16:45:49 <dzimine> what's the context&
16:45:50 <dzimine> ?
16:45:55 <rakhmerov> 2-3 people last week asked about it
16:46:45 <rakhmerov> the point is: they have Icehouse OpenStack where they don't have Keystone API v3 enabled because they think it's still experimental
16:47:08 <rakhmerov> my understanding
16:47:15 <NikolayM> dzimine, it is in my notepad
16:47:30 <rakhmerov> unfortunately, I don't have a comprehensive info on that topic at the moment
16:48:06 <rakhmerov> so they said: can we have Mistral work with v2 or create some workaround for that?
16:48:19 <rakhmerov> the problem is that we use trusts
16:48:36 <rakhmerov> which actually don't work well too for us
16:49:06 <rakhmerov> but at least the concept of trusts seems to be helpful for us
16:49:46 <rakhmerov> so, I'm not expecting us to answer that right now but we need to figure that out within days
16:50:00 <rakhmerov> already create a BP (in discussion status)
16:50:03 <rakhmerov> created
16:50:43 <rakhmerov> you can leave your thoughts in it: https://blueprints.launchpad.net/mistral/+spec/mistral-keystone-api-v2-support
16:50:43 <dzimine> what will it take to support v2? did someone looked at it? NikolayM you know keystone pretty well?
16:51:42 <rakhmerov> the whole problem is in auth token expiration
16:52:12 <rakhmerov> for example, when a trigger runs a workflow we don't have user credentials
16:53:16 <rakhmerov> there are only two ways: authenticate using Mistral's system credentials (from config) or use a previously trust (basically impersonated credentials)
16:53:32 <rakhmerov> the first way is ugly because it may be a security breach
16:54:36 <rakhmerov> honestly, I don't see a good solution at this point
16:55:06 <rakhmerov> technically, if we come up with a solution it should be easy to implement
16:55:58 <rakhmerov> Dmitri, maybe you could discuss this thing with your guys
16:56:11 <dzimine> we don't use much of keystone.
16:56:34 <rakhmerov> yeah, but some of you may have seen a similar problem before
16:56:41 <rakhmerov> ok
16:56:43 <dzimine> and the problem with credentials is not limited to keystone. Talk to Fuel guys, they'll need sets of credentials to all sorts of targets
16:56:54 <rakhmerov> I know
16:56:55 <rakhmerov> yes
16:57:45 <rakhmerov> I heard that keystone folks were planning to drop trusts completely, btw, and introduce a different thing
16:57:55 <rakhmerov> but didn't see any detailed description so far
16:58:14 <rakhmerov> ok, let's finish for today
16:58:28 <rakhmerov> about a minute and a half left
16:58:43 <rakhmerov> thanks for coming
16:59:03 <rakhmerov> Dmitri, let's be in touch with you. We need your input
16:59:12 <rakhmerov> bye
16:59:29 <rakhmerov> #endmeeting