16:01:12 <rakhmerov> #startmeeting Mistral
16:01:12 <gokrokve> hi
16:01:13 <openstack> Meeting started Mon Nov 11 16:01:12 2013 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:14 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:16 <openstack> The meeting name has been set to 'mistral'
16:01:39 <igormarnat> Hey there
16:01:48 <akuznetsov> Hi
16:01:59 <rakhmerov> Let's discuss things happened within the last couple of weeks
16:02:19 <rakhmerov> so we didn't have a meeting last Monday due to summit activities
16:02:49 <rakhmerov> just a couple of words about the summit itself and how everything went regarding Mistral
16:03:06 <rakhmerov> I think it was actually pretty successful
16:03:25 <rakhmerov> we had a lightning talk and an Unconference track
16:04:00 <rakhmerov> and Mistral was also discussed at other design sessions
16:04:41 <rakhmerov> for example, we were at the design session of Heat where they were trying to decide how to switch to TaskFlow
16:05:00 <rakhmerov> and I had a couple minutes to tell briefly about what Mistral is going to be
16:05:24 <rakhmerov> they definitely liked the idea of the project itself and seem to have endorsed it completely
16:05:56 <rakhmerov> and they offered their help if we need
16:06:12 <rakhmerov> I think we will be in touch with them anyway
16:06:46 <gokrokve> Sure.
16:06:54 <gokrokve> We plan to work with Heat team.
16:07:07 <rakhmerov> yep
16:07:10 <gokrokve> Actually we already spoke with Steven about contribution to Heat.
16:07:45 <rakhmerov> there were also a number of people who approached me between the sessions and asked questions about Mistral
16:08:15 <rakhmerov> there was a guy who said "This is really what I need in my work, I'm going to use it when it's done"
16:08:44 <rakhmerov> so I think all went pretty well, I'm glad to realize that
16:09:19 <gokrokve> Cool.
16:09:21 <rakhmerov> Solum guys I believe will be using Mistral too when they get to some more complicated scenarios
16:09:38 <rakhmerov> they will definitely need it
16:09:48 <gokrokve> It actually gives some pressure on us too. We have to deliver something valuable during Icehouse release.
16:10:05 <rakhmerov> yes, I think we'll be able to
16:10:24 <rakhmerov> the game is becoming more interesting :)
16:10:28 <rakhmerov> and exciting
16:10:32 <gokrokve> Which means, that we have to plan Mistral very carefully, to cover important common parts.
16:10:46 <rakhmerov> yes, you're right
16:11:03 <rakhmerov> that's why we got some new folks in our team
16:11:11 <rakhmerov> as akuznetsov
16:11:12 <gokrokve> So, do you have a list of features people asked for?
16:11:29 <gokrokve> Great. Welcome Alex!
16:11:32 <rakhmerov> Alex, welcome to the team!
16:11:41 <rakhmerov> I hope we'll be cool things together
16:11:48 <rakhmerov> * we'll do
16:11:49 <akuznetsov> thank you
16:12:27 <rakhmerov> gokrove, yes, I have some notes after conversations with people, they will be reflected in the documenation
16:13:06 <gokrokve> In BPs?
16:13:14 <rakhmerov> mostly, they're related with some more advanced functionality that we haven't thought so far
16:13:39 <gokrokve> Let's follow OpenStack way. If we have a feature let's document it in BP on the launchpad.
16:13:41 <rakhmerov> yes, as far as BPs we need to start publishing all our ideas as BPs
16:13:48 <gokrokve> So everybody can see it and comment.
16:13:56 <rakhmerov> gokrokve, agree
16:14:00 <rakhmerov> right
16:14:15 <rakhmerov> ok, now let's review the previous action items
16:14:28 <gokrokve> After we have all BPs, we can go to IRC canells asking for review.
16:14:35 <rakhmerov> ok
16:14:38 <ruhe> hi folks. is mistral going to be a rest api service for taskflow?
16:14:48 <gokrokve> Probably we will be able to collect some feedback from other teams, and heat in particular
16:15:12 <rakhmerov> ruhe, not exactly for taskflow
16:15:19 <gokrokve> That not exactly this way. Mistral is an API for workflows orchestration.
16:15:26 <rakhmerov> yes
16:15:28 <gokrokve> It will use taskflow under the hood.
16:15:41 <gokrokve> But it will have more features.
16:15:46 <rakhmerov> much more
16:15:47 <igormarnat> But it gonna be more than rest api for task flow.
16:15:53 <akuznetsov> In OpenStack with Mistral and TaskFlow project will have two way for workflow creation, we should give recommendation in which case Mistral or TaskFlow should be used
16:16:06 <gokrokve> Like even management and scheduled tasks.
16:16:09 <rakhmerov> and address some completely different use cases that can't be addressed by taskflow
16:16:47 <rakhmerov> akuznetsof, good idea. We need to start writing on this
16:16:50 <gokrokve> Yes. We are targeting more high level use cases. Like backup execution by schedule, live migration on event and so on.
16:17:34 <ruhe> could mistral be used as underlying infrastructure for Heat orchestration engine?
16:17:51 <rakhmerov> ruhe, yes, it could
16:17:56 <gokrokve> Mistral will have its own DSL to express task flows. Soy you will not need to create a code, Mistral will generate the code for task flow.
16:18:21 <gokrokve> Actually this was discussed on the summit during Heat design session.
16:18:36 <rakhmerov> we discussed it with Heat team and they positively look at it
16:18:46 <gokrokve> So, yes in general, but when Mistral will have at least some maturity.
16:19:03 <rakhmerov> but they want to start using something pretty soon
16:19:04 <rakhmerov> yes
16:19:24 <gokrokve> That is why we plan to work with Heat team closely, to have some feedback from them.
16:20:01 <rakhmerov> probably they will start using TaskFlow and then they'll continue with the engine for TaskFlow that schedules tasks across Mistral
16:20:25 <gokrokve> They will not probably be able to help writing the code but at least they may help us to make a list of required features.
16:20:26 <rakhmerov> that's going to be another point of integration between TaskFlow and Mistral
16:20:46 <rakhmerov> yes, that's what we need from them mostly
16:21:05 <rakhmerov> those guys I think are too busy to be able to write some code for us
16:21:07 <katyafervent> So the project design will be be as it is now, and there is no changes made after summit. Is it correct?
16:21:15 <rakhmerov> at least that's what I heard from them
16:21:43 <rakhmerov> katyafervent, yes, it's 99% true
16:22:06 <rakhmerov> people accepted our ideas just as they are
16:22:29 <rakhmerov> with some slight changes that are mostly related with some additional advanced features
16:23:17 <rakhmerov> I'll share all of this pretty soon
16:23:25 <katyafervent> rakhmerov, that's great! So now we need to read it deeply
16:23:27 <ruhe> i wonder, what tools are you going to use to store distributed state? zookeeper/sql/smth else?
16:24:01 <gokrokve> That is a good question.
16:24:18 <rakhmerov> there's a bunch of options, at this point it's not decided yet. You can contribute your thoughts if you want :)
16:24:38 <gokrokve> Right now I don't see any intentions to use external tools for that.
16:25:08 <gokrokve> We don't have a line of code, so we can properly design this from initial stage.
16:25:09 <akuznetsov> possibly we should have a plugble system for state storage
16:25:20 <rakhmerov> I think we first need to start diving deeper into design
16:25:33 <rakhmerov> yes, I would love to see a way to abstract it out
16:26:10 <rakhmerov> guys, I would also ask all of you to use openstak-dev mailing list more actively
16:26:24 <rakhmerov> you got an idea - you share it via the mailing list
16:26:35 <rakhmerov> and it could be absolutely anything
16:26:52 <rakhmerov> like "Where do I need to put a comma in this line?"
16:27:05 <rakhmerov> so that people could participate
16:27:41 <gokrokve> Agree.
16:27:45 <rakhmerov> I think we hasitate to do it frequently but we need to change our mindset here
16:27:57 <gokrokve> Yep.
16:28:03 <rakhmerov> communication is everything
16:28:18 <gokrokve> So, Renat will you take an AI to create a list of BP for the features?
16:28:49 <rakhmerov> #action rakhmerov, create a list of Blueprints for Mistral features
16:28:50 <gokrokve> Publish them on Launchpad and them start discussion in mailing list.
16:29:00 <rakhmerov> ok
16:29:23 <rakhmerov> so, now let's get back to some previous action items, just to be consistent
16:29:32 <gokrokve> We also need to strat conversation about design in general.
16:29:37 <igormarnat> We can shortly discuss list of potential topics here. For now there was a question about the way to keep track of state
16:29:41 <rakhmerov> #topic Previous Action Items
16:29:42 <gokrokve> ok
16:30:12 <rakhmerov> 1. rakhmerov, stanlagun, tsufiev, keep working on DSL/API Mistral specification
16:30:16 <igormarnat> Anyway, let's stick to the agenda now and return back to potential questions later
16:30:57 <rakhmerov> this is still in progress and we now have a target date Nov 20 to complete this
16:31:12 <rakhmerov> after the summit we have some new things to include into the specs
16:31:22 <rakhmerov> so, I'll transfer it on
16:31:37 <igormarnat> Let's add one more owner for this AI
16:31:44 <igormarnat> akuznetsov
16:32:09 <rakhmerov> #action rakhmerov, akuznetsov, stanlagun, tsufiev, keep working on DSL/API Mistral specification and include all features discussed at the summit
16:32:12 <rakhmerov> ok
16:32:29 <rakhmerov> 2. rakhmerov, finish detailed description of "Long-running Business Process" use case
16:32:46 <rakhmerov> so regarding this one, I've done it
16:33:31 <rakhmerov> however, we decided not to publish it on the wiki for a while because it's not what we're going to target within the next 6 months
16:34:07 <rakhmerov> I really think, because of the structure of the wiki we have here it could be published actually
16:34:16 <rakhmerov> and accessible via "Read more" link
16:34:33 <rakhmerov> I personally don't see any harm in doing that
16:34:44 <rakhmerov> what do you think?
16:35:12 <rakhmerov> I'm talking about that use case where we calculate salary using several enterprise systems
16:35:59 <rakhmerov> anyway, this documentation will be evolving all the time
16:36:26 <rakhmerov> if you agree, I'll create an AI to publish it
16:37:11 <rakhmerov> gokrokve, igormarnat?
16:37:27 <gokrokve> Yes. publish it. But add a note that this is not a goal. It is just an example how to use it in general as a generic engine.
16:37:40 <rakhmerov> sure, ok
16:37:57 <igormarnat> We can publish it, no issue, though I don't see a lot of value in that for now:)
16:38:10 <igormarnat> Anyway, let's do it
16:38:14 <rakhmerov> #action rakhmerov, publish detailed description of "Long-running business process" use case on wiki
16:38:23 <rakhmerov> yes, it's a 10 min thing to do :)
16:38:27 <rakhmerov> ok
16:38:31 <rakhmerov> the next one
16:38:46 <rakhmerov> 3. stanlagun, finish detailed description of "Post Deployment Configuration" use case
16:38:49 <rakhmerov> done
16:39:02 <rakhmerov> but under a different name
16:39:40 <rakhmerov> #info "Cloud Environment Deployment" use case, https://wiki.openstack.org/wiki/Mistral/Cloud_Environment_Deployment_details
16:40:19 <rakhmerov> 4. ativelkov, collect the info about use case related to setting up a network (not a high priority)
16:40:32 <rakhmerov> in progress, not a high priority
16:40:48 <rakhmerov> #action ativelkov, collect the info about use case related to setting up a network (not a high priority)
16:41:04 <rakhmerov> 5. rakhmerov, prepare the first version of Roadmap and the development plan
16:41:29 <rakhmerov> the first version of Roadmap is done and event shared at the summit
16:41:43 <rakhmerov> but I still didn't share it anywhere else
16:42:00 <igormarnat> So here comes the next AI for us
16:42:18 <rakhmerov> what ways of sharing it would you recommend guys? Except wiki, of course
16:42:23 <igormarnat> To put together AIs and agreements achieved in HK and publish them in mailing list
16:42:35 <rakhmerov> yes
16:43:17 <rakhmerov> #action akhmerov, put together AIs and agreements achieved in HK and publish them in mailing list
16:44:08 <rakhmerov> the Roadmap we have now is kind of a draft, how can we discuss it with the community in the best way?
16:44:12 <rakhmerov> mailing list?
16:44:22 <rakhmerov> or some sort of a Blueprint?
16:44:25 <igormarnat> Let's discuss them prior to publishing though in order to see if me and Gosha and Alexander can add anything
16:44:38 <rakhmerov> yes, ok
16:45:12 <rakhmerov> #action gokrokve, igormarnat, akuznetsom review Roadmap and add whatever you think needs to be added in there
16:45:45 <rakhmerov> #action rakhmerov, share Roadmap with the community to start discussing it
16:46:38 <rakhmerov> #topic Free discussion
16:46:53 <rakhmerov> anything else guys that you'd like to discuss?
16:46:59 <igormarnat> Great. Let's return back to the list of questions we had
16:47:13 <igormarnat> The first one was the way to track states
16:47:20 <rakhmerov> ok
16:47:45 <igormarnat> ruhe, akuznetsov, what else was there?
16:48:01 <ruhe> the only thing i wanted to mention...
16:48:15 <ruhe> there is a project https://github.com/airbnb/chronos which might be a good source of inspiration :)
16:48:21 <rakhmerov> so I see it could be encapsulated in a subsystem of Mistral from the very beginning so that we may vary the actual storage whenever we want
16:48:46 <rakhmerov> #info https://github.com/airbnb/chronos
16:48:51 <rakhmerov> thanks!
16:49:41 <rakhmerov> you mean in the context of storing a distributed state?
16:49:52 <ruhe> storing and executing
16:49:57 <rakhmerov> ok
16:50:18 <ruhe> chronos uses http://mesos.apache.org/ which itself might be a good engine for distributed task execution
16:50:23 <akuznetsov> we should define what part of workflow functionality in Mistral and which in TaskFlow
16:51:28 <rakhmerov> #action rakhmerov, akuznetsov, NikolayM review https://github.com/airbnb/chronos and http://mesos.apache.org/  with regard to storing and executing distributed tasks
16:52:04 <rakhmerov> akuznetsov, yes, I believe TaskFlow is currently missing some stuff required for Mistral
16:52:28 <rakhmerov> I think we need to identify that as soon as possible
16:52:56 <igormarnat> rakhmerov, not only that, but we might want to describe explicitly what's the difference between them and when one use Taskflow, when does he use Mistral
16:53:11 <igormarnat> Description of what's need to be added to Taskflow is a separate topic
16:53:22 <rakhmerov> and start discussing it with TaskFlow folks so that they either implement that themselves or let us contribute, or both
16:53:43 <rakhmerov> ooh, apologize. I may have misunderstood that
16:53:58 <rakhmerov> yes, got it now
16:54:35 <igormarnat> Are there any news from our big and powerful prototyping team? What did we prototype already? :)
16:55:28 <rakhmerov> #action rakhmerov, kuznetsov, harlowja clearly describe the high-level separation of functionality between TaskFlow and Mistral in terms of addressed use cases
16:55:35 <rakhmerov> sounds good enough?
16:56:06 <rakhmerov> ok
16:56:10 <rakhmerov> 4 mins left
16:56:15 <igormarnat> LGTM
16:56:23 <rakhmerov> ok
16:57:29 <rakhmerov> looks like we have a bunch of AIs this time :)
16:57:33 <rakhmerov> a lot of work
16:58:06 <rakhmerov> anything else?
16:58:45 <rakhmerov> ok, they we'll end for today
16:59:04 <rakhmerov> thanks to everyone! I'll see you all soon
16:59:05 <igormarnat> Ok, thanks for coming, everyone. TTYL, BFN:)
16:59:13 <rakhmerov> #endmeeting