16:01:12 #startmeeting Mistral 16:01:12 hi 16:01:13 Meeting started Mon Nov 11 16:01:12 2013 UTC and is due to finish in 60 minutes. The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:01:14 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:01:16 The meeting name has been set to 'mistral' 16:01:39 Hey there 16:01:48 Hi 16:01:59 Let's discuss things happened within the last couple of weeks 16:02:19 so we didn't have a meeting last Monday due to summit activities 16:02:49 just a couple of words about the summit itself and how everything went regarding Mistral 16:03:06 I think it was actually pretty successful 16:03:25 we had a lightning talk and an Unconference track 16:04:00 and Mistral was also discussed at other design sessions 16:04:41 for example, we were at the design session of Heat where they were trying to decide how to switch to TaskFlow 16:05:00 and I had a couple minutes to tell briefly about what Mistral is going to be 16:05:24 they definitely liked the idea of the project itself and seem to have endorsed it completely 16:05:56 and they offered their help if we need 16:06:12 I think we will be in touch with them anyway 16:06:46 Sure. 16:06:54 We plan to work with Heat team. 16:07:07 yep 16:07:10 Actually we already spoke with Steven about contribution to Heat. 16:07:45 there were also a number of people who approached me between the sessions and asked questions about Mistral 16:08:15 there was a guy who said "This is really what I need in my work, I'm going to use it when it's done" 16:08:44 so I think all went pretty well, I'm glad to realize that 16:09:19 Cool. 16:09:21 Solum guys I believe will be using Mistral too when they get to some more complicated scenarios 16:09:38 they will definitely need it 16:09:48 It actually gives some pressure on us too. We have to deliver something valuable during Icehouse release. 16:10:05 yes, I think we'll be able to 16:10:24 the game is becoming more interesting :) 16:10:28 and exciting 16:10:32 Which means, that we have to plan Mistral very carefully, to cover important common parts. 16:10:46 yes, you're right 16:11:03 that's why we got some new folks in our team 16:11:11 as akuznetsov 16:11:12 So, do you have a list of features people asked for? 16:11:29 Great. Welcome Alex! 16:11:32 Alex, welcome to the team! 16:11:41 I hope we'll be cool things together 16:11:48 * we'll do 16:11:49 thank you 16:12:27 gokrove, yes, I have some notes after conversations with people, they will be reflected in the documenation 16:13:06 In BPs? 16:13:14 mostly, they're related with some more advanced functionality that we haven't thought so far 16:13:39 Let's follow OpenStack way. If we have a feature let's document it in BP on the launchpad. 16:13:41 yes, as far as BPs we need to start publishing all our ideas as BPs 16:13:48 So everybody can see it and comment. 16:13:56 gokrokve, agree 16:14:00 right 16:14:15 ok, now let's review the previous action items 16:14:28 After we have all BPs, we can go to IRC canells asking for review. 16:14:35 ok 16:14:38 hi folks. is mistral going to be a rest api service for taskflow? 16:14:48 Probably we will be able to collect some feedback from other teams, and heat in particular 16:15:12 ruhe, not exactly for taskflow 16:15:19 That not exactly this way. Mistral is an API for workflows orchestration. 16:15:26 yes 16:15:28 It will use taskflow under the hood. 16:15:41 But it will have more features. 16:15:46 much more 16:15:47 But it gonna be more than rest api for task flow. 16:15:53 In OpenStack with Mistral and TaskFlow project will have two way for workflow creation, we should give recommendation in which case Mistral or TaskFlow should be used 16:16:06 Like even management and scheduled tasks. 16:16:09 and address some completely different use cases that can't be addressed by taskflow 16:16:47 akuznetsof, good idea. We need to start writing on this 16:16:50 Yes. We are targeting more high level use cases. Like backup execution by schedule, live migration on event and so on. 16:17:34 could mistral be used as underlying infrastructure for Heat orchestration engine? 16:17:51 ruhe, yes, it could 16:17:56 Mistral will have its own DSL to express task flows. Soy you will not need to create a code, Mistral will generate the code for task flow. 16:18:21 Actually this was discussed on the summit during Heat design session. 16:18:36 we discussed it with Heat team and they positively look at it 16:18:46 So, yes in general, but when Mistral will have at least some maturity. 16:19:03 but they want to start using something pretty soon 16:19:04 yes 16:19:24 That is why we plan to work with Heat team closely, to have some feedback from them. 16:20:01 probably they will start using TaskFlow and then they'll continue with the engine for TaskFlow that schedules tasks across Mistral 16:20:25 They will not probably be able to help writing the code but at least they may help us to make a list of required features. 16:20:26 that's going to be another point of integration between TaskFlow and Mistral 16:20:46 yes, that's what we need from them mostly 16:21:05 those guys I think are too busy to be able to write some code for us 16:21:07 So the project design will be be as it is now, and there is no changes made after summit. Is it correct? 16:21:15 at least that's what I heard from them 16:21:43 katyafervent, yes, it's 99% true 16:22:06 people accepted our ideas just as they are 16:22:29 with some slight changes that are mostly related with some additional advanced features 16:23:17 I'll share all of this pretty soon 16:23:25 rakhmerov, that's great! So now we need to read it deeply 16:23:27 i wonder, what tools are you going to use to store distributed state? zookeeper/sql/smth else? 16:24:01 That is a good question. 16:24:18 there's a bunch of options, at this point it's not decided yet. You can contribute your thoughts if you want :) 16:24:38 Right now I don't see any intentions to use external tools for that. 16:25:08 We don't have a line of code, so we can properly design this from initial stage. 16:25:09 possibly we should have a plugble system for state storage 16:25:20 I think we first need to start diving deeper into design 16:25:33 yes, I would love to see a way to abstract it out 16:26:10 guys, I would also ask all of you to use openstak-dev mailing list more actively 16:26:24 you got an idea - you share it via the mailing list 16:26:35 and it could be absolutely anything 16:26:52 like "Where do I need to put a comma in this line?" 16:27:05 so that people could participate 16:27:41 Agree. 16:27:45 I think we hasitate to do it frequently but we need to change our mindset here 16:27:57 Yep. 16:28:03 communication is everything 16:28:18 So, Renat will you take an AI to create a list of BP for the features? 16:28:49 #action rakhmerov, create a list of Blueprints for Mistral features 16:28:50 Publish them on Launchpad and them start discussion in mailing list. 16:29:00 ok 16:29:23 so, now let's get back to some previous action items, just to be consistent 16:29:32 We also need to strat conversation about design in general. 16:29:37 We can shortly discuss list of potential topics here. For now there was a question about the way to keep track of state 16:29:41 #topic Previous Action Items 16:29:42 ok 16:30:12 1. rakhmerov, stanlagun, tsufiev, keep working on DSL/API Mistral specification 16:30:16 Anyway, let's stick to the agenda now and return back to potential questions later 16:30:57 this is still in progress and we now have a target date Nov 20 to complete this 16:31:12 after the summit we have some new things to include into the specs 16:31:22 so, I'll transfer it on 16:31:37 Let's add one more owner for this AI 16:31:44 akuznetsov 16:32:09 #action rakhmerov, akuznetsov, stanlagun, tsufiev, keep working on DSL/API Mistral specification and include all features discussed at the summit 16:32:12 ok 16:32:29 2. rakhmerov, finish detailed description of "Long-running Business Process" use case 16:32:46 so regarding this one, I've done it 16:33:31 however, we decided not to publish it on the wiki for a while because it's not what we're going to target within the next 6 months 16:34:07 I really think, because of the structure of the wiki we have here it could be published actually 16:34:16 and accessible via "Read more" link 16:34:33 I personally don't see any harm in doing that 16:34:44 what do you think? 16:35:12 I'm talking about that use case where we calculate salary using several enterprise systems 16:35:59 anyway, this documentation will be evolving all the time 16:36:26 if you agree, I'll create an AI to publish it 16:37:11 gokrokve, igormarnat? 16:37:27 Yes. publish it. But add a note that this is not a goal. It is just an example how to use it in general as a generic engine. 16:37:40 sure, ok 16:37:57 We can publish it, no issue, though I don't see a lot of value in that for now:) 16:38:10 Anyway, let's do it 16:38:14 #action rakhmerov, publish detailed description of "Long-running business process" use case on wiki 16:38:23 yes, it's a 10 min thing to do :) 16:38:27 ok 16:38:31 the next one 16:38:46 3. stanlagun, finish detailed description of "Post Deployment Configuration" use case 16:38:49 done 16:39:02 but under a different name 16:39:40 #info "Cloud Environment Deployment" use case, https://wiki.openstack.org/wiki/Mistral/Cloud_Environment_Deployment_details 16:40:19 4. ativelkov, collect the info about use case related to setting up a network (not a high priority) 16:40:32 in progress, not a high priority 16:40:48 #action ativelkov, collect the info about use case related to setting up a network (not a high priority) 16:41:04 5. rakhmerov, prepare the first version of Roadmap and the development plan 16:41:29 the first version of Roadmap is done and event shared at the summit 16:41:43 but I still didn't share it anywhere else 16:42:00 So here comes the next AI for us 16:42:18 what ways of sharing it would you recommend guys? Except wiki, of course 16:42:23 To put together AIs and agreements achieved in HK and publish them in mailing list 16:42:35 yes 16:43:17 #action akhmerov, put together AIs and agreements achieved in HK and publish them in mailing list 16:44:08 the Roadmap we have now is kind of a draft, how can we discuss it with the community in the best way? 16:44:12 mailing list? 16:44:22 or some sort of a Blueprint? 16:44:25 Let's discuss them prior to publishing though in order to see if me and Gosha and Alexander can add anything 16:44:38 yes, ok 16:45:12 #action gokrokve, igormarnat, akuznetsom review Roadmap and add whatever you think needs to be added in there 16:45:45 #action rakhmerov, share Roadmap with the community to start discussing it 16:46:38 #topic Free discussion 16:46:53 anything else guys that you'd like to discuss? 16:46:59 Great. Let's return back to the list of questions we had 16:47:13 The first one was the way to track states 16:47:20 ok 16:47:45 ruhe, akuznetsov, what else was there? 16:48:01 the only thing i wanted to mention... 16:48:15 there is a project https://github.com/airbnb/chronos which might be a good source of inspiration :) 16:48:21 so I see it could be encapsulated in a subsystem of Mistral from the very beginning so that we may vary the actual storage whenever we want 16:48:46 #info https://github.com/airbnb/chronos 16:48:51 thanks! 16:49:41 you mean in the context of storing a distributed state? 16:49:52 storing and executing 16:49:57 ok 16:50:18 chronos uses http://mesos.apache.org/ which itself might be a good engine for distributed task execution 16:50:23 we should define what part of workflow functionality in Mistral and which in TaskFlow 16:51:28 #action rakhmerov, akuznetsov, NikolayM review https://github.com/airbnb/chronos and http://mesos.apache.org/  with regard to storing and executing distributed tasks 16:52:04 akuznetsov, yes, I believe TaskFlow is currently missing some stuff required for Mistral 16:52:28 I think we need to identify that as soon as possible 16:52:56 rakhmerov, not only that, but we might want to describe explicitly what's the difference between them and when one use Taskflow, when does he use Mistral 16:53:11 Description of what's need to be added to Taskflow is a separate topic 16:53:22 and start discussing it with TaskFlow folks so that they either implement that themselves or let us contribute, or both 16:53:43 ooh, apologize. I may have misunderstood that 16:53:58 yes, got it now 16:54:35 Are there any news from our big and powerful prototyping team? What did we prototype already? :) 16:55:28 #action rakhmerov, kuznetsov, harlowja clearly describe the high-level separation of functionality between TaskFlow and Mistral in terms of addressed use cases 16:55:35 sounds good enough? 16:56:06 ok 16:56:10 4 mins left 16:56:15 LGTM 16:56:23 ok 16:57:29 looks like we have a bunch of AIs this time :) 16:57:33 a lot of work 16:58:06 anything else? 16:58:45 ok, they we'll end for today 16:59:04 thanks to everyone! I'll see you all soon 16:59:05 Ok, thanks for coming, everyone. TTYL, BFN:) 16:59:13 #endmeeting