08:06:53 <d0ugal> #startmeeting mistral
08:06:54 <openstack> Meeting started Fri Aug 17 08:06:53 2018 UTC and is due to finish in 60 minutes.  The chair is d0ugal. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:06:55 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
08:06:55 <d0ugal> Morning
08:06:57 <openstack> The meeting name has been set to 'mistral'
08:06:58 <d0ugal> Sorry I'm late!
08:07:11 <d0ugal> https://etherpad.openstack.org/p/mistral-office-hours
08:07:20 <d0ugal> PING rakhmerov, apetrich, bobh, mcdoker181818, akovi, hardikjasani
08:07:30 <rakhmerov> I'm here
08:07:33 <apetrich> o/
08:08:50 <d0ugal> Hows everyone doing?
08:08:55 <d0ugal> Anything specific you'd like to discuss?
08:09:46 <openstackgerrit> Dougal Matthews proposed openstack/mistral stable/rocky: Removes non needed parameter passed in magnum client creation  https://review.openstack.org/592924
08:11:46 <rakhmerov> d0ugal: good )
08:11:55 <rakhmerov> d0ugal: trying to find out if I can actually go to Denver
08:12:03 <rakhmerov> checking with Kendall
08:12:10 <d0ugal> rakhmerov: cool, fingers crossed :)
08:12:17 <rakhmerov> we'll see
08:12:23 <rakhmerov> hopefully I'll get a reply today
08:12:36 <d0ugal> We have four security bugs thanks to some testing at OVH.
08:12:44 <rakhmerov> yeah
08:12:46 * apetrich is deep into mistral-tempest-plugin
08:13:08 <d0ugal> rakhmerov: Have you looked at them?
08:13:14 <rakhmerov> nope
08:13:16 <d0ugal> :)
08:13:18 <rakhmerov> well, only one of them
08:14:07 <rakhmerov> d0ugal: from my side, just a small update. I temporarily switched back to performance work and postponed scheduler as I planned. I want to fix one serious performance problem in the Mistral architecture
08:14:28 <d0ugal> Which one? :)
08:14:59 <rakhmerov> https://bugs.launchpad.net/mistral/+bug/1783708
08:14:59 <openstack> rakhmerov: Error: malone bug 1783708 not found
08:16:04 <rakhmerov> ooh, which one you mean what? :)
08:16:10 <rakhmerov> security bug or performance problem
08:16:11 <rakhmerov> ?
08:16:14 <d0ugal> rakhmerov: I actually meant which perf issue
08:16:18 <d0ugal> but I was also interested in the bug :)
08:17:07 <rakhmerov> ok, so
08:17:14 <rakhmerov> performance problem is the following
08:18:01 <rakhmerov> due to our non-blocking transactional model we have issues with things like completing workflows and triggering joins
08:19:24 <rakhmerov> we have to use self-scheduling periodic jobs that check some conditions
08:19:42 <rakhmerov> and the problem with them is that if we do them too often it's a load on DB
08:20:15 <rakhmerov> if too rarely, then there are big delays between say when a workflow is logically completed and marked as completed
08:20:40 <d0ugal> makes sense
08:20:52 <rakhmerov> we kinda try to predict a delay when another iteration of a checking job should run but the algo is very very unreliable
08:20:53 <d0ugal> I have noticed joins can be a little slow, but we only use them in a couple of places
08:21:07 <rakhmerov> and probably it's hard to make it reliable, too many things that it depends on
08:21:13 <rakhmerov> yeah
08:21:15 <rakhmerov> so
08:21:22 <rakhmerov> we hit a corner case
08:21:38 <rakhmerov> we have workflows with 300+ tasks
08:21:51 <rakhmerov> where every task is a subworkflow!
08:21:58 <d0ugal> haha, nice
08:22:14 <rakhmerov> now imagine how much time we lose on delays that occur for each subworkflow
08:22:19 <rakhmerov> seconds for each
08:22:26 <rakhmerov> many minutes for the entire thing..
08:22:32 <rakhmerov> just on watings
08:22:35 <rakhmerov> waitings
08:22:36 <d0ugal> Yeah, that is bad :)
08:22:38 <rakhmerov> yeah
08:22:49 <rakhmerov> and so I'm eager to fix that somehow
08:23:00 <rakhmerov> so thinking now how because it's fundamentally not easy
08:24:22 <d0ugal> It does sound complicated, but also should be interesting.
08:26:35 <rakhmerov> yeah
08:26:37 <rakhmerov> it is )
08:31:06 <rakhmerov> d0ugal: did you try to find out how many people are ready to go to Denver for Mistral sessions
08:31:19 <rakhmerov> I'm just wondering if it makes sense for me to go at all )
08:31:27 <d0ugal> rakhmerov: I only checked with the core team, and nobody could go.
08:31:37 <d0ugal> Other than maybe you :)
08:31:51 <rakhmerov> I can go alone, of course, and have fun but alone.. )
08:31:52 <d0ugal> I know rbrady and thrash|g0ne will be there - but I think they will be focused on other sessions.
08:31:59 <rakhmerov> and may be participate in other sessions
08:32:01 <d0ugal> rakhmerov: indeed, I do wonder if it would be useful.
08:32:05 <rakhmerov> yeah
08:32:19 <rakhmerov> d0ugal: can you check again may be?
08:32:28 <d0ugal> Right, it would be good for crossover sessions and maybe you can meetup with some mistral people (maybe make the mistral session very short?)
08:32:28 <rakhmerov> via ML for example
08:32:32 <d0ugal> rakhmerov: Sure.
08:32:48 <rakhmerov> d0ugal: yeah, I still see the point
08:33:03 <rakhmerov> ok, if they can cover flights/hotel I'll go
08:33:26 <d0ugal> rakhmerov: did you email Kendall?
08:33:57 <rakhmerov> yes
08:34:07 <d0ugal> I hope they have not already started removing Mistrals room. It could get confusing with me sending this email now...
08:36:27 <d0ugal> rakhmerov: email sent - so hopefully we get some replies :)
08:38:28 <d0ugal> rakhmerov: Can you take a look at the security bugs? Do you think any of them are urgent and must be fixed in Rocky?
08:38:34 <d0ugal> https://launchpad.net/mistral/+milestone/rocky-rc2 - you can see them all here
08:39:02 <rakhmerov> d0ugal: ok
08:39:31 <d0ugal> I guess they are only really important if Mistral is publicly accessible
08:39:37 <d0ugal> or at least with users you don't trust :)
08:41:33 <rakhmerov> yeah, looking..
08:43:57 <rakhmerov> https://bugs.launchpad.net/mistral/+bug/1666565 seems very easy to fix
08:43:57 <openstack> rakhmerov: Error: malone bug 1666565 not found
08:44:11 <rakhmerov> we could probably backport it to rocky
08:46:27 <d0ugal> rakhmerov: Yeah, agreed.
08:46:32 <d0ugal> I can probably do that now actually
08:46:37 <rakhmerov> :)
08:46:42 <d0ugal> I got distracted looking at the YAML bug
08:47:14 <rakhmerov> hah
08:47:26 <rakhmerov> https://bugs.launchpad.net/mistral/+bug/1699786 is interesting and suprising to me
08:47:27 <openstack> rakhmerov: Error: malone bug 1699786 not found
08:47:54 <rakhmerov> I wasn't aware of this asymmetry
08:48:03 <rakhmerov> shouldn't also be a problem to fix
08:48:05 <d0ugal> rakhmerov: indeed, that surprised me too :)
08:48:47 <rakhmerov> yeah
08:49:18 <rakhmerov> I'm not sure though what is our criteria based on which we decide if a bug needs to be backported
08:49:32 <rakhmerov> if simplicity, then yes, this one is for rocky too
08:49:51 <rakhmerov> importance? Well, not sure on that
08:50:07 <rakhmerov> but I assume for some users it may be quite important
08:50:24 <rakhmerov> who use auth and many tenants
08:50:25 <d0ugal> I guess we should backport them if we think it is realistic (not too complicated) and safe (not backwards incompatible risks)
08:50:39 <rakhmerov> right
08:50:40 <rakhmerov> hm..
08:50:52 <rakhmerov> this is not compatible though..
08:51:03 <rakhmerov> but definitely a bug
08:51:07 <d0ugal> yeah
08:51:28 <d0ugal> We need to fix them all anyway, so I guess we can do that and then see if the fix can bbe backported.
08:51:51 <d0ugal> Once we have a fix it will be much easier to judge :)
08:56:56 <rakhmerov> d0ugal: agreed
08:57:07 <openstackgerrit> Dougal Matthews proposed openstack/mistral stable/rocky: Update .gitreview for stable/rocky  https://review.openstack.org/590266
08:57:08 <openstackgerrit> Dougal Matthews proposed openstack/mistral stable/rocky: Make Tripleo non-voting on Rocky  https://review.openstack.org/592933
08:57:08 <openstackgerrit> Dougal Matthews proposed openstack/mistral stable/rocky: Revert "Make Tripleo non-voting on Rocky"  https://review.openstack.org/592934
09:00:14 <d0ugal> #endmeeting