14:00:23 #startmeeting nova-scheduler 14:00:24 Meeting started Mon Jul 13 14:00:23 2015 UTC and is due to finish in 60 minutes. The chair is n0ano. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:25 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:27 The meeting name has been set to 'nova_scheduler' 14:00:34 anyone here to talk about the scheduler? 14:01:07 anyone here to talk about the scheduler? 14:01:25 edleafe, are you mocking me :-) 14:01:32 whoops - sorry - there was a lag 14:01:41 internet is flaky today :( 14:01:42 o/ 14:02:04 I blame everything on the internet (except for those things I blame on MSoft) 14:02:12 jaypipes said he won't be able to make it today 14:02:36 oh well, hopefully next time 14:02:40 I have the internet under my desk - it seems fine - I just looked 14:03:12 PaulMurray: well, maybe the problem's that your desk is too far away from my laptop 14:03:23 PaulMurray, have you check that the pipes are free of obstruction? 14:03:49 bauzas, YT 14:04:19 PaulMurray: use some of this: http://j.mp/1Hqlida 14:04:34 well, let's get started 14:04:39 n0ano: yeah, bauzas is celebrating Bastille Day Eve 14:04:50 edleafe, I clicked a link like that already today - I'm not falling for that one twice in one day 14:05:02 * PaulMurray embarrased 14:05:07 PaulMurray: but you can trust *me*! 14:05:09 edleafe, ahh, makes sense, we'll give him all the ARs then 14:05:23 #topic Liberty specs 14:06:15 if the priority page is up to date we only have 2 priority specs that are still open https://etherpad.openstack.org/p/liberty-nova-priorities-tracking 14:06:30 1- https://review.openstack.org/#/c/179224/ 14:06:44 2- https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/request-spec-object,n,z 14:07:24 n0ano: the second is just a list of patches, not specs? 14:07:30 actually 1 is a patch and 2 is a spec so, assuming we can get reviews on these, I think we're in good shape as far as priority tasks are concerned 14:08:47 well, 2 seems to start with https://review.openstack.org/#/c/145528/ 14:09:09 * n0ano typing too fast 14:09:28 2 is the space but there is a link to the patches for it on the priority page 14:09:52 last I talked to bauzas on that he felt confident he would get the patches out in time and didn't need any help on them 14:12:03 I'm not `too` concerned, I think it's just a matter of doing the work for now although 1 (destination on migrations) needs some reviews 14:14:22 unless anyone has anything else on priorities let's move on 14:14:35 move on! 14:14:45 #topic mid-cycle meetup 14:15:16 I have something new that I would like to add to the mid-cycle agenda 14:15:27 I've been having talks off and on with jaypipes about a radical change to the whole compute_node resource tracking and scheduler updating. 14:15:28 edleafe, cool, what is it? 14:15:37 The main idea is to create a Cassandra deployment to hold all the information. 14:15:45 This will allow as-close-to-immediate updates as possible. 14:15:54 I want to talk about this at the mid-cycle. This is still just a POC, so I want to get an idea of what sort of situations would be convincing that this needs to be explored further; e.g., running multiple schedulers w/o racing 14:16:07 I met with some of the Cassandra people to get a reality check on my ideas, and they agreed that this is a good fit for what Cassandra offers, and would be a great deal more efficient for this than Rabbit/MySQL currently gets us. 14:16:28 edleafe, would this be an internal implementation or would it affect the APIs 14:16:47 It's a pretty radical change, so I want to have some hard data before even thinking about writing code for nova 14:17:06 n0ano: it would affect how things work internally a lot 14:17:17 n0ano: it would not affect external APIs at all 14:17:23 sounds like a perfect topic for the mid-cycle, have you updated the etherpad with this? 14:17:34 not yet 14:17:47 edleafe, I would like to know more, I won't be at the mid cycle 14:17:52 just had some more email conversations with Jay this morning 14:17:59 wanted to run it by the group first 14:18:33 I'm not familiar with Cassandra so I'd have to research that a little before I can intelligently comment 14:19:01 n0ano: understood 14:19:26 my biggest concern would be APIs but, if it doesn't modify them, even if it involves a lot of change, that's basically just implmentation 14:19:39 edleafe, I would like to know why it makes a big difference - I assume you mean more than just the communication 14:20:00 I spoke with the tech people at DataStax (the company behind Cassandra) and they are willing to have a tech hangout where people can ask questions about Cassandra 14:20:28 PaulMurray: for one, it would eliminate the raciness of running multiple schedulers 14:20:43 PaulMurray: two, it would eliminate the need for compute to update the scheduler 14:21:36 three, it would allow for a more HA setup, since there wouldn't be a single point of failure 14:22:08 PaulMurray: and you bring up my reason for discussing this at the midcycle 14:22:28 I'd like to get a list of known pain points and/or limitations of the current design 14:22:50 and then design tests that would show the improvement (if any) 14:22:55 edleafe, I know what cassandra is, so I would like to understand if it impacts the compute node end - clearly you have to get data into cassandra 14:23:14 if this is successful, by the time Tokyo rolls around, we could have a much more informed discussion 14:23:31 PaulMurray, kind of what I was thinking, compute may not have to update the scheduler but it has to update something 14:23:57 I think a public email thread before the meetup so everyone has a change to think about this would be good 14:24:19 PaulMurray: n0ano: yes, exactly. Compute would post its status to Cassandra, and the scheduler(s) would have that (almost) immediately available 14:24:34 n0ano: sure, that is the plan 14:24:56 edleafe, as opposed to compute update the scheduler and the scheduler immediately has the data :-) 14:25:02 n0ano: the one thing I *don't* want to do is get mired down in implementation concerns before we even have a chance to see if it is better 14:25:22 n0ano: for some values of _immediate_ 14:25:30 edleafe, is it as fast as /dev/null 14:25:35 :) 14:25:55 any change is painful, and we're really good at identifying why this will be hard 14:26:06 I want this to be a POC 14:26:18 I'm willing to hold off comment until I know more about Cassandra and you start the email thread 14:26:25 I want people to say "If it can do x, I might consider it further" 14:26:37 edleafe, I would stick clear of words like "immediately" and identify what the difference is 14:26:49 looking forward to it 14:26:49 if I run the POC and it isn't better, then no harm done 14:27:01 PaulMurray: well, sure 14:27:13 edleafe, I'd be more interested in `this is a current problem and this is why the new proposal fixes that problem' 14:27:14 PaulMurray: this is a little less formal that an email to the ML 14:27:24 n0ano: yes!!! 14:27:31 edleafe, I assume you have seen this https://www.youtube.com/watch?v=b2F-DItXtZs 14:27:32 that's what I want from the midcycle 14:27:52 PaulMurray: heh 14:28:19 PaulMurray: but yeah, that's the perception people have 14:28:38 they lump Cassanda, MongoDB, redis, etc., all into the same 'NoSQL' camp 14:29:26 edleafe, what I mean is it helps to understand what is actually happening and what is being changed 14:29:51 PaulMurray: sure, but that's getting ahead of ourselves a bit, IMO 14:30:15 PaulMurray: like n0ano said, let's identify the current limitations, and try something new 14:30:17 we were moving towards using an in memory copy of data, so it would be interesting to see what is different in what yo upropose 14:30:31 if that helps, great. If not, let's try something else 14:31:36 certainly an interesting idea, let's see where it goes 14:31:53 #action edleafe to add Cassandra for the scheduler to the mid-cycle agenda 14:32:10 * n0ano wonders why my actions never do anything 14:32:22 n0ano, ... and edleafe to start an ML thread 14:32:33 #action edleafe to start a mailing list thread to lay the groundwork for Cassandra discussion at mid-cycle 14:32:42 PaulMurray: way ahead of you :) 14:32:55 not in my partial order 14:32:57 :) 14:33:00 OK, let's move on 14:33:02 n0ano: actions don't echo in the meeting. They just appear in the summary 14:33:27 edleafe, really? I hate UI with no positive feedback, oh well 14:33:32 moving on... 14:33:36 #topic opens 14:33:46 anything new for today? 14:34:12 n0ano, I wasn't here last week so I didn't get to say I finished the resource tracker objects 14:34:28 not new, but here's what jaypipes wrote before the meeting: " 14:34:34 PaulMurray, cool, are you awaiting reviews? 14:34:34 Unfortunately, I need to go pick my wife up at the airport in Tampa and so will miss our weekly scheduler IRC meeting. Not much news from my end, other than I will try to finish up the resource objects patch series this week, and continue to do reviews on request-spec patch series." 14:35:27 n0ano, no, but just noticed I didn't remove self.conductor_api - so I will do one more to do that :) 14:35:44 PaulMurray, always something :-) 14:35:47 n0ano, its only the variable and the import 14:37:18 OK, I'm hearing crickets 14:38:10 one final thought, I think we should cancel next week's meeting, that'll be a travel day for the mid-cycle, follow up on IRC & ML till then 14:38:37 n0ano: +1 14:38:38 n0ano, edleafe sure - I will not be there, so enjoy and I'll catch up in a couple of weeks 14:38:52 tnx everyone, we'll talk again soon 14:38:55 PaulMurray: there will be some sort of audio hookup 14:38:55 #endmeeting