14:59:45 #startmeeting scheduler 14:59:46 Meeting started Tue Jul 30 14:59:45 2013 UTC and is due to finish in 60 minutes. The chair is n0ano. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:59:47 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:59:49 The meeting name has been set to 'scheduler' 15:00:02 show of hands, anyone here for the scheduler meeting? 15:00:57 o/ 15:02:02 jog0, just you & me so far, this could be quick :-) 15:02:14 hi 15:03:15 o/ 15:03:54 well, while we're waiting for people... 15:04:00 #topic administrivia 15:04:34 I will be out the months of Sept & Oct (back in time for Hong Kong), anyone want to chair this meeting during those 2 months? 15:05:26 n0ano: I would offer but, the time of day doesn't always work for me 15:05:57 kind of the same for me. I can take it sometimes, but I know I'll be out a few of those weeks. 15:06:15 hi 15:06:16 we don't need an asnwer today, I just wanted to get it out and we'll find someone, I don't want the meeting to die on the vine. 15:06:38 moving on to more interesting stuff 15:06:45 hi all 15:06:46 =) 15:06:54 #topic multiple active scheduler policies/drivers 15:07:08 n0ano, to remember this point, can you "action" it? 15:07:12 glikson, jog0 I believe you've been active on the list on this, can you summarize? 15:07:47 I can try 15:07:48 #action need new moderator for months of Sept & Oct 15:07:55 jgallard, tnx, good point 15:08:00 n0ano, thanks :) 15:08:31 we have a blueprint and a patch submitted. there have been several concerns with the current design approach. 15:09:02 seems that the main one was whether we need API-driven management of policies, or we can start with something more simple for Havana 15:10:09 especially given the short timeline remaining for Havana seems like starting simple with the option to make it more flexible in future would be the way to go 15:10:23 is the issue config files? 15:11:20 simple isn't always good either. we have to keep in mind that any 'simple' short term solution will have to have a migration path to the next thing etc 15:11:31 yes (I think). whether defining policies in nova.conf is good enough for Havana (similarly to cinder multi-backed support) 15:11:53 IMHO, defining policies in nova.conf is not good enough for Havana 15:12:32 jog0: agree, but I don't expect major issues with migration.. 15:12:35 are they really going to be so dynamic that this is an unreasonable restriction? 15:12:48 I don't expect them to be dynamic in most cases 15:13:06 sure, they may not be dynamic but we don't want more config options in nova.conf 15:13:26 adding policies typically means adding new kinds of hardware, or changing overall cloud management policies.. 15:13:36 jog0, give the config file has well defined sections, what's wrong with putting them there 15:13:41 adding this to nova.conf means making nova.conf even more complex 15:14:16 n0ano: its not about the right section or not. We want things to use the REST APIs when possible 15:14:38 jog0, +1 15:14:42 in fact, adding APIs and DB support adds a lot to the overall complexity.. 15:14:47 jog0, yes, APIs are the ultimate goal but is using the config file that much of a problem for Havana 15:15:12 well I think this is missing a big piece, cross project policies 15:15:28 such as cinder and neutron related 15:15:40 jog0: fully agree, but it shouldn't prevent us from solving an important subset of the problem in Havana 15:16:07 I disagree, I would like to see a plan for a better solution, and how this gets us on the right track 15:16:58 seems to me we all want the APIs but think that's too complex a change for Havana 15:17:02 If there is a roadmap for how to get to the better solution, and this is an temporary solution then sure. Also what about russellb's idea of using flavors 15:17:25 jog0: do you think we can come up with such a plan without a discussion at the summit? 15:17:38 maybe this is something that should be targeted for Icehouse and not Havana 15:17:44 glikson: true 15:17:47 jog0: I accepted his idea 15:17:59 glikson: so using flavors sounds likea good first step 15:18:03 the patch under review uses flavor extra specs 15:18:07 its doesn't change things in any major way 15:18:23 and addresses most of your basic requirments as far as I can tell 15:18:23 the extra spec specifies the policy 15:19:17 jog0: in theory, we can specify all the scheduler parameters (driver, filters, etc) in the flavor -- but I think it would be an abuse.. 15:19:37 also much harder in terms of compatibility and migration going forward 15:19:53 glikson: yeah 15:20:20 glikson, yes, and complicated in term of flavors management and update too 15:20:25 well I think there is a bigger issue at hand. Which is there are several competing scheduler blueprints and no overall plan 15:21:15 jog0, that should be handled by a session at the next summit, trying to come up with a comprehensive plan 15:21:25 jog0: IMO, the proposed design is flexible enough to seamlessly move to different mechanism for persistency and lifecycle of policies, when we know what exactly we are trying to achieve. 15:22:01 glikson: the issue is when there are a bunch of BPs pulling the scheduler in different directions we end up with a mess 15:22:44 we kind of had that at the last summit but it just wound up being a quick review of multiple BPs, not an overall plan 15:22:46 jog0: in general, I agree that some sort of overall architecture and roadmap would be useful 15:22:48 yes, its too late to work on agree before the summit. but its a good time to figure out the proposal for the summit 15:23:18 the duedate for feature proposal is Aug 23? 15:23:21 Some of the best summit sessions are the ones where there is a well thought out proposal that gets discussed 15:23:29 shanewang1: no new BPs already 15:23:31 jog0: I will surely vote for having a discussion on this at the summit 15:23:50 glikson: a discussion over a fully proposed plan 15:23:55 let's not get too side tracked, we're diverting from multiple schedulers into overall scheduler design 15:24:22 n0ano: sorry, I just think its hard to have this discussion without the overall one 15:24:24 both are good topics but let's finish the multiple scheduler discussion first 15:25:15 jog0, we're not going to finalize the overall picture today so I think it's still useful to discuss the specific issues 15:26:04 trust me, I'm adding a `overall scheduler plan' as a topic (most likely for next week when we have a chance to think about it). 15:26:22 n0ano: no we aren't but we can start that discussion, perhaps schedule it off meeting 15:26:32 jog0: the question is whether this particular BP, which has been discussed at high level at the last summit and approved, should be the one to "suffer" from this. It doesn't introduce any revolutionary changes, and does not seem to be contradicting any other major design goal that we may potentially want to enforce.. 15:26:42 jog0, +1 (we're in violent agreement) 15:26:58 glikson: can you link to the patch 15:27:18 https://review.openstack.org/#/c/37407/ 15:28:27 glikson: I thought you said this used the flavors, not a list of policies 15:29:13 jog0: flavor specifies the policy 15:29:40 glikson, then where are the actual policies defined 15:29:46 the details of each policy is in nova.conf 15:30:17 ahh, now I see the issue, that does seem to be a bit of an abuse of the nova.conf 15:30:25 n0ano: yup 15:30:43 why? 15:31:06 specify the default driver/filters is ok, but adding couple of extra ones is not? 15:31:07 you are now putting a potentially unbounded amount of info in the config file, not what it was defined for 15:31:55 n0ano: we want things like policies to be controlled via APIs when possible 15:32:18 this makes things like RBAC and domains work nice 15:32:18 n0ano: in theory.. 15:32:22 hummm, it's not the same thing for instance when you configure several cinder backend into cinder.conf ? (multiple sections into cinder.conf) 15:32:37 jgallard: indeed, it is 15:32:39 in that case, its not an issue 15:32:50 jgallard: isn't that saying this machine has these backends 15:33:01 and this is different per machine, and is only local. 15:33:02 that's where we 'stolen' the idea 15:33:18 while here we are saying I have these global policies and I am defining them in the nova.conf file 15:33:54 where an admin API user cannot touch them even if the cloud operator wanted that 15:33:55 I'm with jog0 on this one (he's expressing my concerns better) 15:33:56 jog0: not really -- you can potentially control all of your storage pools via a single machine running cinder-volume 15:34:57 in fact, having a lot of storage pools, with dynamics, is a much more reallistic scenario than having lots of scheduling policies.. 15:34:59 glikson: sure, but that is an edge case of this? and adding new hardware is different than changing a policy 15:36:01 note, just because cinder works this way doesn't mean we agree with that design, I'd say cinder is wrong in this case 15:36:15 jog0: can you have an example, what kind of scheduling policies do you have in mind when thinking about having lots of them? 15:37:05 glikson: who knows, but why can't I as a owner of a cloud say, you as a seperate domain and maybe a virtual private cloud inside of mine cannot set your own arbitrary scheduling policies 15:38:45 jog0: you surely can, but it is not supported by the patch we've submitted. we support many other use-cases, which do not require this kind of programmability. we don't need to support all the possible policy-related use-cases in a single patch, right? 15:39:43 glikson, as long as you have a way of ultimately supporting all the use cases starting simple is fine 15:39:55 I am not aware of any inherent problems with our implementation that would prevent anyone from extending it to support better progammability 15:39:57 glikson: anyway, I think the next step here is: 15:40:26 clearly explain your current patch in great detail, and push to the ML thread and see what happens 15:40:34 on ther other hand, I am also not aware of any alternative implemetation that would solve the bigger problem in Havana time frame.. 15:41:22 glikson: this isn't always about the first implementation proposal wins, its about the 'right one,' however that is defined 15:41:34 jog0: you mean, use cases or implementation? 15:41:44 glikson: implementation 15:42:05 does this patch was not already discussed on the ML? 15:42:26 jgallard: the patch has had several revisions since then, not sure if that has changed anything major 15:42:34 there is no single "right one" in most cases. if this solves a real problem, and it seems to be flexible enough to extend to support additional use-cases -- this should be enough, IMO.. 15:42:45 jog0, ok 15:43:00 jog0: we addressed some of Russell's suggestions. 15:43:04 it is a nova meeting or a ceilometer meeting? 15:43:12 hk_peter, nova 15:43:23 glikson: needs to be re-reviewed now? 15:43:41 @n0ano thx 15:43:59 hi russel. yes, would appreciate if you could take another look (and maybe remove your -2) 15:44:14 glikson: well for one thing the commit msg needs work 15:44:24 ack, will look 15:44:53 is there anybody can help to design the screen for nova's schduler setup? 15:44:53 russellb: you may also take a look at the discussion we just had here, in the last 1/2 hour.. 15:46:38 jog0: ok 15:47:18 so, are we winding down on this subject (subject to mailing list activity coming up)? 15:48:03 taking silence as assent... 15:48:07 #topic opens 15:48:50 I wanted to talk about instance groups (but gary doesn't seem to be here) and/or simple way to improve the scheduler but I don't think there's enough time 15:48:57 we'll defer those to next week. 15:49:08 anyone have anything new to bring up toda? 15:49:09 can i say something about my project pandora? i scare i am disturbing 15:49:38 hk_peter, is this scheduler related? 15:49:45 kind of 15:49:57 go ahead 15:49:57 but i really want to say something about pandora background first 15:50:04 hk_peter, got a link? 15:50:27 hk_peter, I was going to say, is this something you should start on the dev mailing list first? 15:50:57 pandora is a admin console http://peter.kingofcoders.com , we are *NOT* forking horizon, we have the horizon API. We want to create a better GUI, so customer is easier to make the decision to purchase openstack 15:51:38 hk_peter: is this opensource? 15:51:47 yes, completely open source, apache license 15:52:00 why not push code up into horizon? 15:52:09 hk_peter, sounds to me like you `are` trying to replace horizon, you'd better expect a certain amount of controversy 15:52:15 two days ago, hong kong openstack community chairman want to submit our news to openstack blog, but refused, we are very down. 15:52:20 hk_peter, why not just enhance/change horizon 15:52:32 I just want to say, pandora rely on horizon, we are not forking it. 15:52:42 Pandora is java, hard to push code into horizon 15:53:01 Pandora is not web base, i think our team can do better in a standalone app. 15:53:35 what is the link with scheduling? 15:53:36 hk_peter, do you have a detailed design you can point us to (I'm not liking what I'm hearing so far I have to tell you) 15:53:42 Honestly, i still not sure how to submit code review to horizon team, we want to enhance the scheduler, give it a nice UI 15:54:23 here you go http://peter.kingofcoders.com/?p=442 15:55:12 hk_peter, we don't have time to discuss this more today, I strongly suggest you raise this on the dev mailing list 15:55:21 thanks n0ano 15:55:27 anything else new anyone want to raise? 15:55:30 take you time 15:56:00 hk_peter: the short of it is you can't adjust QOS scheduling today via any REST APIs 15:56:18 well there are a few ways starting to show up, but we are discussing that here 15:57:09 if we give it a REST APIs, letting people/third party program to reprogram the scheduler, isn't it great? 15:57:20 hk_peter: see above 15:57:58 hk_peter, in a word, no, that's not necessarily a good idea 15:58:16 anyway, times up, so I'll thank everyone and we'll talk again next week. 15:58:27 #endmeeting