14:00:13 <edleafe> #startmeeting nova_scheduler
14:00:14 <openstack> Meeting started Mon Feb  5 14:00:13 2018 UTC and is due to finish in 60 minutes.  The chair is edleafe. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:14 <edleafe> #link Agenda for this meeting https://wiki.openstack.org/wiki/Meetings/NovaScheduler#Agenda_for_next_meeting
14:00:15 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:18 <openstack> The meeting name has been set to 'nova_scheduler'
14:00:22 <edleafe> Good UGT morning!
14:00:26 <edleafe> Who's here?
14:00:43 <takashin> o/
14:01:02 <efried> @/
14:01:15 <edleafe> comb your hair, efried!
14:01:26 <cdent> oh hi
14:03:23 <edleafe> Huh, looks like it will a quick meeting
14:03:24 <edleafe> #topic Specs & Reviews
14:03:40 <edleafe> Since Feature Freeze is past, not much to say about these changes, so I'll just paste 'em here for posterity, and if anyone has anything to say about them, go for it.
14:03:52 <edleafe> #link Provider Tree series, starting with: https://review.openstack.org/#/c/537648/
14:03:55 <edleafe> #link Nested RP traits selection: https://review.openstack.org/#/c/531899/
14:03:58 <edleafe> #link Granular resource requests review: https://review.openstack.org/#/c/517757/
14:04:01 <edleafe> #link Remove microversion fallback: https://review.openstack.org/#/c/528794/
14:04:04 <edleafe> #link Use alternate hosts for resize: https://review.openstack.org/#/c/537614/
14:04:09 <edleafe> Anyone want to discuss any of these?
14:04:25 <efried> I just edited to include spec links
14:05:08 <edleafe> efried: gotcha
14:05:21 <efried> We can be getting a jump on reviewing ready code.
14:05:26 <efried> even if we can't merge it.
14:05:38 <edleafe> true, true
14:06:01 <jaypipes_> o/ sorry for being late
14:06:15 <edleafe> hey there jaypipes_!
14:06:43 <edleafe> #topic Bugs
14:06:43 <edleafe> #link Placement bugs https://bugs.launchpad.net/nova/+bugs?field.tag=placement
14:06:52 <edleafe> A few new ones this week
14:06:53 <edleafe> efried seems to have found issues with generation consistency
14:07:29 <efried> I think I have fixes proposed for most of the ones I opened.
14:07:36 <efried> There's still at least one that's going to need some discussion.
14:07:45 <efried> Not sure if we want to do that here or wait til the PTG
14:08:27 <edleafe> Unless it's a simple thing, I'd prefer to wait for PTG so that we can get several eyes on it
14:08:34 <cdent> I made a spec for the aggregates one: https://review.openstack.org/#/c/540447/ and including some editorializing in the alternatives section
14:08:41 <efried> It's simple.  It's not easy.
14:08:51 * efried clicks
14:09:26 <edleafe> #link Add generation support in aggregate association https://review.openstack.org/#/c/540447/
14:10:26 <edleafe> efried: it sounds like we need a general cleanup of how we handle generations
14:10:38 <efried> Yes
14:10:53 <edleafe> BRB - gotta move my car
14:10:55 <jaypipes> edleafe: cleanup?
14:11:23 <efried> More of a sweep to make sure we've got all the holes filled - on both sides (server & report client)
14:11:29 <jaypipes> efried: is it cleanup or just making sure we are consistent in exposing the generation when changing things related to the provider?
14:11:33 <jaypipes> gotcha
14:12:10 <efried> jaypipes: On the client side, there's a couple reviews out there for bugs.
14:13:04 <efried> one of them you've already been reviewing.
14:13:26 <efried> Then there's this one: https://review.openstack.org/#/c/539712/
14:13:37 * edleafe is back
14:14:00 <jaypipes> efried: k
14:14:15 <efried> hm, maybe that's the only one related to generation conflicts so far.  The rest we need to discuss eventually.
14:14:35 <edleafe> Which kind of leads us to...
14:14:36 <edleafe> #topic Open discussion
14:14:38 <edleafe> One thing on the agenda:
14:14:47 <edleafe> Should allocations fail if there is a generation mis-match for a provider, even if the provider still has sufficient available inventory?
14:14:50 <edleafe> #link https://bugs.launchpad.net/nova/+bug/1719933
14:14:51 <openstack> Launchpad bug 1719933 in OpenStack Compute (nova) "placement server needs to retry allocations, server-side" [Medium,Triaged] - Assigned to Jay Pipes (jaypipes)
14:14:52 <edleafe> Could we perhaps increment the generation on a successful allocation even if the generation is older?
14:15:13 <edleafe> We don't need to answer this today
14:15:28 <jaypipes> edleafe: it should fail and let the caller decide.
14:15:30 <edleafe> But it came up in discussion last week, and I didn't want it to get lost
14:15:31 <efried> This is related to https://bugs.launchpad.net/nova/+bug/1746373
14:15:32 <openstack> Launchpad bug 1746373 in OpenStack Compute (nova) "Placement APIs with missing conflict detection" [Undecided,New] - Assigned to Eric Fried (efried)
14:16:04 <cdent> jaypipes: why should allocations fail, though?
14:16:09 <edleafe> jaypipes: but if there is sufficient inventory, why should it fail?
14:16:14 <efried> Sorry, maybe I missed something.  Does bug 1719933 imply that we're using generations in allocation APIs?  Cause afaict, we're not.
14:16:15 <openstack> bug 1719933 in OpenStack Compute (nova) "placement server needs to retry allocations, server-side" [Medium,Triaged] https://launchpad.net/bugs/1719933 - Assigned to Jay Pipes (jaypipes)
14:16:20 * bauzas waves super late
14:16:24 <cdent> efried: they are used only server-side
14:16:54 <jaypipes> cdent, edleafe: it should "fail" in so much as a 409 Conflict is returned and allows the caller to retry if it wants. i.e. what we already do in the claim_resources() process.
14:16:57 <efried> Swhat I thought.  So we're always incrementing based on what's in the db, not what comes in on the request, right?
14:17:10 <efried> Or does the allocation_request contain a generation (and I missed it)?
14:17:32 <cdent> at the api level it acquires a generation, and then can concurrent update fail
14:17:32 <edleafe> I think that this would be part of an overall discussion of how generations are used/exposed/etc. for PTG
14:18:12 <jaypipes> ack
14:18:25 <efried> Right now the doc for PUT /allocations/{c} 1.12-  says generation is ignored.
14:18:26 <cdent> jaypipes: sure, but we talked (you even TODO'd) about allowing a few server-side retries, and it was in the discussion of how that was actually more complex than we initially thougt that we realized this generation issues
14:18:48 <jaypipes> cdent: ok
14:19:03 <cdent> and at the moment I can't think of any reason why we would want to take a generation, it ought to just work (if there's room)
14:19:28 <edleafe> yeah, if we can retry on generation mismatch, what is having a generation involved getting for us?
14:19:59 <edleafe> Food for thought, and we have 3 weeks to digest until PTG
14:20:00 <efried> ...and doesn't figure in the POST /allocations API at all (at least based on the doc).
14:22:07 <cdent> edleafe: did you make an explicit entry about this on the ptg etherpad (I think you said you did, but I can't remember for sure)?
14:22:10 <edleafe> efried: the generation is part of the RP part of the Allocation, no?
14:22:21 <efried> nope
14:22:36 <efried> In PUT, it's there but ignored.  In POST, it's not there.
14:22:39 <cdent> of the Allocation object, it is
14:22:42 <edleafe> cdent: not yet. I figured let's mention it here, and if it wasn't clarified, we'd PTG it
14:23:10 <efried> cdent: I do not see it.
14:23:28 <cdent> efried: ask me after the meeting and I'll point it/explain it
14:23:29 <efried> Anyway, I believe I did add this to the etherpad; let me go find the line no.
14:23:54 <efried> L61
14:24:34 <edleafe> Added
14:24:54 <efried> edleafe: Duplicate
14:24:57 <edleafe> It's kind of a dupe
14:24:59 <edleafe> jinx
14:25:06 <efried> at least move it next to the other one.
14:25:16 <cdent> (just to note: "more than one thread managing allocations for a single consumer" is the crux of that biscuit, and something I thought we were disallowing, so is the real root of the topic)
14:25:18 <efried> There is a little bit of non-overlap.
14:26:49 <edleafe> So it seems to me that we need to, as a group, define the areas where generation is critical to ensure data integrity, and where it isn't.
14:27:00 <edleafe> And then make sure that the code matches that
14:27:55 <efried> just so
14:28:32 <edleafe> OK, then, anything else for Open Discussion?
14:28:49 <cdent> yeah, couple things
14:29:03 <cdent> I just wrote a blog post summarizing placement api changes:
14:29:08 <cdent> #link queens summary https://anticdent.org/placement-queens-summary.html
14:29:17 <edleafe> Ah, yes, thanks for that
14:29:20 <cdent> that's _only_ api changes. can do more if people ask
14:29:39 <edleafe> I'm 2/3 of the way through my blog post on Alternate Hosts
14:29:49 <cdent> And last week I wrote a different posting, which I would appreciate some feedback on, especially from jaypipes (if only from a long term history point of view)
14:29:57 <bauzas> cdent: I like the idea to just tell about API changes
14:29:58 <cdent> #link placement extraction https://anticdent.org/placement-extraction.html
14:30:34 <cdent> I'd like to think that _someday_ we can do the extraction, but without some shared discussion on the topic, it will never get traction
14:30:51 <cdent> and we'll always be saying "after X"
14:30:52 <bauzas> _someday_ is a reasonable target
14:30:58 <edleafe> And I think the longer we wait to do that, the more difficult it will be
14:31:10 <bauzas> do we still consider that as a necessary thing ?
14:31:32 <edleafe> I do
14:31:34 <bauzas> I mean, if other projects can use openstackclient or directly call the Placement API, why is that such a big deal ?
14:31:42 <cdent> bauzas: it's necessary for the health of openstack, even if there are no technical reasons
14:31:43 <bauzas> from a technical perspective I mean
14:31:46 <cdent> (but there are technical reasons)
14:32:03 <edleafe> OpenStack could all be in Nova :)
14:32:11 <bauzas> cdent: do you feel placement was less prioritary than other features in Nova ?
14:32:32 <cdent> bauzas: if you haven't had a chance to read the posting, it covers various things
14:32:43 <cdent> bauzas: no, actually, I think that placement takes _too much_ priority in nova
14:32:53 <cdent> and that nova would benefit by having placement not in it
14:32:57 <bauzas> cdent: that's certainly a good point
14:33:08 <bauzas> cdent: too much traction
14:33:19 <efried> Development of placement is currently being guided entirely by nova, and therefore (despite intentions to the contrary) primarily for the benefit of nova.
14:33:23 <bauzas> cdent: but the thing is, I feel we're still on a firedrill
14:33:41 <cdent> an artificial firedrill
14:33:44 <bauzas> and the interface between Nova and Placement isn't fully sold
14:34:00 <cdent> that interface shouldn't matter to extraction, should it?
14:34:04 <bauzas> for example, consider efried's point on generic request expression
14:34:16 <bauzas> cdent: well, a single repo eases things, right?
14:34:19 <cdent> no
14:34:20 <cdent> no
14:34:20 <cdent> no
14:34:32 <cdent> totally disagree with that
14:34:45 <bauzas> honestly, I was way less interacting with placement that cycle than the others
14:34:55 <cdent> (this :) )
14:34:55 <bauzas> so I need to catch up
14:35:24 <bauzas> but still, I'm a bit considering how it can be difficult for things to coordinate if we split
14:35:26 <bauzas> anyway
14:35:32 <bauzas> let's not overdiscuss that now
14:35:45 <bauzas> it's a pretty serious meat for disussions at the PTG :-)
14:36:04 <bauzas> tbc, I'm not opposed to the split
14:36:05 <cdent> bauzas: if you've got comments on the blog posting, before the ptg, if you could leave them there that would help move things forward before we get to the ptg
14:36:18 <bauzas> cdent: I'll certainly read the blogpost
14:36:32 <cdent> if this coming ptg is anything like the last one, speculative discussion about what we _might_ do will be shut down by people who want to talk only about stuff we are definitely going to do
14:36:44 <cdent> so need to discuss as much speculatively beforehand
14:36:53 <edleafe> good point
14:36:54 <cdent> bauzas: comments at least as important as reading
14:37:14 <cdent> edleafe: yeah, not just about extraction, but most topics
14:37:39 <bauzas> roger this
14:37:42 <edleafe> "If it isn't going to be completed in this cycle, why should we talk about it?"
14:38:18 <edleafe> Anyway, let's continue the discussion on the blog post
14:38:26 <cdent> ✔
14:38:28 <edleafe> Anything else before we wrap things up?
14:38:36 <cdent> no sir
14:38:45 <edleafe> Going once...
14:39:02 <edleafe> Going once...
14:39:10 <edleafe> oops - twice!
14:39:24 <edleafe> OK, thanks everyone!
14:39:24 <edleafe> #endmeeting