21:01:55 <markmcclain> #startmeeting crossproject
21:01:57 <openstack> Meeting started Tue Jul  7 21:01:55 2015 UTC and is due to finish in 60 minutes.  The chair is markmcclain. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:01:58 <tpatil> o/
21:01:59 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:01:59 <j^2> ,/
21:02:02 <stevebaker> \o
21:02:02 <openstack> The meeting name has been set to 'crossproject'
21:02:06 <gordc> o/
21:02:16 <dtroyer> o/
21:02:18 <johnthetubaguy> o/
21:02:35 <mtreinish> o/
21:02:39 <markmcclain> #link https://wiki.openstack.org/wiki/Meetings/CrossProjectMeeting Agenda
21:02:40 <hogepodge> o/
21:02:46 * geoffarnold lurking
21:02:48 <markmcclain> #info The M-Release will be called Meiji (明治)
21:02:55 <etoews> o/ ping me if needed
21:02:59 <markmcclain> #link http://lists.openstack.org/pipermail/openstack-dev/2015-July/068718.html
21:03:01 <edleafe> o/
21:03:32 <Rockyg> o/
21:03:39 <mestery> o/
21:04:34 <notmyname> here
21:04:45 <stevebaker> https://www.youtube.com/watch?v=Eau5KnckMTA
21:04:46 <markmcclain> We're also in need of volunteers to chair this meeting in late July and August... feel free to add your name to the wiki.  Members of TC will help you get setup
21:04:58 <markmcclain> #topic Horizontal Team Announcements
21:05:31 <ttx> On the release management side, two things:
21:05:35 <ttx> I summarized the final plan for stable releases last week:
21:05:41 <ttx> #link http://lists.openstack.org/pipermail/openstack-dev/2015-July/068400.html
21:05:53 <ttx> I also started working on overhauling the release model tags, to match the new models we support in Liberty, and make it clearer what the options are:
21:05:59 <ttx> #link https://review.openstack.org/#/c/198789/
21:06:08 <ttx> Still WIP but early comments welcome
21:06:14 <ttx> EOF
21:06:31 * nikhil_k sneaks in
21:06:41 <david-lyle> ls
21:07:07 <redrobot> o/
21:07:25 <johnthetubaguy> ttx: I never saw python-**client as aligned with a cycle, but I guess it is a little bit
21:08:08 <fungi> as dependencies of api servers it is
21:08:42 <fungi> hence the reason we eventually ended up with stable branches so we could release stable updates under caps to guard against breaking changes
21:08:44 <gordc> johnthetubaguy: i think it is now since we created all the stable branches to match requirements.
21:08:52 <gordc> err that ^
21:09:16 <johnthetubaguy> ah, good point, I guess they are aligned now, if only in a dependency sense
21:09:41 <fungi> takes us back to the whole "you can use it as a client library, but also our servers use it to talk to one another" paradigm which is somewhat irreconcilable
21:09:41 <ttx> johnthetubaguy: it used to not have stable branches at all. It now has
21:09:51 <ttx> so it's kind of linked now
21:10:04 <johnthetubaguy> ttx: yeah, not sure I had clocked in impact of that till just now
21:10:05 <ttx> Anyway, I think we can move on
21:10:10 <johnthetubaguy> yep
21:10:51 <markmcclain> any other announcements?
21:11:13 <markmcclain> #topic New API Guidelines ready for cross project review
21:11:23 <markmcclain> #link  http://lists.openstack.org/pipermail/openstack-dev/2015-July/068682.html
21:11:55 <markmcclain> etoews: listed 7 guideline ready for final review please make sure to take a look
21:12:28 <markmcclain> not certain there's any other discussion, so we can move on
21:12:32 <markmcclain> #topic Is Eventlet Best Practices ready for TC?
21:12:40 <markmcclain> #link https://review.openstack.org/#/c/154642/
21:13:04 <markmcclain> This is not a spec in the true meaning... really a collection of best practices
21:13:21 <ttx> unless there is some -1 vote on this I guess we should consider it as ready as it will ever be and approve it at TC level
21:13:29 <ttx> next week
21:13:49 <markmcclain> ttx: agreed just wanted to give a final look before requesting it on the TC agenda
21:14:16 <markmcclain> #topic Return request ID to caller
21:14:20 <markmcclain> #link https://review.openstack.org/#/c/156508/
21:14:23 <tpatil> Now, we have 2 solutions to return request id from client
21:14:27 <johnthetubaguy> for the record, I love the pattern of sharing the best practices
21:14:32 <tpatil> 1) Add a new method get_previous_request_id to return X-Openstack-Request-Id
21:14:39 <markmcclain> johnthetubaguy: ++
21:14:44 <tpatil> 2) Register a callback method with the client which will be invoked after it gets a response from OpenStack service (This solution is suggested by Brant), Thanks Brant for this wonderful solution
21:14:55 <tpatil> Implementation wise, both the solutions requires minimal changes to the respective python client code base
21:15:07 <tpatil> Need feedback on which way to go
21:15:10 <Rockyg> FYI.  There are a bunch of old BPs that seem to implement some of this stuff (and I can't find them right now)
21:15:40 <bknudson> I'm fine with either one. I'm just not a big fan of thread-local, probably due to not having used it much.
21:15:48 <Rockyg> It appears there was an attempt in 2014/2014 to make this happen and some of it is coded already.
21:16:08 <tpatil> I'm inclined towards Solution #2
21:16:47 <jungleboyj> I would like to see this added.  Don't have a strong feeling on the implementation.
21:17:05 <lifeless> ugh
21:17:13 <lifeless> both those options are poor
21:17:16 <lifeless> can I propose #3?
21:17:23 <lifeless> return structured data
21:17:29 <Rockyg> So, the requestIDs might already exist in some/all project and only need access added to clients
21:17:39 <lifeless> don't call forward, and don't use thread locals. Neither are needed.
21:17:40 <Rockyg> lifeless: +1
21:17:56 <lifeless> I have -very- strong feelings about this.
21:18:05 <tpatil> lifeless: We have considered that before, but due to compatibility issue we had to drop that idea.
21:18:11 <Rockyg> syslog RFC has a structured field in the header
21:18:20 <lifeless> tpatil: What compatibility issue?
21:18:23 <johnthetubaguy> I know there was talk of not breaking backwards compatibility, but I am sure we can do both somehow
21:18:38 <lifeless> tpatil: we're going to end up paying for whatever API is written up forever.
21:18:56 <tpatil> lifeless: if we decide to return structured data from client to caller, then we would need to make several changes to the OpenStack core project using clients.
21:19:02 <lifeless> tpatil: there are lots of ways to gracefully change return types - kw flags to signal, new method names - being two of the most common
21:19:08 <nikhil_k> #2 has a problem that you may not get the req id even after getting a correct response from the API. you can see a 5xx in the req-id call and tell the user wrongly that request has failed.
21:19:15 <lifeless> tpatil: yes, and you'll need to make several changes to the OpenStack core projects to do this anyway.
21:19:44 <lifeless> tpatil: hidden global behaviours (which #1 and #2 are) are bug-traps.
21:20:12 <nikhil_k> agree, somewhat
21:20:44 <tpatil> nikhil_k: we can call callback method only if case of error > 400
21:20:49 <johnthetubaguy> so the other thing, is maybe just add this in the new SDK, to speed up the deprecation of the older clients?
21:20:51 <lifeless> IIRC the apis generally return a dict anyhow right ?
21:20:58 <lifeless> so just add another key in the dict
21:21:26 <johnthetubaguy> lifeless: honestly thats what I assumed this was before I read the spec
21:21:37 <lifeless> [I may be remembering the wrong layer]. Anyhow, happy to do design closer to the code - can you point me at the specific methods being called that are a problem ?
21:22:34 <johnthetubaguy> tpatil: do we have a prototype we could look at, that might get an answer quicker?
21:22:59 <tpatil> for solution #2, we can provide POC code by tomorrow.
21:23:52 <lifeless> tpatil: #2 is also not acceptable
21:24:09 <lifeless> tpatil: I'm sorry to be so hard-and-fast here, but this is fundamental plumbing affecting everything
21:24:36 <bknudson> I think a lot of keystoneclient methods return lists rather than a dict
21:24:38 <lifeless> tpatil: Design the right API, and then we can figure out how to get from A to B.
21:24:39 <tpatil> lifeless: what's the issue with solution #2? can you please elaborate more on this?
21:24:46 <bknudson> like list_users
21:25:10 <lifeless> tpatil: its got all the same quirks as TLS does - e.g. its going to impact client sharing across threads
21:25:32 <johnthetubaguy> lifeless: are you thinking about having some "response" wrapper returned that include the return value and the request-id, and maybe some flag when you create the client to say, wrap responses to include the request id, so the client opts into the new return values?
21:25:33 <lifeless> tpatil: the clients have been stateless from the callers perspective so far as I understand it
21:25:34 <bknudson> if you want to implement tls in your callback then go ahead.
21:26:10 <tpatil> lifeless: even in case of client sharing across threads, in the callback method we have response object which details out everything like URL and request id.
21:26:12 <lifeless> tpatil: they wrap a specific auth token, but are not more stateful than that. This is a bug change.
21:26:38 <lifeless> tpatil: which then forces the receiver of that callback to be globally synchronised with all threads of execution so that it knows what to do with the thing it received.
21:26:51 <nikhil_k> win 49
21:27:07 <lifeless> johnthetubaguy: for instance, yes.
21:27:26 <tpatil> lifeless: what's the issue with solution #1?
21:27:54 <Rockyg> point of information:  Anyone look at http://git.openstack.org/cgit/openstack/oslo.middleware/tree/oslo_middleware/request_id.py?
21:28:00 <lifeless> tpatil: stateful and thus racy: if a decorator were to make a second request before the first obtained its request id, we'd lose it
21:28:04 <johnthetubaguy> lifeless: I think I get your point, that approach is cleaner than suddenly having the client become secretly statefull
21:28:25 <lifeless> side effects make code hard to reason about
21:28:35 <tpatil> lifeless: that's right, but as a developer you shouldn't be doing that..wrong coding practice.
21:28:43 <lifeless> and create traps. I'm giving concrete examples of the traps that can occur.
21:28:55 <johnthetubaguy> Rockyg: you mean the thing as used here: https://github.com/openstack/nova/blob/master/etc/nova/api-paste.ini#L88
21:28:57 <lifeless> tpatil: as a developer we shouldn't be creating such traps for our future selves to suffer from.
21:29:20 <Rockyg> yup
21:29:35 <bknudson> lifeless: so what would your proposal look like? change every API to return an object that has a .request_id attribute?
21:29:45 <Rockyg> johnthetubaguy: It seems Nova implemented it a while ago....
21:29:59 <johnthetubaguy> Rockyg: yeah, its been working well for us, AFAIK
21:30:07 <lifeless> bknudson: there's a few ways that come to mind. One would be to yes, put an attribute on the existing responses.
21:30:10 <bknudson> we'd probably have to provide a wrapper that gets returned.
21:30:26 <johnthetubaguy> Rockyg: my life would be a living hell without it
21:30:28 <Rockyg> johnthetubaguy: shouldn't this be the model for other projects?
21:30:36 <lifeless> bknudson: another would be to be returning structured data that includes both the api value, and the metadata about the transaction [of which requestid is the first]
21:31:16 <lifeless> Rockyg: thats the other side
21:31:18 <bknudson> lifeless: can that be backwards compatible?
21:31:21 <nikhil_k> ummmm
21:31:22 <dhellmann> lifeless: some, if not all, of the clients return objects not dictionaries
21:31:31 <johnthetubaguy> Rockyg: I guess I had wrongly assumed most other folks had already adopted that middleware, when they could, but yeah, its the other side of the problem here
21:31:31 <bknudson> otherwise you're rewriting every application
21:31:31 <dhellmann> the rest APIs return dictionaries, but the client turns them into classes
21:31:36 <lifeless> Rockyg: it ensures that in the server there is a request id created and that it gets back to the client
21:31:50 <nikhil_k> lifeless: what other transation details? mind sharing a couple?
21:32:13 <Rockyg> lifeless:  sort of a linked list first key: orig, ID, second ke : caller id, new  ....last key last id, end
21:32:13 * dhellmann catches up with scrollback
21:32:50 <johnthetubaguy> https://github.com/openstack/python-novaclient/blob/master/novaclient/v2/keypairs.py#L47
21:32:59 <lifeless> nikhil_k: I don't have any- I'm talking extensability. E.g. what if we wanted to know the HTTP return code (even though its a success)
21:33:04 <johnthetubaguy> yeah, I think we return objects not dicts
21:33:07 <lifeless> nikhil_k: like - was it a cached value
21:33:34 <nikhil_k> I see, yeah that seems relevant
21:33:38 <bknudson> I assume list() returns a list of objects
21:34:47 <lifeless> https://github.com/openstack/python-novaclient/blob/master/novaclient/base.py#L64
21:34:50 <lifeless> is _list
21:35:52 <lifeless> yes,
21:35:55 <lifeless> it returns a list of objects
21:36:08 <lifeless> so one entirely compatible way would be:
21:36:15 <lifeless> class ListWithMeta(list):
21:36:32 <lifeless> def __init__(self, values, metadata):
21:36:40 <lifeless> super(ListWithMeta, self).__init__(values)
21:36:46 <lifeless> self.metadata = metadata
21:36:47 <bknudson> use paste!
21:36:51 <lifeless> its 4 lines
21:36:54 <lifeless> you can live :)
21:37:04 <lifeless> just use that in _list
21:37:26 <lifeless> and stash whatever we want in .metadata - e.g. a dict with keys like request-id
21:37:31 <johnthetubaguy> lifeless: yeah, it seems simple enough
21:37:50 <dhellmann> does the neutron client return objects, or does it still use simple data structures?
21:37:53 <tpatil> lifeless: good solution
21:38:04 <lifeless> its not 10000% risk-free, but it should cover most issues
21:38:10 <Rockyg> ++
21:38:12 <bknudson> I thought the goal was to not have to make a lot of changes to the code
21:38:22 <lifeless> take a similar approach across the board at the point actual calls are made
21:38:36 <dhellmann> bknudson: I think with lifeless' proposal only the client code would change, because we would be returning compatible objects that had more features
21:38:40 <markmcclain> dhellmann: dictionaries
21:38:44 <johnthetubaguy> bknudson: it may have been, but it should really be to make life easy of the users of the client
21:39:02 <bknudson> as long as someone's willing to do the work I'm fine with it
21:39:09 <dhellmann> markmcclain: ok, so those will need to be DictWithMeta or something
21:39:13 <bknudson> maybe a chance to have the clients use some library code
21:39:37 <bknudson> because the novaclient code looks very similar to the keystoneclient code
21:39:41 <stevebaker> some clients return dicts, others real objects
21:39:44 <markmcclain> dhellmann: nearly all responses are in an enclosing dict, so a metavar could be added
21:39:57 <dhellmann> markmcclain: that's another way to do it
21:39:59 <lifeless> so look
21:40:10 <lifeless> I understand the minimum-size-of-change argument
21:40:29 <lifeless> but we can't build robust scalable and *easy to change* code on a bed of hidden state and fragile things
21:40:45 <lifeless> the client patterns are -old-. They didn't anticipate where we are today.
21:41:16 <markmcclain> dhellmann: http://paste.openstack.org/show/353082/
21:41:52 <johnthetubaguy> so I can't help thinking about adding this in the newer SDK, and just killing off the older ones, but that also feels like its cheating
21:41:52 <dhellmann> lifeless: the challenge is with pushing the needed changes through all of the consuming projects
21:42:11 <dhellmann> johnthetubaguy: I think the sdk team does not really want it to be tied up in inter-server communication pattersn
21:42:31 <lifeless> dhellmann: OTOH request-id is something clients want
21:42:58 <lifeless> dhellmann: I filed one of the originating bugs here to have that available to debug things easily from shell
21:43:08 <dhellmann> lifeless: I agree. I'm not sure we need all of the patterns you're foreseeing, though
21:43:11 <lifeless> dhellmann: (without having to manually read the raw HTTP coming back)
21:43:19 <lifeless> dhellmann: sure
21:43:25 <lifeless> so what do we want to achieve here?
21:43:33 <bknudson> it would be interesting to see a sample implementation of proposal #3.
21:43:45 <lifeless> the neutron dict approach would be amenable to a new key
21:43:53 <johnthetubaguy> dhellmann: not sure we agreed on what to use for inter-server coms yet, but I still think end users will want the request-id
21:44:02 <lifeless> but having something consistent and usable across the board would be ideal IMO
21:44:05 <bknudson> there's probably some APIs that don't return anything now
21:44:07 <Rockyg> johnthetubaguy: ++
21:44:10 <dhellmann> I've been trying to balance the needs of this one new feature with the amount of effort I know it will be to make potentially breaking changes to the way the clients work
21:44:12 <bknudson> so those would have to change to return something
21:44:35 <bknudson> it's not a problem with backwards-compat, but just the amount of work reqd to find all these and change them.
21:44:54 <lifeless> so
21:44:59 <bknudson> we'll also have to find a way to get the request ID when an exception is raised
21:45:18 <lifeless> you could start with the servers, from there identify the api calls actually made cross-process, fix just those in a consistent pattern
21:45:21 <dhellmann> we can add extra attributes to exceptions pretty safely
21:45:23 <johnthetubaguy> bknudson: same solution right?
21:45:25 <lifeless> yah
21:45:28 <lifeless> same solution
21:46:20 <bknudson> yes, it's easy to a field to an exception, but you have to also change all the code that raises exception to set the value and I don't know how much code that is
21:46:24 <dhellmann> lifeless: looking at the way a lot of this code relies on base class methods, I think once they fix those to start returning enhanced objects it's going to apply to a lot of the calls
21:46:26 <johnthetubaguy> so Nova can be a fun source of cross server communication, if you are looking for a place to start looking :)
21:46:27 <lifeless> I'd like to be able to write a wrapper function e.g. 'call_and_log' that would make the call, log the request id, and be transparent
21:46:28 <bknudson> to add a field
21:46:41 <tpatil> definitely, it's going to be a big change though
21:47:00 <lifeless> bknudson: huh, no. You only need to change the places its raised in the client, because you're adding the request id client side
21:47:05 <lifeless> bknudson: much more contained
21:47:17 <dhellmann> lifeless: I think he means there may be many such places, though
21:47:40 <lifeless> dhellmann: ah. I assumed grep within a client would be fairly easy.
21:48:06 <dhellmann> easy to find, harder to update and test
21:48:22 <dhellmann> but maybe we're getting into details we don't need to worry about right now
21:48:42 <markmcclain> dhellmann: kind of seems we're headed that direction
21:48:47 <lifeless> so if you look at the novaclient code
21:48:50 <johnthetubaguy> I think we should prototype this, and see how bad it really is
21:48:58 <dhellmann> this new proposal to return "enhanced" versions of the objects feels a little hacky, but I'd like to see what it really looks like
21:49:15 <johnthetubaguy> nova has a base resource, so it might be just adding it there
21:49:20 <dhellmann> if it's really transparent, then I think it is better than the thread local solution
21:49:36 <tpatil> let me work on this solution and come up with the POC code by next week
21:49:46 <Rockyg> Sounds like Nova is the code base to be the guinea pig ;-)
21:49:50 <johnthetubaguy> dhellmann: agreed it feels hacky, but it seems a better trade off, but I could imagine changing my mind when I saw code
21:49:52 <lifeless> tpatil: cool, thank you
21:50:00 <bknudson> tpatil: yes, thanks!
21:50:06 <dhellmann> johnthetubaguy: right
21:50:11 <Rockyg> Excellent tpatil!
21:50:13 <tpatil> Thanks everyone
21:50:34 <markmcclain> good discussion everyone
21:50:44 <markmcclain> #topic Open discussion
21:50:53 <johnthetubaguy> tpatil: thanks for pushing on this, its really needs doing, appreciate it!
21:51:16 <tpatil> johnthetubaguy: That's on my plate for this release from long time
21:51:34 <markmcclain> #link https://wiki.openstack.org/wiki/Travel_Support_Program
21:52:01 <markmcclain> There's about a month left to apply, so if know candidates who this will benefit make sure they know about the deadline
21:52:12 <markmcclain> Anything else for this week?
21:53:14 <ttx> Volunteers for chairing the next two weeks ?
21:53:35 <markmcclain> nikhil_k will leading next week unless the wiki is stale
21:53:42 <johnthetubaguy> I think I signed up for the end of the month
21:53:46 <ttx> oh, right weeks after
21:53:56 <nikhil_k> i am on schedule
21:53:59 * ttx goes to bed then
21:54:01 <markmcclain> johnthetubaguy: yep see you on the list
21:54:11 <markmcclain> we do need a volunteer for the 21st
21:54:24 <markmcclain> johnthetubaguy: thanks for volunteering
21:54:46 <Rockyg> 21st is OSCon
21:55:03 <johnthetubaguy> hmm, its the Nova midcycle too, oops
21:55:11 <bknudson> next week is keystone midcycle
21:55:56 <markmcclain> yeah... so if there's a topic that's of big interest and conflicts with a mid-cycle make sure to note it on the wiki
21:56:06 <markmcclain> so we can get the right people here to discuss
21:56:37 <markmcclain> Thanks all for joining this week.  Have a good rest of your week.
21:56:40 <markmcclain> #endmeeting