21:02:32 <ttx> #startmeeting crossproject
21:02:33 <openstack> Meeting started Tue Mar  3 21:02:32 2015 UTC and is due to finish in 60 minutes.  The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:02:34 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:02:36 <dhellmann> o/
21:02:37 <ttx> Our agenda for today:
21:02:38 <openstack> The meeting name has been set to 'crossproject'
21:02:43 <ttx> #link http://wiki.openstack.org/Meetings/CrossProjectMeeting
21:02:46 <harlowja_> \o
21:02:51 * morganfainberg is lurking here.
21:02:58 <ttx> #topic OpenStack developer doc vs. cross-project Specs
21:03:11 <ttx> So... The suggestion of writing an OpenStack coding guide was first raised on the "Eventlet best practices" openstack-spec:
21:03:18 <ttx> #link https://review.openstack.org/#/c/154642/
21:03:22 <bknudson> I'd like the developer docs to look like docs rather than specs.
21:03:28 <harlowja_> i think dhellmann should write another book that we can use
21:03:32 <harlowja_> ^ joke
21:03:33 <ttx> Basically it appeared more as a reference set of coding guidelines than as a clear problem to solve with a beginning and an end
21:03:38 <ttx> dhellmann raised the following thread to discuss the idea:
21:03:42 <ttx> #link http://lists.openstack.org/pipermail/openstack-dev/2015-February/057816.html
21:03:47 <ttx> Reaction was generally positive
21:03:48 <annegent_> bknudson: what does look like mean
21:03:59 <ttx> but then there was the question of when guidelines should be spec material and when should they be coding guide material
21:04:12 <ttx> And it's true that guidelines tend to start their lives as specs (as we make sure the exiting code catches up with the new guidelines)
21:04:20 <ttx> existing*
21:04:26 <ttx> ...and end their lives as guidelines for future code being written
21:04:27 <bknudson> annegent_: the specs look like a list of short docs, whereas a doc is something you can read through.
21:04:35 <ttx> So I guess there are 3 approaches:
21:04:45 <ttx> 1- Use specs for everything, no coding guide (and expect people to find rules for future code in the specs repo)
21:04:58 <ttx> 2- Use spec for drafting rules and the initial effort of converting existing code, then copy relevant parts of the spec to the coding guide so that future code is more likely to follow the rules
21:05:11 <ttx> 3- Draft rules in the coding guide, then create spec to track the effort of converting the existing code to the newly-defined rules (if any)
21:05:12 <annegent_> bknudson: ok, thanks for the explanation
21:05:24 <ttx> Personally I find (1) not really future-proof
21:05:26 <notmyname> isn't this why we have hacking?
21:05:39 <asalkeld> +1 to 3, but same repo as the specs
21:05:42 <ttx> we can't expect future developers to sift through specs to find which ones happen to contain guidelines that they should apply to code they'll write
21:05:50 <dhellmann> notmyname: some things are easier to explain in prose
21:06:06 <ttx> I think (3) makes more sense process-wise than (2)
21:06:07 <harlowja_> ttx sounds like we just need to organize specs more like a 'book' ?
21:06:17 <ttx> They could technically live in the same repository, although the "openstack-specs" name feels a bit misleading then
21:06:21 <nikhil_k> == ttx
21:06:39 <ttx> harlowja_: well, specs tend to be implemented and "completed", while rules live forever
21:06:41 <asalkeld> ttx: we have docs in code repos
21:07:01 <annegent_> yeah I'd personally like to see a centralized cross-project coding guide separate from specs
21:07:08 <harlowja_> well i hope rules don't live forever, the python community seems to change python drastically every few years, lol
21:07:14 <ttx> so I'd argue they are different beasets, aven if there can be specs about implementing rules (especially the first time to catch up)
21:07:30 <sdague> annegent_: yeh, this seems like a development handbook that would be in the openstack manuals constellation
21:07:31 <thingee> o/
21:07:50 <sdague> because things like "eventlet best practices" are one of the proposed bits
21:08:23 <asalkeld> except, having it in the same repo might make it easier to convert specs into docs
21:08:26 <ttx> sdague: bnemec argues that "eventlet best practices" is the same animal as the "Log guidelines"
21:08:28 <annegent_> sdague: except for the reviewers should be experts (not necessarily all doc team members)
21:08:38 <dhellmann> sdague: the testing guidelines and logging guidelines could easily be moved over to that doc, too
21:08:38 <sdague> ttx: I'm unconvinced
21:08:43 <dhellmann> this feels a bit bikesheddy - we've had pretty good luck so far with a "policy" subsection of the oslo-specs repo, and that feels like it would be a much easier route than creating a whole new document
21:08:51 <ttx> i.e. a set of guidelines with a spec to describe catch-up phase
21:09:00 * devananda notes the existence of a similar-ish guide in oslo.db for using sessions
21:09:04 <dhellmann> sdague: the guidelines parts of those documents would move, but the "to do" list wouldn't
21:09:09 <annegent_> dhellmann: guess it depends on the goals -- getting agreement across projects? Or helping contributors?
21:09:37 <sdague> annegent_: right, it's a different audience
21:09:40 <dhellmann> devananda: sure, we do have docs for the libs, too. bnemec's thing on eventlet could go into oslo.concurrency, except I'm not sure anyone would find it there because it's about eventlet not oslo.concurrency
21:09:41 <sdague> specs are about agrement
21:09:57 <ttx> dhellmann: so you would follow option 2 ?
21:09:59 <notmyname> maybe it's because this started with eventlet, but I'm not a big fan of a central set of guidelines (rules) for 500+ openstack repos to follow. syntax/hacking stuff is fine, but "here's how to use this 3rd party library" doesn't seem like a good idea for rules
21:10:04 <bknudson> #link http://specs.openstack.org/openstack/openstack-specs/
21:10:07 <bknudson> there's the repo now
21:10:19 <devananda> notmyname: except for when it is
21:10:23 <dhellmann> ttx: I prefer option 1 actually
21:10:37 <dhellmann> notmyname: it's not prescriptive, it's advice
21:10:42 <sdague> a developer handbook is a contributor onboarding would be kind of handy. Not everyone has been around these things for ever
21:10:52 <notmyname> devananda: my point is that I don't know when that happens, if ever
21:10:56 <devananda> notmyname: plenty of libraries produce docs on how to use them well
21:10:58 * harlowja_ also prefers they not be called rules (software changes folks, practices change, ...); but i guess thats just my personal feeling
21:11:00 <ttx> dhellmann:  but moving spec to a specific directory under specs ?
21:11:00 <dhellmann> sdague: sure, and that's different than what we've had proposed so far as specs
21:11:03 <devananda> notmyname: eg, sqlalchemy
21:11:20 <jungleboyj> sdague: +2
21:11:27 <dhellmann> ttx: yeah, in oslo-specs we just have a "policy" dir but we could use "devref" or something here
21:11:49 <notmyname> devananda: nope. not gonna fly. that's not how openstack works. every "guideline" that starts optional has turned into something that is strongly pressured to conform to
21:12:07 <notmyname> dhellmann: ^^ (sorry not devananda)
21:12:33 <bknudson> so what's the argument for having a separate repo?
21:12:34 <dhellmann> notmyname: do you think that's the case for the existing guidelines?
21:12:51 <devananda> I'd be happy to see shared guides // devref material, but I agree that the audience is different from our cross-project specs
21:13:00 <ttx> notmyname: I guess you object to the concept of cross-project specs anyway ?
21:13:02 <devananda> however the set of folks who should review it is not all that different
21:13:03 <notmyname> devananda: I'm fine with a library producing docs on how to use it well. that's good. I'm not ok with having a prescription for every openstack repo
21:13:09 <notmyname> ttx: no, I don't
21:13:13 <annegent_> bknudson: one argument pro separate repo would be for a review group
21:13:27 <ttx> notmyname: they sometimes contain guidelines, too
21:13:35 <notmyname> dhellmann: I'd point to last years "TC gap analysys" as a way to get projects to conform to guidelines
21:13:37 <dhellmann> notmyname: in the case of the eventlet thing, it's more about saying "if you do this you're going to have bugs, but this other thing works" isn't it?
21:13:44 <morganfainberg> notmyname: if we have learned lessons "this is bad because x" hat is t bad documentation. Doesn't mean there are exceptions to it.
21:13:47 <devananda> notmyname: sure. I do not think it should be proscriptive and mandated across all projects
21:13:53 <bknudson> annegent_: that's a good one.
21:14:04 <devananda> notmyname: however, the eventlet discussion, aiui, originally started because folks want to mandate a change
21:14:15 <dhellmann> notmyname: I don't think that's a bad thing.
21:14:16 <notmyname> it touches a nerve because I've seen it many times. ;-)    /me feels the same way about the api working group
21:14:20 <morganfainberg> But having reasonable guidelines that a lib let's you do but we know causes issues unless you work around it isn't negative.
21:14:55 <devananda> gotta step away for 5 min
21:15:29 <asalkeld> notmyname: there are user's that don't like that each project does things in different ways
21:15:54 <bknudson> for developers it's also confusing when you switch between projects.
21:15:54 <Rockyg> There are operators who don't like that projects aren
21:15:56 <harlowja_> thinks we should just have guidelines/lessons learned (don't call them rules, things that people must follow); if people still want to shoot there feet off, thats there choice
21:16:03 <Rockyg> t consistenet in the OpenStack "space"
21:16:04 <notmyname> harlowja_: that's better
21:17:04 <jungleboyj> bknudson: +1
21:17:06 <harlowja_> Rockyg imho to facilatate a healthy opensource community thats just going to have to be one of the balances that may not always be perfect (conformity via rules, vs innovation)
21:17:30 <dhellmann> the eventlet document was created after someone wanted to patch an oslo library to do something custom with eventlet because their app was not properly initializing eventlet
21:17:34 <Rockyg> A developer ref gives newbies a good set of guidelines to follow that are from lessons learned.  If you're senior and really good, you can judge when exceptions are reasonable
21:17:48 <dhellmann> the oslo team rejected that patch, and started trying to provide more guidance for using eventlet
21:18:24 <asalkeld> so the devref if for newbies, and we need a thing that is more aimed at experts
21:18:24 <dhellmann> they're not rules, per se, but if you're not following them then there will be problems with using libs that expect you to be following them (not just from oslo)
21:18:30 <Rockyg> Jarlowja: if the community finds it too hard to manage, they go elsewhere.  Then the projects become just for developers
21:18:36 <dhellmann> asalkeld: yeah, it sounds like we have 2 audiences
21:18:37 <notmyname> dhellmann: which goes to what harlowja_ said.
21:18:45 <Rockyg> manage in use.
21:19:09 <harlowja_> Rockyg sure, thus the balance that is not easy to maintain
21:19:14 * devananda returns
21:19:21 * harlowja_ afaik the same thing happened with hadoop
21:19:43 <bknudson> even the openstack-specs repo says it's for "Cross-Project Specifications and Policies" -- but there are no policies yet.
21:19:52 <ttx> notmyname: so you'd prefer solution (1) ?
21:20:23 <harlowja_> 'Policies' sounds to much like rules imho
21:20:30 <asalkeld> yeah
21:20:34 <ttx> "Guidelines" sounds better
21:20:40 <Rockyg> The cross-project specs should be used to create devref.  If a spec supersedes old spec, it gets deprecated and the doc gets modified.
21:20:46 <jungleboyj> Seems like we need to stay away from anything that sounds like 'rules'.
21:20:55 <asalkeld> "Implementation suggestions"
21:20:58 <dhellmann> Rockyg: I would like to avoid a bunch of busy work moving things around
21:20:59 <jungleboyj> ttx: +2
21:21:03 <bknudson> we do have rules -- e.g., api stability guidelines.
21:21:17 <harlowja_> u just called that guidelines, not rules, lol
21:21:22 <bknudson> https://wiki.openstack.org/wiki/Governance/Approved/APIStability
21:21:23 <thingee> ttx: +1
21:21:28 <notmyname> ttx: I think "Specs" should be for pieces of work that are defined and get merged to a specific repo (or set of repos). if we want advice for libraries somewhere, then I don't think it should be in a specs repo
21:21:29 <morganfainberg> You could do the same thing that is the API ref docs. Publish the devref from the same repo. Like keystone has for the api spec vs specs for features.
21:21:35 <eglynn_> well "guidelines" is certainly friendlier than hard rules
21:21:44 <ttx> We could do (1) the other way around. Move completed, irrelevant specs to a "completed" directory once they are done. If a spec is in the "active" directory it's still "current". Solves the discoverability issue
21:21:45 <morganfainberg> If you want to have the same repo with devref in it.
21:22:02 <morganfainberg> That way you kind of strike middle ground.
21:22:05 <Rockyg> notmyname ++
21:22:22 <notmyname> ttx: that's what we do in swift actually (have a "done" folder for finished specs"
21:22:28 <ttx> notmyname: but you don't want them in a coding guide either
21:22:35 * bnemec wanders in late
21:22:46 <Rockyg> ttx:  that's what I intended ;-)
21:23:07 <bknudson> having the devref and spec in the same repo means that you can update the devref along with the spec (in the same review), so when the spec is approved/merged the devref is, too.
21:23:32 <notmyname> ttx: perhaps it's the "guide" word. it depends on how that document is done. I do not want it with eg pep8 rules. I'm completely ok with "here's what we've learned about the best way to use evenlet or to [not] do DB migrations etc"
21:23:46 <asalkeld> can we publish this stuff differently, even though they live in the same repo?
21:24:27 <Rockyg> OK.  Developer's Guide
21:24:29 <morganfainberg> notmyname: I think that is what is being pitched here. That was my understanding.
21:24:45 <ttx> morganfainberg: +1
21:24:58 <bnemec> Yeah, it's eventlet best practices, not eventlet commandments. :-)
21:24:59 <morganfainberg> Here is what we know/learned. Don't do x unless you are aware of y because $reason
21:25:17 <Rockyg> morganfainberg ++
21:25:22 <dhellmann> I thought using the same repo would let us actually produce something useful quickly, but if it's just going to mean more arguing then let's create a separate reference guide repository.
21:25:50 <morganfainberg> bnemec: I like there eventlet commandments better (as a name) :P
21:26:05 <notmyname> morganfainberg: yup, that's great. just not in a "this is how you must code in an openstack project"
21:26:25 <bnemec> dhellmann: +1.  I'd prefer to go with the one repo thing like oslo-specs, but not so much that I want to endlessly bikeshed over it.
21:26:50 <dhellmann> we have spent more time talking about it than it would have taken me to set up a new repo, so let's not waste any more time
21:26:55 <morganfainberg> dhellmann: or a separate directory in the same repo. Like the API spec would work. Just can't be a "spec" as in "work to be done". Because that is not meaningful or easy to parse for someone looking for reference.
21:26:59 <SlickNik> dhellmann: ++
21:27:18 <ttx> bnemec: how about writing a blogpost instead
21:27:54 <asalkeld> ttx: there is no review or acceptance then
21:27:58 <morganfainberg> We should just make it an easy way to search/find/reference regardless of where it is. A blog post is as good as a repo to start.
21:28:07 <devananda> blog post --
21:28:16 <ttx> asalkeld, devananda : that was sarcasm
21:28:17 <bnemec> It would have hurt this document a lot to have me do it as a blog post.
21:28:24 <devananda> ttx: oh :)
21:28:29 <bnemec> Heh
21:28:43 <morganfainberg> But just pick a place for it ;). I think we have over discussed this.
21:28:56 <ttx> no kidding
21:29:02 * asalkeld not stressed about this, do *something* and it will probably be fine
21:29:03 <morganfainberg> It doesn't belong as a "work to be done"-spec. That is all that needed to be said.
21:29:12 <dhellmann> #action dhellmann create openstack/developer_reference repository
21:29:14 <devananda> dhellmann: how about oslo/developer-guides (or change name as appropriate) to host developer-contributed-and-reviewed reference material
21:29:15 <bknudson> if we have a separate repo are there different rules for appoval?
21:29:19 <devananda> hah
21:29:21 <Rockyg> Cool thing about spec and doc in same repository is that you can have spec=chapter where appropriate
21:29:29 <dhellmann> ttx: let's move on
21:29:31 <ttx> dhellmann: ++
21:29:40 <devananda> ++
21:29:45 <morganfainberg> bknudson: start with the same rules as we have for the current repo. Update as needed.
21:29:49 <ttx> We have reached the end of my timebox (and patience) for this anyway
21:29:52 <bnemec> +1
21:29:59 <ttx> #topic Replacing eventlet: how to make progress (or not)
21:30:23 <ttx> So we have two specs for separate approaches in getting rid of eventlet:
21:30:25 <harlowja_> sooooo
21:30:27 <ttx> #link https://review.openstack.org/#/c/153298/
21:30:30 <ttx> #link https://review.openstack.org/#/c/156711/
21:30:31 <asalkeld> groan
21:30:40 <ttx> and I feel like those are getting nowhere
21:30:50 <ttx> and I'd like to set a constructive next step for those
21:31:01 <dhellmann> asalkeld: if you all don't pick, oslo is going to pick for you because eventlet is a constant source of trouble for us
21:31:14 <ttx> but then I'm not sure we are on the right day for that
21:31:24 <ttx> I feel like we are stagnating on this, in particular with two specs diluting the essential question of why we should even consider moving off eventlet at this point
21:31:33 <ttx> To unblock ourselves, I suggested that the two proposers actually collaborate to answer that essential question
21:31:43 <ttx> Because having two parallel specs answering that question in slightly different terms doesn't really help making progress imho
21:31:43 <harlowja_> well they are fundamentally different solutions ;)
21:31:50 <asalkeld> out of those 2, i pick asyncio
21:31:51 <ttx> Which is why I proposed that the two specs should converge on a single spec with two implementation alternatives (a metaspec)
21:31:54 <dhellmann> harlowja_: the premise of wanting to move is the same, though
21:31:59 <ttx> Describe the problem with eventlet once, and propose solutions in comparable terms
21:32:06 <ttx> Rather than separately stating that your approach is superior and sending everyone away, wondering why we even want to change eventlet in the first place
21:32:28 <asalkeld> but after using yield for ~year for scheduling - i still am not a fan
21:32:34 <ttx> But then neither author seems interested in that approach
21:32:37 <eglynn_> ttx: have you run the metaspec idea past haypo?
21:32:43 <harlowja_> sure, seems fair, although there will still be the my approach is different better than yours part, lol
21:32:51 <ttx> eglynn_: I did. He prefers to prove his point with code
21:32:53 <harlowja_> due to the fundametnally different ways these work
21:33:04 <harlowja_> lifeless wants CSP instead :-P
21:33:09 <eglynn_> ttx: yeah, I suspected that might be the case
21:33:13 <ttx> I just fear he will pile up a lot of code and get stopped after wasting a lot of work
21:33:22 <devananda> ttx: I would appreciate the clear description of why we need to move off of eventlet
21:33:29 <ttx> devananda: me too
21:33:30 <devananda> and why now()
21:33:30 <dhellmann> harlowja_: "CSP"?
21:33:37 <sdague> devananda: ++
21:33:49 <lifeless> communicating sequential processes
21:33:56 <dhellmann> lifeless: thanks
21:34:04 <devananda> dhellmann: i will object strongly to oslo making a change that forces projects to fundamentally change their implementation / threading model
21:34:07 <ttx> devananda: I mean, we used to have a Python3 long-term issue motivating the change and kind of balancing its high cost
21:34:08 <harlowja_> dhellmann http://goless.readthedocs.org/en/latest/  (ya, what lifeless said)
21:34:17 <lifeless> I linked to several sources in my review comment
21:34:18 <devananda> dhellmann: i'm not sure if that's what you meant, but that's how I read it
21:34:19 <sdague> lifeless: isn't that basically the rewrite in go approach?
21:34:19 <dhellmann> devananda: then it's best to get involved in this discussion :-)
21:34:30 <lifeless> sdague: no
21:34:50 <lifeless> sdague: I'm not a fan of go in general, but the concurrency model stuff is really nice
21:35:16 <devananda> ttx: right. _used_to_. also that didn't motivate us strongly enough over the last two years, so why now() ?
21:35:27 <harlowja_> lifeless +1 the concept is nice and would be great to make happen; i'm just not sure the practicality of it happening
21:35:29 <dhellmann> devananda: ftr, I'm opposed to approaches that require major changes in the applications
21:36:16 <lifeless> sdague: one of the ways in which it is nice is that it is very similar to what we do now (since we do relatively little synchronised workloads anywa)
21:36:19 <dhellmann> devananda: part of this is looking ahead to python 3, where eventlet continues to be only mostly supported
21:36:20 <devananda> dhellmann: I thought so. hence my uncertainty in my interpretation above
21:36:47 <lifeless> eventlet has been a source of bugs
21:36:47 <sdague> dhellmann: I thought the python3 situation was rapidly improving
21:36:52 <lifeless> some major and very hard to diagnose ones
21:37:07 <lifeless> thats the why, I don't think there is disagreement over this, is there?
21:37:08 <sdague> lifeless: so has sqlalchemy
21:37:13 <dhellmann> sdague: it is improving, but I don't know about rapidly or completely
21:37:39 <harlowja_> lifeless and then u can get into the concurrency model taskflow uses/enables (which isn't CSP but is more like concurrent dataflow-like), ha
21:37:42 <lifeless> sdague: yup, and there are folk muttering about that long-term too, but lets pick one battle at a time
21:37:47 <ttx> FWIW I am having network issues
21:38:11 <devananda> when we use a library this heavily and then find issues, is abandoning it better than fixing it?
21:38:26 <ttx> devananda: exactly. I fear that harlowja_  and haypo are moving to solutions and failed to convince anyone there was a problem
21:38:28 <sdague> yeh, I mean libvirt has had monster hard to diagnose bugs
21:38:29 <dhellmann> devananda: do we have volunteers to go work on fixing it?
21:38:57 <lifeless> devananda: depends if the issues are due to implementation or concept - eventlets issues are arguably not implementation
21:38:57 <harlowja_> i'm just proposing a spec, with ideas, i leave it up to the community to pick the solution ;)
21:39:01 <dhellmann> it's fine to say we should work upstream, but if we have no one actually interested in doing that then it's not actually a solution
21:39:03 <devananda> dhellmann: do we have confidence that the proposed alternatives don't *also* have problems that we'll only find when every project is using it at openstack-levels of scale?
21:39:05 <ttx> And each spec presents a slightly different view on what "the problem" is
21:39:13 <notmyname> swift contributors have often made upstream patches to eventlet and haven't had a problem getting them accepted
21:39:39 <dhellmann> devananda: I agree, that's part of the discussion we should be having
21:39:59 <lifeless> devananda: the scale thing confuses me; any one python process is still very limited in size in Openstack
21:40:12 <sdague> lifeless: the developer scale
21:40:13 <dhellmann> notmyname: the issue isn't with eventlet's willilngness to take patches, it's with the lack of people to make the patches in the first place
21:40:18 <lifeless> devananda: do you mean 'many contributors that don't grok the library' ?
21:40:34 <devananda> lifeless: IIRC some of the issues we've had only showed up intermittently, ie, in the gate
21:40:36 <harlowja_> ttx so i think i can work with haypo on creating the unified 'what is the issue' part; and then i guess have huge sections for  the different ways to 'solve' it
21:40:39 <sdague> if you have super awesome developers that don't make mistakes, your underlying implementation doesn't matter
21:41:01 <ttx> harlowja_: I /think/ that would be more likely to succeed
21:41:02 <notmyname> dhellmann: I'm saying that in that when we need something for eventlet, we've made patches and they've been accepted. and swift hasn't seen any scale problems with it
21:41:04 <sdague> if you have 1000+ contributors + 2 million lines of extant code... things are different
21:41:07 <harlowja_> ttx the ways to solve it list will not be comprehensive ( lifeless could for example add a CSP section, i could add another section for taskflow dataflow concurrency...)
21:41:17 <devananda> lifeless: i still haven't seen a compelling reason to switch off of eventlet -- as much as I personally would prefer to have proper threading
21:41:20 <devananda> sdague: exactly
21:41:22 <ttx> harlowja_: I'd very much want to see a CSP section
21:41:45 <harlowja_> and a taskflow dataflow one to, and ... (there are alot of models that could work, ha)
21:41:45 <dhellmann> notmyname: yes, and I'm saying that I don't have anyone willing to help with the python 3 port and so that's going to become an issue if the eventlet team doesn't actually finish it -- I'm trying to ensure we have a path into the future
21:42:10 <notmyname> dhellmann: didn't their latest release fishing that up? 0.17 IIRC
21:42:19 <sdague> 0.17.0, which is in the gate, passes their python 3 tests
21:42:23 <dhellmann> notmyname: they've claimed support before
21:42:44 <sdague> it would probably be interesting to find a project which we could light up on it and see if it passes our tests
21:42:46 <devananda> harlowja_: thanks - I'd really like to understand the justification for even considering mandating a change that would affect all the projects like this
21:42:47 <harlowja_> http://en.wikipedia.org/wiki/Dataflow_programming (for those who wonder wtf dataflow is, lol); taskflow is something like this
21:42:51 <dhellmann> but they were missing some monkeypatching or had installation issues or some other thing
21:42:56 <ttx> harlowja_: but then, that's only my suggestion in incvreasing chances -- feel free to go your own way :)
21:43:18 <harlowja_> ttx well this is a cross-project thing, i can't go and make projects do it (and thats not even possible, i am only 1 man, lol)
21:43:18 <devananda> dhellmann: also, just to make sur ei'm on the same page, what is the impact if oslo were to support both eventlet and asyncio? has that been discussed and I just missed it?
21:43:33 <ttx> OK, 3 more minutes and we'll cover misc other specs
21:43:43 <dhellmann> devananda: asyncio will require a level of code change to actually be useful that means we don't want to support both
21:43:56 <lifeless> dhellmann: won't we have to to enable migration ?
21:44:01 <dhellmann> we would have to support converting things to coroutines
21:44:03 <lifeless> dhellmann: /whatever/ we choose?
21:44:09 <devananda> lifeless: exactly ..
21:44:13 <harlowja_> devananda supporting native threads, and eventlet threads is do-able imho (but then eventlet...)
21:44:18 <dhellmann> lifeless: this is why I don't think asyncio is a reasonable approach
21:44:20 <harlowja_> but asyncio throws a wrench in
21:44:22 <asalkeld> do we have a port of eventlet that works on top of asyncio?
21:44:27 <lifeless> yes
21:44:29 <dhellmann> harlowja_: right, I think moving directly to regular threads is more realistic
21:44:38 <lifeless> haypo has done that
21:44:45 <lifeless> and its in the spec
21:45:00 <harlowja_> afaik its asyncio ontop of eventlet, not the reverse
21:45:11 <asalkeld> lifeless: does that have the same issues?
21:45:17 <dhellmann> it runs on the eventlet loop, but does not do the monkeypatching part, right? so you still have to convert application code to coroutines to actually achieve any concurrency
21:45:25 <lifeless> AIUI he has a eventlet mainloop that is an asyncio adapter
21:45:29 <lifeless> as part of the migration strategy
21:45:38 <asalkeld> o, we need it the other way around
21:45:59 <bknudson> keystone has solved the problem already... deprecate eventlet.
21:46:16 <harlowja_> https://github.com/1st1/greenio i thought right?; i thought that just plugged eventlet/greenlet into asyncio
21:46:17 <morganfainberg> bknudson: that works for api surface but not for conductor etc
21:46:19 <sdague> bknudson: that's only possible for things that *only* have an API service
21:46:20 <harlowja_> so u still need asyncio
21:46:24 <dhellmann> right, the problem we have with eventlet is mostly caused by the use of the monkeypatching and not by eventlet's underlying code
21:46:49 <lifeless> dhellmann: except the race conditions around closed fd's
21:46:58 <ttx> I don't really expect us to come to a conclusion here :)
21:47:01 <dhellmann> sdague: oslo.messaging has a threading executor, so apps that have no wsgi interface could port to threads
21:47:08 <ttx> Let's move on to cover a few other topics
21:47:15 <sdague> dhellmann: sure
21:47:20 <morganfainberg> But I would love to see eventlet and custom wsgi layers go away for the API surface across all of OpenStack (personal preference). Conductor, compute, etc different issue.
21:47:23 <sdague> dhellmann: but keystone doesn't do that  :)
21:47:24 <dhellmann> lifeless: I don't know the background there
21:47:25 <lifeless> dhellmann: which I think *may* be fixed. Assuming no more occurences
21:47:28 <ttx> but I feel a cross-project design summit session to discuss this
21:47:29 <harlowja_> or was it http://aioeventlet.readthedocs.org/ i can't remember, ha
21:48:04 <ttx> #topic Other openstack-specs discussion
21:48:05 <harlowja_> lifeless afaik it was eventlet/greenlet under asyncio; so u still need to be using asyncio concepts (or triolloius concepts)
21:48:07 <dhellmann> ttx: we should have a lot more discussion online first, and get harlowja_ and haypo to pull together the justification for changing at all so we can at least reach some agreement on that
21:48:10 <sdague> ttx: well, if we can't get folks to write the meta spec, I don't think that that sesssion will be productive either
21:48:19 <ttx> #undo
21:48:20 <openstack> Removing item from minutes: <ircmeeting.items.Topic object at 0x7f41590>
21:48:22 <harlowja_> dhellmann i'll see what i can do
21:48:32 <dhellmann> harlowja_: thanks
21:48:32 <ttx> sdague: right
21:48:34 <harlowja_> np
21:48:40 <ttx> #topic Other openstack-specs discussion
21:48:46 <ttx> * Add library stable release procedures/policy (https://review.openstack.org/#/c/155072/)
21:48:50 <ttx> We discussed that one last week, but it didn't really trigger a lot of new reviews
21:48:58 <ttx> I suspect it's more lack of interest or opinion, than a problem with the spec, though
21:48:59 <lifeless> harlowja_: I'll help, ping me?
21:49:05 <harlowja_> lifeless sure
21:49:46 <ttx> So I propose we keep it there for another week then consider moving on to rubberstamping
21:50:04 <ttx> Read and review if you care!
21:50:10 <ttx> * Managing stable branch requirements (https://review.openstack.org/#/c/159249)
21:50:13 <dhellmann> figuring out the right approach for stable releases of libraries is going to be more important now that we're capping (and eventually possibly pinning) dependencies in stable branches
21:50:20 <ttx> This one is the outcome of a discussion that happened on the ML at:
21:50:26 <ttx> #link http://lists.openstack.org/pipermail/openstack-dev/2015-February/057367.html
21:50:30 <ttx> Business summary:
21:50:34 <ttx> - Only a handful of people work on stable branches
21:50:45 <ttx> - Stable branches get broken all the time due to their code being incompatible with a new version of a dependency being published on the Internet (abcd>=1.0)
21:50:59 <ttx> - Recently we added version caps (abcd>=1.0,<=1.1.7) so that stable branches would not pull recent crap, and all was awesome
21:51:09 <ttx> - That can still fail due to uncapped dependencies of dependencies though
21:51:23 <ttx> - jogo's proposal is to compute pinned versions all of dependencies, record that in stable branches (as requirements.gate) and use that in stable branch gating
21:51:41 <adam_g> i've picked up 159249 from jogo and have started a new draft just this afternoon. there are some technical challenges to figure out but i think the idea is solid
21:51:42 <ttx> - master branches are unaffected by this change and continue to potentially consume new crap, to make sure we don't get stuck with stale deps in future development
21:52:20 <ttx> I think it's pretty straightforward, and I wonder if we really needed to go through the openstack-specs process for this
21:52:28 <ttx> it just neds someone to JFDI
21:52:36 <devananda> ttx: everything you just said seems good to me
21:52:46 <ttx> but then we want to document things we do
21:52:56 * ttx blames dhellmann kilo motto
21:53:04 <adam_g> ttx, we need to think carefully about it. this introduces new ways we can wedge ourselves
21:53:13 <jungleboyj> Just as long as we stop breaking the stable branches.
21:53:20 <dhellmann> ttx: I asked jogo for a spec so we could work out the details before making changes, because a big part of the reason we had so much trouble this cycle was making changes to how requirements work without understanding the side-effects
21:53:22 <ttx> adam_g: any example ?
21:53:43 <dhellmann> we could have done that within the requirements group, but since it also affects stable-maint and eventually the projects producing client libraries I thought an XP spec made sense
21:53:55 <ttx> dhellmann: ok
21:54:27 <adam_g> ttx, specifically, how we manage and sync the .gate files. these will end up being  more explicit than anything we've used in the past, and we risk making things even more restrictive
21:54:29 <dhellmann> adam_g: the ability to wedge ourselves is why I prefer caps over pins; it's easy to add a != entry to the file
21:54:30 <jogo> adam_g: thanks for taking over
21:54:32 <ttx> adam_g: if that introduces new way we can wedge ourselves, those should be documented in the spec ?
21:54:56 <ttx> adam_g: should we consider it still WIP ?
21:55:19 <bknudson> are we expecting lots of changes in the pins after the initial one?
21:55:24 <adam_g> ttx, the spec is still a WIP, yes. im in favor of the spec process for this change
21:55:28 <ttx> We can make progress on patchsets in a smaller circle (stable/relmgmt/QA)
21:55:45 <ttx> I thought jogo unWIPped it
21:55:58 <ttx> which signals it's ready for others to review/approve
21:55:59 <adam_g> bknudson, the version pins are generated based on caps (or other constraints) in requirements.txt
21:56:05 <dhellmann> bknudson: that depends on how many backports we have
21:56:05 <ttx> maybe we should reWIP it
21:57:42 <adam_g> ttx, sorry. ive been out for the last week, was trying to pick this spec up today.  will hopefully have something ready for broader  next week
21:57:51 <adam_g> *broader review
21:58:00 <ttx> adam_g: I sugegst you re-WIP it while you analyze consequences
21:58:08 <adam_g> ttx, sure
21:58:26 <ttx> you might need jogo's help to do that
21:58:30 <adam_g> jogo, ya ^
21:58:44 <ttx> OK, We'll reconsider when it's unWIPped
21:58:46 <ttx> #topic Open discussion & announcements
21:58:50 <ttx> We had 1:1s syncs today in #openstack-relmgr-office, logs at:
21:58:54 <ttx> #link http://eavesdrop.openstack.org/meetings/ptl_sync/2015/ptl_sync.2015-03-03-09.00.html
21:59:01 <ttx> Next week I'll be at the Ops Summit, so if someone wants to pick up chairing this meeting, let me know
21:59:09 <ttx> Based on the agenda we might just skip it anyway
21:59:18 <ttx> Anyone else coming to the Ops Summit ?
21:59:20 <dhellmann> I can do it, if we need a meeting
21:59:26 <ttx> dhellmann: thx!
21:59:38 <morganfainberg> Yay dhellmann!
21:59:44 * notmyname will be at the ops summit
21:59:59 * morganfainberg will not be at the ops summit.
22:00:07 <ttx> I know sdague and mordred will be too
22:00:39 <ttx> and thingee
22:00:44 <sdague> and mtreinish
22:01:22 <ttx> alright, we should be able to conspire^Wdrink together
22:01:36 <ttx> and that is the end
22:01:37 <ttx> #endmeeting