Monday, 2020-03-02

*** dhellmann_ has joined #openstack-tc00:07
*** dhellmann has quit IRC00:08
*** dhellmann_ is now known as dhellmann00:08
*** jamesmcarthur has quit IRC00:26
*** jamesmcarthur has joined #openstack-tc01:35
*** jamesmcarthur has quit IRC02:12
*** jamesmcarthur has joined #openstack-tc02:17
*** jamesmcarthur has quit IRC02:21
*** jamesmcarthur has joined #openstack-tc02:36
*** jamesmcarthur has quit IRC03:02
*** slaweq has joined #openstack-tc04:17
*** slaweq has quit IRC04:22
*** slaweq has joined #openstack-tc05:20
*** slaweq has quit IRC05:24
*** evrardjp has quit IRC05:35
*** evrardjp has joined #openstack-tc05:35
*** slaweq has joined #openstack-tc05:35
*** slaweq has quit IRC05:40
*** lpetrut has joined #openstack-tc07:02
*** lpetrut has quit IRC07:03
*** lpetrut has joined #openstack-tc07:03
*** iurygregory has joined #openstack-tc07:28
*** witek has joined #openstack-tc07:46
*** slaweq has joined #openstack-tc07:53
*** jaosorior has joined #openstack-tc08:16
*** rpittau|afk is now known as rpittau08:18
*** tosky has joined #openstack-tc08:23
*** dmellado has quit IRC09:23
*** dmellado has joined #openstack-tc09:26
*** dmellado has quit IRC09:33
*** dmellado has joined #openstack-tc09:35
*** jaosorior has quit IRC09:35
*** tetsuro has joined #openstack-tc11:12
*** rpittau is now known as rpittau|bbl11:32
*** tetsuro has quit IRC11:50
*** Luzi has joined #openstack-tc11:53
*** rpittau|bbl is now known as rpittau13:34
*** tosky_ has joined #openstack-tc13:47
*** tosky has quit IRC13:47
*** tosky_ is now known as tosky13:47
openstackgerritMerged openstack/governance master: Add QA upstream contribution opportunity  https://review.opendev.org/70663714:32
mnasertc-members: https://review.opendev.org/#/c/710020/ had a few revisions, appreciate reviews again14:44
*** beekneemech is now known as bnemec15:00
*** Luzi has quit IRC15:01
* fungi sighs at http://lists.openstack.org/pipermail/openstack-discuss/2020-March/012928.html15:05
smcginnisI was not aware of that stance by the team. This may require more discussion.15:08
jrollsmcginnis: yeah, it's been a thing for a while :(15:15
smcginnisI don't like it, but it does potentially highlight a huge risk.15:15
jrolleg. https://docs.openstack.org/releasenotes/glance/train.html#relnotes-19-0-0-stable-train15:15
jroll>Documentation examples were changed from openstack commands back to glance. This should help avoid the frustration of glance-community maintaining different client than what is referred in examples. ‘python-glanceclient’ is and will be the reference implementation of Images API and the team will implement all API changes to the relevant client version of the cycle as well.15:16
smcginnisThe OSC team is drastically understaffed at the moment.15:16
mugsieits been like this for a while15:16
smcginnisAnd I don't believe it is able to handle keeping up with the current needs of the projects that are part of it.15:16
*** lpetrut has quit IRC15:18
jrollyeah, that's the excuse, is that OSC devs haven't implemented everything glanceclient supports15:19
fungion the other hand, projects are choosing to maintain their own separate clients rather than help15:19
fungiit's this siloing which is why we fail at working together as a communityty15:19
mugsiejroll: the main issue for a long time was the inabliity to pipe a file into OSC afaik15:20
evrardjpwell I complained on this ... a year ago?15:20
mugsieevrardjp: aye15:20
mugsieit is a long running thing15:20
jrollfungi: right, that's why I said excuse instead of reason :)15:20
evrardjpI think the solution would have been to do the convergence of clients as a community goal, like I initially proposed15:20
evrardjpeven if it was too ambitious...15:21
*** jamesmcarthur has joined #openstack-tc15:21
smcginnisThat will take more than one cycle to do. Likely more than 5 cycles.15:22
smcginnisMaybe we should get something scheduled for Vancouver to have a larger discussion about this.15:22
zanebevrardjp: I think I counted at least 3 goals that the Glance team has not completed since Queens15:24
zanebso I'm not sure that would help15:24
evrardjpisn't that a big signal? :p15:25
gmannbut what is option even big signal?15:25
evrardjpmaybe we should have something out there to say "please help glance, they don't even have resources to support using our official common client"15:25
zanebwe basically did15:26
gmannwe have been keep saying since many years15:26
evrardjpwe said glance need help15:26
evrardjpI am focusing on a different point here15:26
gmannyeah, osc is one things but there are or may be other in future15:26
evrardjpI don't disagree15:26
zaneband the team is fairly constrained... but if you look at their PTG output they're talking about new features to work on and not even discussing all the goals they've missed15:27
evrardjpzaneb: is that because other projects ask for those features?15:27
*** jamesmcarthur has quit IRC15:27
zanebI don't remember what they were so can't comment15:28
toskyfungi: sorry, but sometimes project tried to help with no result (see microversion for OSC, which is blocking cinder)15:28
toskyfungi: please don't put all the blame on the projects15:28
evrardjpI mean, I have no doubt they try to be good citizens :)15:28
*** jamesmcarthur has joined #openstack-tc15:28
toskythat said, I don't agree with the full denial way that glance decided to follow15:28
fungitosky: i don't, i see it as a possible sign that we might need to find ways to make it easier for them to work on osc instead of on their individual clients15:29
evrardjpI don't want us to be fingerpointing on who is to blame in this story15:29
evrardjpif possible15:29
evrardjp:)15:29
evrardjpI would rather think of a way forward15:29
zanebfungi: on that note ftr I still think it's a mistake to have in-tree and out-of-tree plugins for OSC. If everything were out of tree it would be easier for all projects to develop their own15:30
zanebmugsie has been trying to fix that for years15:30
fungithey choose not to work on osc, but that could certainly be due to friction making it too hard for them to do so15:30
fungizaneb: yep, i think i agree15:30
fungi(but splitting my attention with a conference at the moment)15:31
mugsiezaneb: yeah, I stand by my plugins for all manirfesto15:31
evrardjpif everything was in-tree you would have a larger team15:31
evrardjpif everything was out of tree everything is simple15:31
evrardjp"simple"15:31
toskyevrardjp: the first part is not necessarily true15:31
evrardjptosky: so is the second :p15:32
toskyevrardjp: the second has been proven to be at least consistent15:32
evrardjpit's all about side effects here15:32
evrardjptosky: maybe not in the details, but in the interface indeed15:32
evrardjpmy point is : we can talk about in-tree, out-of-tree, but it won't change the current state15:33
zanebif you tell a team they can choose whether to maintain 2 clients or 1, they'll rarely choose 215:33
jrollmugsie: ++15:33
evrardjpzaneb: totally agree15:33
evrardjpwhich is why I wanted to push for the story "everyone uses osc" and that's it15:34
zanebif you tell a team they can maintain their own client or fight to get reviews from another team on the official client, they'll often choose to maintain their own15:34
evrardjpyes.15:34
evrardjpNIH syndrome too15:34
toskyevrardjp: I think that can be solved only by either a) from the OSC core team, providing the missing building blocks needed by other teams or b) giving more power to the other teams15:34
mugsiebut - if you make them be aplugin, but not have the ability to generate cli docs - they also will think twice15:34
zanebmugsie: or maybe the OSC team will think twice about their plugin support once they have no CLI docs15:35
evrardjptosky: I don't disagree :)15:36
zaneb(relevant issue: https://bugs.launchpad.net/python-openstackclient/+bug/1735016 )15:36
openstackLaunchpad bug 1735016 in python-openstackclient "Plugin documentation is worthless" [Undecided,New]15:36
zanebmugsie: you can at least generate them locally, they just won't appear in the OSC docs15:37
zanebe.g. https://docs.openstack.org/python-heatclient/latest/cli/index.html15:38
toskyzaneb: uhm, about that, I didn't even know there was centralized documentation15:39
toskyin sahara we have had our own OSC doc for ages (from the same commands): https://docs.openstack.org/python-saharaclient/latest/cli/reference.html15:39
zanebtosky: https://docs.openstack.org/python-openstackclient/latest/cli/command-list.html15:40
zanebif you are in the blessed group15:40
*** rm_work has joined #openstack-tc15:41
* asettle laughs at the "plugin documentation is worthless" bug title15:41
asettleAnother zing, zaneb15:41
* zaneb didn't make any friends with that one15:42
asettleShocking.15:42
asettle:p15:42
evrardjphaha15:42
evrardjpI am glad zaneb is not here to make friends.15:43
evrardjpthis is serious bizniz15:43
asettle"However, those projects relegated to the plugin ghetto get only this word salad:"15:43
asettleahahhahahahaa15:43
zanebI am! I am! I plead incompetence15:43
evrardjplol15:43
asettleAbsolutely fuckin' cacklin15:43
asettleWhat a bug report15:43
asettleOn a scale of one to 10, how annoyed were you when you wrote this?15:44
zanebit was a bad day, clearly15:44
asettle2017 was rough on us all man15:44
zanebI can check my diary if you like15:44
* evrardjp clicks the subscribe button15:44
rm_workfor the record, having been in several conversations with glance folks about this issue, it definitely doesn't seem a case of "we're trying, but there's a lot of friction and it's too difficult"... it definitely felt like "nah, didn't you hear? OSC is dead, people are supposed to just use their own plugins again"15:44
asettlezaneb, absolutely15:45
evrardjpI heard something different in the past rm_work15:45
rm_workI almost screamed at the guy I was talking to last time but opted to just walk away15:45
evrardjpthat sounds a fair move that many practiced15:45
evrardjp:D15:45
asettlerm_work, interesting. It definitely was the former last time I spoke to the PTL.15:45
rm_workhe just seemed willfully oblivious <_<15:45
evrardjpbut I think there is a big problem, and I am trying to find a way forward15:45
evrardjpasettle: I got the same experience15:46
*** jamesmcarthur has quit IRC15:46
evrardjpthanks for saying this rm_work15:46
rm_workit also seemed very much like "well, the OSC team hasn't added all our features, so why should we use it?"15:46
* rm_work shrugs15:47
smcginnisTo be fair, there are about 1.5 people actually working on glance.15:47
rm_worklol yes15:47
mordredlet me know if I can help out in any way with this issue15:47
asettleYeah, that's true. And has been the case for quite some time.15:47
smcginnisSo anything that is not a quick and easy update for anything but core functionality is just not going to happen.15:47
rm_workbut it's not like it is actually that much work to add support for projects in OSC15:47
evrardjpmordred: I am trying to find what are the real next steps15:47
zaneboh hey, it looks like that bug was fixed and I can close it15:47
evrardjpbecause when it will be about to say "who will work on this, everyone will disappear"15:47
evrardjpzaneb: lol15:47
smcginnis"but it's not like it is actually that much work to add support for projects in OSC" < based on past experience, I have to disagree with that statement.15:48
evrardjpit's not a word salad anymore?15:48
mordredevrardjp: awesome. the landscape has changed a bit too, since we're trying to use sdk and not python-*client in OSC now15:48
evrardjpthat's true15:48
mordredso where in the past one of the reasons for being minimal in OSC core was due to library proliferation hell15:48
mordredthe situation on the ground is markedly different now15:48
rm_workOctavia added support. the hard work was actually adding SDK support, but we didn't have to go that route IIRC15:48
mordredso maybe we shoudl all circle back around and figure out what we can do differently15:48
evrardjpmordred: which is why I think it's now question on what are the next steps15:48
mordredyup15:48
rm_workso has that changed, and SDK is now mandatory?15:48
smcginnismordred: ++15:49
evrardjpmordred: that does sound like a community goal we had pushed forward with lbragstad and I, one or two years ago :p15:49
mordredrm_work: it's not mandatory - but it IS the direction we're trying to head - because otherwise installing a working OSC winds up installing a hundred bazillion packages - when all that it's doing is making REST calls15:49
* evrardjp gets old15:49
mordredevrardjp: :)15:49
mordredevrardjp: the longer you're around the more everything comes back15:49
evrardjpYou know what you're talking about :p15:50
mordredI wouldn't go that far )15:50
mordred:)15:50
evrardjpso the next logical step  -- for the folks wanting to help glance -- is to make sure that everything that the glanceclient is doing is integrated into sdk, correct?15:52
zanebah, they moved to storyboard and then fixed it: https://storyboard.openstack.org/#!/story/173501615:52
rm_workpart of me wants to just go freaking implement whatever is actually missing15:52
evrardjprm_work: yeah, well, establish a delta, and getting it done?15:53
rm_workbut another part of me feels like that team doesn't deserve to go the shitty route and have a bunch of other folks do the work for them <_<15:53
rm_workbut yep, at least establishing a delta should be doable15:53
rm_workthough I worry about hidden bits that aren't obvious where functionality might be broken even though everything looks the same as far as keywords/args15:54
rm_workre: the start of that whole email thread15:54
rm_workthough it turns out it's broken the same in both clients, so eh :D15:54
*** lpetrut has joined #openstack-tc15:55
rm_workmy suggestion to them in the past was, continue recommending the use of OSC, and report bugs as they come up, and get them fixed! which apparently was too onerous15:56
rm_workbecause it seems like most of the actually important functionality is there... we don't use anything but OSC for management of images in our cloud and have never had issues15:56
rm_workit's not like the glance API has actually changed in like 3 years15:56
mordredas I mentioned in the thread too, gtema has a patch up for migrating osc from glanceclient to sdk15:57
rm_workso step 1 might be to review/merge that15:57
mordredwhich will get things like support for clouds that require uploading to switch first and whatnot15:57
rm_workto get us moving in the right direction15:57
mordred++15:57
*** dmellado has quit IRC15:57
mordreds/switch/swift/15:57
evrardjpyeah that sounds awesome15:58
evrardjpmordred: are there other places where osc is not using the sdk still?15:59
evrardjpSorry I am not aware of the lay of the land15:59
*** dmellado has joined #openstack-tc16:00
toskyevrardjp: sahara for sure (as it switched to osc long ago)16:00
mordredevrardjp: many - it's still mostly using *client - but we're finally at the point where we're comfortable making teh switch. glanceclient-ectomy was the first switch of existing *client usage16:00
mordred https://review.opendev.org/#/c/699416/ fwiw16:01
rm_workcool, will look16:03
rm_workyeah Octavia just did its initial OSC work via SDK I believe...16:03
* rm_work double checks16:03
*** jamesmcarthur has joined #openstack-tc16:04
*** jamesmcarthur has quit IRC16:04
*** jamesmcarthur has joined #openstack-tc16:05
rm_workhmm apparently I'm wrong :D16:05
rm_workso, probably we're due for switching to SDK -- I can probably look at that, we're used to being a guinea pig / canary :)16:06
evrardjphaha16:06
mugsiedesignate is the same, we migrated before SKD was really a thing16:06
mugsieSDK*16:06
*** lpetrut has quit IRC16:06
evrardjpthis isn't a coal mine, you won't die from this.16:06
mordredwait. I'm not supposed to be sitting in a coal mine when I work on openstack?16:07
* mugsie narrator voice: this was a foreshadowing moment 16:07
evrardjphahah16:07
evrardjpmordred: you didn't see my previous workplace apparently.16:07
evrardjpoh also, I am a mining engineer by formation.16:08
evrardjpI totally see you in a mine mordred.16:08
rm_workI mean, Octavia has two mascots, after all...16:08
rm_workhttps://usercontent.irccloud-cdn.com/file/C1rMwjI7/IMG_20200303_010838.jpg16:09
clarkbre osc, I wonder if some of the disconnect here is that osc has startup overhead that the *clients don't have due to lack of plugin loading in those specific clients.16:10
clarkbdefinitely the biggest issue using osc today as a user is its slowness16:10
clarkb(just about everything else about it is great though)16:11
rm_workslow as in ... because it has to make like 4 calls to get going (auth, service-catalog, etc) before making the actual call you meant to do? :D16:12
fungiplugin entrypoint discovery via pkg_resources i think?16:13
rm_workah, that kind of slow16:13
clarkbrm_work: that is part of it, and reusing tokens does dramatically improve performance too. But the other part of it is the plugin entrypoint loading by pkg_resources16:13
fungiwhen you're calling osc over and over not in its persistent interactive mode16:13
fungi(read: you're using devstack)16:14
clarkbit has to scan all location in your PYTHONPATH, then list package details for every package found then it sorts every package. This means it gets slower if you have more packages installed and if you have slow disk16:14
evrardjpclarkb: didn't you write a bug and/or propose a fix for that?16:14
rm_workfungi: or you're running a bunch of scripts in CI/CD that just calls `openstack <blah>` a ton :D16:14
clarkbthe *clients have the same token+catalog overhead but no the entrypoint slowness16:14
evrardjpI remember you proposing an alternative16:14
fungirm_work: yup16:14
rm_work(which is what we do)16:14
evrardjpor maybe I am wrong?16:14
clarkbevrardjp: its actually slower in the python2 case and only marginally faster in the python3 case16:14
evrardjprm_work: that's what everyone does16:15
evrardjp:p16:15
clarkbevrardjp: the updated python lib to load entrypoints uses the same underlying algorithm, they just do it a bit more efficiently16:15
clarkb*more efficiently on python316:15
clarkbthe real fix imo is to stop loading entrypoints at all16:15
fungibut the slowdown is still geometric with the number of modules installed16:15
clarkbfungi: yes16:15
evrardjpfungi: ofc16:15
fungieven with the new implementation16:15
clarkbwe might also want to look at caching tokens again16:16
clarkbbecause that does have a big impact too16:16
mordredthat's one of the reasons we don't do any entrypoint plugins in sdk - the cost. I think unwinding that completely from osc would be a _large_ effort, but one we could certainly talk about in the new sdk world. probably a good topic for the PTG16:17
*** dmellado has quit IRC16:18
*** dmellado has joined #openstack-tc16:20
*** witek has quit IRC16:23
clarkbmordred: one idea I had was to stop using entrypoints for the in tree commands at least16:25
mordredyeah - I thought some work had happened in that direction16:25
clarkbbut I think you'd have to do 90% of the work to untangle those from the plugin system as you would to stop using entrypoints for plugins alltogether16:26
clarkb(so not easy or quick fix)16:26
mordredyeah16:26
mordredclarkb: I thought there was some work in stevedore that was done to allow an optimized path for things like the in-tree plugins ... but maybe that effort just didn't get finished16:26
mordredI think we should _definitely_ have a session on this topic at the PTG - although we obviously don't have to wait until then to work on things - but it seems like figuring out blockers, plans and pain points would be useful to everyone16:28
rm_workis someone planning to do the necessary rebasing on that glance->SDK patch?16:28
mordredrm_work: gtema is on PTO - so if we want to move it forward before he's back someone else should probably step in16:30
rm_worki'll take a crack at it really quick16:32
mordredrm_work: cool, thanks!16:34
*** diablo_rojo has quit IRC16:40
*** diablo_rojo has joined #openstack-tc16:40
*** dmellado has quit IRC16:46
*** dmellado has joined #openstack-tc16:49
*** rpittau is now known as rpittau|afk16:59
*** dmellado has quit IRC17:01
fungitim says we should make it a cycle goal. who am i to argue with tim?17:06
mordredfungi: yeah. tim is one of the main reasons I've been pushing on this... if it'll make tim happy, that seems like the best motivation17:07
*** dmellado has joined #openstack-tc17:07
evrardjp:)17:10
rm_workcycle goal for OSC support seems good to me, FWIW :D17:11
evrardjp\o/17:11
evrardjphistory truly repeats itself!17:11
evrardjphaha17:11
*** dmellado has quit IRC17:19
*** dmellado has joined #openstack-tc17:25
rm_workwaiting on local pep8 and this rebase is done17:26
mordredrm_work: sweet17:27
*** evrardjp has quit IRC17:35
*** evrardjp has joined #openstack-tc17:35
rm_workdone, posted. not too bad.17:40
rm_workI think it maybe needed more work than just a rebase though?17:43
rm_workwill take a look after tests run17:43
*** iurygregory has quit IRC18:19
*** KeithMnemonic has joined #openstack-tc18:35
dhellmannmordred , clarkb , rm_work, evrardjp : dtroyer and I came up with a plan to make in-tree plugins static in OSC, but I never had time to build it in part because the details were a bit more complicated than I thought.19:33
dhellmannMy initial idea was to just have a table built into the code for well-known commands and fall back to the plugin scan if something wasn't found.19:33
dhellmannBecause of the multi-layer loading to handle API versioning, that proved more challenging to actually implement than I thought it would be.19:34
mordreddhellmann: ah - ok, so at least I wasn't imagining you having worked on it at least19:34
mordredat least19:34
mordrednot sure why I said at least twice19:34
mordredbut a third seemed like a worthy addition19:34
dhellmannI suggest looking into dropping the use of plugins entirely, and just build the argparse command tree you'd need. Although that's going to have the same problem, that some options are only available when some API versions are in use.19:35
dhellmannSo maybe dropping that behavior would make the other work easier.19:35
mordredyeah. it might be that we need to register all of the argparse commands regardless of whether they'd work19:35
dhellmannit's not even the commands, it some options to some commands19:36
mordredand do later validation19:36
mordredyeah19:36
mordredargparse things :)19:36
dhellmannyou get a different version of the "noun verb" level plugin based on API version19:36
mordredyeah19:36
dhellmannalthough if I had my druthers, I'd say force everyone to go do the work in gopherlib and build a go client because that's going to result in better API design over the long term. We have way too many APIs that are "put a bunch of JSON here, good luck figuring out what it should hold"19:37
dhellmannthat's unlikely to be a widely popular opinion, though :-)19:37
mordredyeah - many other dragons there :)19:37
mordredbut - I think we've got a good basis for dealing with teh API in sdk - and in sdk we actually present the same interface all the time no matter what the remote api version is19:38
mordredso maybe as part of sdk-ificiation we can remove the osc behavior of presenting version-dependent options19:38
mordredthat will probably be more compicated than just saying it though19:38
dhellmannprobably, yeah19:39
mordreddhellmann: thanks though - that's super helpful in terms of knowing areas that have been investigated and areas to go poke at19:39
dhellmannthere's an etherpad from the last PTG in Denver somewhere with maybe more details19:39
*** jamesmcarthur has quit IRC19:40
mordredawesome19:40
*** jamesmcarthur has joined #openstack-tc19:40
*** jamesmcarthur has quit IRC19:45
mnaseri think it's time to seriously consider moving away from the PTL model20:00
mnaseri've discussed with a few people the idea of actually having something like "maintainers"20:00
mnaserwhich could be nothing other than the actual core reviewers for example20:01
mnaserand that way, the responsibility of the project and the tasks of being a PTL is delegated between multiple people20:01
mnaserand then leave it up to the teams to split work between them20:02
mnaseresp with nova in the state it is in, and that no one really wants to volunteer to PTL afaik.20:02
smcginnisDiffusion of responsibility is a much larger risk than the ability of a PTL to delegate responsibilities.20:05
cmurphymnaser: my worry would be that we lose accountability among the maintainers, everyone will think that someone else will take care of X20:05
fungialso core reviewership is currently conferred from the ptl, a prerequisite would be to determine how maintainers/core reviewers are identified by an established chain of governance20:06
mnasersmcginnis, cmurphy: i think both of you have a point here, but it seems like teams generally share that responsibility between them from what i've noticed20:06
clarkbfungi: re that I've often wondered if we should have more formal openstack maintainer type roles. Basically epople that do the work that sdague and mtreinish and mriedem and jogo were doing when they worked on openstack20:06
mnaserthe way i'd imagine it is something inside the meeting "hey did anyone get the email about X?" "ok, Y will take an action item of taking care of it"20:06
fungifrom a governance perspective, the osf bod identifies an elected body of tc members to govern official projects, and the tc in turn identifies ptls (normally by proxy from holding elections for those seats), then the ptls identify core reviewers and other delegations as they see fit20:07
clarkbthen maybe we aren't competing among projects for resources but can pool them together. But similarly would need updates to how responsibility is conferred20:07
fungiwhat i describe is the current model, to be clear20:08
njohnston_I think this might work ok for larger teams - along the lines of how the liaison system effectively sharded some of the PTL's communication and representation responsibilities - but I don't know that it would help much for medium to small projects.20:08
mnasermedium to small projects can have one 'maintainer'20:08
mnaserthat's something that the team can sort out between them together in a way20:08
fungigovernance-wise there still needs to be some form of delegation from the tc to the people responsible for each project (whether they're ptls or maintainer groups)20:09
mnaserthere hasn't been much times that i've needed (as tc) to reach out to "the ptl" and usually reaching out to the team helps20:10
mordredfungi: I disagree with your description. the osf bod does not identify an elected body of tc members. the osf bylaws define the exist of both a board of directors and a tc who have various electorates20:10
fungimordred: yes, your description is more accurate20:11
mnaser"Project Team Leads (“PTLs”) manage day-to-day operations, drive the team goals and resolve technical disputes within their team. Each team should be self-managing by the contributors, and all disputes should be resolved through active debate and discussion by the community itself. However if a given debate cannot be clearly resolved, the PTL can decide the outcome. Although the TC is generally not involved in20:11
mnaserteam-internal decisions, it still has oversight over team decisions, especially when they affect other teams or go contrary to general OpenStack goals."20:11
fungimordred: and hidden therein is also the fact that the relevant sections of the bylaws require agreement of the tc to change them20:11
mordredfungi: that said - I believe the PTLs and team structure does derive from the TC delegation - so I think your overall point is solid20:12
mordredfungi: yup20:12
mnaseri really think that stuff can just be delegated a layer down to the team (based on what it's defined right now).  it's not like the tc is running after ptls about releases, or about hitting freeze targets, etc20:12
mordredfungi: it is one of the reasons TC is a very unfortunate name20:12
fungimnaser: what's still missing there is who says who's on those maintainer teams? presumably the tc, or the tc defines some mechanism through which that choice is proxied20:13
fungisince the tc is ultimately still responsible for them20:13
fungiif that decision is proxied, the proxy needs definiting20:13
fungier, defining20:13
mnaserfungi: yep, agreed.20:13
fungithe current proxy is contributor elections in each team to pick a ptl20:14
fungiso that's the challenging bit to define, i think20:14
mnaserperhaps a ptl can just become a 'maintainer' or maybe a 'project owner' -- which we can have multiple of20:16
fungithe "can become" part is what i'm saying is the challenge. how does someone "become" a maintainer, formally?20:17
mnaseryeah, i don't know about that part yet20:18
fungiwe can't punt and say "it's just the core reviewer model" because 1. officially ptls are the folks who say who can be a core reviewer right now, and 2. not all teams have the same sorts of core reviewer sets20:18
mnaserbut thoughts and ideas are welcome20:18
njohnston_fungi: The only way I can think of is for maintainer status to be something nominated and elected from within the project's contributor base20:18
funginjohnston_: that's far from the only way i can imagine, but it's certainly one which would be familiar relative to the current model20:19
mnaseranyone familiar with some of the k8s governance and perhaps how they deal with this?20:20
njohnston_fungi: Fron a practical perspective with a contributor base that is used to having a say in their project management through elections, any method that does not use elections would come across as disempowering20:20
mnasernjohnston_: does it though?  if i dont merge any of your patches in a cycle, you technically cant have a say :-P20:20
fungialso a core reviewer can stack the current ptl electorate with folks who are likely to vote for a specific candidate by approving changes from them20:21
fungibut i think either of those are pathological malfunction of the team structure which ultimately should be escalated to the tc for resolution20:22
njohnston_Agreed. And hopefully diversity of core contributors would help protect against that in most cases20:23
fungii think some of the subtext here is also what happens if nobody volunteers to lead the nova team now that efried has announced his departure. is nova "too big to fail"? or will the project continue running just fine without any formal leadership? or does the tc need to find a new model which empowers the nova team when nobody wants ultimate ptl responsibilities?20:24
dansmith#2 please20:24
dansmithwell,20:24
njohnston_I think this would have some traction if it was phrased as “instead of one PTL, many PTLs”20:25
dansmith#1 is fine too, but it sounds like that can't happen because of so much boilerplate20:25
fungicurrently, #2 means it continues sailing along as no longer part of openstack because there's no delegation from the tc to the folks running the project20:25
njohnston_fungi: and is there something about the nova PTL position that is uniquely difficult?20:25
funginjohnston_: i'm not, nor have i every been, ptl of nova so i'm unable to answer your question with any authority20:26
mnaseri've seen the nova team consistently share responsibilities with things like running PTGs, project updates, etc20:26
dansmithfungi: just to put you on the spot... do you really see nova being excluded from openstack because of something like this?20:28
dansmithI mean, not to put too much special sauce on nova, but I would think that nova not being part of openstack would be an outcome that doesn't benefit anyone, really20:28
fungidansmith: that was my "too big to fail" allusion. i don't see any outcome where nova is no longer formally part of openstack as a successful result20:28
dansmithfungi: well, you said #2 would imply nova not being openstack20:29
fungiyep. consider it included as completeness20:29
* dansmith doesn't get it20:30
dansmithbut that's okay20:30
fungianother way to phrase this would be we need to find a governance model which formally delegates the running of the project from the bylaws and tc20:30
mnaseri think dansmith is trying to mostly find the leastway with the least amount of friction so he can focus on getting stuff done without having to worry about juggling "who's gonna wear the PTL hat"20:30
fungione option might be that the tc collectively takes on pt duties for nova, because the tc is ultimately responsible for it20:31
fungi(to be honest i also don't see that as being a great option)20:31
dansmithmnaser: yeah, we're kinda out of people to spend a bunch of time doing that kind of stuff, IMHO20:31
fungi"that stuff" being facilitating ptg discussions, tracking completion for cycle goals, proposing releases...20:32
fungiare those things nova is deciding are irrelevant?20:32
mnaserfungi: i think that's a two part answer20:32
mnaserfirst -- those aren't documented PTL roles "technically"20:33
fungi(the folks working on nova collectively that is. it's hard to say what "nova decides" when it's no longer a euphemism for what "nova's elected ptl decides")20:33
mnaserwe historically say they are but the governance docs dont really say much about that20:33
fungiyeah, i'm curious what people think they're responsible for by volunteering to be a ptl (or a maintainer, or whatever replaces the ptl)20:34
njohnston_does nova have a team that is “the core of the cores” analogous to the neutron drivers team?20:34
mnaserand secondly, personally i've seen that responsibility mostly shared across teams, which is something to decide between them20:34
dansmithnjohnston_: we used to, but it basically died and was squashed in because it "wasn't fair"20:34
njohnston_ah ok20:34
fungimnaser: the tc currently expects ptls will find someone to do those things though, right?20:36
mnaserin my experience, that isn't what ends up happening :(20:36
mnaseror at least, most of the smaller teams20:36
mnaseri think if we focus on empowering teams so they can organize themselves on who wants/can deal with what20:36
mnaserit'll be easier for us and them20:37
fungihypothetical, nova decides it's going to ignore a cycle goal... i won't make up a specific example it's a thought exercise... what's the solution? and say it's not nova but... glance. is the solution the same?20:37
fungior should we reframe how we look at cycle goals?20:39
njohnston_if we decide to move to a maintainer system, do we just move the problem down the stack?  i.e. cores a, b, and c don't want to be PTL because they don't want to do the summit project update or organize the PTG discussions.  So we make them a maintainer team, and then they do all the PTL stuff except noone does the project update or organize PTL discussions?20:39
clarkbnjohnston_: thats sort of why I wonder if global maintainers makes more sense20:39
fungikeeping i mind that ptl attendance and project updates are entirely optional20:39
clarkbnjohnston_: empower people interested in the work (assuming they exist) to do it for more than a single silo20:40
fungier, ptg attendance20:40
dansmithnjohnston_: fwiw, we often have non-PTL people doing or participating in the project update, and organizing ptg discussions.. that's never really been a struggle20:40
mnaserits not like we can do much about it *even* if the project has a ptl20:40
mnaser(wrt cycle goals)20:40
fungiright. is there anything we actually hold a ptl accountable for, as a community, other than formally delegating the list of people allowed to merge changes in the project they represent?20:41
mnaserhttps://governance.openstack.org/tc/reference/charter.html#project-team-leads20:41
mnaseri dont think we even formally delegate the list of people allowed to merge changes :p20:41
mnaserif anything ... "Each team should be self-managing by the contributors"20:42
fungiyep, good point20:42
dansmithbtw, are you people mostly worried about the "now until the next election" period, or the next election having no candidated?20:43
dansmiths/d//20:43
mnaserdansmith: for me personally, both20:43
fungithat would also fall into resolving disagreements. the ptl is delegating the ability to assess agreement on when a change is suitable to merge20:43
mnaseri'm pretty sure nova might not be the only ptl-less project come next elections20:43
dansmithif not then it seems like redefinition might be worthwhile20:44
njohnston_We have functional documentation - https://docs.openstack.org/project-team-guide/ptl.html - that basically resolves down to "make sure the right people are cores, the right people are liaisons, the right people are managing releases, the right people are doing project updates/forum sessions, and look at the user survey"20:45
mnaseri wonder if it might make sense for maintainers to be people that are not necessarily core members20:46
dansmithI dunno if it's really unclear, or more specifically a nova thing, or people are just papering over it,20:46
mnaserbecause someone might be interested to contribute... but in helping coordinate releases only, or runining project updates20:46
evrardjpmnaser: this is very close to what I discussed with you in the past I see. I intended to write that down in the ideas repo, but maybe you can go ahead and draft it?20:46
mnaserif i'm drafting this it's going straight to openstack/governance :p20:46
dansmithbut being the PTL does bring a lot of focus from everyone who wants their feature to be merged, or blessed, or someone to appeal those things not happening, etc20:47
dansmithperhaps that's just something we have to figure out how to break out of while retaining the single point person that the rules require, I dunno20:47
mnaseri get the feeling that being nova's ptl is a lot more than other projects20:47
fungisay i'm coming to this new (and it's a question i hear from people at companies who are assessing whether they should risk getting involved in a project)... who decides whether my proposed changes will be merged, how do i know their decision will be impartial, and who do i escalate my concerns to if i discover they aren't?20:48
dansmithbut I don't think that you guys convincing yourselves that the rules don't actually require someone to do "that much work" is going to change anything :)20:48
mnaserfwiw https://etherpad.openstack.org/p/tc-train-ptg L17420:48
mnaserbut it looks like none of that really ended up helping the idea of leaderless projects20:49
mnaserbut instead just focused on "this isn't that big of a deal and its not a long of work"20:49
fungidansmith: i believe the ptls of nova have done a vast amount of necessary work. what i question is whether getting rid of the ptl means that work no longer needs to get done20:49
njohnston_dansmith: Right, ifyou aren't doing the work then you're spending a bunch of time coordinating (at best) or hounding (at worst) other people to get the work done.  Non-trivial investment of capacity is going to happen one way or another.20:49
mnaseri actually think the factor of "its not my problem" helps resolve the overload of stuff the ptl needs to deal with20:50
dansmithfungi: not saying it doesn't all need to get done (some definitely doesn't) but being the focal point for that is not great20:50
fungidoes replacing the ptl with a group of people solve the pressure by turning it into a shell game where nobody can figure out who to pressure to merge their change?20:50
mnasera ptl saying "its not my problem" is "a bad ptl" (im putting quotes here because that might be the general thought), a maintainer saying "its not my problem" isn't an issue, just find another one20:50
evrardjpmnaser: if it's embryonic there is no reason to have it in governance. If it's ready go ahead.20:50
mnaserif you cant convince a single maintainer20:50
dansmithfungi: do you think people should be pressured to merge changes?20:50
mnaserit probably doesn't make sense to push20:50
* evrardjp continue catching up20:50
fungidansmith: i don't think people should be pressured to merge changes. maybe the solution to that problem isn't getting rid of the ptl20:51
dansmithfungi: I don't really think we need to have a supreme court of final appeal to get something merged20:51
mnaseri think at the end of the day, people *will* go to the ptl, and the fact it's one single individual, it all ends up on their plate20:52
dansmithfungi: well, you seem to keep reverting back to "who do I have to pay to get this merged?" kinda thing20:52
dansmithyup20:52
fungimaybe it should be the tc's responsibility to tell contributors to shut up and stop pestering the nova ptl to contest merging a change20:52
dansmithnot only that,20:52
fungidansmith: i come back to it because it seems to be the crux of the pain20:52
dansmithbut entities involved, people and companies, kinda expect the PTL to be the last buck on policies and responsibility for those that get selected20:52
mnaseri think it's that but also some other misc stuff that you seem to sign yourself up without agreeing to20:53
fungijust because i acknowledge that's some of what makes the role painful doesn't mean i think it's a good thing, but it might still be a reality20:53
mnaseraka "can somoene take care of the project update?" vs "well no one is volunteering for a project update and i'm the ptl so.."20:53
mnaserfeels like it'll kinda force a bit more cooperation across people somehow.20:53
mnaseri think there's *a lot* of relief to be done if there was more than one name on that list20:54
fungidansmith: but to pick that pain point, does it magically go away if there's no ptl? or do the folks applying that pressure find someone else to be the focus of their pestering?20:54
mnaseryes, they can try and ping different cores, and cores can say "sorry, not interested"20:54
dansmithno, it doesn't go away, but it becomes more of a "find someone who cares about this, because I do not" sort of thing20:55
dansmithwhich I think is easier to deal with20:55
dansmithobviously the PTL could take a hard line on that20:55
mnaserbut no PTL ever does20:55
dansmithbut you know.. it'd be difficult20:55
dansmithmnaser: right20:55
mnaserand even if they did, they'd go to another core20:55
mnaserwhich would come back to the PTL20:55
dansmithI'm like the most negative person on the project... I can say no to anything.. and it gets really exhausting20:57
mnaserdansmith: i'm glad you said yes to not running an UPDATE on all instance rows in train upgrades then \o20:57
mnaser\o/20:57
dansmithheh20:57
evrardjpdansmith: and? How is that linked to governance?20:58
evrardjpsorry but I fail to see the big picture, it sounds like team organisation20:58
fungidansmith: no, that's great. i'd rather hear form the person with the strongest opinions on why something doesn't work than the most amenable/apathetic one20:58
dansmithevrardjp: it's not20:58
mnaseri agree that it's not team organization :\20:58
evrardjpbut I understand some roles are tedious, and we should help make them less tedious20:58
fungiso to try to summarize, the impetus to no longer elect a ptl for nova is because nobody wants to be the focus of pressure for everyone who wants their fixes/features merged, and nova should be able to assess team consensus collectively without a fallback to resolve disputes20:59
*** jamesmcarthur has joined #openstack-tc20:59
dansmithfungi: I'm not really that person either.. I'm not demanding some other plan be in place, I'm just in here chatting with you people.. I think the fact that we need an interim ptl for a short period of time and nobody is willing to step up is just a symptom worth noting20:59
* mnaser suggests that every TC member at least sits in Nova PTG for a couple hours at least20:59
evrardjpdansmith: that we can probably all agree on :)20:59
dansmithfungi: no dude, I'm saying that's one reason I think nobody wants to do it.. please don't take my words as speaking for the whole team21:00
fungidansmith: until you mentioned it, i hadn't considered that pressure from prospective contributors to deny changes was the biggest pain point for a nova ptl21:00
mnaseri think everyone would have a lot more perspective being on those21:00
dansmithI don't even know that it's the biggest, it's just one21:00
fungibut it's one you've heard, from within the team21:00
mnasersome of the suggestions that come in are so far developed and you have to sit and say "what you're doing doesn't add up"21:00
mnaserand people having pressure from their organizations to deliver on the features inside upstream21:01
*** njohnston_ is now known as njohnston21:02
fungiprobably this is a discussion which should happen on the ml21:07
fungito get broader input21:07
mnaseryep21:08
mnaseri just like to start in IRC to get a small idea first21:08
evrardjpthat's basically what I proposed above :p21:08
dansmithfungi: if/when you do, please don't say "dansmith said $thing was a/the/etc problem"21:08
fungidansmith: no, no, we should *restart* the discussion on the ml is what i really meant ;)21:09
dansmithI think you took and ran with that one thing I brought up as a not-just-the-documented-responsibilities example and turned it into the or one of the most important things, and it's really not, it's just part of it.. and an example21:09
dansmithk21:09
evrardjpdansmith: tbh, I would expect you to chime in on the thread :p21:09
dansmithevrardjp: you might not want to hold your breath :)21:09
fungii wouldn't want to put words in anyone's mouth, if you want to raise opinions on the ml thread that's for you to do21:09
evrardjp:)21:10
fungidansmith: though i do hope someone's still around in nova to explain what ptl responsibilities specifically discourage them from offering to be ptl21:10
dansmithfungi: well, I'm sure you can imagine that all the people sitting around not volunteering for it are doing so with reasons and not lack thereof :)21:11
* ttx waves from his vacation spot21:11
mnaserlolol21:12
mnaseri was just thinking of ttx missing all this :)21:12
fungiyep, and i want to learn what those reasons are, so the community can evaluate ways to possibly find solutions to those reasons21:12
ttxmnaser: come on, you raised it now by design21:12
ttxbut I see everything21:12
mnaserttx: you're *welcome*21:13
mnaserwould you have preferred on your way back from vacation instead?21:13
ttxre: no more PTLs, the two main issues qare:21:13
ttxwhat happens if core reviewers disagree ? Do we default to doing nothing?21:13
ttxand how do we get critical  functions filled, like release liaison (release signoff) ?21:13
ttxinternet connection spotty here21:14
ttxSo ideally we'd keep the "bucket stops here" function but remove the expectation that this is the person to ping to police core reviewers and get review priority21:15
ttxI mean, we could say that the TC arbitrates disputes between core reviewers...21:16
ttxbut I'm not sure that's actually desirable21:16
fungii'd be interested to hear ideas on how to convince folks that they shouldn't pester the ptl to get changes reviewed21:17
ttxmaybe remove the PTL "core police" function21:17
fungithat doesn't seem like something we tell people to do in the first place21:17
mnaserttx: i feel like at our current velocity i doubt we're moving things so quickly that someone is very unhappy about fixing disagreements21:17
mnaserman i cant write today, jetlag hitting hard21:17
ttxmnaser: used to be that project teams were pretty strongly against TC getting into their affairs... maybe times have changed21:18
mnaserwhat i mean is i dont think PTLs are doing a lot of "clearing out disagreements" and maybe that can just be escalate to the TC *at the very worst*21:18
mnaseri dont think PTLs enjoy being a tiebreaker either.21:18
mnasermaybe rather than the TC *getting* into their affairs, we can help facilitate the conversation if there are two strong opinions/sides21:19
ttxmnaser: frankly I think there is more promise in decoupling teams from review core areas21:19
ttxmnaser: your other suggestion21:19
mnaserpersonally during the whole placement thing, i really found it useful to just sit with everyone and be like "ok cool, lets come up with a plan"21:19
mnaseri think it took like 10-15 minutes to come up to something that everyone agreed to and jot it down21:19
ttxanyway, I'm not around this week, so feel free to burn everything down21:20
ttxI might get snowed in anyway21:21
fungittx: so get rid of per-project core review teams in favor of openstack-wide maintainer teams?21:22
ttxfungi: no, have teams around functional domains and core reviews around sections of code21:22
fungithough you should really get back to vacationing and reply another time ;)21:22
* fungi needs to get back to being at a conference and not ignoring it21:23
ttxI should get back to vacationing21:23
ttxmnaser: I'll read your thread or your governance charter change proposal when I get back :)21:24
ttx(I agree something has to evolve, I'm just not sure what yet)21:24
mnaserttx: get back to vacationing! :P21:25
*** e0ne has joined #openstack-tc21:29
*** e0ne has quit IRC21:43
mnasertc-members: http://lists.openstack.org/pipermail/openstack-discuss/2020-March/012950.html21:48
*** slaweq has quit IRC22:09
*** slaweq has joined #openstack-tc22:11
*** slaweq has quit IRC22:16
*** slaweq has joined #openstack-tc22:33
*** slaweq has quit IRC22:39
*** slaweq has joined #openstack-tc22:41
*** slaweq has quit IRC22:46
*** slaweq has joined #openstack-tc23:11
*** jamesmcarthur has quit IRC23:16
*** slaweq has quit IRC23:16
*** tosky has quit IRC23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!