Thursday, 2018-11-08

*** dhellmann has joined #openstack-tc00:07
*** dangtrinhnt_x has joined #openstack-tc01:12
*** dangtrinhnt_x has quit IRC01:46
*** mriedem has quit IRC01:50
*** diablo_rojo has quit IRC01:53
*** ricolin has joined #openstack-tc02:20
*** fungi has quit IRC03:06
*** fungi has joined #openstack-tc03:09
*** fungi has quit IRC03:10
*** mrhillsman has joined #openstack-tc03:19
*** fungi has joined #openstack-tc03:40
*** fungi has quit IRC03:41
*** fungi has joined #openstack-tc03:45
*** diablo_rojo has joined #openstack-tc06:20
*** chenyb4 has joined #openstack-tc07:10
*** alexchadin has joined #openstack-tc07:39
*** diablo_rojo has quit IRC08:08
*** shrasool has joined #openstack-tc08:30
*** shrasool has quit IRC09:11
*** jpich has joined #openstack-tc09:14
*** e0ne has joined #openstack-tc09:50
*** dtantsur|afk is now known as dtantsur09:55
*** chenyb4 has quit IRC11:00
evrardjpo/12:27
*** dtantsur is now known as dtantsur|brb12:32
*** weshay is now known as weshay_ruck12:59
jrollchoo choo!13:01
*** e0ne has quit IRC13:03
evrardjpjroll: :)13:09
evrardjpit's official!13:09
evrardjpalmost13:09
evrardjpwe're gonna eat trains for x years more (counting extended maintenance)13:09
dhellmanntc-members: I think all of the slides for our presentation monday are done. feedback welcome: https://docs.google.com/presentation/d/1wcG7InY2A5y67dt5lC14CI1gQHGZ3db8-yshOu-STk8/edit#slide=id.g46a6072f4a_4_22913:24
gmanndhellmann: on slide#8, i see kolla-ansible is highest commit, is it considered in kolla?13:32
gmanndhellmann: on slide#9 should we include tempest plugins release also ?13:36
*** e0ne has joined #openstack-tc13:37
dimsdhellmann : nice work! slides look great13:39
dhellmanndims : zaneb, TheJulia, and smcginnis did the work on their sections13:39
dhellmanngmann : good questions. I think ttx and smcginnis worked together on those slides13:40
dimszaneb, TheJulia, smcginnis : awesome!13:40
ttxgmann: I used the Kolla "13:41
ttxgroup" in stackalytics13:41
ttxso it should13:42
ttxThe others spread their work across a /lot/ of repos13:42
TheJuliawe have many many many repos13:43
ttxdhellmann: slide 10 is obviously incomplete. Not sure if smcginnis planned to fill it13:44
dhellmannhmm, yeah13:44
gmannttx: yeah, got it. thanks13:45
gmanndhellmann zaneb, TheJulia, smcginnis, ttx  nice work. good slides and stats.13:48
TheJuliaI'm tempted to write a script that goes through all the git histories and does the math, excludes procedural/translation patches and calculates the numbers for the slide, I did mine with the whole data so naturally a bit watered down13:50
evrardjpTheJulia: I kinda like that, I feel like raw data should go into a dashboard. Those dashboards can be used by anyone later.  I think this can be extended to show hotspots of risks, etc.13:52
evrardjpbut I am getting ahead there.13:53
evrardjpanyway, nice slides!13:53
dhellmannTheJulia : there's some stuff in goal-tools that may help with that. I used it for the presentation in Vancouver, and then imported CSV data into google docs to make charts14:01
dhellmannit's all very hacky, so don't judge14:01
*** jpich has quit IRC14:02
*** irclogbot_3 has quit IRC14:02
*** tdasilva has quit IRC14:02
*** fanzhang has quit IRC14:02
*** jaosorior has quit IRC14:02
*** tonyb has quit IRC14:02
*** scas has quit IRC14:02
*** eandersson has quit IRC14:02
TheJulianever :)14:02
*** fanzhang has joined #openstack-tc14:03
TheJuliaevrardjp: I kind of like that idea... but it is also all very situational, the issue I kind of felt with the existing data was the only way to reveal the 50-60 revision patches would be to evaluate each item per repo14:04
*** jpich has joined #openstack-tc14:04
*** tonyb has joined #openstack-tc14:07
*** jaosorior has joined #openstack-tc14:08
evrardjpTheJulia: I don't think there is 'one solution fits all' for this kind of things, but at least if we could share all those hacks into one location, it would prevent people from rewriting their own14:13
*** mriedem has joined #openstack-tc14:13
evrardjpI can already see 4 ppl having their own tooling for getting stats, pretty sure we can work on that all together14:14
dhellmannevrardjp : yeah, that was part of the point of starting to dump a bunch of code into the goal-tools repo14:17
evrardjpcool14:17
*** alexchadin has quit IRC14:39
lbragstaddhellmann those slides look nice14:44
dhellmannlbragstad : thanks!14:46
evrardjpoffice hours time?15:00
zanebo/15:00
fungilooks that way15:00
evrardjpgmann: are you there for the email? :D15:00
ttxohai15:01
gmanndone15:01
gmanno/15:01
* ttx is packing up15:01
fungiyeah, same15:01
evrardjpgmann: thanks!15:01
evrardjpttx: already?15:01
ttxevrardjp: I'll be in Berlin starting Friday15:02
*** dtantsur|brb is now known as dtantsur15:02
evrardjpttx: oh that sounds fair then :p15:02
* lbragstad pops in for office hours15:04
dhellmanno/15:05
*** tdasilva has joined #openstack-tc15:11
dhellmannTheJulia : do you want to +1 this patch to add tenks? https://review.openstack.org/#/c/600411/115:11
TheJuliadone15:13
dhellmanntc-members: we could use another review on https://review.openstack.org/616189 too15:13
dhellmannTheJulia : thanks15:13
dhellmanntc-members: it looks like we're close to ready with zaneb's technical vision proposal; it would be good to have that lined up for approval for next week's discussion: https://review.openstack.org/#/c/592205/15:15
ttxdhellmann: added my review on the 61618915:16
dhellmannttx: thanks15:17
openstackgerritMerged openstack/governance master: Add tenks under Ironic governance  https://review.openstack.org/60041115:22
openstackgerritMerged openstack/governance master: update home page for openstackclient  https://review.openstack.org/61618915:23
zanebtc-members: we need to discuss what changes we want to make to https://review.openstack.org/#/c/613145/15:25
dhellmannwhich comments do you want to start with?15:29
zanebdhellmann: so you're saying we should test the exact versions that LTS distros are going to ship, and not worry about newer versions...15:30
dhellmannwe should focus on those; I consider anything else a bonus15:31
zanebcdent is saying we should only test newer versions...15:31
dhellmannyeah, I think that's severely misguided15:31
dhellmannand would be a disservice to our users15:31
zanebme too fwiw15:31
dhellmannI don't mind including those, but I definitely don't think we should limit ourselves to those15:31
dhellmannand I don't place a high priority on it15:31
zanebbut distro folks like coreycb and zigo want both current and newer versions15:31
zaneboh, and folks on the ML wanted to avoid testing too many versions at once15:32
dhellmannI'm ok with that. My position isn't a strong no. It's just about where I place the emphasis.15:32
dhellmannand if someone wants us to include a version, they should be prepared to help set up the images needed15:32
zaneb+115:33
dhellmannthe concern about testing too many versions is based on a lack of understanding of where our CI resources are actually being spent15:33
dhellmannunit test jobs take a tiny amount of resources compared to some of our much longer running jobs15:34
dhellmannso it's something to be aware of, but it's not a reason to aggressively drop jobs15:34
ttxOn the Python version testing question I'll admit that I had trouble making sense of the overlapping proposals and which one(s) should actually be reviewed15:34
dhellmannyeah, we have 3-4 patches changing different aspects of things and I think we want to either merge those or get them into a series so the progression is clear15:35
fungii feel like as long as we're not also stuck catering to outdated python versions for prior distro lts releases (what is the benefit to keeping stein compatible with the python3.5 shipped on ubuntu 16.04 lts, especially when we still test python2.7 anyway?), it's fine to test the current ubuntu lts and some future interpreter as well15:35
zanebso I guess the main question is if we want to stick with min-max testing (fewer CI resources; slightly higher risk) or also unit test versions from specific distros that fall in the middle of the range (more CI, but slightly lower risk)15:35
gmann+1, unit test jobs does not add much time on overall time/resource15:35
ttxis 613145 our best shot as a consensus ?15:35
dhellmannfungi : it's not yet clear to me which version of python 3 RHEL 8 will have, so I don't want us to drop 3.5, yet15:35
fungidhellmann: it's not clear there will be a future ibm rhel15:36
fungi(at least not to me at any rate)15:36
dhellmannif I could get anyone to give me a version number I would be more definitive about dropping, so for now it's just conservative stance15:36
dhellmannI don't know if the dates for RHEL 8 release are public, but I would personally expect that to happen before the acquisition goes through15:36
dhellmannand I haven't heard anything internally that makes me believe RHEL will be going anywhere15:36
zanebfungi: nobody knows the future, but IBM have announced that they intend to run RH as a standalone unit. it's very unlikely that they'd discontinue RHEL15:37
fungiwell, at any rate, i would not be especially happy if we declare that we're keeping stable/stein compatible with ubunt 16.04 lts because it means that much longer before we'll eventually be able to stop worrying about that particular platform15:37
dhellmanncan we run 3.5 on bionic?15:37
zanebdhellmann: I believe not15:38
dhellmannor 18.04 or whatever the right designation is15:38
dhellmannok15:38
fungiit wouldn't be a python version that users are deploying15:38
zanebbionic has 3.6 & 3.7 IIUC15:38
zanebfungi: for unit tests that's less crucial15:38
fungiwe could build python from source for whatever distro we want in that case15:39
dhellmannfwiw, I am perfectly OK with us saying that when we reach this stage for future transitions, we would expect someone from RH to set up a CentOS image for running those tests15:39
dhellmannit's a bit short notice to do that now, so I'm hoping we can have a more gradual transition15:39
dhellmannI will see what I can learn about python versions in RHEL 815:39
fungii'm more concerned that we won't be able to stop relying on ubuntu-xenial images once we cease testing stable/rocky15:39
fungibecause we'll be stuck testing stable/stein on it too15:40
dhellmannsure, I understand15:40
dhellmannI will try to determine if it's safe to drop them for stein now15:40
dhellmannor at least before the release15:40
fungii'm more than a little miffed that we're catering to a closed development process and making wild guesses as to what python version their open-core release model is going to suddenly pop out with no warning15:41
fungiif red hat can start letting us (the public) see their actual package choices for the next release it would make sense to track and plan to support them15:42
dhellmannhmm. I don't like it either. I'm not sure I would call it "open core"15:42
fungier, well, not openly-developed at any rate15:42
dhellmannI think "not publicly announced" is reasonable. They seem  particularly concerned about making public announcements for some reason.15:43
funginot making a public announcement and not letting people see what you're working on (and potentially participate in it) are different things altogether15:43
dhellmanntrue15:44
evrardjpI think we are losing track of conversation there15:44
dhellmannok, I have confirmed that it is safe to drop 3.5 support.15:44
fungievrardjp: not really. ubuntu lets us know what python versions they're considering. red hat doesn't. i'm fine planning ahead on not-entirely-certain ubuntu python3 plans but not so much mums-the-word rhel python3 plans15:45
evrardjpI also believe that _even if our employers (except for a few select ones) should adapt to the produced community codes, not the other way around_ like it's done for other software15:45
evrardjpeven if these are our employers we are talking about, the distros should adapt*15:45
evrardjpor should I say vendors15:46
evrardjpfungi: but does that matter?15:46
dhellmannzaneb : so I think that leaves us with saying 3.6 for sure, and 3.7 as a future version?15:46
fungidhellmann: thanks for confirming. that makes this somewhat easier15:46
dhellmannI agree this is frustrating. Frankly it's frustrating for a lot of us internally too.15:47
dhellmannI would support wording to the effect of "python versions known to be available in the next LTS" or whatever and then if RH won't say then we have explained why we're not going to worry about it.15:47
fungii can only begin to imagine how frustrating it is for you :/15:47
fungiand yes, that seems like a reasonable statement15:48
dhellmannlet's get the basics settled and we can add that as a follow-up15:48
fungiwe can't make plans to support a distro release if the distro won't give us some inkling of what target we need to try hitting15:48
dhellmannthen I can point my management team at the patch15:48
evrardjpyay15:49
dhellmannthat won't be a fun thread :-/15:49
evrardjpI guess...15:49
evrardjpgood luck in advance :p15:50
fungii mean, it's not just about python versions15:50
dhellmannof course15:50
fungiwhen nova needs to decide what versions of libvirt to drop support for, same problem15:50
dhellmannI'd be happy with an idea of a base version number for those things15:50
evrardjpyeah but nova goes for a review, ask for companies to step in, and is more a community discussion of 'how far can we go together'15:51
evrardjpdhellmann: could you clarify what you mean there?15:52
evrardjpI have the impression we are talking about the same thing, but with different vocabulary15:52
dhellmanncdent, jroll: could one of you add me to the cross-project-spec-liaisons group, please: https://review.openstack.org/#/admin/groups/1372,members15:52
dhellmannevrardjp : by "base version" I mean a guess at a minimum. So I don't know what version of python will be in rhel 8, but I know it will be higher than 3.5 (so at least 3.6)15:53
dhellmannbasically give me the >= version for the dependency15:53
dhellmannusually that's enough15:53
jrolldhellmann: I'm not able to add people there15:53
dhellmannhmm, I guess it's not self-editing15:53
evrardjpthat was my understanding. but you plan for the future, not the present15:54
dhellmannjroll : thanks, I'll move that discussion to -infra15:54
jroll:)15:54
fungithat group is managed by cross-project-spec-liaisons-chair15:55
fungiwhich seems to have dhellmann in it15:55
fungiso he should be able to alter it15:55
fungihttps://review.openstack.org/#/admin/groups/1371,members15:55
dhellmannoh, frickler just added me to that group15:55
dhellmannsee #openstack-infra for details15:56
fungicool, you should be all set then15:56
dhellmannzaneb : did you get enough of an answer from that discussion?15:59
zanebdhellmann: I think so... let's make the change we discussed to test every version that appears in a supported distro + the newest version we can find if that's different16:00
dhellmannthe newest version available in a system package somewhere?16:01
dhellmannsystem/distro whatever16:01
dhellmannI don't want us to write a policy that implies we're committing to building our own16:01
zanebyes16:01
zaneb"the latest released version that is available in any distribution we can feasibly use for testing"16:02
dhellmannwfm16:03
zanebplenty of outs there if we need them ;)16:03
dhellmannprojects must support the other versions, and may support the latest version, if the image is available16:03
dhellmanngmann : is the qa team prepared to coordinate the image update work? ("coordinate" not always "do")16:04
dhellmannideally it would be done by distro staff working with/on the qa team16:05
gmanndhellmann: i doubt. having less people in OpenStack QA having image update experience/knowledge16:05
dhellmannyeah16:05
dhellmannthat will be the next challenge16:06
gmannif i am not wrong, is it infra doing till now ?16:07
dhellmanninfra has, yes. I'm sure they would help someone else learn to do it.16:07
gmannagree but main issue in QA is very few resources and their bandwidth.16:09
dhellmannsure, maybe I wasn't clear in what I was asking16:09
dhellmannif there was a document to explain how to do it, and there was someone willing to do the work, would the QA team *coordinate* that work by helping to work out the schedule when the change should happen, helping to communicate the change to the community, etc.16:10
fungiodds are we won't have problems finding volunteers to get images building for ubuntu 20.04 lts and centos-next16:10
dhellmannI certainly hope not16:11
fungimost of the coordination we're going to need is confirming that we have those images in advance of when openstack will need them, and keeping tabs on jobs switching from old to new images16:11
dhellmannI think it would be good to have a team responsible for the activity, rather than treating it as a special one-off thing every time16:11
fungisure16:11
dhellmannright16:11
fungijust saying it doesn't have to be qa team members doing the work to make those images available16:12
dhellmannthat feels like a fit in terms of mission for the qa team, so if the promise is that no work is expected without volunteers to build the images, I'm hoping gmann will say yes to the coordination part16:12
dhellmannright, I think we're saying the same thing16:12
dhellmannI think the folks doing the image work *become* qa team members for the duration16:12
fungia fine way to look at it16:13
dhellmanneven if that's their only contribution, they would do the work within that team16:13
persiaIs there a structural reason for it not to be "the QA team", or just something about the current size of the QA team?  My thought is that if the QA team is willing to accept more folk to also do that, it might make sense to have one larger team rather than two small ones.16:13
dhellmannpersia : it's a resource issue for the current members16:13
dhellmannthat's why I'm trying to position it as just a coordination task16:13
persiadhellmann: Then I think I'm arguing with you, that new folk should become QA members to build images.16:13
fungimostly trying to assuage gmann's concerns of "less people in OpenStack QA having image update experience/knowledge"16:14
dhellmannpersia : yeah, that's what I'm saying16:14
persia(or maybe "arguing alongside .." is clearer in writing)16:14
dhellmannthey just might not stick around and do other work, which is fine16:14
dhellmannand the existing qa members who have no vested interest in this task aren't expected to  pick up the slack if volunteers from the distros don't show up16:14
dhellmannyes, I think we agree :-)16:14
fungithose images are used system-wide. opendev will want to be able to run jobs on them, as will zuul, starlingx too, and airship, maybe eventually even kata16:15
dhellmannfungi : I agree that's a concern. We're going to have to do some training, which means we need to assemble the volunteers to receive (and give) it, which means we need to start making noise about this policy change ASAP16:15
fungiso it's more about collaborating to make sure someone has done the work to make them available16:15
persiafungi: Do you feel there is a need for the images to be provided by the (semi-) external CI vendor, rather than describe those other projects as using components from OpenStack and maybe contributing back?16:15
dhellmannok, maybe I misunderstood what clarkb had said about this in the past, which I thought was that the infra/opendev team was no longer going to do the work16:16
persiaI also remember reading something like that (which may have no relation to any text in archives)16:16
fungidhellmann: the infra/opendev team is no longer going to force openstack to stick to its testing policies16:16
dhellmannah16:16
fungithat's the biggest change16:17
dhellmannI interpreted that as also meaning creating the images, but I can see how it wouldn't have to include that change, too16:17
fungiin the past _we_ drove the changing of jobs over from running on older to newer images, pre-testing to make sure they wouldn't break openstack projects, et cetera16:17
*** dklyle has quit IRC16:17
persiaThe side effect of this is that in the event that the opendev policies diverge from openstack, someone in openstack needs to pick up the slack16:17
fungis/_we_/infra/16:17
dhellmannyeah, I'm fine with us just doing a hard change of platform at the start of each cycle, and fixing the fallout16:18
dhellmannsomeone else might want a more nuanced plan, though :-)16:18
dhellmannI think gmann did say the QA team would deal with that for the devstack jobs16:18
fungiso we would of course want the qa team to collaborate with the opendev infra team and distro volunteers to make sure images it wants to run jobs on are made available in the system, but those popular enterprise distro images are going to be valuable beyond openstack so i think it's fair for openstack not to bear all of the burden on making those images viable16:19
dhellmannthat seems completely reasonable16:20
dhellmannI'm not sure where we write down those expectations16:20
fungiwe had people really excited to make ubuntu-bionic images usable well before bionic even officially released16:20
openstackgerritZane Bitter proposed openstack/governance master: Resolution on keeping up with Python 3 releases  https://review.openstack.org/61314516:21
zanebdhellmann: let's try that ^16:21
fungidhellmann: worst case it's up to the distros who want us to test on them to work with the opendev infra team to make sure they're available in a reasonable amount of time, else openstack can certainly choose to stop testing on those particular platforms16:21
fungiit's how we've managed the images for fedora, opensuse, gentoo and so on16:22
dhellmannfungi : right, that's exactly the dynamic I'm trying to get us to set up16:22
fungivolunteers invested in those distros worked to make sure the images were buildable and could run jobs16:22
fungidebian too16:23
*** dklyle has joined #openstack-tc16:23
zanebfungi: ++16:23
fungieven the centos images to a great extent16:23
fungiubuntu has sort of gotten a free ride, but if push comes to shove i expect we would look to them to help us make those happen too16:23
fungisince the bulk of the opendev services run on ubuntu, the opendev infra team has had a vested interest in being able to run its own ci jobs on that particular platform anyway16:25
gmannsorry got disconnected. reading ...16:25
dhellmannzaneb : 2 questions about wording clarity, but otherwise I think that looks pretty good16:26
dhellmannI have to step out for a bit, so I'll catch up and read it more closely when I return16:26
gmanndhellmann: fungi got it now. I am ok with QA team to coordinate image work/availability between infra/opendev/distro -> openstack community.16:27
gmannand it mostly fit for devstack scope.16:28
fungiawesome, thanks gmann!16:28
gmannif we have some doc/guide for that i can put that link on QA doc side.16:29
fungisince it's something that's tended to only happen once every couple of years and the circumstances in openstack change more rapidly than that, we've done it different ways every time16:31
clarkbfungi: dhellmann: right I think the issue is that the last two times infra did this for openstack there was lots of complaining. We are happy to provide the platform but the actual transition is really openstack's thing and if openstack doesn't like how we've done it in the past I'd prefer openstack figure it out itself16:31
fungii'm not sure detailed documentation of the process is a super effective use of anyone's time16:32
fungiit would have to be updated more frequently than it was used, and would inevitably fall out of date and have to be figured out again when the time came anyway16:32
clarkbthe upside to when infra was doing it is I felt like we spent far less time on this in the past :)16:35
smcginnisttx: I thought you added that Rocky highlight slide. I seem to remember talking very briefly about it and wondering if we needed to recap the cycle highlight compilation that Anne had done.16:41
smcginnisI think we can probably drop the slide unless there are key points to call out with those.16:42
*** fanzhang has quit IRC16:46
*** e0ne has quit IRC16:48
ttxsmcginnis: I'm fine with dropping it. But then the retrospective feels a bit short, for two people... If you feel comfortable presenting the contributors data I compiled, I'm fine not presenting16:48
*** fanzhang has joined #openstack-tc16:49
ttxThe key on those graphs is to convey that one developer can make a significant impact. Because with large numbers there is always the temptation to think that there is no point is contributing since there are so many contributors already and one more would not make a difference16:52
ttxSo Doug presented data to reinforce that point last time, and the slides for this meeting are a way to make that point again16:53
ttxfrom a different angle16:53
ttxSo if you feel ok presenting that data, it's probably better for you to present the whole segment and drop my name from it16:54
persiaThe number "17" is really exciting.  That means that if someone can only generate a single patch per fortnight, they still end up in the "more active" category.17:01
ttxyes!17:30
ttxA lot of people say "it's hard for us, and there is little point in contributing a drop in the ocean" -- but that's really a false perception. A couple of hours per week will make a significant difference already17:31
persiaI'd probably say "a day a fortnight", being as it's easy to burn through a couple hours a week, especially if not concentrated.17:32
persiaBut I recognise that not everyone likes the word, so maybe "a day every couple weeks"?17:32
*** jpich has quit IRC17:34
persiaAlternately put: "If your organisation can only donate 10% of an FTE to openstack development, this is enough to make a significant impact, and will likely cause your organisation to appear on lists of the more active contributors" or so.17:34
*** diablo_rojo has joined #openstack-tc17:48
*** irclogbot_3 has joined #openstack-tc17:59
*** zaneb has quit IRC18:23
*** ricolin has quit IRC18:24
* TheJulia reads the interesting discussion19:04
TheJuliafungi: I really really really like that idea19:04
* TheJulia goes to the airport19:06
*** diablo_rojo has quit IRC19:12
*** diablo_rojo has joined #openstack-tc19:15
*** mriedem has quit IRC19:32
*** mriedem has joined #openstack-tc19:59
*** e0ne has joined #openstack-tc20:19
*** e0ne has quit IRC20:22
*** mriedem has quit IRC20:41
*** dklyle has quit IRC20:45
*** dklyle has joined #openstack-tc20:45
*** mriedem has joined #openstack-tc20:45
*** dklyle has quit IRC20:46
*** mriedem has left #openstack-tc21:18
*** diablo_rojo has quit IRC21:50
*** zaneb has joined #openstack-tc21:52
*** diablo_rojo has joined #openstack-tc22:03
*** mriedem has joined #openstack-tc22:36
*** mriedem has quit IRC22:46
*** dklyle has joined #openstack-tc23:44

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!