15:00:11 <TheJulia> #startmeeting ironic
15:00:11 <openstack> Meeting started Mon Mar  9 15:00:11 2020 UTC and is due to finish in 60 minutes.  The chair is TheJulia. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:12 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:14 <openstack> The meeting name has been set to 'ironic'
15:00:15 <TheJulia> Greetings everyone!
15:00:16 <iurygregory> o/
15:00:17 <dtantsur> o/
15:00:18 <JayF> o/
15:00:20 <etingof> \o
15:00:21 <rpittau> o/
15:00:22 <arne_wiebalck> o/
15:00:23 <TheJulia> Welcome to our weekly agenda!
15:00:26 <cdearborn> o/
15:00:26 <TheJulia> err, meeting!
15:00:30 <rloo> o/
15:00:32 <TheJulia> Well this meeting is going off wonderfully already
15:00:41 <erbarr> o/
15:00:50 <khansa> o/
15:00:54 <rpioso> \o
15:00:57 <TheJulia> Our agenda can be found on the wiki, as always
15:00:58 <mgoddard> \o
15:00:59 <TheJulia> #link https://wiki.openstack.org/wiki/Meetings/Ironic#Agenda_for_next_meeting
15:01:08 <TheJulia> #topic Announcements / Reminders
15:01:34 <TheJulia> #info Rocky has entered extended maintenance and no more releases will be performed.
15:01:58 <uzumaki> o/
15:02:08 <TheJulia> #info This week is R-9, so we're beginning to run out of time for this cycle.
15:02:15 <dtantsur> oh, again?
15:02:55 <TheJulia> #info Reviewers, please spend some time reviewing, we have a number of items people are pushing on and I'm sure everyone will appreciate some additional reviews
15:03:15 <TheJulia> dtantsur: 3 weeks until non-client library freeze
15:03:35 <dtantsur> so, metalsmith, sushy and ironic-lib
15:03:43 <TheJulia> dtantsur: Realisticaly we'll need to cut our release in 6 weeks.
15:03:47 <dtantsur> I guess sushy is particularly active
15:03:51 <dtantsur> yeah
15:04:18 <openstackgerrit> Merged openstack/bifrost master: Prepare to switch CentOS 8 job voting again  https://review.opendev.org/703954
15:04:23 <dtantsur> and feature freeze in around 4 weeks, I guess?
15:04:31 <TheJulia> I was hoping to have a write-up of the mid-cycle posted by now, but haven't quite gotten to it
15:04:48 <TheJulia> dtantsur: realistically yeah
15:05:00 <dtantsur> TheJulia: if you share an etherpad/gdoc with me, we can work on the contents together
15:05:19 <iurygregory> I can help if necessary also o/
15:05:26 <TheJulia> dtantsur: that being said, if there is limited scope or sufficient confidence in a feature I'm less concerned about a hard/formal feature freeze
15:05:43 <TheJulia> Much appreciated guys, expect a link later today :)
15:05:52 <dtantsur> yeah, I'm not advocating for a hard freeze, rather a soft freeze as previously
15:05:52 <iurygregory> ack
15:05:58 <TheJulia> dtantsur: ++
15:06:09 <TheJulia> Does anyone else have anything to announce this week?
15:06:34 <TheJulia> Seems we had no action items so we can skip that section of the meeting
15:06:50 <dtantsur> ++
15:07:02 <TheJulia> Are we good to proceed onward!?
15:07:07 <rpittau> let's
15:07:16 <dtantsur> choo-choo!
15:07:27 <TheJulia> #topic Review sub-team status reports
15:07:34 <TheJulia> #link https://etherpad.openstack.org/p/IronicWhiteBoard
15:08:02 <TheJulia> Starting at line 298.
15:09:09 <TheJulia> I've not had spoons to work on the whitepaper, but the doc is there. Hopefully the next month will be nice and quiet!
15:09:22 <dtantsur> I don't think I've got anything for the whitepaper in the end :(
15:09:23 * arne_wiebalck hopes that, too
15:10:04 <TheJulia> we need to copy and attribute some content sadly. We expected people proposing content to do so... that was hopeful I think.
15:10:10 <TheJulia> I guess nobody wants to take the role of the editor
15:10:42 <TheJulia> Anyway, moving on!
15:10:47 <TheJulia> dtantsur: thanks for the update on deploy steps!
15:11:00 <arne_wiebalck> the node retirement item can be closed, no?
15:11:01 <dtantsur> np, it's fun to work on :D
15:11:11 <TheJulia> tzumainn awesome progress on multitenancy!
15:11:12 <dtantsur> arne_wiebalck: the sdk patch hasn't merged yet
15:11:17 <tzumainn> TheJulia, thanks!
15:11:19 <dtantsur> or has it?
15:11:24 <dtantsur> tzumainn++
15:11:24 <rpittau> arne_wiebalck: the openstacksdk patch is not merged yet, but it should be today
15:11:28 <arne_wiebalck> I think it has.
15:11:42 <arne_wiebalck> almost, ok
15:11:47 <TheJulia> in the gate \o/
15:12:25 <TheJulia> Reviews of https://review.opendev.org/#/c/672780/ would be much appreciated!
15:12:25 <patchbot> patch 672780 - ironic-specs - Update L3 based deployment spec - 36 patch sets
15:12:27 <rpittau> I need to fix one small thing in the functional test, but will do in a follow-up
15:13:11 <arne_wiebalck> nice, thanks!
15:13:21 <TheJulia> stevebaker is not aorund, but I'm wondering if we've reached a forward path on the wsme work?
15:13:28 <TheJulia> s/aor/aro/
15:13:43 <dtantsur> I haven't seen new patches on that topic
15:14:58 <TheJulia> Looks like the software raid patches are ready for review
15:15:05 <arne_wiebalck> I rebased and tested the UEFI RAID patches on real h/w.
15:15:25 <arne_wiebalck> Yes, are good to go from my POV.
15:15:46 <TheJulia> arne_wiebalck: the standalone raid tests are failing on the ironic patch, so something might be breaking it on that patch
15:16:10 <TheJulia> Are we basically good python3 wise for this cycle?
15:16:22 <arne_wiebalck> TheJulia: ah, the ironic one?
15:16:27 <TheJulia> arne_wiebalck: yeah
15:16:50 <iurygregory> TheJulia, mostly
15:16:53 <dtantsur> python 3.. has someone checked what is happening in the metalsmith CI?
15:17:10 <rpittau> I think we're missing still some stuff
15:17:15 <iurygregory> tenks
15:17:40 <mgoddard> tenks has patches ready to go
15:17:52 <iurygregory> mgoddard, yeah =)
15:17:57 <TheJulia> mgoddard: could you add it to the whiteboard? :)
15:18:02 <mgoddard> https://review.opendev.org/#/c/695881 and https://review.opendev.org/#/c/710100
15:18:03 <patchbot> patch 695881 - tenks - CentOS 8 support - 20 patch sets
15:18:04 <patchbot> patch 710100 - tenks - Bump ansible version constraints to >=2.7,<2.10 - 5 patch sets
15:18:06 <mgoddard> TheJulia: sure
15:18:07 <rpittau> dtantsur: I was going to check metalsmith, didn't have time today
15:18:11 <TheJulia> Thanks!
15:18:15 <dtantsur> nice, thanks
15:18:41 <rpittau> and we're still waiting for the fix to centos8 libvirt pkg for bifrost, somewhat related
15:19:01 <TheJulia> So cross-project wise, one thing to raise awareness. The nova team found an issue with the hash ring logic. Turns out someone renamed one of their machines after the deployment was already online to something with mixed case.
15:19:32 <TheJulia> I noticed we kind of had a similar bug around conductor start-ups, so I put up a patch for that. Since nobody has reported it, I'm not too worried about getting the fix merged quickly.
15:20:07 <rloo> oh. interesting (or not) :-(
15:20:08 <arne_wiebalck> TheJulia:do you have a link to a bug?
15:20:10 <TheJulia> Anyone have anything else to ask questions about on the list or note status ?
15:20:27 <TheJulia> arne_wiebalck: https://review.opendev.org/#/c/711680/
15:20:28 <patchbot> patch 711680 - nova - Lowercase ironic driver hash ring and ignore case ... - 3 patch sets
15:20:39 <arne_wiebalck> TheJulia: thanks!
15:21:07 <TheJulia> https://review.opendev.org/#/c/711765/ <-- what I posted after looking at our code
15:21:07 <patchbot> patch 711765 - ironic - Make reservation checks caseless - 2 patch sets
15:21:53 <TheJulia> Are we good to proceed ?
15:21:59 <dtantsur> ++
15:22:13 <openstackgerrit> Harald Jensås proposed openstack/ironic master: Additional IP addresses to IPv6 stateful ports  https://review.opendev.org/700002
15:22:13 <TheJulia> #topic Deciding on priorities for the coming week
15:22:26 <TheJulia> #link https://etherpad.openstack.org/p/IronicWhiteBoard
15:22:36 <TheJulia> Line 165
15:23:44 <TheJulia> iurygregory: looks like the IPE patches merged, is there anything else needed for that right now?
15:24:29 <iurygregory> TheJulia, I have some backports to stable/train, and I noticed that we don't have the documentation for the ipe (I'm working on it)
15:25:21 <TheJulia> iurygregory: Thanks, and thanks for noting the backports
15:25:43 <TheJulia> I really hope folks have some review bandwidth this week. :(
15:25:58 <iurygregory> TheJulia, I just added the patches in "Potential candidates for backport"
15:26:15 <TheJulia> Are we good with the list of patches today?
15:26:28 <rloo> what does the 'for the next meeting' mean?
15:26:30 <dtantsur> I guess we can remove the "For the next meeting" line?
15:26:34 <rloo> L 227
15:26:41 <dtantsur> rloo: stuff that I proposed earlier today
15:26:50 <rloo> heh, thx dtantsur
15:26:55 <TheJulia> Removed
15:27:04 <rloo> quick question wrt backports. now that rocky is extended maintenance.
15:27:19 <rloo> i see at L244, 'agreed... to take fixes to stein'.
15:27:19 <TheJulia> rloo: sure
15:27:39 <rloo> i'm not sure when 'last week' was. But is that the agreement?
15:28:04 <dtantsur> I'm not sure either, but my take would be that backports to any non-EM releases are recommended, Rocky and Queens are optional
15:28:13 <TheJulia> ++
15:28:18 <dtantsur> non-EM releases are Train and Stein
15:28:33 <dtantsur> Pike and Ocata are likely dead already, and nobody has bothered with them for a while
15:28:50 <rloo> i'm good with that. if no disagreements, i'll update that wording and link to this discussion.
15:29:05 <TheJulia> rloo: thanks for spotting that!
15:29:10 <dtantsur> rloo++
15:29:30 <TheJulia> Are we good to proceed to discussion items?
15:29:40 <openstackgerrit> Riccardo Pittau proposed openstack/metalsmith master: [DNM] Testing CI  https://review.opendev.org/711935
15:29:41 * dtantsur is good
15:29:45 <iurygregory> ++
15:30:16 <TheJulia> #topic Discussion
15:30:27 <TheJulia> We have a single topic, which relates to what we were just actually talking about!
15:30:39 <iurygregory> yay
15:31:08 <TheJulia> dtantsur inquires if we should keep the ocata and pike branches in Extended Maintenance or stop supporting them completely as in closing the stable branches and no longer accepting patches for those branches.
15:31:48 <rloo> what do other projects do?
15:31:54 <dtantsur> good question
15:31:59 <rloo> (ok, nova!)
15:32:09 <rloo> my heros. ha ha.
15:32:11 * dtantsur looks at nova
15:32:26 <dtantsur> I see that nova still has stable/ocata
15:32:28 <rpittau> if we're not going to to backports, why should we keep them open?
15:32:55 <rloo> how does it hurt to keep them open?
15:32:55 <TheJulia> rpittau: Well right now, it is up to us if we want to accept patches on those branches
15:32:57 <dtantsur> on the other hand, they have pretty recent patches to ocata: https://review.opendev.org/#/q/project:openstack/nova+branch:stable/ocata
15:33:00 <TheJulia> we just won't cut the EM brancehs
15:33:03 <TheJulia> branches
15:33:07 <dtantsur> rloo: it doesn't, maybe just wrong messaging
15:33:42 <TheJulia> The thing we kind of need to keep in mind is that it is OKAY to drop test jobs if they are a burden for EM branches
15:33:42 <dtantsur> like, I'm not going to revive the CI on Pike and Ocata. If anybody wants to - be my guest :)
15:33:50 <rloo> i suppose if some bug comes up that someone really wants fixed, and if it eg coincides with a nova patch in ocata?
15:33:54 <rpittau> extended maintenance means that there are community members maintaining it
15:33:58 <rloo> i thought we already turned off CI for pike and ocata.
15:33:59 <TheJulia> Like EM grenade tests, we didn't need anymore.
15:34:10 <iurygregory> this is also related if we care about holding the configurations in ironic-tempest-plugin e.g. https://github.com/openstack/ironic-tempest-plugin/blob/0cbe05eff6b1618e0cba700e246680f91e029832/ironic_tempest_plugin/config.py#L55
15:34:19 <dtantsur> ah, right, iurygregory raises a good point
15:34:23 <iurygregory> https://github.com/openstack/ironic-tempest-plugin/blob/0cbe05eff6b1618e0cba700e246680f91e029832/ironic_tempest_plugin/config.py#L216
15:34:46 <dtantsur> ironic-tempest-plugin is branchless, so in theory it has to support all open branches
15:35:00 <dtantsur> I've heard something about neutron (?) capping their tempest plugin
15:35:01 <rpittau> I guess we need to keep queens compatibility
15:35:16 <TheJulia> rpittau: the TC's intent is that the branches in EM are open for largely non-core community maintainers to submit patches to.
15:35:22 <dtantsur> RH has business interest in queens, so I'm up for keeping it alive for now
15:35:26 <rloo> can we change the rules so that the plugin only supports... (what?) branches?
15:35:36 <TheJulia> I'm all for saying "if it is in EM, we no longer need it in tempest"
15:35:50 <rloo> yeah, what TheJulia sez ^^!
15:35:51 <dtantsur> that will mean that the tempest jobs will no longer pass there
15:36:01 <dtantsur> OR we'll have to cap the plugin (which is not entirely impossible)
15:36:02 <rpittau> TheJulia: I understand that, just thinking even if we have patches submitted but no core reviewing or taking care of the CI, well.....
15:36:18 <dtantsur> and this ^^^
15:36:36 <rpittau> maintaining CI for 4 releases is already an humungous job
15:36:51 <dtantsur> oh yes
15:36:52 <TheJulia> dtantsur: well, is the burden of keeping CI working worth it. At some point only unit tests are going to work for some of these branches no matter what we do and how many hours we pour into them
15:36:54 <openstackgerrit> Merged openstack/bifrost master: Do not use tests as filters  https://review.opendev.org/711815
15:36:59 <rpittau> humongous*
15:37:10 <dtantsur> TheJulia: right, but then have do we merge something there?
15:37:22 <dtantsur> I'm not going to +2 a change that doesn't pass at least some testing
15:38:08 <TheJulia> If we want to hold the requirement of some integration tests working, then where do we cut that off at?
15:38:20 <openstackgerrit> Dmitry Tantsur proposed openstack/bifrost stable/train: Do not use tests as filters  https://review.opendev.org/711938
15:38:55 <TheJulia> dtantsur: I think the TC expectation was that at some point really old branches that would never be released again may only end up with unit testing
15:38:59 <TheJulia> but that it is also up to each team
15:39:16 <rloo> let's just cut them off then. there aren't enough people here to maintain all that, and if we won't +2 if there are no tests, then no use. Is that how everyone feels, no one will +2 w/o (functional) tests?
15:39:16 <TheJulia> I'm kind of neutral and all for reducing pain
15:39:32 <dtantsur> realistically, we don't see a lot of patches there
15:39:38 <dtantsur> (to put it mildly)
15:39:52 <dtantsur> if there was a strong interest in e.g. ocata, I would probably force myself into maintaining it
15:40:11 <rloo> so.. maybe turn off. leave proposed patches around. people can merge those to their local copies?
15:40:15 <dtantsur> but the last patch was proposed to ocata in August
15:40:28 <TheJulia> rloo: that was also the expected intent at some point
15:40:33 <rloo> i mean if we can' guarantee that a patch works, we don't want to +2. so it is going to be people-beware...
15:40:58 <TheJulia> I don't think there is strong interest in maintaining ocata... nor really pike
15:41:12 <dtantsur> exactly
15:41:17 <TheJulia> so whatever reduces the pain, I'm all for
15:41:30 <dtantsur> we can make it a documentation issue: document which branches we maintain to which extent
15:41:34 <dtantsur> similar to releases.o.o
15:41:43 <rloo> is it worth talking to nova to get their viewpoint on why they added changes to ocata recently? wow, just looking now, and it was quite recent.
15:42:19 <dtantsur> maybe just somebody cares to? :)
15:42:36 <rpittau> maybe there was an interest downstream
15:43:11 * rloo continues to be impressed with nova
15:43:16 <dtantsur> note that ironic had been a fast-moving project until very recently
15:43:26 <TheJulia> dtantsur: I say we update the docs, we drop tests to minimal on those branches, we document we're not intending to merge anything to those branches, and move on
15:43:43 <rloo> fwiw, we are using ocata downstream but I don't expect changes to be backported to that...
15:43:45 <dtantsur> maybe even skip dropping tests. if somebody wants to fix them - cool!
15:43:55 <TheJulia> dtantsur: cool! Love it!
15:44:31 <TheJulia> rloo: at that point, I guess you've got downstream branches already and most fixes are environment/local need specific
15:45:01 <rloo> TheJulia: i make us feel the pain of being on an old branch by manually backporting things that we need...
15:45:31 * TheJulia wonders if there is a movie reference that is appropriate...
15:46:30 <TheJulia> Anyway, I'm all for minimal pain, lets just document stance and move on.
15:46:55 <TheJulia> Since we have no RFEs listed nor the Baremetal SIG has anything listed, I suggest we go directly to Open Discussion
15:47:23 <dtantsur> #action dtantsur to document supported branches
15:47:34 <TheJulia> dtantsur: thanks!
15:47:37 <dtantsur> unless somebody beats me to it
15:47:43 <iurygregory> so we support till queens right? =)
15:47:49 <rpittau> iurygregory: yeah
15:48:03 <iurygregory> awesome =)
15:48:50 <rloo> to be clear, we were talking about ocata & pike, right?
15:48:55 * iurygregory updates ironic-tempest-plugin since ocata will be no longer supported
15:49:11 <rpittau> rloo: yep
15:49:29 <rloo> ok, wanted to make sure 'til queens' included queens :)
15:49:38 <rpittau> :)
15:49:39 <iurygregory> rloo, yeah =) sorry
15:49:47 <TheJulia> \o/
15:49:56 <TheJulia> If there are no objections, Open Discussion time?
15:50:18 <rpittau> let's open the open discussion
15:50:25 <dtantsur> openly
15:50:55 <TheJulia> #topic Open Discussion
15:50:59 <TheJulia> So how is everyone?!?
15:51:23 <rpittau> oh gosh.... reserve question ?
15:51:30 <TheJulia> lol
15:51:34 <dtantsur> Worried, tired. Nothing new :)
15:51:50 <TheJulia> *sigh* I know that feeling
15:52:43 <rpittau> one thing I have, I'm switching the jbos in ironic to dib but there is an issue with ironic-tempest-ipa-wholedisk-bios-agent_ipmitool that really bugs me, I was thinking to leave it to tinycore until we can figure that out ?
15:52:59 <dtantsur> do you have a clue what is happening there?
15:53:13 <TheJulia> On a plus side, hopefully I can stop taking tamiflu in a few days.
15:53:24 <rpittau> dtantsur: https://4ae2de5d02c97f2930b7-ff2d888c3307ad9718a14b305143c473.ssl.cf2.rackcdn.com/710278/13/check/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool/419eb12/controller/logs/ironic-bm-logs/node-0_console_2020-03-05-16%3A01%3A18_log.txt
15:53:26 <dtantsur> that's an improvement!
15:53:44 <dtantsur> oh
15:53:49 <rpittau> yeah :/
15:53:52 <dtantsur> rpittau: I seem to recall that DIB-built images hate having >1 NICs
15:54:05 <openstackgerrit> Mark Goddard proposed openstack/tenks master: WIP: Support linux bridge in addition to OVS  https://review.opendev.org/711682
15:54:06 <dtantsur> can it be the cause here?
15:54:17 <dtantsur> I recall setting NIC number to 1 for these jobs
15:54:24 <openstackgerrit> Iury Gregory Melo Ferreira proposed openstack/ironic-tempest-plugin master: Change default driver  https://review.opendev.org/711941
15:54:29 <rpittau> maybe, I see it's failing checking for 2 macs
15:54:30 <TheJulia> That is... alarming
15:54:46 <TheJulia> could lookup be broken?
15:54:57 <rpittau> mmm
15:55:06 <openstackgerrit> Merged openstack/ironic master: Expose allocation owner to additional policy checks  https://review.opendev.org/707307
15:55:08 <dtantsur> mm, yeah, the NIC problem would manifest itself as "no route to host" or something like that
15:55:19 <dtantsur> 404 from lookup smells fishy
15:55:34 <TheJulia> very fishy
15:55:50 <rpittau> I think I'll split the patch anyway, we can move forward with the others, they look fine
15:55:58 <iurygregory> ++
15:56:02 <dtantsur> yeaahh
15:56:03 <TheJulia> it is supposed to split on a command and look them up individually... at least that is what I remember of it
15:56:14 <dtantsur> broken lookup doesn't explain why only this job fails
15:56:20 <iurygregory> moving most of the jobs is awesome already...
15:56:59 <TheJulia> rpittau: splitting soudns like a good idea because we could also be increasing hypervisor load with some of the job changes
15:57:14 <rpittau> TheJulia: exactly
15:58:54 <dtantsur> wrapping up?
15:59:04 <TheJulia> Yup, Does anyone have anything else?
15:59:41 <TheJulia> Thanks everyone! Have a wonderful week!
15:59:45 <dtantsur> thank you!
15:59:49 <rpioso> TheJulia: ty
15:59:49 <rpittau> thanks!
15:59:51 <iurygregory> ty
16:00:24 <TheJulia> #endmeeting