15:00:34 <TheJulia> #startmeeting ironic
15:00:35 <openstack> Meeting started Mon Apr  6 15:00:34 2020 UTC and is due to finish in 60 minutes.  The chair is TheJulia. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:35 <TheJulia> o/
15:00:35 <iurygregory> o/
15:00:36 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:38 <openstack> The meeting name has been set to 'ironic'
15:00:40 <dtantsur> o/
15:00:43 <rpittau> o/
15:00:45 <mgoddard> \o
15:00:50 <kaifeng_> o/
15:00:50 <rpioso> \o
15:00:51 <ajya> o/
15:01:02 <TheJulia> arne_wiebalck: I think the ipa patch merged, the alternative is waiting an entire cycle because it is a breaking change for older IPA images
15:01:10 <TheJulia> Good morning everyone!
15:01:11 <etingof> o/
15:01:25 <TheJulia> Our agenda can be found on the wiki!
15:01:27 <TheJulia> #link https://wiki.openstack.org/wiki/Meetings/Ironic#Agenda_for_next_meeting
15:01:30 <arne_wiebalck> o/
15:01:39 <TheJulia> We have a number of items, so we should go ahead and get started!
15:01:53 <TheJulia> #topic Announcements / Reminders
15:01:53 <rloo> o/
15:02:15 <arne_wiebalck> TheJulia: ok, let's take this after the meeting if you have a moment
15:02:36 <erbarr> o/
15:02:59 <TheJulia> April 9th is the Ussuri-3 milestone which means Cycle highlights are due this week. If there is anything people wish for me to specifically mention, please let me know ASAP.
15:03:25 <TheJulia> Openstack's RC1 deadline is April 23rd, which means we need to cut our release for the ussuri cycle before then.
15:03:35 <TheJulia> Any questions on that?
15:03:47 <TheJulia> Does anyone heave anything else to announce or remind us of?
15:04:02 <rpittau> TheJulia: next monday is bank holiday I think
15:04:10 <iurygregory> ironic-tempest-ipa-wholedisk-direct-tinyipa-multinode is back voting =)
15:04:11 <dtantsur> and Friday too for some of us
15:04:27 <iurygregory> ++ for dtantsur said
15:04:39 * TheJulia senses she needs to take a day or two off
15:04:39 <rpittau> yep, but for next monday I guess we'll have to cancel the meeting
15:04:59 <TheJulia> That seems like it is viable
15:05:01 <TheJulia> Any objections?
15:05:13 <cdearborn> o/
15:05:29 <cdearborn> (sorry - showing up late - that was not an objection)
15:05:34 * TheJulia hears no objections :)
15:05:36 <rajinir> o/
15:05:42 <TheJulia> cdearborn: I figured that :)
15:05:49 * iurygregory was about to ask if it was XD
15:06:22 <TheJulia> #agreed Ironic will not be holding a meeting on the 13th because of the Holiday.
15:06:34 <TheJulia> Can I get a volunteer to send a note to the mailing list?
15:06:53 <dtantsur> I can do it
15:06:55 <iurygregory> \o
15:07:05 <dtantsur> iurygregory: go ahead if you want to :)
15:07:11 <TheJulia> Excellent! Thanks!
15:07:18 <iurygregory> dtantsur, if you are ok with it =)
15:07:32 <dtantsur> I'm always ok with less work ;)
15:07:55 <iurygregory> action item on me =)
15:08:17 <TheJulia> #action iurygregory to send a note to the mailing list indicating we will not be meeting next week.
15:08:24 <TheJulia> Onward!
15:08:27 <TheJulia> #topic Review action items from previous meeting
15:08:59 <TheJulia> We had one action item last week, which was for me to send out another doodle for whitepaper meeting, but sadly I did not upon starting to perceive the stress everyone was under.
15:09:10 <TheJulia> arne_wiebalck: Is this what your topic item is on, trying to figure out a time to revisit?
15:09:34 <arne_wiebalck> TheJulia: yes, I think we should keep the momentum
15:09:55 <arne_wiebalck> TheJulia: there were quite some additions and reviews
15:09:59 <TheJulia> arne_wiebalck: would you be willing to send out the doodle?
15:10:02 <TheJulia> ++
15:10:06 <arne_wiebalck> TheJulia: sure
15:10:11 <TheJulia> arne_wiebalck: excellent!
15:10:20 <TheJulia> Okay then, ward if there are no objections!
15:11:03 <TheJulia> #topic Review subteam status reports
15:11:13 <TheJulia> #link https://etherpad.openstack.org/p/IronicWhiteBoard
15:11:17 <TheJulia> Line 287
15:12:01 <TheJulia> mgoddard: dtantsur: looks like the deploy steps stuffs could use an update on the whiteboard.
15:12:16 <dtantsur> it's probably pretty much correct, lemme see
15:13:17 <TheJulia> arne_wiebalck: Looks like software raid only has just the doc patch and the ironic patch we were just talking about left. \o/
15:13:50 <dtantsur> done
15:13:53 <TheJulia> That seems like it is it to me. Is everyone read to proceed onward?
15:13:55 <TheJulia> dtantsur: thanks!
15:13:59 <arne_wiebalck> TheJulia: \o/
15:14:13 <iurygregory> ++
15:14:41 <TheJulia> Looks like ?rpittau? is adding some stuff to CI improvements
15:14:54 <rpittau> just one thing, done
15:15:01 <TheJulia> thanks
15:15:05 <TheJulia> Onward!
15:15:12 <TheJulia> #topic Deciding on priorities for the coming week!
15:15:20 <TheJulia> #link https://etherpad.openstack.org/p/IronicWhiteBoard
15:15:43 <TheJulia> Starting at line 130
15:16:03 <TheJulia> On a positive note, a ton of patches merged last week! Lets keep up the momentum because I need to write the cycle highlights :)
15:16:03 <dtantsur> I've added a few dependency clean-ups there, although most of them have been approved (thanks!)
15:16:13 * TheJulia cleans up the merged items now
15:19:17 <TheJulia> How does that look for everyone?
15:19:38 <rpittau> ok
15:19:49 <iurygregory> ok
15:20:14 <dtantsur> LGTM
15:20:15 <TheJulia> Seems like a lot, but 5 or 6 of them are already in-flight or will be approved quickly because they are really small patches.
15:20:49 <TheJulia> Anyway, if we're good with that, I guess it is time to proceed to discussion!
15:20:56 <TheJulia> Any objections?
15:21:07 <iurygregory> ++
15:21:19 * TheJulia takes that as no :)
15:21:25 <TheJulia> #topic Discussion
15:21:39 <TheJulia> First topic is rpittau's regarding the bioic OVMF package.
15:21:45 <TheJulia> #link https://bugs.launchpad.net/ubuntu/+source/edk2/+bug/1821729
15:21:46 <openstack> Launchpad bug 1821729 in edk2 (Ubuntu Cosmic) "UEFI in ovmf package causes kernel panic" [Undecided,Invalid]
15:21:51 <TheJulia> rpittau: the microphone is yours
15:21:52 <rpittau> thanks
15:22:09 <rpittau> I'm testing the ovmf native package on bionic using dib
15:22:30 <openstackgerrit> Verification of a change to openstack/python-ironic-inspector-client failed: Make oslo.i18n an optional dependency  https://review.opendev.org/717730
15:22:31 <openstackgerrit> Verification of a change to openstack/python-ironic-inspector-client failed: Get rid of the oslo.utils requirement  https://review.opendev.org/717731
15:22:32 <openstackgerrit> Verification of a change to openstack/python-ironic-inspector-client failed: Move osc-lib to test-requirements  https://review.opendev.org/717733
15:22:45 <rpittau> at the moment whtat I'm seeing is that the node gets the ip but it hangs before getting the NBP
15:22:51 * TheJulia glares at openstackgerrit bot... :)
15:22:59 <rpittau> this is what I'm seeing http://paste.openstack.org/show/791675/
15:23:29 <rpittau> thee console log stops right after teh Station IP address
15:23:45 <rpittau> and the only anomaly is the tftpd error in journal
15:23:51 <TheJulia> connection tracking/firewalling blocking the return connection?
15:24:21 <rpittau> I checked the firewall but I can't see anything wrong, apparently, also nothing changed on that side :/
15:24:45 <TheJulia> Well bionic did make default connection tracker helper chagnes
15:24:57 <TheJulia> But I thought we accounted for that in our CI jobs
15:25:08 <rpittau> yes, and we have a workaround for that in place
15:25:32 <rpittau> otherwise many other things would not work
15:25:38 <TheJulia> yeah
15:26:16 <rpittau> it could be that ovmf now requires another rule in iptables, but I can't really say what
15:26:16 <TheJulia> connection refused makes me think that the tftp daemon is not able to send packets to OVMF code running in the VM which makes me think that maybe they broke their tftp code?!?
15:26:36 <iurygregory> O.o
15:26:36 <TheJulia> I feel like the socket needs to be tcpdumped
15:26:42 <iurygregory> woa
15:26:49 <TheJulia> just tcpdump the entire thing and see what we've got going back and forth
15:27:17 <TheJulia> if we don't see ovmf emitting a icmp packet to the TFTP server address, then we know it is hypervisor firewalling
15:27:39 <rpittau> ok, I'll see to that
15:27:51 <TheJulia> rpittau: the ipv6 ci job has an example of using tcpdump in it if memory serves
15:28:05 <TheJulia> Hopefully that helps
15:28:10 <rpittau> the thing that's making me crazy is thtat updating the package and restarting the server creation works
15:28:22 <TheJulia> ugh :(
15:28:49 <TheJulia> Well, lets at least get a tcpdump.. because if ovmf is telling the server "send me the file on this port" and then responds with ICMP, it is clearly an OVMF bug
15:28:55 <rpittau> ok
15:29:18 <TheJulia> Are we good to move on?
15:29:25 <hjensas> both updating package and restarting server create togheter. Or both those are workaround?
15:29:26 <rpittau> I am :)
15:29:58 <rpittau> hjensas: starting with xenial pkg, creating node, then desotryong node, upgrading package, creating node again works
15:30:20 <rpittau> creating node directly with bionic package does not work
15:30:30 <dtantsur> Oo
15:30:31 <rpittau> s/node/server
15:30:40 <hjensas> rpittau: wow. Thanks, crystal clear.
15:30:44 <TheJulia> :(
15:31:06 <TheJulia> Next topic is dtantsur's on his proposal to make ironic a top-level opendev project.
15:31:13 <TheJulia> #link http://lists.openstack.org/pipermail/openstack-discuss/2020-April/013757.html
15:31:15 <openstackgerrit> Merged openstack/ironic master: Change multinode job to voting  https://review.opendev.org/717348
15:31:16 <openstackgerrit> Merged openstack/python-ironicclient master: Make oslo.i18n an optional dependency  https://review.opendev.org/717729
15:31:17 <TheJulia> dtantsur: the floor is yours
15:31:18 <openstackgerrit> Merged openstack/python-ironicclient master: A standalone CLI for ironic  https://review.opendev.org/717273
15:31:30 <dtantsur> well, I mostly wanted to make sure that all parties here have read and commented
15:31:46 <dtantsur> I know some of us have discussed it previously, but the discussion stalled
15:32:02 <dtantsur> a lot of interesting stuff in the thread, highly recommend
15:32:21 <TheJulia> That thread is also a little hard to read because there is a strong undertone of hurt and fear
15:32:31 <iurygregory> yeah
15:32:39 <dtantsur> not without that indeed. but also quite a few useful thoughts.
15:32:47 <TheJulia> Indeed
15:33:04 <dtantsur> I think it's important that people who care about ironic (many of them are present) read, think and comment
15:33:18 <dtantsur> otherwise the thread will stall and end up with nothing
15:33:19 <TheJulia> ++
15:33:36 <dtantsur> even if we don't end up breaking up from openstack, I'm sure there can be a lot of useful follow-ups
15:33:57 * dtantsur needs to provide a new round of responses to people, send spoons!
15:34:21 <TheJulia> It is a discussion that needs to happen, and I love that it returning us to the long debated what is openstack thread
15:35:20 <dtantsur> I seem to be great at opening cans of worms :D
15:35:25 <TheJulia> dtantsur: I'm running low, but maybe http://www.cutestpaw.com/articles/50-cute-cats-make-your-life-happier/ might help
15:35:32 * hjensas likes that the thread started stirring a bit about how to "get to market" that opentack is a collection of services. That can be used standalone or just 1 or 2.
15:35:46 <dtantsur> TheJulia: that may work, thanks!
15:36:01 <dtantsur> hjensas: that's a possible useful outcome
15:36:15 <TheJulia> hjensas: and it is highlighting a lot of major disconnects that need mending... somehow
15:37:21 <dtantsur> indeed
15:37:47 <TheJulia> Anyway, dtantsur thank you for raising the topic, and if you need more cut pet links, I'll happily dig for them because I need spoons too.
15:37:56 <dtantsur> more cut pet links \o/
15:37:59 <dtantsur> np :)
15:38:17 <iurygregory> cut pet links should be shared =)
15:38:27 <TheJulia> #agreed All cute pet photos should be shared.
15:38:35 <TheJulia> Shall we proceed onward?
15:38:37 <rpioso> The MAAS web site link etingof shared provided insight into what we're competing with: https://maas.io/
15:39:21 <dtantsur> that's me following the thread in question: http://www.cutestpaw.com/wp-content/uploads/2011/11/Kitten.jpg
15:39:34 * kaifeng_ actually investigated and compared with ironic before
15:39:35 * etingof thinks MaaS is a tough competitor, at least from marketing and PR standpoints
15:39:49 <dtantsur> rpioso: exactly. somebody (mnaser?) asked how the side should look - this is how probably
15:39:56 <dtantsur> * site
15:40:02 <TheJulia> And we have no marketing or PR. Or to put it more precisely, we are the marketing and PR team as well.
15:40:11 <dtantsur> kaifeng_: I'd be curious to hear about your experience!
15:40:40 <dtantsur> TheJulia: yeah, we cannot get to the same level.. but we may get a nice landing page!
15:40:51 * arne_wiebalck was also quite intrigued by the https://rebar.digital/ link etingof posted earlier
15:40:53 <TheJulia> dtantsur: interesting you note that, there was an effort that was pushed many moons ago to try and simplify project landing pages and not have them full of links of descriptie text
15:41:20 <kaifeng_> My feeling on maas is it is a project to quickly bring a data center into a public cloud
15:41:21 <dtantsur> for that we need to *have* project landing pages
15:41:22 <TheJulia> it flopped of course because some still wanted the index style landing page
15:41:43 <dtantsur> I don't believe docs.o.o/$project is
15:42:04 <TheJulia> no, it isen't, and the intent of the docs team was to try and craft them into that
15:42:10 <TheJulia> Sadly, things change too
15:42:21 <TheJulia> Anyway, lots of good discussion, and we should move onto the SIG!
15:42:29 <dtantsur> ++
15:42:34 <TheJulia> #topic Baremetal SIG
15:42:51 <TheJulia> arne_wiebalck: Is there anything else to discuss that we didn't cover earlier?
15:43:15 <arne_wiebalck> TheJulia: Not relly. Thanks everyone for contributing, looks much better now!
15:43:18 <arne_wiebalck> really
15:43:34 * etingof will push his bits in a bit
15:43:46 <arne_wiebalck> etingof: tanks!
15:43:55 * arne_wiebalck cannot type
15:44:01 <TheJulia> dtantsur: fyi, this is me right now http://www.cutestpaw.com/wp-content/uploads/2011/11/To-infinity-and-beyond.jpeg needing more coffee
15:44:01 <dtantsur> I'm a bit behind on my part, sorry
15:44:13 <TheJulia> Same, I'm getting pulled in many directions
15:44:18 <dtantsur> looks like you indeed :)
15:44:31 <rpittau> same :/
15:44:41 <TheJulia> I should have time this week, we've not been running the 3d rpinters around the clock since Saturday morning.
15:45:20 <TheJulia> printers
15:45:31 <TheJulia> Anyway, onward to Open Discussion
15:45:34 <TheJulia> #topic Open Discussion
15:45:47 <dtantsur> I have a follow-up question to https://review.opendev.org/#/c/717273/
15:45:48 <patchbot> patch 717273 - python-ironicclient - A standalone CLI for ironic (MERGED) - 9 patch sets
15:45:57 <dtantsur> first, yes, we now have a 'baremetal' command without OSC
15:46:04 <TheJulia> \o/
15:46:08 <TheJulia> NobodyCam: ^^^^^
15:46:17 <dtantsur> second, what would people think if I enable a way for the same command to add ironic-inspector-client commands?
15:46:38 <dtantsur> i.e. if python-ironic-inspector-client is installed, then 'baremetal introspection blah-blah' is a thing
15:46:42 <TheJulia> dtantsur: Might be interesting, do you think you could get it up in the next 24 hours?
15:46:56 <dtantsur> TheJulia: tomorrow morning quite likely
15:47:03 <TheJulia> I say do it!
15:47:19 <NobodyCam> Good morning folks and WooHoo
15:47:24 <dtantsur> I got close to it, but then stumbled upon an issue in ironic-inspector-client
15:47:28 <dtantsur> I think I can work around it
15:47:47 <TheJulia> Okay, awesome
15:47:49 <TheJulia> Thanks dtantsur !
15:50:12 <TheJulia> Do we have anything else to discuss?
15:50:46 <iurygregory> justa heads up for the cores to look at the changes in the ironic-tempest-plugin https://review.opendev.org/712041
15:50:46 <patchbot> patch 712041 - ironic-tempest-plugin - Split Tempest tests between ipmi and redfish - 8 patch sets
15:50:47 <iurygregory> =)
15:51:03 * etingof pulls the mic
15:51:17 <iurygregory> go for it etingof o/
15:51:17 <TheJulia> iurygregory: I may have already added those..
15:51:19 <TheJulia> etingof: ?
15:51:29 <etingof> it turned out that ironic can't vmedia boot Supermicros because they do vmedia it in their own OEM way. what we could do is to introduce a new sushy extension package - sushy-oem-supermicro. so I am wondering how useful that would be...?
15:51:30 <iurygregory> TheJulia, oh thanks =)
15:51:48 <dtantsur> etingof: not just package, also a new driver, right?
15:51:56 <dtantsur> living in ironic-staging-drivers since it's not vendor-backed?
15:52:13 <etingof> well, ideally - just the mediation layer well-hidden inside sushy extension...
15:52:17 <dtantsur> s/driver/hardware type/ (heh, of all people, I'm not supposed to confuse these)
15:52:35 <dtantsur> etingof: the way sushy works now, OEM stuff won't be magically picked up
15:53:16 <etingof> yeah, ideally that should be magical whenever possible
15:53:36 <etingof> but, yes, explicit vendor driver would do as well
15:54:11 <dtantsur> I'm trying to understand the scope of your proposal
15:54:19 <etingof> it seems the magic is not impossible
15:54:31 <dtantsur> do you propose to also modify sushy and/or the redfish hw type to magically pick up OEM changes?
15:54:37 <dtantsur> similarly to what we discussed in GVA?
15:55:19 <etingof> yeah, that seems ideal to me... look, if sushy/can learn from BMC what kind of vendor is that, sushy could probably pick the right extension to mediate...
15:55:35 <dtantsur> yeah, or it can be a driver_info field
15:55:42 <dtantsur> I'd read a detailed RFE on this
15:55:58 <etingof> right
15:56:08 <TheJulia> ++
15:56:31 <etingof> perhaps the case of supermicro could serve as a precedent for sushy OEM automation...
15:56:35 <TheJulia> yeah, I'm not exactly groking what and how at the moment so I think something written might be helpful
15:57:07 <rpioso> etingof: Would the iDRAC vmedia diffs benefit from the mediation layer?
15:57:47 <dtantsur> I guess it depends on what is going to be proposed?
15:57:50 <etingof> rpioso, ideally, if we could hide all idrac-redfish differences in sushy extension...
15:58:58 <rpioso> etingof: We would still have a need to be able to offer our vendor value add. Not sure how that would be handled.
15:59:25 <etingof> rpioso, like what?
15:59:29 <dtantsur> via selling hardware? ;)
15:59:35 <dtantsur> sorry, couldn't resist
15:59:48 <rpioso> dtantsur: heh
16:00:01 <dtantsur> I don't think "you need a non-standard driver for our hardware" is a selling point actually
16:00:02 <etingof> for the ironic operator it is important to deploy the machine
16:00:11 <etingof> dtantsur++
16:00:13 <rpioso> etingof: Server Configuration Profile (SCP).
16:00:20 * rpioso looks for the story
16:00:31 <dtantsur> ah, you mean, you have more features coming?
16:00:44 <dtantsur> this is fine. I think the current discussion is to make all base features of 'redfish' hw type work
16:00:47 * TheJulia wonders if we are done with the meeting
16:00:51 <dtantsur> we probably are
16:01:02 <rpioso> etingof: https://storyboard.openstack.org/#!/story/2003594
16:01:27 <etingof> TheJulia should now dismiss us
16:01:37 <TheJulia> Excellent!
16:01:51 <TheJulia> Have a wonderful week everyone! Stay safe! And remember to share cute pet photos!
16:02:01 <TheJulia> Also photos of ferrets. My wife <3s them.
16:02:05 <TheJulia> #endmeeting