17:00:50 <jroll> #startmeeting ironic
17:00:51 <openstack> Meeting started Mon Jul 25 17:00:50 2016 UTC and is due to finish in 60 minutes.  The chair is jroll. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:52 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:54 <openstack> The meeting name has been set to 'ironic'
17:00:55 <TheJulia> o/
17:00:56 <devananda> o/
17:00:56 <vdrok> o/
17:00:57 <rpioso> o/
17:01:05 <stendulker> o/
17:01:05 <lucasagomes> o/
17:01:06 <skramaja> hello
17:01:29 <rloo> o/
17:01:42 <davidlenwell> o/
17:01:45 <jroll> hey everyone
17:01:51 * jroll gets started
17:01:55 <milan> \o
17:01:57 <jroll> #topic Announcements and reminders
17:02:13 <krtaylor> o/
17:02:16 <jroll> #info ironic now has the supports-upgrade and follows-standard-deprecation tags \o/
17:02:38 <devananda> woot!
17:02:46 <NobodyCam> o/
17:03:01 <devananda> Many thanks to everyone who's worked on upgrade testing!
17:03:12 <jroll> #info some discussion on nova/ironic interactions at nova's midcycle last week, please do read:
17:03:14 <jroll> #link http://lists.openstack.org/pipermail/openstack-dev/2016-July/099922.html
17:03:28 <NobodyCam> oh great news from the mid cycle jroll
17:03:40 <thiagop> o/
17:03:59 <jroll> any other announcements or reminders?
17:04:25 <thiagop> OneView CI has been down due to power outage problems that affected underlying infra
17:04:40 <thiagop> fixing it today, should be running soon
17:05:02 <jroll> cool
17:05:07 <stendulker> iLO CI is also down due to infra getting shifted
17:05:22 <stendulker> should be up shortly
17:05:27 <jroll> I noticed dell CI is failing very quickly as well
17:05:38 <jroll> like, 7 second runs
17:05:46 <jroll> I assume that isn't an ironic issue :)
17:06:35 <jroll> ok, moving on
17:06:41 <jroll> #topic subteam status reports
17:06:51 <jroll> those are here
17:06:53 <jroll> #link https://etherpad.openstack.org/p/IronicWhiteBoard
17:06:56 <jroll> line 81 or so
17:07:31 <jroll> I'll give folks a few to update / look over those
17:08:03 * natorious waves o/
17:08:22 <Madasi> o/
17:09:00 <rloo> jroll: wrt the network isolation/nova patch, what does 'may still be able to..' mean? If the patch is reviewed and is good to merge? Or something else?
17:09:38 <jroll> rloo: we missed the non-priority feature freeze, people seem supportive but haven't got confirmation from PTL
17:09:59 <rloo> jroll: ok, so we should review it asap then.
17:10:11 <rloo> jroll: also, is that the only outstanding patch (w/o portgroups?)
17:10:14 <jroll> rloo: indeed
17:10:16 <jroll> and yes
17:10:53 <rloo> jroll: gotcha, thx.
17:11:40 <jroll> looks like a lot of progress this week \o/
17:12:28 <rpioso> jroll: The Dell hardware is being changed, so builds are broken.  We are looking into it.
17:12:34 <jroll> rpioso: noted, thanks
17:13:53 <jroll> okay, any questions/comments on the subteam updates?
17:14:49 <jroll> #topic Alternatives for the proposed approach at https://review.openstack.org/#/c/331564/11/specs/approved/boot-params.rst
17:14:54 <jroll> #link https://review.openstack.org/#/c/331564/11/specs/approved/boot-params.rst
17:14:55 <lucasagomes> that's me :-)
17:14:56 <jroll> lucasagomes: you're up
17:14:58 <jroll> :D
17:15:03 <lucasagomes> so, looking for altenatives/suggestions here
17:15:12 <lucasagomes> The tl;dr of what the spec is proposing is: Having a way to the users to pass custom kernel command lines when configuring the bootloader for local boot
17:15:39 <lucasagomes> and current it proposes to use the standard way to do it with grub, which is editing the /etc/default/grub and then regenerating the grub.conf
17:15:51 <lucasagomes> the thing is that, in Ironic, we consider touching the files in the user image as a layer violation
17:16:14 <lucasagomes> and the alternatives that I can think of sucks
17:16:16 <jroll> is there any reason this can't be done with port-boot automation and a reboot?
17:16:35 <lucasagomes> that's one alternative, have cloud-init to set it and reboot the node
17:16:35 <jroll> (and maybe have ironic do the initial grub.conf to skip the first reboot)
17:16:45 <jroll> cloud-init, or ansible, chef, etc
17:17:01 <lucasagomes> yeah, the only thing is that we could avoid that if it was configured by Ironic
17:17:05 <lucasagomes> no extra reboot, which saves time
17:17:15 <lucasagomes> and also is a bit weird to have a node to reboot twice after active
17:17:26 <jroll> if ironic did the initial config you could skip the extra reboot
17:17:32 <lucasagomes> yes
17:17:43 <jroll> just a bit of a duplication thing
17:18:02 <lucasagomes> another alternative is to use "grubby" which is a command line tool that can add/remove kernel parameters from the grub bootloader
17:18:08 <lucasagomes> but ubuntu doesn't package it :-/
17:18:12 <NobodyCam> I kinda felt that it wasn't a layer violation as we are installing the boot loader anyway, so I looked at it like just some configuration of that
17:18:28 <lucasagomes> NobodyCam, yeah, that's kinda way I wanted to bring it here
17:18:37 <lucasagomes> I know it does touches the files in the image, but, that's the standard interface
17:18:43 <devananda> in some cases, we're installing the bootloader (partition image)
17:18:52 <devananda> but not in all cases
17:18:56 <natorious> what about images that don't use grub?
17:19:01 <jroll> right, but in no cases are we touching files in the rootfs
17:19:08 <devananda> right
17:19:11 <jroll> which seems to be the primary objection
17:19:12 <lucasagomes> devananda, exactly, that's what this spec is proposing. For when installing the bootloader for a partition image
17:19:15 <jroll> which I tend to agree with
17:19:19 <JayF> That's the first of many
17:19:26 <JayF> my second objection would be in line with natorious's comment
17:19:31 <jroll> sure
17:19:38 <JayF> essentially we'd be adding an Ironic feature that is tightly coupled into the image
17:19:39 <lucasagomes> natorious, currently we only support grub instalation as the bootloader for patition image
17:19:43 <JayF> which seems like a pretty bad idea
17:19:50 <lucasagomes> natorious, for full disk image you can use whatever
17:19:51 <natorious> it might be a good time to start full in on efi
17:20:03 <natorious> which would open up options
17:20:12 <JayF> Another question I have is, why is this an Ironic-specific problem/
17:20:15 <natorious> bc that would be its own partition
17:20:26 <JayF> Why shouldn't this be solved via nova, using more standard ways of configuring instances/
17:20:34 <jroll> JayF: good question, what does nova do here? do they do it?
17:20:43 * lucasagomes don't know
17:20:54 <JayF> jroll: I don't think they support anyhting like this.
17:20:57 <natorious> nova doesn't need efi boot w/ legacy emulation
17:21:14 <natorious> and pygrub
17:21:24 <jroll> natorious: I mean, how does a user give kernel args to nova, and what is done with them?
17:21:32 <JayF> jroll: which is kinda my point; if nova supported it and we needed to implement it, I'd be slightly less weirded out by it, but as it sits, it seems like we're implementing a feature that has a small scope of usefulness while extending ironic's scope considerably
17:21:52 <jroll> lucasagomes: also, who is meant to put the data in instance_info? operator? nova?
17:22:07 <skramaja> even if set in nova, it has to be updated to grub via another service or agent by reading it..
17:22:14 <lucasagomes> skramaja, ^ is the author here
17:22:21 <jroll> oh
17:22:34 <skramaja> and then reboot..
17:22:37 <jroll> skramaja: I understand that, I'm trying to figure out how this works as a whole
17:22:53 <skramaja> but there are services which are dependent certain args..
17:23:04 <skramaja> like DPDK depnds on iommu
17:23:09 <natorious> I didn't think nova passes in kargs outside of pygrub
17:23:11 <jroll> ah, I see, the operator sets the instance_info
17:23:18 <skramaja> before enbling DPDK, immou has to be enabled..
17:23:25 <clarkb> as someone that has needed to do this before (but with nova and VMs) we jsut edited our iamges and provided them to nova and it worked fine
17:23:30 <devananda> jroll: nova sets instance_info, not operator ...
17:23:32 <jroll> which... doesn't seem terribly useful to the user?
17:23:36 <JayF> clarkb: that's kinda what I'm feeling the proper solution is
17:23:37 <jroll> devananda: line 147
17:23:44 <clarkb> which worked fine for us eg nova/ironic don't need to know or care
17:23:46 <JayF> clarkb: which is why I'm pushing back against ironic fiddling in etc
17:23:48 <devananda> clarkb: ++
17:23:55 <jroll> clarkb: yep, makes sense
17:24:00 <skramaja> clarkb: JayF modifying image is not scalable..
17:24:12 <devananda> skramaja: building your own image is very scalable
17:24:31 <skramaja> because in case differential clusters, u need to build multiple images..
17:24:34 <clarkb> (we set kernel boot params to limit available system memory fwiw)
17:24:59 <skramaja> we can use introspection rules via ironic to set specific args for isolcpus..
17:25:03 <devananda> skramaja: yea, build one image per hardware type, with what ever kernel params you need baked into it
17:25:47 <TheJulia> or build local tooling to edit an existing image
17:26:00 * lucasagomes would prefer that ^ over creating loads of images
17:26:30 <skramaja> devananda: it is not just one set of parameters.. ther are multiple criterias we need to multiple combinations.. cpu isolatio, hugepages, iommo
17:26:49 <TheJulia> lucasagomes: I think it is something that has been done in the community several times too :)
17:27:10 <lucasagomes> TheJulia, probably more people have hit that yeah
17:27:18 <JayF> It sounds like this might be a larger issue though
17:27:25 <JayF> given clarkb pointed out they had it with nova and vms
17:27:26 <devananda> skramaja: and you can't change these from within the instance after its been deployed -- and just reboot it?
17:27:44 <JayF> which gives me even more pause to Ironic doing it w/o Nova cooporation or at least some consensus on the ML
17:28:00 <skramaja> devananda: yes.. it is not possible because certain drivers loading like DPDK depends on immou flags
17:28:05 <skramaja> and hugepages enabling..
17:28:48 <jroll> right, my primary objections are: 1) reaching into the instance and touching /etc, and 2) that there's no interface to the user here, only the operator can set it
17:29:07 <jroll> if we can solve those, I'm okay with it
17:29:14 <jroll> otherwise I'd prefer to leave this to post-boot automation
17:29:27 <JayF> jroll: what about 3) thinking about how something like this fits in the larger ecosystem
17:29:42 <JayF> from clarkb's comment, it's clear this isn't unique to ironic, so I'm loathe for us to move on it w/o a larger consensus
17:29:43 <devananda> jroll: if only the operator can set it, this proposal doesn't make any sense to me
17:29:47 <jroll> JayF: that ties into (2) imo
17:29:59 <lucasagomes> jroll, 1) we can add a dependency on "grubby" so we don't touch files directly, but may not work on all distros
17:30:09 <jroll> devananda: indeed, except for deployments where op==user
17:30:12 <devananda> if the operator is setting kernel boot flags persistently on a Node, that has no bearing ...
17:30:13 <JayF> lucasagomes: we're still modifying, even if we use an interface to it
17:30:14 <devananda> yea, well...
17:30:28 <JayF> lucasagomes: it wouldn't change the nature of my objection
17:30:30 <lucasagomes> JayF, but we do modify it when we install the bootloader
17:30:39 <JayF> lucasagomes: not the root fs, only the /boot, correct?
17:30:42 <devananda> jroll: I agree with your objections
17:30:42 <lucasagomes> we currently mount the partition and run grub-install
17:30:46 <jroll> also, it isn't clear from this spec if this is only for partition images or also whole disk images
17:30:55 <lucasagomes> JayF, right and grubby does not modify /etc either
17:31:00 <lucasagomes> only grub.conf afaik
17:31:14 <JayF> lucasagomes: if it doesn't change etc/default/grub whatever it does would be non-persistent
17:31:19 <JayF> lucasagomes: and wouldn't really fix the issue
17:31:21 <lucasagomes> the downside of it is that, it's not persistent on a kernel update
17:31:55 <lucasagomes> JayF, right, that's why I added it to the meeting, cause the alternatives to changing default/grub are not very ideal
17:32:02 <TheJulia> I also agree with almost every objection raised thus far.  I feel like reaching into the image is a bad idea... if we want to provide a modular interface to do_something(), thats different and can be site/deployment specific... and truthfully that is what this feels like, pushing a environmental requirement into the deployment process
17:32:32 <JayF> TheJulia: "pushing a environmental requirement into the deployment process" <-- A+++ summary
17:32:42 <devananda> TheJulia: agreed
17:32:52 <TheJulia> JayF: I've started and stopped typing that like 10 times now, so I finally just typed it
17:32:56 <lucasagomes> ok so post-deploy scripts
17:33:11 <lucasagomes> skramaja, even if not ideal, is it viable ?
17:33:26 <devananda> and furthermore, there already exist ways to achieve the desired goal (bake the needed parameters into an image)
17:33:27 <skramaja> lucasagomes: i have to check
17:33:35 <JayF> lucasagomes: I specifically suggested in IRC, that getting something in the configdrive and using an agent to set the options was an alternative I could be on board with. Then Ironic sets the opts for firstboot, and the agent persists them
17:33:49 <JayF> lucasagomes: but that would require coming through nova and doing the work there first, since they own/manage configdrives
17:33:49 <devananda> which we support and maintain the boundary between deployment process and tenant workload
17:33:56 <jroll> JayF: which would depend on nova having an interface for this :)
17:33:59 <jroll> yeah
17:34:11 <JayF> lucasagomes: (although admittedly telling someone to go to nova first is slightly more helpful than telling them to screw off, today, lol)
17:34:21 <lucasagomes> :D
17:34:40 <lucasagomes> yeah, def we should see in nova how an interface for it would look like in parallel
17:34:56 * jroll wishes folks wouldn't complain about other openstack project teams here :/
17:34:58 <lucasagomes> ok, so we can proceed from here, thanks for the ideas/suggestions/comments all
17:35:14 <NobodyCam> thank you for bringing it up lucasagomes
17:35:20 <jroll> yeah, thanks lucasagomes :)
17:36:09 <jroll> okay, next topic
17:36:15 <jroll> there's a couple RFE reviews, should be quick
17:36:27 <jroll> #topic RFE: Validate iLO SSL certificates in the iLO driver
17:36:44 <jroll> I wish this was called "use custom CA certs to validate iLO SSL certs", but oh well
17:37:03 <jroll> #link https://bugs.launchpad.net/ironic/+bug/1599710
17:37:03 <openstack> Launchpad bug 1599710 in Ironic "[RFE] Validate iLO SSL certificates in the iLO driver" [Wishlist,In progress] - Assigned to Shivanand Tendulker (stendulker)
17:37:11 <jroll> this adds a ca_cert option for ilo communication
17:37:18 <stendulker> jroll: I can change it
17:37:20 <jroll> makes sense to me, any objections?
17:37:27 <jroll> stendulker: I'll comment on the review :)
17:37:37 <stendulker> jroll: thank you :)
17:37:51 <TheJulia> no objections here
17:37:52 <thiagop> nope
17:38:17 <JayF> jroll: not a complaint, just they're more busy than we are :)
17:38:23 <jroll> specs cores? JayF devananda rloo lucasagomes NobodyCam ?
17:38:38 <lucasagomes> no objections either
17:38:44 <NobodyCam> none here
17:38:47 <rloo> i'm fine
17:38:52 <jroll> cool, approving
17:39:10 <jroll> and the second RFE on the agenda was approved earlier today
17:39:10 <JayF> I have +2s on that spec already
17:39:12 <jroll> woo
17:39:25 <rloo> jroll: yeah, sorry, i forgot to delete that from the agenda
17:39:26 <JayF> Wait, a question then
17:39:34 <JayF> I thought you weren't supposed to approve an RFE that had a spec
17:39:36 <JayF> until the spec landed
17:39:40 <jroll> there's a spec for this?
17:39:42 <JayF> but this spec is blocked because the RFE isn't approved?
17:39:48 <jroll> it's a code patch
17:39:50 <jroll> not a spec
17:39:56 <jroll> afaict
17:40:00 <JayF> lol, I'm not used to being core /o\
17:40:01 <jroll> https://review.openstack.org/#/c/338791
17:40:02 * thiagop remembers deadlocks
17:40:03 <jroll> lol
17:40:05 <JayF> saw my +2 and presumed spec
17:40:22 <lucasagomes> #link https://review.openstack.org/#/c/338791/
17:40:24 <vdrok> :D
17:40:26 <lucasagomes> it's code
17:40:36 <jroll> stendulker: posted comments, if you update that I'm happy with it
17:40:46 <jroll> okay, let's move on
17:40:47 <stendulker_> jroll: Will update it
17:40:49 <jroll> #topic open discussion
17:40:51 <jroll> stendulker_: thanks :)
17:41:08 <jroll> anyone have anything for open discussion?
17:41:21 <stendulker_> jroll: there is a config drive related RFE for quiet some time https://review.openstack.org/#/c/338791
17:41:43 <jroll> stendulker_: wrong link?
17:41:46 <JayF> stendulker_: that's a wrong link, I think? That's what we just looked at
17:41:47 <stendulker_> jroll: Would you please comment on fate of that as well
17:42:02 <stendulker_> jroll: https://launchpad.net/bugs/1599710
17:42:02 <openstack> Launchpad bug 1599710 in Ironic "[RFE] Validate iLO SSL certificates in the iLO driver" [Wishlist,In progress] - Assigned to Shivanand Tendulker (stendulker)
17:42:08 <thiagop> The dynamic allocation for OneView drivers code was refactored to ease the life of you guys reviewing it. If you'd be so kind as to take a look, it is much appreciated: https://review.openstack.org/#/c/286192/
17:42:11 <jroll> stendulker_: also wrong link?
17:42:16 <JayF> stendulker_: that's literally what we just talked about and approved?
17:42:31 <stendulker_> jroll: https://bugs.launchpad.net/ironic/+bug/1493328
17:42:31 <openstack> Launchpad bug 1493328 in Ironic "No Config drive support for whole disk images deployed using iscsi based deploy" [Medium,In progress] - Assigned to Shivanand Tendulker (stendulker)
17:42:47 <thiagop> btw, TheJulia: Thanks for noting that. I'm looking into it yet today.
17:43:00 <TheJulia> thiagop: no problem :)
17:43:01 <stendulker_> Corresponding review https://review.openstack.org/#/c/230924/
17:43:03 <jroll> stendulker_: really?! O_o
17:43:34 <jroll> wow, how did we miss that
17:43:34 <stendulker_> jroll: review raised since Oct'15
17:44:19 <JayF> stendulker_: I'll take a look at that patch today
17:44:20 <jroll> stendulker_: sounds like a bug to me, no need for RFE approval
17:44:22 <jroll> but
17:44:30 <jroll> would cores help push that through please?
17:44:33 <jroll> that's a big miss :(
17:44:40 <stendulker_> it touches all ironic-lib, ironic and ipa
17:44:47 <jroll> yeah
17:44:47 <stendulker_> all three patches in review
17:44:56 <jroll> stendulker_: can you link the other two in the bug?
17:45:15 <devananda> oh - ouch
17:45:18 <stendulker_> jroll: they are
17:45:31 <stendulker_> jroll: same bug id being used
17:45:38 <jroll> stendulker_: I only see two, 230924 and 225115
17:45:54 <devananda> it looks like last update to the ironic patch was 6 weeks ago?
17:46:28 <stendulker_> jroll: ipa is not linked. Will do that https://review.openstack.org/#/c/296466/
17:46:31 <jroll> yeah, that could use an update
17:46:34 <jroll> stendulker_: thanks
17:46:51 <devananda> stendulker_: if you can update these, I'll add them to my review queue
17:46:52 <stendulker_> ironic lib is a dependency
17:46:53 <jroll> stendulker_: so if you can get the ironic and IPA patches updated, I'd love to get this done soon
17:46:56 <jroll> ah right
17:47:07 <stendulker_> ironic patch cannot merge without ironic lib
17:47:07 <jroll> okay let's push the ironic-lib patch through, and I'll release it
17:47:08 <JayF> I'll look over that ironic-lib patch today
17:47:25 <JayF> this aligns with our desire to get those shell scripts outta ipa too
17:47:27 <stendulker_> ironic lib-> ironic -> ipa
17:47:28 <JayF> it's a win all the way around
17:47:33 <stendulker_> is the dependency list
17:47:43 <devananda> also, is it worth setting up gate job for this configuration, perhaps limited to ironic-lib or IPA patches?
17:48:16 <jroll> devananda: I think someone is working on whole disk image + iscsi CI
17:48:23 <jroll> and configdrive is always enabled in CI
17:48:26 <devananda> cool
17:48:56 <stendulker_> Also any help on thsi short spec would be useful https://review.openstack.org/#/c/230274/
17:49:11 <stendulker_> https://launchpad.net/bugs/1526755
17:49:11 <openstack> Launchpad bug 1526755 in Ironic "[RFE] Baremetal provisioning using UEFI secure boot mode" [Wishlist,In progress] - Assigned to Shivanand Tendulker (stendulker)
17:49:37 <stendulker_> Few cores have already reviewed it.
17:50:02 <devananda> stendulker_: are there plans for other drivers to support uefi-secure-boot ?
17:50:24 <stendulker_> devananda: do not know
17:50:26 * devananda looks towards the other driver maintainers
17:51:12 <thiagop> devananda: not in short term for OneView
17:52:05 <jroll> hm, seems like nova shouldn't have anything to do with the boot mode for the deploy ramdisk
17:52:26 <jroll> I see others are okay with this thuogh, so someone correct me if there's a reason for that
17:52:34 <stendulker_> secure boot enabled deploy ramdisk support is already present in DIB
17:52:51 <devananda> stendulker_: the spec seems to indicate that secure boot is a property of the Nova flavor
17:53:13 <devananda> stendulker_: but doesn't this also depend on the user image?
17:53:19 <stendulker_> devananda: yes. That is already there when iLO drivers enabled secure boot
17:53:23 <jroll> devananda: this is about the deploy ramdisk
17:53:36 <devananda> ah. gotcha
17:53:38 <stendulker_> devananda: yes. user image also needs to be secure boot enabled
17:53:46 <jroll> which makes me wonder why nova cares :)
17:53:50 * jroll will take it to the spec
17:53:55 <stendulker_> devananda: partition image creation support is available in DIB
17:54:11 <stendulker_> devananda: whole disk image creation patches under review in DIB
17:54:31 <stendulker_> jroll: thank you
17:55:17 <jroll> any other topics? 5 minutes left
17:55:21 <NobodyCam> just a quick request for reviews on https://review.openstack.org/#/c/272658
17:55:23 <NobodyCam> :;p
17:55:37 <devananda> I'll comment on the spec too. I'm not sure why this is exposed to users orthogonally to the security of the instance image
17:56:28 <stendulker_> devananda: thank you for the review
17:58:37 <NobodyCam> Great meeting Thank you all :)
17:58:54 <jroll> yep, thanks all
17:58:56 <jroll> #endmeeting