17:00:47 #startmeeting ironic 17:00:48 Meeting started Mon Jul 11 17:00:47 2016 UTC and is due to finish in 60 minutes. The chair is devananda. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:49 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:00:50 o/ 17:00:50 hellooo o/ 17:00:52 The meeting name has been set to 'ironic' 17:00:53 o/ 17:00:57 o/ 17:00:58 o/ 17:00:58 o/ 17:01:02 o/ 17:01:02 o/ 17:01:03 o/ 17:01:05 o/ 17:01:10 o/ 17:01:20 o/ 17:01:24 #topic announcements 17:01:29 o/ 17:01:40 o/ 17:01:41 hi 17:01:49 o/ 17:02:03 #info jroll and nobodycam are out / missing the meeting today, and asked me to run it 17:02:09 o/ 17:02:37 of course, our agenda is at the usual place in the wiki, for those who are new, it's here: 17:02:40 #link https://wiki.openstack.org/wiki/Meetings/Ironic 17:02:46 who said they were allowed to have a vacation ? 17:03:01 o/ 17:03:05 davidlenwell: who said they were on vacation? :) 17:03:05 JayF: welcome back :) 17:03:16 :D 17:03:23 rloo: touche 17:03:33 I don't have any announcements other than that, and didn't get a chance to touch base with jroll yet this week 17:03:41 anyone else have announcements? 17:03:45 rloo, hah 17:04:07 i think dtantsur mentioned that we released ironic-lib? 17:04:38 yep 17:04:46 newton 2 is this week 17:04:50 we've actually released everything today, except for ironic itself 17:04:53 and IPA as well no? 17:04:58 yes, IPA as well 17:05:18 jroll wants to hold ironic release until we get networking in 17:05:38 Which I believe is reasonable and I suspect will take the bulk of the meeting discussing 17:05:44 #info newton-2 milestone is this week 17:06:00 #info nova feature freeze is behind us now, and nova midcycle is next week 17:06:06 how does newton-2 milestone affect ironic (or does it?) 17:06:15 OneView CI is passing for some troubles since the enablement of tempest job for agent driver 17:06:19 rloo: it doesn't directly affect us -- we release independently of those 17:06:28 I'm debugging it right now 17:06:37 rloo, not directly, it just served us a reminder to check if we need a release 17:06:46 devananda: ok. but nova FF does; the console feature won't get in until Ocata. 17:06:50 so we took this chance and release some cool stuff :) 17:06:55 rloo: right 17:07:04 dtantsur: cool is good :) 17:07:13 ok, moving on 17:07:18 #topic subteam status reports 17:07:24 #link https://etherpad.openstack.org/p/IronicWhiteBoard 17:07:45 oh, one more thing to announce: https://review.openstack.org/340335 17:07:50 rloo: devananda: Same with rescue; going to miss until ocata 17:08:01 we're applying for a couple more tags based on our new shiny grenade testing 17:08:03 couldn't get the spec reviewed on the ironic side quick enough to even get the code going well :( 17:08:10 namely, supports-upgrade and follows-standard-deprecation 17:08:15 dtantsur: nice! (and finally)! 17:08:18 starting around line 80 in that 'pad, if you're responsible for one of the subteams and haven't already updated it, please do so now 17:08:22 dtantsur: oh! awesome 17:08:55 o/ 17:08:57 dtantsur: is the grenade job now voting/gating? 17:09:05 devananda, it is 17:09:06 yup 17:09:19 has been for a couple of weeks now, I think? We landed the voting before I left 17:09:21 Grenade for Inspector? Or Ironic? 17:09:33 I know Ironic is voting. Not sure if Inspector job is up and running or not. 17:09:36 jlvillal, ironic 17:09:41 jlvillal, inspector one is running and NOT voting yet 17:09:59 jlvillal, we also have to fix grenade to be able to run our tests at all; currently it does not pick tempest plugins 17:10:19 dtantsur: that's awkward ... 17:10:40 devananda, see https://review.openstack.org/337372 17:11:18 heh 17:12:11 lucasagomes: i added something to the enhanced root device hints subteam report 17:12:20 rloo, heh I was about to write something similar 17:12:22 so thanks :D 17:12:26 lucasagomes: :D 17:12:49 notifications report has been updated. not much to report, just needs reviews, rebased it again today 17:13:01 JayF: on the rescue work, can we land all the ironic parts during Newton at least? 17:13:22 dtantsur: wrt py 3.5. Do you know if any/all of the ironic stuff runs in py 3.5? 17:13:35 devananda: I can't land anything until I get more reviews on the spec :( 17:13:40 rloo, we don't run on any pythons 3 yet 17:13:54 dtantsur: OH. I thought we ran on py 3.4. :-( 17:13:57 rloo, in a production sense of "run" ofc.. 17:13:59 rloo, dtantsur: fwiw, the services work in py35 on xenial (that's my current dev env) 17:14:01 however the client does not 17:14:02 dtantsur: should we open bugs against those or something? 17:14:03 devananda: the code is essentially done, just needs to be put up when the spec lands. The Nova changes are trivial enough to potentially get in post-freeze too, but right now the trajectory is that it'll be done around T :P 17:14:04 rloo: that;s unittests 17:14:35 vdrok: I thought it was more than unit tests. 17:14:42 devananda, do you have a bug for this 3.5 problem with client? 17:14:49 rloo, for now it's only the unittests... maybe in the future we can add some functional ones on py3 as well 17:14:54 dtantsur: haven't filed it yet 17:15:18 wrt the work for py 3.5 then, we should track it somehow. is it an RFE? 17:15:44 rloo: I'd say it's just bugs. no need for an RFE, we should just put up CI jobs and fix things until they pass 17:15:47 I think it's a bug 17:15:52 ++ 17:16:05 so, we have the unit test jobs for Py35, they seem green 17:16:08 ok, a bug then. Someone want to volunteer to file one unless we have already? 17:16:10 the difference between an RFE and a non-RFE bug are borderline trivial at this point anyway :) 17:16:18 JayF: exactly 17:16:29 and we don't need a bug to track that we have bugs ;) 17:16:32 rloo: I'd suggest the initial bug would be: Ironic should run integratino tests in a py35 environment 17:16:38 rloo, everyone who finds an issue should file a bug ;) 17:16:47 when those tests are done, we should file bugs about anything that makes them fail 17:16:52 so what i'm looking for is a list of tasks/things-to-do so we are py35. 17:16:52 JayF, it's not a great bug, until the whole openstack has it... 17:17:09 has = has it reported and working on it 17:18:51 seems like we've got all the status updates now, so, moving on 17:18:52 rloo: so I see unittests for both ironic and client are green 17:19:22 and we don't have any functional/integration tests on py3 17:19:35 vdrok: thx 17:19:38 thanks everyone for your work on / reporting status of all the subteams! 17:20:04 #topic is network_interface a hardware type 17:20:21 so is it? :) 17:20:27 rloo: this is your topic, however, sambetts and Sukhdev_ and I were also discussing it in the meeting just before this one 17:20:42 devananda: and ...?? 17:20:43 I don't think it is, because it can vary based on the environment 17:20:54 devananda, any consensus ? 17:20:55 but the desire was to keep it similar to minimize confusion 17:21:16 my concern is that keeping it similar, might cause more confusion. 17:21:35 or at least, they are different, and we need to be clear to show the difference. 17:21:57 TheJulia raised a good point that volume connector interface is going to be similar to network interface, in that it doesn't necessarily depend on the server or its OOB management device 17:21:59 and that is a possibility, and I do agree with you, however we need to decide and move forward since this also came up during the midcycle 17:22:22 but it also *might* depend on hardware type... 17:22:29 is the deploy interface similar too then? 17:22:30 dtantsur: indeed, it might 17:22:31 Yes, and only the deploying operator will know that 17:22:41 yeah, like with deploy and inspect interfaces 17:22:59 I'd prefer we treat the volume interface the same, but I'm fine with treating the network interface differently 17:23:21 +1 17:23:22 I personally think that a hardware_type should have some interfaces which can have defaults and some that can't but can list support for interfaces 17:24:01 e.g. volume_connector and network_interface don't have sane defaults but a hardware_type can say I support X, y and Z 17:24:03 dtantsur: aiui, the driver composition reform says that each hardware interface can specify a default, but doesn't allow for the operator to set a system wide default. if it did, would that solve this issue? 17:24:06 sambetts: that sounds reasonable 17:24:26 sambetts: but the hardware_type can't restrict the choice of the user 17:24:56 devananda, kind of. operator-driver defaults will increase inconsistency between clouds; and then for network_interface there is no vendor default 17:24:57 TheJulia: the spec says it can iirc :) 17:25:08 it can and will ;) 17:25:12 vdrok: :( 17:25:30 sambetts: without allowing for a hardware / vendor default, then the operator is forced to supply a choice -- the exact same choice, in many cases -- for every Node they enroll 17:25:58 devananda: right, so thats the problem with that solution 17:26:10 but it would remove all the confusion around those interfaces 17:26:11 vdrok: dtantsur: well, I guess a user could always define a new hardware type :) 17:26:13 well, it restricts the possible choices, but the user/operator can specify the actual interface implementation (one of the possible choices) for a particular node 17:26:18 TheJulia, of course :) 17:26:25 if the operator is allowed to set a config-level default for those *_interfaces, it would improve usability 17:26:55 for cases where ironic is running in a homogeneous environment (eg, all nodes should use Neutron for the network_interface) 17:26:56 devananda: what if they configure a default not supported by a particular hardware_type they support 17:26:58 devananda, but then users can no longer rely on vendor-provided defaults, right? because in every cloud an operator could override them 17:27:23 so enrolling a node with only providing "ilo-gen8" can mean different things on different clouds 17:27:34 we'd have to validate that default against all the hardware types 17:27:46 to ensure its a validate fall back too 17:27:50 valid* 17:27:53 sambetts, good point 17:28:03 what if there is no default, we only offer a default for those who choose or need the capability 17:28:06 it seems like we have two cases: 1. use vendor-provided default or disabled if not specified; 2. use vendor-provided default or if not provided, use system-wide config default. 17:28:06 are we talking about all the interfaces or some particular like network 17:28:35 vdrok: I'm thinking specifically of network and volume interface right now 17:29:06 rloo: I think #1, although I could see #2 being part of someone shipping a pre-configured/tested physical product 17:29:07 ah, so you only want this to affect these 2 interfaces, not things like "power" etc? 17:29:14 because, in most cases, these do not depend on the Node or its BMC 17:29:20 even though if we do it for "volume", I see point in doing the same for "deploy" 17:29:27 while this brings inconsistency, that makes sense 17:29:51 another solution I highlight in my comment in on spec, are that we make Ironic smart enough to try and validate via the validate function each of the interfaces that hardware_type lists until you get one that validates successfully 17:30:05 dtantsur: as far as enrollment behaving differently on different clouds -- we're debating whether to expose that difference to the operator doing the enrollment or not 17:30:09 and we make the validate functions smart enough to realise if they'll work in that environment or nto 17:30:25 sambetts, sounds very complex 17:30:27 devananda, hmm, how? 17:30:41 dtantsur: eg, in one cloud, I may need to pass network_interface=flat while in another cloud I will need to pass network_interface=neutron -- for every node 17:30:51 it's almost like we have a hardware_type 17:30:52 by renaming the field I guess :) 17:30:53 and an environment_type 17:31:10 because those are essentially the two axis that the drivers pivot on 17:31:14 dtantsur: OR I use the same enrollment code in each case, because the CONF is different in each cloud, and there is a default that works in each environment 17:31:19 devananda, ah yeah. I was asking because I assumed you propose operator defaults for all interfaces, not only volume/deploy/network 17:31:28 (do you?) 17:31:56 JayF: agreed, although some nodes may not have x environmental capability do to some $reason, so they can't be expected to apply against everything in inventory 17:32:05 JayF, OH :) it makes sense, but I'm scared to imagine how it might look like in the end 17:32:25 dtantsur: I'm suggesting we only do this for, as JayF has termed it, environment_types, because it doesn't make sense to me for the ilo_gen8 hardware type to dictate what sort of network provider I use with it 17:32:30 You'd almost have to have ops define environment_types in the API or in a config file for something like that to work 17:32:36 Well Ironic can calculate the environment right? 17:32:42 It just seems like those terms can help us talk about it more sanely, right? 17:32:47 e.g. hit keystone, see if neutron is there 17:33:04 sambetts: not entirely, there could be invisible physical constraints 17:33:14 sambetts: also ensure we're admin there? 17:33:14 or planned architectural constraints 17:33:38 sambetts: the keystone service list won't indicate whether neutron is configured with admin access to the TORs 17:35:05 so, we're going to end up with 2 kinds of *_interface fields, right? a new group will contain volume, deploy and network, IIUC 17:35:22 and this new group will use operator default, not vendor default, right? 17:35:49 dtantsur: can't the vendor_default value be 'use-operator-default'? 17:35:49 to boil this down to a concrete suggestion for the thing we're currently blocking the neutron integration work on -- should we add an operator-settable default value for network_interface? 17:35:59 so this is what I mean by using the validate function, e.g. flat network will validate in that case because it does need any extra info to opertate, but neutron interface won't validate because it requires local link information to validate so Ironic trys to validate the interface, and picks flat because it valdiates 17:36:08 rloo, I think it's a bit of overkill 17:36:14 devananda: IMO it's easy to say "no" now and add it later, but hard to say "yes" now and go back on that later 17:36:24 devananda: I think it can be added later on as an iterative change 17:36:26 devananda: so that means my vote would be no; but it's a very mild preference 17:36:48 JayF, then what happens when a user enrolls a node without network_interface? 17:37:01 if we don't have an operator-settable default value (via a config), does it mean it (network stuff) isn't useable? 17:37:16 no, we infer it from dhcp provider for now 17:37:49 but if someone writes a custom network interface implementation, they will have to set it in every node they need 17:38:12 Perhaps the solution should be this: network_interface and volume_connector interface, are part of the hardware_type but don't have vendor set default, those defaults are opertator set, and when the conductor loads the operator set default is validated against all the loaded hardware_types to make sure they all support it, so we can guarentee a lowest common denominator functionality 17:38:12 for me, whether we have the config or not, i want to make sure we think (or don't think) network interface is part of the set of hardware type interfaces. because if it isn't, we should rename it all so it doesn't have 'interface' in their names, etc. 17:39:48 rloo: network (and volume, etc) are python interfaces, loaded bu the Node's driver object 17:39:51 is it possible that a particular hardware_type might actually want to specify a network_interface or volume_connector interface? Ie, do we want to disallow vendor from setting a default? 17:40:01 rloo: I'm not sure how the distinction you're pointing out warrants renaming them 17:40:12 hardware_types need to list supported interfaces for network/volume_connector 17:40:17 devananda: yes, but in the wording etc, we use 'interface'. if we think they shouldn't be part of hardware type interfaces, we might be better off calling them 'drivers'. 17:40:41 "hardware_types need to list supported interfaces for network" <- that's what I'd love to avoid 17:40:47 devananda: the distinction/wording (to me) is important to be able to clearly talk/describe about these things. 17:40:54 e.g. I have a hardware specific network_interface, that won't work on other peoples hardware, so I want to list it as a supported interface 17:41:10 rloo: +1 that's part of why I suggested the environment/hardware split 17:41:11 rloo: potentially except I think that would "look" awesome to a vendor, but to an operator it might be a headache 17:41:26 sambetts has an interesting point 17:41:43 sambetts: dtantsur devananda maybe have not-supported instead? :) 17:41:47 rloo: w/r/t distinction, I agree, although clear, concise documentation could also help reduce potential confusion 17:41:58 in some cases, the network_interface _is_ dependent on the phys hardware, and that list of supported network interfaces should be defined by the hardware driver / vendor 17:42:22 devananda, yeah, in the same fashion is deploy interface: it's generic for all in-tree hw types, but may be vendor-specific too 17:42:40 provided they are available upstream.. 17:42:54 vdrok: we can not define the set of not-supported 17:43:14 vdrok: if you do that then every hardware_type except my own will have to list my interface, which is horrible and won't work for out of tree 17:43:35 yeah, that;s a bad idea 17:43:38 which is why i was suggesting that network_interface is a hardware_type interface, but that we allow HardwareTypeFoo to specify default_network_interface as 'foo-net-impl' or 'use-operator-setting' 17:43:48 or 'None' for disabled 17:44:30 rloo: That is an interesting possibility, and could be done later as to not block the current work, at least I think it could be done later 17:44:56 rloo: I don't think we should allow to disable network interface, we have noop.. 17:44:59 rloo: then all my nodes depending on hardware_type will either have to be manually overidden or not 17:45:15 as i see it, most servers (and thus their HardwareType class) won't have any opinion on the current set of NetworkInterfaces (noop, flat, neutron) 17:45:18 they'll all work 17:45:46 except for specific hardware (eg, blades, some cisco things, etc) that will only work with their specific NetworkInterface class 17:45:55 if hardware_types just list supported_network_interfaces = [] and then we have an operaotr default which is validated to by the lowest common denomintor of all loaded hardware_types we should be fine right? 17:46:16 where 'use-operator-setting' is get the value from a config option. 17:46:47 heads up, I'm going to time box this discussion in 5 minutes, so there's room for open discussion too 17:47:08 I think sambetts suggestion is good 17:47:23 I also think it would be good from a validation standpoint 17:47:30 can we recap what we decided..? 17:47:36 ++ for a recap 17:47:48 sambetts: can you recap for us? 17:47:54 also everyone is free to propose an amendment to the driver composition spec ;) 17:47:55 I can see some possibile issues of it though, but I think for most cases it will work, and for the cases where it might not, the operator likely has an ironic expert on-hand 17:48:23 * rloo didn't think anything was decided 17:48:43 * TheJulia thinks we have different things we are talking about 17:48:50 I do not think we've reached a concensus here, but a couple ideas seem possibly good 17:48:54 (or different primary concerns rooted in the same place) 17:49:01 AIUI: leave the field as network_interface, allow hardware types to not set default_network_interface and use the config option, validating that this value is in supported_network_interfaces? 17:49:03 TheJulia: yea, and that concerns me a bit 17:49:30 * dtantsur would prefer a patch to the spec 17:49:49 possibly recap and continue on mail... 17:49:53 ok 17:49:56 lazy_prince, ++ 17:50:03 I'll review this and post a summary to the ML shortly 17:50:09 thank you devananda 17:50:15 thx devananda 17:50:25 * rloo thinks a short voice meeting would be useful after recap/ML if ML doesn't move fast enough. 17:50:31 * TheJulia agrees 17:50:37 * TheJulia actually perfers voice at this point 17:50:44 i don't want this holding up the network stuff for too long 17:50:44 * lazy_prince too agrees.. 17:50:50 agreed 17:51:12 #action devananda to post a summary of this discussion to the ML, since we didn't reach any consensus here, with possible voice meeting to follow 17:51:16 #topic open discussion 17:51:18 rloo: there does seem to be some consensus with keeping the name at least and no longer blocking 17:51:18 * Sukhdev_ likes the last few statements :-) 17:51:22 My suggestion is this: hardware_types list supported_network_interfaces = [], but no default network_interface, there is a default network_interface in the config file, validated on conductor load to be supported by all loaded hardware_type, node created by with overriding the network interface will fall back to this interface, but operators can still override a specifc node to a different 17:51:28 network_interface as long as its supported by the hardware_type and enabled 17:51:32 i have a topic related to ironic notifications that i'd like to bring up for open discussion 17:51:38 for context, ironic will optionally send notifications about certain events like node provision state and power state changes over the message queue when the code for it lands https://github.com/openstack/ironic-specs/blob/master/specs/approved/notifications.rst 17:51:46 when someone wants to add a new notification, should they go through the spec process, just make an RFE, or neither (just put up code)? 17:51:51 perhaps it should depend on the number of notifications that would be put on the queue during relatively normal operation 17:52:09 mariojv: is it configurable which notifications are emitted? 17:52:18 devananda sambetts: can we take a vote for what time works for folks for the voice call so that we can get these patches merged? 17:52:24 sambetts: good summary! so close to my topic change, sorry about that 17:52:29 somewhat; they're not configurable on a per-notification basis, but each notification has a priority 17:52:43 rloo: you can configure the minimum priority of notifications you'd like to see 17:52:56 rloo: so if you set the config option to error, you only get error and critical notifications 17:53:02 rloo: if you set to debug, you get all notifications 17:53:05 mariojv: I think just like anything else, it'd depend on the size/intensity of the change. I want us to think of notifications more like log messages than anything else; and as long as we manage the levels/priorities properly, it shouldn't have a major impact on folks 17:53:25 JayF: agreed. i think it's one of those "not a problem until it becomes a problem" type things 17:53:26 Sukhdev_: that's going to take up more meeting time. I'll add a doodle to the mail, but it'll have to be responded to quickly so we can set it up w/in the next two days 17:53:27 mariojv: so a new notification seems like a smallish feature. if it isn't configurable, they will show up. so it needs a reno? which means small RFE/bug. (no spec). My thinking. 17:53:45 true, maybe a reno would be good 17:53:55 devananda: let's set up the meeting for tomorrow. we need to move fast :) 17:54:18 rloo, I think we have a v2 meeting tomorrow already, no? 17:54:21 mariojv: I agree with JayF, it is entirely dependent upon the size of the change and should likely be determined in the review process 17:54:38 lucasagomes: not all of us would attend both meetings. should be ok. 17:54:39 mariojv: I would like to treat them like LOG messages 17:55:05 devananda: if LOG messages, then just a bug. 17:55:13 rloo: exactly 17:55:21 this sounds like a good enough consensus to me. thanks 17:55:28 "ironic doesn't LOG when $thing happens" // "ironic doesn't emit a notification when $thing happens" 17:55:36 don't want to interrupt too much, but quick FYI krtaylor wanted to ask for comments from Ironic driver owners on https://review.openstack.org/330270 so if you're a driver owner please take a look!! 17:56:04 mjturek1: thanks - it's open discussion time! you're not interrupting :) 17:56:10 :) 17:56:23 mjturek1: do you know if kurt sent out email about that patch? might be useful to do so. 17:56:43 rloo: not sure if he did, I'll recommend it tho 17:56:48 sambetts ^ cisco driver is mentioned in there 17:57:45 pretty sure I already adding the Cisco ones to there 17:58:13 I did they are @L3120 17:58:26 krtaylor: ^^ 17:59:44 something i wanted to ask but forgot. apart from the networking stuff, is there anything we might want in better shape before the nova midcycle next week? 18:00:15 we will need to continue at #openstack-ironic 18:00:19 Also, I'll be there along with jroll, so if you all have something nonobvious you want mentioned at the Nova mid-cycle, lmk 18:00:20 time's over here 18:00:26 thanks for the discussion everyone - we're just about out of time. 18:00:32 thanks! 18:00:32 thanks \o 18:00:33 see you next week! 18:00:35 see ya 18:00:43 #endmeeting