13:00:44 <baoli> #startmeeting PCI Passthrough 13:00:45 <openstack> Meeting started Tue Jan 28 13:00:44 2014 UTC and is due to finish in 60 minutes. The chair is baoli. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:00:46 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 13:00:49 <openstack> The meeting name has been set to 'pci_passthrough' 13:01:31 <heyongli> hi, 13:01:44 <baoli> Hi there 13:01:45 <irenab> hi 13:02:18 <rkukura> hi 13:02:51 <baoli> #topic https://wiki.openstack.org/wiki/Meetings/Passthrough#Agenda_on_Jan_28th.2C_2014 13:04:11 <heyongli> ijw, i saw your comments on my patch, but that's not new patch, i just restore it from abandon. just remind.. 13:05:00 <sadasu_> #nick sadasu 13:05:07 <baoli> Let's start with icehouse schedule 13:05:13 <heyongli> sure 13:05:43 <baoli> how long do we have for coding, unit testing, review, etc? based on the schedule? 13:06:10 <heyongli> one month? 13:06:25 <sadasu_> icehouse-3 is march 6th 13:06:39 <irenab> freeze? 13:07:01 <sadasu_> yes, same date for the freeze 13:07:11 <rkukura> neutron features need to be in review by 2/18 13:07:29 <irenab> so we have about 3 weeks? 13:08:29 <baoli> hmm, it's going to be tough given all the unit test cases we have to come up with in the same time 13:08:51 <baoli> rkukura, is there any leeway? 13:09:00 <sadasu_> yes, how are we splitting up tasks? 13:09:13 <rkukura> baoli: exceptions can be granted 13:09:17 <heyongli> very tight, for review in nova, every core is busy now. 13:09:31 <rkukura> idea is to have initial version of patch in review by then, not necessarily merged 13:09:42 <irenab> lets see what is minimal set of changes we need in nova for basic support 13:10:04 <baoli> rkukura, that's good. 13:10:22 <baoli> So it doesn't have to include everything, but to start with the review process 13:10:41 <sadasu_> does nova have a similar initial review date? 13:10:45 <irenab> if we manage most of the definitions via neutron port, what is required from nova? 13:11:15 <baoli> heyongli, can you put down details about the nova work breakdown? I agree with Irena about the minimal set we need from generic pci support 13:11:40 <heyongli> baoli, the date? 13:12:08 <baoli> Yongli, which date you are talking about 13:13:04 <heyongli> i just don't know what's your mean. yunhong and i can meet the review date. 13:13:43 <baoli> heyongli, that's great. But in order for the rest to be tested, we may need your patch earlier. 13:13:48 <rkukura> nova feature proposal deadline is same as neutron's, 2/18. 13:14:08 <baoli> I put a couple of dependencies:Dependencies on Nova Generic PCI-Passthrough Support of PCI attribute sriov_group in the PCI passthrough device list Support of PCI stats based on sriov_group 13:14:20 <heyongli> baoli: sure, will be release in 2 or 3 day, i think but without API. 13:14:22 <rkukura> nova blueprint approval deadline is 2/4 13:14:49 <heyongli> rkukura, this is the real problem. 13:14:51 <baoli> heyongli, that sounds terrific! 13:15:28 <rkukura> heyongli: Getting BPs approved? 13:15:37 <heyongli> baoli: but without API, pci flavor almost can not work now... we might need extend alias for this ? 13:15:47 <heyongli> rkukura, yes. 13:16:13 <baoli> heyongli, does the dependencies make sense? we don't need the flavor for now 13:16:54 <baoli> all we need is to tag the device with an attribute called sriov_group, and be able to scheudle based on that 13:16:59 <irenab> baoli: I may miss something, what is instead of PCI flavor? 13:17:03 <heyongli> baoli, that fine to me. 13:17:39 <heyongli> irenab: without flavor, we can not request the pci. but schedule and stats can be done. 13:18:00 <baoli> irenab, since we don't need to enhance the --nic option, we don't need the flavor. The request spec is created based on the sriov_group that is associated with the neutron port 13:18:29 <baoli> I thought I made it clear in the wiki from Jan. 27th 13:18:41 <heyongli> baoli, i miss sth? we don't need the --nic? or just for temp? 13:18:52 <irenab> baoli: so we need neutron port to get additional attribute sriov_group? 13:19:02 <baoli> heyongli, in the initial release, we don't need to 13:19:15 <baoli> in later release, we'll add that 13:19:26 <irenab> baoli: you probably did, just not sure I understood what is required to set and where 13:19:32 <heyongli> baoli, ok 13:20:03 <baoli> irenab, a device is tagged with the physical net that we have configured in neutron 13:20:09 <irenab> Can make a short summary of initial support to be sure we are all aligned? 13:20:12 <heyongli> another topic, the mark thing: i want it to like : find the device by flavor name or flavor spec, how do you think about this approch? this solution is quick for implement. 13:20:58 <baoli> heyongli, this doesn't make sense since I could have two ports coming from the same flavor 13:21:01 <irenab> any sriov related definition should be done on neutron net or port? 13:21:19 <heyongli> baoli: so it's identical, right? 13:21:38 <baoli> heyongli, I suggest to make it generic. Any uuid is fine 13:22:05 <baoli> heyongli, I described a simple approach in our original google doc. 13:22:14 <baoli> we can take it offline. 13:22:36 <heyongli> baoli: if you have idea, bring it up to a mail thread? ok i will i check it 13:22:52 <baoli> Irena, let me recap from the wiki in here: 13:23:07 <baoli> 1. configure physical net 13:23:22 <heyongli> baoli: ok let's us do in mail, offfline. 13:23:28 <irenab> baoli: may I interup with clarification questions? 13:23:41 <baoli> irenab, go ahead 13:24:17 <irenab> where do you mean to configure the physical network? physically, config files? 13:24:42 <baoli> as how we do it now in the plugin config file 13:24:57 <irenab> ok, go ahead 13:25:04 <heyongli> that's Ian's tag system in config file, the pci_information, i think. 13:25:25 <irenab> baoli: currentlyon neutron defs, right? 13:25:29 <baoli> so 2. neutron net-create <name>. this neutron network will be associated with the provider network 13:25:49 <irenab> physicla net from (1) ? 13:25:55 <baoli> this network will have the provider net name, seg id, seg type, etc. 13:26:19 <baoli> You can do neutron net-create <name> --provider-net name=, 13:26:26 <baoli> I forgot the exact syntax 13:26:41 <irenab> baoli: its fine, go ahead 13:26:52 <rkukura> list physical_network in neutron server's network_vlan_ranges config, and in openvswitch-agent's bridge_mappings config, then use --provider:physical_network when creating network 13:27:31 <baoli> openstack@devstack-16:~$ neutron net-show mynet +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | id | 94d7caea-2c8b-4cf0-835e-c2b3a6e2de7c | | name | m 13:27:55 <baoli> | provider:network_type | vlan | | provider:physical_network | physnet1 | | provider:segmentation_id | 202 13:28:12 <irenab> baoli: fine 13:28:15 <baoli> somehow, the copy/paste doesn't work. But that's from a neutron net-show 13:28:44 <baoli> so 3. neutron port-create --vnic-type direct <net> 13:29:06 <baoli> when you create the port, it's created from the network that is created in step 2. 13:29:12 <baoli> So a port is on a physical net 13:29:30 <irenab> baoli: assuming ML2, port is created with vif_type not bound 13:29:47 <baoli> 4: nova boot --flavor <> --image <> --nic port-id=<port-uuid-in-step-3> 13:30:10 <irenab> baoli: missing the nova config with regards to SRIOV 13:30:28 <baoli> since a device has been tagged with the phyiscal net, and we need a port from the same net in the 'nova boot' commande 13:31:06 <irenab> so in nova.conf in pci_white_list you put tag= physnet1 ? 13:31:13 <baoli> so step 0: define the pci whitelist so that each sriov port is tagged with theire physical net using the attribute srivo_group 13:31:50 <irenab> heyongli: is it supported by you patches? 13:32:01 <heyongli> irenab: yes, pci_infomation 13:32:12 <heyongli> attach any tag you want. 13:32:19 <baoli> In order to do that, the two dependencies I listed in the wiki need to be met 13:32:24 <irenab> heyongli: great 13:32:39 <baoli> irena, does it make sense overall? 13:32:51 <irenab> baoli: so the two dependencies resolve the nova shceduling part? 13:33:04 <baoli> irenab, that's what I believe 13:33:06 <irenab> baoli: I think yes, it makes sense. 13:33:37 <baoli> irenab, glad that we are on the same page 13:33:37 <irenab> so to take it to parts we need to make work: 13:34:09 <irenab> baoli: after few month on IRC, it should happen eventually :-) 13:34:36 <irenab> 1. neutron = add vnic_type 13:35:10 <irenab> neutron => add PCI related attributes to the port and get it back for port_create/update/show 13:35:21 <rkukura> I've only been on these IRC meetings for a week - Is the srivo_group an attribute of the flavor, or not? 13:35:54 <irenab> rkukura: it will be initially the tag per pci_device 13:35:57 <baoli> rkukura, sriov_group is a pci attribute, you then can define a flavor using the attribute 13:36:16 <baoli> but we don't need flavor in the initial release 13:36:48 <rkukura> I'm just trying to understand which part of this forces nova to schedule the instance onto a suitable compute node 13:36:56 <baoli> Those terms are described in ijw/yongli/yunhong's proposal 13:37:02 <irenab> nautron => have ML2 mech driver (or drivers) for SRIOV 13:37:10 <irenab> ^neutron 13:37:24 <heyongli> rkukura, refrence https://wiki.openstack.org/wiki/PCI_passthrough_SRIOV_support#find_specific_device_of_a_instance_based_on_pci_attr 13:37:48 <baoli> rkukura, you are right in that nova uses sriov_group to report pci stats, and schedule based on that 13:37:57 <irenab> is there anything else for neutron? 13:39:22 <baoli> Irenab, I think that's about it. I listed several things on the wiki. Another task is to implement the port-create new arguments 13:39:50 <baoli> rkukura, the new arguments for port-create make sense to you? 13:40:08 <irenab> baoli: I meant it by neutron => add PCI related attributes to the port and get it back for port_create/update/show 13:40:14 <rkukura> baoli: Which proposal? 13:40:37 <baoli> rkukura, heyongli provided the reference to the doc 13:40:41 <heyongli> irenab: i think this is plan B. 13:41:11 <irenab> baoli: is it? 13:41:22 <heyongli> irenab, as we talked, this is a long term goal, right? 13:42:01 <rkukura> I'm not seeing specific port attributes defined in that wiki 13:42:08 <irenab> heyongli: I am not talking on scheduling part, we just need to pass the chosen SRIOV device details to neutron to make it available for agent 13:42:30 <irenab> rkukure: it is mentioned in my doc 13:42:41 <irenab> #link https://docs.google.com/document/d/1RfxfXBNB0mD_kH9SamwqPI8ZM-jg797ky_Fze7SakRc/edit# 13:42:49 <heyongli> irenab, ignore my point, i miss-understood what you talk about. 13:43:10 <baoli> irenab, I kind of understand what you are talking about. The pci information will be available to neutron 13:43:35 <irenab> baoli: so we just need to clarify the list of attributes, its all. 13:43:39 <baoli> In this case, since the port is existing already, nova calls port_update with the pci infor 13:43:41 <irenab> for nova we have: 13:43:46 <rkukura> I'm still advocating using the binding:profile dict attribute for input params to neutron, and a new binding:vif_details dict attribute for output params from the bound MechanismDriver 13:44:20 <baoli> rkukura, I think that we've agreed on that? 13:44:28 <irenab> rkukura: agree with you 13:44:42 <rkukura> I'm hoping this new binding:vif_details attribute can serve the same purpose as the binding:vif_security attribute that has been separately proposed. 13:44:54 <irenab> rkukure: glad you drive it 13:44:56 <rkukura> So one generic dict for input, and one for output 13:45:41 <irenab> we just need to clarify for SRIOV whtt attriburtes will be passed in and out, since filled by nova 13:46:05 <irenab> baoli: so for nova side, there is: 13:46:05 <baoli> Irenab, let's do that offline. 13:46:16 <irenab> baoli: fine 13:46:21 <rkukura> sounds good 13:46:38 <baoli> We can continue afterwards from another channel. 13:47:01 <baoli> Heyongli, are you ok with me doing the nova SRIOV part? 13:47:16 <heyongli> baoli: sure. 13:47:28 <irenab> great, can do on #openstack-neutron. So for nova we have allocate_for_network and vif_driver 13:47:45 <heyongli> but, what the exatly the SRIOV part? the vif? 13:47:45 <sadasu> rkukura: thanks. What is the BP for that? 13:47:49 <baoli> Do we need a separate BP for that? In that case, we also need this new BP approved? 13:48:10 <rkukura> I'm happy to take on any of the generic ML2 and portbinding extension changes, and/or work with irenab on those. 13:48:11 <baoli> Heyongli, the wiki makes it clear 13:48:32 <irenab> rkukura: with you on this 13:48:51 <baoli> So on neutron, we have rkukura and Irenab 13:49:01 <heyongli> baoli: i see some basic pci passthrough part, that confuse me. 13:49:18 <sadasu> I am currently working on the cisco ml2 driver..but I am available also 13:49:20 <baoli> heyongli, can you be more specific? 13:49:27 <rkukura> sadasu: I don't think we have a BP specifically describing binding:vif_details yet, but we can create one if needed 13:49:27 <heyongli> Support of PCI attribute sriov_group in the PCI passthrough device list 13:49:47 <heyongli> this is done by my pci_information patch 13:49:50 <sadasu> rkukura: ok. 13:50:01 <baoli> Heyongli, we are assuming that we can use an attribute called sriov_group\ 13:50:12 <irenab> sadasu: I think we may need generic SRIOV Mech Driver and then extend it according to specific vendor stuff 13:50:12 <heyongli> ok, fine 13:50:16 <baoli> The rest is done by the generic pci passthrough 13:50:29 <baoli> sadasu, that's great 13:50:44 <rkukura> For ML2's binding:profile implementation, seems we want to allow certain changes when unbound but not when bound. I think we can handle this by calling into the bound MechanismDriver to validate binding:profile updates. Does that make sense? 13:51:11 <sadasu> rkukura: +1 13:51:37 <baoli> heyongli, the scope for the nova sriov can be pretty much derived from the POC patch I sent to you. 13:51:58 <heyongli> baoli: that's great, get it. 13:52:05 <baoli> heyongli, cool 13:52:12 <baoli> heyongli, back to the BP question. 13:52:25 <heyongli> ***pci_information support, add tag/extra information to pci devices. 13:52:25 <heyongli> pci_information = { { 'device_id': "8086", 'vendor_id': "000[1-2]" }, { 'e.group' :'gpu' } } 13:52:25 <heyongli> ***pci flavor define attr can be used in the pci flavor and how pci stats report it's pool 13:52:25 <heyongli> pci_flavor_attrs = ['e.group'] 13:52:25 <heyongli> ***pci schduler to support corresponding extra information. 13:52:26 <rkukura> irenab, sadasu: A base MD class might be useful, but lets make sure a mix of vendor's MDs can co-exist in a deployment, so registering separate MDs is probably the easiest way 13:52:40 <heyongli> i add this to meeting wiki for basic pci part support 13:52:44 <irenab> rkukura: agree 13:53:20 <irenab> I think we need some base level for getting all the PCI related port attributes, it maybe done via some utils, no need to duplicate code 13:53:30 <baoli> heyongli, I think that we need a separate BP for nova SRIOV since it's networking related. I guess that I will need to complete the BP and submit it for approval 13:53:52 <heyongli> baoli: sure, +1 13:53:56 <sadasu> irenab, rkukura: I thought the port type driver was fitting beautifully here 13:54:10 <sadasu> but some reason, the implementation of that one has stalled... 13:54:31 <sadasu> so lets assume that as a seperate enhancement that may not be available for icehouse 13:54:32 <irenab> baoli: maybe makes sense to split the VIF driver to separate bp 13:55:09 <rkukura> sadasu: Not sure what you mean by "port type driver"? 13:55:09 <irenab> sadsu: can you port a link to the bp or patch? 13:55:36 <baoli> irenab, we can do that too. But for now, the priority is to get it done 13:55:39 <rkukura> irenab, baoli: Is plan to enhance nova's GenericVIFDriver? 13:55:44 <sadasu> https://blueprints.launchpad.net/neutron/+spec/ml2-typedriver-extra-port-info 13:55:56 <irenab> rkukura: yes 13:56:02 <baoli> rkukura, from the existing implementation, that seems to be right direction 13:56:27 <sadasu> rkukura: yes 13:56:28 <rkukura> irenab, baoli: I agree 13:56:49 <baoli> Ok, a few minutes left 13:57:00 <baoli> Recap: 13:57:15 <rkukura> sadasu: Hoping we can get that BP into icehouse, but I haven't seen discussion of using it for SR-IOV 13:57:17 <irenab> baoli: I think we need to support different VIF types, so it will make it easier to have the common part separated and we can push VIF drivers in parallel. 13:57:52 <sadasu> rkukura: I didn't want to add another dependency in our existing effort.. 13:57:53 <irenab> baoli: so we continue on neutron IRC? 13:58:24 <sadasu> I think at this point I 'll go with your suggestion 13:58:27 <baoli> irenab, I am not sure I understand. Sure, let's continue in #openstack-neutron 13:58:42 <baoli> but recap: 13:58:59 <baoli> heyongli will have generic pci support patch available in a few days 13:59:07 <baoli> baoli work on nova SRIOV 13:59:19 <baoli> rkukura, irenab on neutron. 13:59:34 <heyongli> guy, i may not attend this meeting from tomorrow, yunhong will cover me, and i will occasionally check mail. 13:59:38 <baoli> sadasu is avaialbe as well. 14:00:09 <baoli> heyongli, that's fine. Can you convey the message from this meeting to other folks? 14:00:11 <heyongli> so , if remove my +8 time zone, you guy may have good time slot include yunhong here. 14:00:15 <irenab> baoli: please send the link to bp once you register one 14:00:32 <baoli> irenab, which bp? 14:00:38 <heyongli> baoli: sure, yunhong will cover me, and i will talk with him tormmory, to transfer my work. 14:00:49 <irenab> baoli: nova SRIOV 14:01:04 <baoli> irenab, I'm going to use the existing one 14:01:19 <baoli> continue in neutron IRC? 14:01:23 <irenab> baoli: ok 14:01:25 <baoli> #endmeeting