03:02:29 <Sundar> #startmeeting openstack-cyborg
03:02:30 <openstack> Meeting started Wed Jun 12 03:02:29 2019 UTC and is due to finish in 60 minutes.  The chair is Sundar. Information about MeetBot at http://wiki.debian.org/MeetBot.
03:02:31 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
03:02:34 <openstack> The meeting name has been set to 'openstack_cyborg'
03:02:51 <Sundar> #topic Roll call and agenda
03:02:51 <xinranwang> Hi all
03:02:58 <Sundar> #info Sundar
03:03:06 <wangzhh> #info wangzhh
03:03:10 <Biwei> #info Biwei
03:03:12 <ikuo_o> #info ikuo_o
03:03:34 <yikun> #info yikun
03:03:41 <Sundar> Agenda: https://wiki.openstack.org/wiki/Meetings/CyborgTeamMeeting#Agenda
03:03:43 <xinranwang> #info xinranwang
03:03:54 <Sundar> Anything to add to agenda?
03:04:27 <Sundar> #topic Cyborg CI
03:05:02 <Sundar> As we have said, we need two things to merge Nova patches: Nova functional test with fake Cyborg API, and tempest CI with fake driver
03:05:37 <Sundar> Taking the tempest one first, xinranwang and Biwei are working on the general framework and plugins.
03:06:13 <Sundar> For the fake driver, yikun has kindly agreed. But he needs help. I'll help as much as I know.
03:06:23 <Sundar> It would be great if others can also help him out.
03:06:43 <Sundar> yikun, please feel free to ask questions.
03:07:03 <Sundar> ANy comments or questions on fake driver and/or tempest?
03:07:18 <Li_Liu> #info Li_Liu
03:07:25 <ikuo_o> Thanks Yikun.
03:07:29 <yikun> I still don't know what is the real requirement of fake driver. I assume it's just a driver that will return empty or just pass
03:07:55 <xinranwang> Biwei and I are working on the tempest test case and general framework.  We will test them in real env, once the fake driver is implemented, yikun can use these testcases.
03:08:26 <yikun> and also, we should merge these two patch, :) https://review.opendev.org/#/c/658986    https://review.opendev.org/#/c/658987
03:08:38 <yikun> and then I can append the patch base on these two
03:08:42 <xinranwang> real env means an openstack env with fpga
03:08:49 <Sundar> yikun: The fake driver ill be a derivative of the generic driver, which will return some pre-defined driver OVO for discover()
03:09:52 <yikun> Sundar: ok, will summit a patch this week, and any detail comments can be happened in the patch
03:09:56 <Sundar> It can return success or NULL for other APIs.
03:10:21 <Sundar> yikun: Great, thanks. Please ensure the fake driver is added to the tempest repo.
03:10:25 <yikun> Sundar: got it
03:10:46 <Sundar> Now, for the Nova functional test
03:11:03 <Sundar> Can somebody take that up?
03:11:35 <Sundar> It will just mock the Cyborg APIs called by Nova -- get device profile, post/patch/delete ARQs
03:12:19 <yikun> “Please ensure the fake driver is added to the tempest repo.” not sure why it is tempest repo?
03:12:33 <yikun> I thought it should be in cyborg repo
03:13:22 <Biwei> yikun, there's a cyborg-tempest-plugin repo, living outside of the cyborg repo
03:14:12 <Sundar> The tempest plugins are going to call that fake driver.
03:14:22 <xinranwang> Sundar:  what should fake driver return exactly, the info returned by this fake driver will be used in nova to boot vm, right ?
03:15:19 <Sundar> xinranwang: Yes. Its discover() API should return a pre-defined device OVO object
03:15:26 <yikun> Biwei: OK, https://github.com/openstack/cyborg-tempest-plugin
03:15:43 <xinranwang> Sundar:  tempest has api test and scenario test, those 2 types only interact with API layers, I don't think tempest should call driver directly.
03:15:44 <Sundar> It could define a device with soem fake controlpath-IDs, 1 deployable and an attach handle, all with fake PCI addresses
03:15:51 <xinranwang> Biwei:  what do you think?
03:16:17 <xinranwang> Sundar:  yes, but what info stored in this OVO,  like pci address
03:16:29 <Biwei> tempest will not call driver directly
03:17:26 <Sundar> Can be totally bogus PCI addresses. The VM may not boot up without the hardware.
03:17:51 <Sundar> To quote Eric Fried: "It should be sufficient to wait for the instance to go ACTIVE (according to Nova) (which will happen even if the guest OS boot pukes) and scrape its domain XML to assert that the correct PCI devices are present."
03:18:56 <yikun> Biwei: how to call driver directly, shouldn't it call cyborg api and cyborg api call the fake driver
03:19:00 <Sundar> Biwei Yes. Not sure if the CYborg agent will call driver API in tempest env, as it happens in real env
03:20:19 <Biwei> Sundar in tempest test, we only need to call the cyborg restful APIs.
03:20:25 <Sundar> The API invocation need not even call the driver -- as long as the driver has reported the device(s).
03:21:03 <yikun> Biwei: "Sundar in tempest test, we only need to call the cyborg restful APIs."   < Yes, so the fake driver should be in cyborg repo, right?
03:21:27 <yikun> and tempest should do something config the cyborg to use the fake dirver
03:21:58 <Sundar> Biwei, if there were some way to set up the Cyborg database in tempest env, like I did with initial_setup.sh in pilot code, we may not even need a fake driver perhaps
03:22:27 <Sundar> Have you seen if tempest can run some script to set up the env?
03:22:37 <xinranwang> I have some discussion with others like Alex, Shaohe. We are not sure if the status will switch to active in that case. I test in local with fake device, it failed.
03:22:57 <Biwei> I haven't figured that out yet. I have done the steps including creating flavor and update the flavor with device and all working so far
03:23:39 <Biwei> but not sure how to do the database set up
03:24:44 <xinranwang> tempest run in an installed openstack env where already have cybrog db, every thing same as a real env
03:24:54 <Sundar> xinranwang: True, I think it goes to error state. However, if we can do 'virsh dumpxml' equivalent and look for the hostdev device, that should be good enough
03:26:20 <xinranwang> Sundar: yes, I agree, but nova api will return error directly to tempest. where should we got the xml from virt driver.
03:26:47 <Sundar> Can you run a comand like 'virsh dumpxml'?
03:27:04 <xinranwang> you mean in tempest?
03:27:08 <Sundar> Yes
03:28:49 <xinranwang> I am not sure if libvirt will really boot it, I wonder that libvirt may do pci
03:29:17 <xinranwang> I am not sure if libvirt will really boot it, I wonder that libvirt may do pci check first before boot a vm
03:29:48 <Sundar> I have tried to launch a VM inside a development VM. It will error out but the VM exists in error state as shown by 'openstack server list'
03:30:00 <Sundar> Then we can look at Nova logs
03:31:00 <Sundar> You are right, the instance won't launch -- so can't do 'virsh dumpxml'
03:31:02 <openstackgerrit> tonybrad proposed openstack/cyborg master: Move to releases.openstack.org  https://review.opendev.org/664774
03:31:24 <xinranwang> I know, the xml file is generate before calling libvirt. Have you run virsh list to see if the instance exists.
03:32:43 <Sundar> The 'virsh list' won't show the instance -- so we should scrap that idea. I used to look at Nova logs. But I also modify the LOG statement in Nova virt driver to dump the xml all the time
03:33:35 <Sundar> We should probably ask other projects that use hardware. I'll get back on this.
03:34:58 <xinranwang> Ok, we should find another way.
03:35:04 <Sundar> Even if it fails in a predictable way, we can look for that specific error in n-cpu logs and declare success.
03:35:36 <Sundar> OK, let's table it for now.
03:35:47 <Sundar> #topic Python 3 migration
03:35:48 <xinranwang> Yes, of course. I agree with you. That was what I did when I don't have devices lol
03:36:26 <Sundar> ikuo_o and all: are there any questions or comments on Py3 migration?
03:36:33 <yikun> The fake driver in my mind it's that: we implement a fake driver in *Cyborg driver*, and tempest set the CONF.agent.enabled_drivers to this fakedriver when test setup.
03:36:33 <yikun> And the tempest can test the case which the fake driver can complete, such as some cyborg API, but for Nova-cyborg intergration test, we should run in real env.
03:36:33 <yikun> [1] https://github.com/openstack/cyborg/blob/master/cyborg/agent/resource_tracker.py#L57
03:37:44 <ikuo_o> We just fixed stable/stein cord.
03:38:04 <ikuo_o> and will throw patches in some days.
03:38:16 <Sundar> ikuo_o: Great. Was that much work?
03:39:15 <Sundar> Will look for the patches.
03:39:30 <Sundar> #topic Cyborg client migration
03:39:42 <ikuo_o> Not so much, but we should fix continuously because new patches will come out.
03:40:16 <Sundar> ikuo_o: Can you please let us know where this stands? Do you need any help?
03:41:09 <ikuo_o> I will tell you If I post patchs, off cource, I ask you help if needed.
03:41:39 <ikuo_o> sorry, "this" means cyborg client migration?
03:41:44 <Sundar> Yes
03:42:10 <ikuo_o> We are discussing how to break down tasks.
03:42:36 <Sundar> OK. Good.
03:42:45 <ikuo_o> I think the tasks would be: 1) implement object open via API eg, device_profile, arq
03:42:53 <Sundar> #topic Cyborg specs and patches
03:42:59 <ikuo_o> 2) implement operation (CRUD) open via API
03:43:07 <ikuo_o> 3)implement call point from library users
03:43:13 <ikuo_o> 4)all the functional tests (toX)
03:43:29 <ikuo_o> any advices welcome!
03:43:34 <Sundar> ikuo_o: I would think the first step is to adopt openstack-sdk?
03:44:15 <ikuo_o> yes, It include adopting openstacksdk.
03:44:32 <Sundar> ok
03:44:46 <ikuo_o> we are preparing write storyboard, pls wait.
03:44:51 <Sundar> Re. Cyborg patches, I approved the generic driver and got it merged. Thanks, yikun
03:45:14 <Sundar> Also, approved a bunch of small patches on fixing opendev URL or email addresses
03:46:11 <yikun> Sundar: thanks, but we should first https://review.opendev.org/#/c/658986 first, :)
03:46:39 <Sundar> I have partial fixes for the pilot code comments. Unfortunately, I could not dedicate a few days to get this done completely.
03:47:37 <Sundar> yikun: Agreed. Missed that
03:48:14 <Sundar> Will refresh the pilot code this week with whatever I have.
03:48:33 <Sundar> @all, please do review the specs
03:48:59 <Sundar> Can we aim to get one set of reviews by this week?
03:49:36 <Sundar> #link https://review.opendev.org/#/q/status:open+project:openstack/cyborg-specs+owner:Sundar
03:50:56 <Sundar> wangzhh, Li_Liu, yikun, xinranwang, all: Can we aim to get one set of reviews for the specs by this week?
03:51:59 <ikuo_o> sorry, poor connection
03:52:25 <Sundar> NP, ikuo_o. I was asking if we can get the specs reviewed by this week.
03:53:04 <ikuo_o> ok, I will check logs.
03:53:21 <Sundar> #topic AoB
03:53:43 <Sundar> I think we can skip the Zoom call tomorrow, and use that time to get the specs reviewed
03:54:41 <ikuo_o> all right, thank you Sundar.
03:55:07 <Sundar> Thanks, ikuo_o and all. Have a good day. Bye.
03:55:10 <Sundar> #endmeeting