12:59:30 <efried> #startmeeting Weekly OpenStack driver meeting 12:59:31 <openstack> Meeting started Tue May 2 12:59:30 2017 UTC and is due to finish in 60 minutes. The chair is efried. Information about MeetBot at http://wiki.debian.org/MeetBot. 12:59:32 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 12:59:34 <openstack> The meeting name has been set to 'weekly_openstack_driver_meeting' 12:59:56 <efried> #topic cinder/iSCSI testing efforts 13:00:00 <efried> jay1_ How's it going? 13:01:14 <thorst> not sure if Jay sees it. I'll PM him. But there are issues we think with the v7k 13:01:23 <efried> I PMed him 13:01:28 <jay1_> Hey Eric, still getting connectivity issues 13:01:31 <thorst> so he's going to ask a team where we know it works to see if we can use that temporarily 13:01:40 <thorst> and maybe see what the difference is in the configs between the two 13:01:55 <efried> I talked to tjakobs yesterday (he was on vacation Friday) and he said he would be around today to help if we needed. 13:02:04 <efried> He had this working in his env, right? 13:02:10 <edmondsw> o/ 13:02:26 <jay1_> thorst: got the other SVC, checking the difference.. 13:02:27 <thorst> efried: I don't think he used a v7k 13:02:35 <thorst> I think he built a LVM iSCSI host 13:02:49 <efried> thorst Ugh, here's where my total ignorance comes into play. 13:03:03 <thorst> :-) 13:03:16 <thorst> there isn't just one storage in the world :-p 13:03:40 <thorst> what we've been told to test is storwize 13:03:45 <efried> Yeah, I (kinda) get that; what I don't have a grasp on is what piece of the puzzle we're getting stuck on. 13:03:50 <thorst> which I know PVC has done with iSCSI on KVM 13:03:57 <thorst> yeah, right now it looks like the storage side 13:03:59 <efried> If I'm debugging vif plugging, I don't care if I'm SSP or localdisk, kind of thing. 13:04:10 <thorst> the negotiation between the hosts (storage and compute) is failing 13:04:16 <efried> Okay. 13:04:16 <thorst> looks like its on the storage side 13:04:40 <efried> gfm was helpful the other day. Maybe we can ask him again. 13:05:01 <efried> and yfeng, and darosale. 13:05:10 <efried> Basically, let's get a lot of people involved and see who's useful. 13:05:12 <efried> Cause it won't be me. 13:05:23 <thorst> +2 13:06:02 <jay1_> Yeah.. ideally when I registered the ISCSI host it is still showing the ports as offline, host and svc are not able discover each other. 13:06:41 <thorst> jay1_: Yeah, that could be that the SVC you used (or v7k) only has one wire on it. 13:06:54 <thorst> which is why I'm hoping you can just use what the KVM team uses 13:07:01 <thorst> if it works on KVM it should work for PowerVM 13:07:06 <edmondsw> jay1_ I assume the word "ideally" was a mistake there? 13:07:36 <jay1_> typo :p 13:07:52 <edmondsw> :) np 13:07:57 <edmondsw> making sure I'm following... 13:08:00 <edmondsw> (sort of) 13:08:25 <chhavi> hi all 13:08:50 <chhavi> efried,thorst: iscsiadm discovery is failing on the setup 13:09:20 <efried> Hi chhavi. We were going to try to gather some SMEs to debug this. 13:09:21 <jay1_> thorst, efried: chhavi was also looking into this today. 13:09:49 <efried> Be nice to get it done first thing, so nobody in India has to stay up super late. 13:09:54 <thorst> ok. I think the next actions are clear 13:10:00 <thorst> try with the SVC we know is set up properly. 13:10:20 <efried> Else figure a way to hand it off to the US so we can get it ready for jay1_ to take over in the (India) morning. 13:10:29 <chhavi> yes, first this needs to be debug from the SVC side why the iscsiadm discovery for the iscsi target iqn is failing 13:11:28 <chhavi> another point efried, i think we should fix that as well in nova-powervm is if get_iscsi_initiator is not found, we should not pass the connector with the wwpn 13:12:10 <efried> That sounds like a thing tjakobs could look into? 13:12:18 <thorst> chhavi: why not? 13:12:24 <chhavi> if the connection-type is iscsi, there is no use of passing wwpn, because of that SVC is using that and providing different error logs 13:12:28 <thorst> the host can have both FC and iSCSI connectivity simulatenously 13:12:47 <thorst> it's up to the driver to decide what to return 13:12:53 <thorst> maybe the storwize code has a bug there 13:12:59 <thorst> I'd rather fix the storwize code... 13:13:49 <chhavi> thorst: from the nova side, if user had requested for the the iscsi attachment, and we are passing the connector, why one should send the wwpn 13:14:39 <chhavi> creating the connector is nova task 13:14:45 <thorst> chhavi: but when get_volume_connector is called from the nova manager you have no idea what the BDM's are. 13:14:53 <thorst> so you don't know what they're asking for - iSCSI or FC 13:15:16 <thorst> so you can't try to outsmart the driver. 13:15:37 <jay1_> Just verified on the other SVC which is being used with PKVM, there is no chap secret code set on it. It is blank. 13:16:17 <edmondsw> ugh... that doesn't sound good 13:16:18 <thorst> OK - we may need to work with chhavi and gfm to continue 13:16:34 <edmondsw> which SVC is that? 13:16:39 <chhavi> when i tested with iSCSI earlier also, i never set the CHAP secret code, and i got the iscsiadm as well. 13:17:39 <edmondsw> we should be testing with appropriate security setup, to match a real customer environment. 13:17:47 <chhavi> shall i try to remove the chap secret and retry, if the iscsiadm discovery works. 13:18:06 <chhavi> edmondsw: before we test with CHAP enabled to rule out 13:18:35 <edmondsw> chhavi sure, I'm not saying we have to *only* test with security setup, but we should at least *also* test with security setup 13:18:41 <edmondsw> if not only 13:18:56 <chhavi> yeah, i know 13:18:56 <edmondsw> i.e. testing without security is optional... testing with security is not 13:19:23 <thorst> so, I agree...but lets walk before we run here? 13:19:25 <chhavi> edmondsw: one more thing do you know how to configure CHAP for username/password 13:19:33 <thorst> lets get the negotiation going and then make it more complex 13:19:39 <thorst> we're fighting enough problems already here. 13:19:49 <thorst> once we solve those, we can add the complexity 13:20:02 <edmondsw> thorst right, my horror is more that the pkvm testing was doing this... not powervm 13:20:10 <thorst> sure 13:20:25 <edmondsw> chhavi sorry, no, I don't know how to set that up 13:20:28 <thorst> I just don't want jay1_ or chhavi going down the rabbit hole and delaying this stuff. That is a parallel separate thread 13:20:34 <chhavi> i removed the chap secret for now to see 13:20:56 <edmondsw> thorst yes, I'll ping gfm about it... if someone will tell me which SVC this was 13:21:12 <efried> Agree we should try to get it working at all first. Then get it working right. 13:21:24 <edmondsw> we all agree :) 13:21:28 <thorst> k 13:21:46 * efried tries to figure out how to use meetbot's #agreed tag... 13:22:24 <edmondsw> so... jay1_ or chhavi, which SVC did you get from the pkvm testers that didn't have CHAP setup? 13:22:29 <efried> #agreed Get it working first. Then get it working right. 13:22:41 <efried> (guess we'll see how that shows up in the minutes) 13:22:45 <efried> (docs are not helpful) 13:22:59 <jay1_> edmondsw: you want the Ip of it ? 13:23:10 <edmondsw> sure... something I can use to identify it when I talk to gfm 13:23:26 <jay1_> sure.. let me ping that to you. 13:23:28 <edmondsw> tx 13:23:37 <chhavi> ping me as well 13:24:03 <jay1_> sure folks sending this to you all. 13:24:56 <esberglu_> o/ 13:26:52 <efried> We done with that topic? 13:27:08 <thorst> yep 13:28:06 <efried> #topic In-tree driver 13:28:44 <efried> No movement since last week. Hesitant to poke mriedem/sdague. Don't want to be *too* squeaky. Thoughts? 13:29:08 <thorst> I'd hold off for a few more days 13:29:14 <thorst> after that powervm binge :-) 13:29:20 <thorst> (which was awesome) 13:29:36 <thorst> or, maybe just ask them if you should hold off for a few days 13:29:39 <efried> "a few more days" will take us to forum. 13:29:41 <thorst> I mean, doesn't hurt to ask 13:31:00 <edmondsw> you already have Sean's +2 on https://review.openstack.org/#/c/391288/ and https://review.openstack.org/#/c/409402/ 13:31:12 <efried> Yes. 13:31:25 <efried> Sigh 13:31:42 <efried> #action efried to poke mriedem about https://review.openstack.org/#/c/391288/ and https://review.openstack.org/#/c/409402/ 13:32:20 <edmondsw> I think the other 4 reviews all still need work, right? 13:32:29 <efried> Yes. 13:33:26 <efried> The SSP one doesn't need work. Just a stupid xenserver recheck, and re-look from cores. 13:33:34 <efried> Wouldn't mind in-team +1s there. 13:33:59 <efried> Previous patch set was +2ed by sdague; latest patch set was just a (manual) rebase. 13:34:03 <edmondsw> efried I added myself 13:34:12 <efried> Oh, were you not on there? Sorry. 13:34:22 <edmondsw> nope... I'll try to look later today 13:34:30 <efried> thx 13:35:01 <edmondsw> do we need to say anything about the other 3, or next topic? 13:35:17 <efried> I'm going to wait til after the forum to get re-cranking on those, I think. 13:35:24 <edmondsw> +1 13:35:34 <efried> Unless by some miracle all three of the ready ones get merged, like, today. 13:35:51 <efried> Anything else in-tree? 13:36:13 <thorst> how's the non-powervm driver stuff going? 13:36:31 <edmondsw> thorst non-powervm? 13:36:32 <efried> #topic OOT driver 13:36:44 <efried> Oh, non-powervm. 13:36:49 <thorst> I'll follow up later. Its not powervm related...but the service BP 13:36:54 <thorst> we can chat outside the meeting 13:37:00 <efried> thorst Yeah, let's do that. 13:37:02 <edmondsw> oh, the catalog stuff 13:37:11 <efried> Yaknow, since this is the PowerVM driver meeting. 13:37:17 <thorst> ;-) 13:37:19 <edmondsw> :) 13:37:20 <efried> So OOT driver. 13:37:30 <efried> I reviewed esberglu_'s patches yesterday. Good start, a few comments. 13:37:55 <efried> https://review.openstack.org/#/c/461147/ https://review.openstack.org/#/c/460331/ 13:38:06 <efried> edmondsw Might as well add you to those. 13:38:08 <thorst> I'll review today 13:38:18 <thorst> and agree with edmondsw being on review 13:38:22 <edmondsw> +1 13:38:25 <efried> done 13:38:36 <esberglu_> efried: Yep gonna fix those up. Gonna continue to work on the backports from in-tree this week 13:38:43 <edmondsw> are any of these for carrying our in-tree changes forward into the OOT driver? 13:38:51 <efried> edmondsw Yes, both of them. 13:38:55 <edmondsw> awesome 13:38:57 <efried> And the pending ones esberglu_ mentions above. 13:39:54 <efried> Any other OOT work in the offing? Bugs? Blueprint work? Do we want to talk about the SR-IOV metrics? 13:39:56 <thorst> I know that there is one patch that we need to review for svenkat today 13:40:02 <efried> thorst link? 13:40:22 <efried> Not on my list (at least not if svenkat is the author) 13:40:28 <thorst> I don't see it, he just brought it up in a call but I don't see it anywhere in the queue 13:40:49 <thorst> ahh, I see it now 13:40:58 <thorst> https://review.openstack.org/#/c/460672/ 13:41:05 <thorst> ymadhavi proposed it on their behalf 13:41:07 <thorst> you +2'd 13:41:09 <thorst> so I'm on point here. 13:41:21 <svenkat> yes.. i added my +1 to it 13:41:22 <thorst> nevermind! 13:41:36 <efried> Added edmondsw 13:41:38 <efried> for form's sake 13:41:54 <esberglu_> I don't know if you saw in slack yesterday but the SRIOV utilization data is currently available from phyp 13:42:32 <efried> esberglu_ Needs to be added to REST, right? 13:42:37 <efried> changh on the hook for that? 13:43:37 <edmondsw> I see 2 changes with the same subject and associated bug: the one above and https://review.openstack.org/461653 13:43:57 <edmondsw> why is that? 13:43:58 <esberglu_> efried: I was talking to him about it. I can finish up that convo today, got distracted by CI stuff yesterday 13:44:19 <efried> edmondsw Is the latter the ocata cherry-pick? 13:44:25 <efried> yeah 13:44:47 <edmondsw> efried ah, yeah... so we're not waiting for it to merge first 13:45:01 <efried> edmondsw Waiting for what to merge? 13:45:13 <efried> oh, waiting for master to merge before cherry-picking? 13:45:14 <edmondsw> the master change, before backporting 13:45:23 <edmondsw> not a big deal 13:45:26 <efried> Nah, this one's pretty trivial, we're pretty confident it's not gonna change since the +2. 13:45:37 <edmondsw> sure 13:46:03 <edmondsw> especially since thorst just +W'd master :) 13:46:24 <efried> #action thorst +W https://review.openstack.org/461653 13:46:39 <efried> Anything else OOT? 13:47:23 <efried> #topic CI 13:47:23 <thorst> nada from me 13:47:25 <efried> esberglu_ Go. 13:47:29 <esberglu_> CI died last night 13:47:37 <esberglu_> Well not died but the runs are all failing 13:47:55 <esberglu_> Looks to be cell related (which I still don't really understand that wel) 13:48:24 <esberglu_> HostMappingNotFound: Host 'powervm-ci-powervm-devstacked-4112' is not mapped to any cell 13:48:42 <esberglu_> I think we should be able to just do that nova-manage discover_hosts thing hopefully 13:48:51 <esberglu_> Gonna start debugging that right after the meeting 13:49:07 <efried> k 13:49:15 <efried> Couple other things to keep an eye on: 13:49:29 <esberglu_> Other that that there are some tests that are failing on and off. I think after the summit I'm gonna devote a week/sprint 13:49:31 <efried> Live migration tests started failing last night. We probably have those disabled. 13:49:57 <esberglu_> to debugging those intermittent failures and trying to work on the temp. disabled tests in the skip lists 13:50:07 <efried> http://lists.openstack.org/pipermail/openstack-dev/2017-May/116160.html 13:50:32 <efried> Also take a look at https://review.openstack.org/#/c/461716/ to see if it affects us. 13:51:28 <esberglu_> efried: Yeah I tried a manual run with that the other day. It failed stacking, but looked like a problem connecting to git.o.o 13:51:59 <efried> esberglu_ Okay, I guess keep it on yer watch list. 13:52:03 <esberglu_> Should be able to try again in the background today 13:52:15 <efried> Anything else CI? 13:52:56 <esberglu_> Nope 13:53:03 <openstackgerrit> Merged openstack/nova-powervm master: Deallocate network on reschedule https://review.openstack.org/460672 13:53:16 <efried> #topic Open discussion 13:53:21 <efried> Anything? Anyone? 13:53:22 <thorst> esberglu_: keep fighting the good fight! Great job with the CI :-) 13:54:04 <esberglu_> Haha thanks 13:54:30 <edmondsw> efried, does that OOT network deallocate change need to make it into one of our in-tree driver patches? 13:54:30 <efried> Okay, if nothing else... 13:54:42 <edmondsw> or are we not far enough along there to need to worry about that? 13:54:45 <efried> edmondsw Not until we have networks in tree ;-) 13:54:49 <edmondsw> right 13:55:02 <edmondsw> I think one of your outstanding patches is for that, no? 13:55:12 <chhavi> thorst: does neo host network is configured to receive all packets 13:55:13 <efried> Yeah, good idea to slap a comment on https://review.openstack.org/#/c/422512/ 13:55:22 <edmondsw> will do 13:55:30 <chhavi> i am just wondering if the neo network can be an issue for iscsi discovery 13:55:45 <chhavi> as per the error it says the neo is not able to receive the packets 13:55:57 <efried> edmondsw Done. 13:56:10 <efried> #endmeeting