14:59:53 <scottda> #startmeeting cinder_testing
14:59:53 <openstack> Meeting started Wed Aug 10 14:59:53 2016 UTC and is due to finish in 60 minutes.  The chair is scottda. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:59:54 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:59:56 <cFouts> o/
14:59:56 <openstack> The meeting name has been set to 'cinder_testing'
14:59:57 <ntpttr__> o/
15:00:00 <smcginnis> scottda: Shoot, I still have a todo from the last meeting I haven't done. Sorry.
15:00:04 <xyang> hi
15:00:09 <patrickeast> Hi
15:00:23 <eharney> hi
15:00:26 <DuncanT> Hi
15:00:58 <_alastor_> hi
15:01:10 <akerr> o/
15:01:12 <scottda> smcginnis: That's ok, since you haven't put up a devref patch to define the tests, we can Discuss the definitions some more today!!
15:01:22 <smcginnis> scottda: ;)
15:01:23 <scottda> I'm kidding, of course.
15:01:32 <scottda> #link http://gorka.eguileor.com/manual-validation-of-cinder-aa-patches/
15:01:53 <scottda> I've been playing with geguileo 's manual tests for HA-ActiveActive ^^^
15:02:24 <scottda> Good stuff there, if anyone else wants to play around with it...
15:02:58 <scottda> I intend to start scripting some of that, probably today. I'll post a link to github when I'm going...
15:02:59 <openstackgerrit> Tom Swanson proposed openstack/cinder: Dell SC: Requests async header is invalid  https://review.openstack.org/353551
15:03:08 <e0ne> hi
15:03:36 <scottda> e0ne: I think this week's Thursday QA meeing is friendly to your TimeZone? Can you attend?
15:03:53 <tommylikehu> time to start
15:04:03 <e0ne> scottda: yes, I'm going to addend it tomorrow
15:04:17 <Swanson> Hi All. Requests updated to 2.11.0 and smote my driver.  https://review.openstack.org/353551 adds two 's to fix it. Quick review?
15:04:21 <scottda> e0ne: Looks like 0900 UTC
15:05:56 <scottda> e0ne: It would be great if you could get some QA core eyes on this for adding multi-backend list to tempest.conf: https://review.openstack.org/#/c/325895
15:06:14 <scottda> That has one core +2, so hopefully it is close/ready
15:06:14 <e0ne> scottda: ok. added to my list
15:06:39 <jbernard> i would like to raise awareness of rbd replication patch https://review.openstack.org/333565
15:06:51 <jbernard> it's not quite done yet, but the big pieces are there
15:07:03 <scottda> We
15:07:07 <scottda> We
15:07:18 <jbernard> any feedback and/or testing is much appreciated
15:07:33 <scottda> We've no real agenda for today...does anyone have anything about cinder testing to discuss?
15:07:55 <geguileo> jbernard: Any instructions on how to easily test it?
15:08:43 <geguileo> scottda: I will be updating the HA A/A patches later today, as I've seen a problem with queueing of retype and migrate
15:08:45 <jbernard> geguileo: I will get those posted
15:09:04 <e0ne> jbernard: what ceph version is required for this feature?
15:09:18 <geguileo> scottda: I'll update the .conf file in my site to point to the latest patch chain
15:09:38 <jbernard> e0ne: >= jewel
15:09:52 <scottda> geguileo: Ok. I'll look for the updates and re-test
15:09:56 <patrickeast> Maybe something of interest, I added an experimental pipeline for my ci so if you comment "run Pure Storage experimental" it will run with multi-node active active using a cluster of c-vols
15:10:09 <smcginnis> patrickeast: Getting all fancy now, huh? :)
15:10:15 <eharney> patrickeast: nice
15:10:26 <patrickeast> Smcginnis: haha something like that
15:10:38 <scottda> #info patrickeast added an experimental pipeline for my ci so if you comment "run Pure Storage experimental" it will run with multi-node active active using a cluster of c-vols
15:10:52 <xyang> patrickeast: how many c-vol nodes are in your cluster
15:10:54 <e0ne> jbernard: noted, thanks
15:11:03 <patrickeast> xyang: just two
15:11:06 <jbernard> e0ne: thanks
15:11:38 <scottda> patrickeast: Do you know offhand the status of patches to get multi-node into the upstream gate?
15:12:37 <patrickeast> Scottda: not really, there are some changes I had to make (linked on the testing etherpad)
15:12:45 <scottda> patrickeast: Does you experimental job pull in AA/HA from master, or does it grap some later patchset from in-review queue?
15:13:08 <nikesm> hi,
15:13:29 <patrickeast> scottda: it uses whatever is up for review, so to get the full effect run it on the last patch of the ha series
15:13:45 <scottda> patrickeast: Cool.
15:14:14 <patrickeast> Although my ci is borked right now... So wait like a day to try it out :p
15:14:20 <scottda> haha
15:14:22 <nikesm> self.backend_name = self.configuration.safe_get('volume_backend_name')  what will be the value of self.backend_name if volume_backend_name is not defined in cinder.conf
15:14:31 <ntpttr__> I have a quick question about testing rolling upgrades - if I had OpenStack running with say two of every cinder service, and upgraded them one at a time from mitaka to master running tempest in between each upgrade would that be good validation? Starting with the db upgrade as well
15:14:53 <ntpttr__> or is there any other step that would need to be done there
15:15:46 <DuncanT> ntpttr__: That sounds reasonable to me. You might need something more than tempest runs, since tempest cleans everything up between runs - something simple that creates some volumes, leaves some attached during the upgrade, etc
15:16:08 <ntpttr__> DuncanT: ah good idea, I can do that manually
15:16:20 <ntpttr__> or write some sort of script
15:16:47 <scottda> Does grenade do anything like that? create and attach volumes, then upgrade, and see if all is well?
15:16:54 <DuncanT> I always recommend a script, you rarely end up doing such tests once. It need not be complicated though
15:17:16 <DuncanT> No idea about grenade, not gotten into it, sorry
15:18:27 <scottda> ntpttr__: dulek may have more detailed test scenarios for rolling upgrade
15:18:46 <scottda> but I don't see him here.
15:18:47 <ntpttr__> scottda: do you know the status of the grenade tests right now? There's a patch up to get those in right
15:19:33 <ntpttr__> scottda: yeah, I'm going to be working with him some on this, I think I'm going to have a meeting with him on the topic sometime soon
15:19:41 <scottda> ntpttr__: grenade for rolling upgrades? I'm not sure
15:20:36 <scottda> I think the controversy is about how the test matrix can easily explode with something like rolling upgrades
15:22:19 <scottda> Other test things to discuss?
15:22:22 <DuncanT> I think what we need is a thorough test set we can run on demand, like the experimental queue
15:22:52 <DuncanT> It isn't reasonable to try to get good, detailed upgrade coverage in the normal gate
15:23:05 <ntpttr__> DuncanT: +1
15:23:08 <scottda> DuncanT: Agreed. And perhaps something that must be done in a local lab, if the tests take too long or take too many resources.
15:24:30 <DuncanT> Certainly it should be easier to start that way
15:25:55 <scottda> ntpttr__: Would you consider documenting your (and dulek 's ) test scenarios and samples? It'd be nice to have even the manual steps you are doing documented.
15:26:05 <openstackgerrit> Tom Swanson proposed openstack/cinder: Dell SC: delete_live_volume payload incorrect  https://review.openstack.org/350262
15:26:19 <ntpttr__> scottda: yeah, I'd be happy to
15:26:39 <scottda> #action ntpttr__ will document manual tests and examples for rolling upgrades
15:26:41 <ntpttr__> I'm still thinking about what deployment solution would be best for this sort of testing
15:27:20 <ntpttr__> we're looking at kolla because it has upgrades playbooks, but we'd have to hack it a bit to get it to stop after upgrading services to run tempest and do other tests
15:27:25 <openstackgerrit> Tom Swanson proposed openstack/cinder: WIP: Dell SC: Live Volume Autofailover  https://review.openstack.org/350782
15:27:59 <DuncanT> ntpttr__: I had some good luck with containers. The nice thing is, you only need to build the HEAD one, the rest can be (and even are better if they are) the same ones from previously
15:28:07 <Swanson> xyang, thanks!
15:28:23 <xyang> Swanson: np
15:28:32 <DuncanT> ntpttr__: I wasn't using kolla but it is what I'd look at now
15:28:58 <ntpttr__> DuncanT: cool, I'll give it a try and keep notes on how it goes
15:29:56 <scottda> What else, folks?
15:31:18 <scottda> We can be done, then. thanks everyone.
15:31:26 <scottda> #endmeeting