16:00:01 <smcginnis> #startmeeting Cinder
16:00:02 <openstack> Meeting started Wed Aug  3 16:00:01 2016 UTC and is due to finish in 60 minutes.  The chair is smcginnis. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:03 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:05 <openstack> The meeting name has been set to 'cinder'
16:00:06 <smcginnis> ping dulek duncant eharney geguileo winston-d e0ne jungleboyj jgriffith thingee smcginnis hemna xyang tbarron scottda erlon rhedlind jbernard _alastor_ bluex vincent_hou kmartin patrickeast sheel dongwenjuan JaniceLee cFouts Thelo vivekd adrianofr mtanino yuriy_n17 karlamrhein diablo_rojo jay.xu jgregor baumann rajinir wilson-l reduxio wanghao thrawn01 chris_morrell watanabe.isao,tommylike.hu
16:00:11 <e0ne> hi
16:00:14 <rhedlind> hi
16:00:14 <_alastor_> o/
16:00:14 <mtanino> hi
16:00:14 <erlon> smcginnis: hey
16:00:17 <ntpttr> o/
16:00:18 <jungleboyj> Hello.
16:00:18 <scottda> hey
16:00:18 <eharney> hi
16:00:19 <geguileo> Hi! o/
16:00:19 <smcginnis> https://wiki.openstack.org/wiki/CinderMeetings
16:00:19 <tommylikehu> hi
16:00:22 <adrianofr> hi
16:00:27 <cFouts> hi
16:00:32 <smcginnis> Hey all
16:00:38 <Swanson> hi
16:00:44 <baumann> hi
16:00:48 <xyang1> hi
16:00:54 <smcginnis> #topic Announcements
16:00:55 <flip214> hI
16:00:55 <merooney> hi
16:00:56 <patrickeast> hi
16:01:06 <hemna> mornin
16:01:18 <dulek_> o/
16:01:21 <tommylikehu> evening in asia
16:01:28 <smcginnis> Privsep support has been enabled in rootwrap, so our libraries can be released again.
16:01:35 <smcginnis> One requested for os-brick.
16:01:36 <dulek_> tommylikehu: We're using UGT. ;)
16:01:44 <hemna> smcginnis, w00t
16:01:46 <tommylikehu> ok~
16:01:51 <e0ne> great news!
16:02:02 <smcginnis> We'll probably need a cinderclient release some time soon as well.
16:02:19 <tommylikehu> +1
16:02:24 <sheel> hi
16:02:25 <smcginnis> I think we are about three weeks away from non-client library freeze.
16:02:38 <smcginnis> Then one more week for client library freeze.
16:02:55 <hemna> ok
16:02:58 <tbarron> hi
16:03:00 <e0ne> #link http://releases.openstack.org/newton/schedule.html
16:03:04 <smcginnis> So let me know if/when anything critical gets in either of those and we can make sure we have a release done to get it available.
16:03:11 <hemna> if we can get some more eyes on the windows connectors in os-brick then, that'd be nice
16:03:20 <hemna> I know Lucian has been trying hard to get those in for a while
16:03:24 <smcginnis> hemna: +1
16:03:49 <smcginnis> Other than that, the usual...
16:03:54 <smcginnis> #link https://etherpad.openstack.org/p/cinder-spec-review-tracking review tracking
16:04:06 <smcginnis> Let's try to get through those priorities.
16:04:25 <smcginnis> OK, that's all I have for announcements.
16:04:35 <smcginnis> #topic Removal of bdd driver
16:04:39 * bswartz is here late
16:04:41 <smcginnis> DuncanT: Take er away
16:04:53 <DuncanT> So this test has been failing since forever
16:05:04 <e0ne> DuncanT: not forever
16:05:15 <e0ne> DuncanT: we've fixed one issue last month
16:05:15 <tommylikehu> how long
16:05:17 <smcginnis> Quite a long time then. :)
16:05:18 <hemna> forever - 1
16:05:22 <DuncanT> Ok, well as far back as my search went
16:05:23 <hemna> so almost forever
16:05:30 <scottda> DuncanT: I've told you a million times, don't exaggerate
16:05:41 * bswartz lols
16:05:43 <DuncanT> I think it should be treated as any other failing CI
16:05:52 <e0ne> DuncanT: we have a fix for job in tempest and we're hitting with tempest team to get it merged
16:06:15 <e0ne> DuncanT: I can fix job today by skipping 4 tests in job config
16:06:17 <hemna> DuncanT, +1
16:06:21 <DuncanT> e0ne: Ok, can you post a link up to that, please?
16:06:24 <smcginnis> e0ne: Do you have a link to that tempest patch?
16:06:32 <e0ne> DuncanT: or get tempest patch merged
16:06:36 <e0ne> #link https://review.openstack.org/#/c/332066/6
16:06:37 <DuncanT> I'd rather see the skips added that the test ocnstantly failing, personally
16:06:46 <e0ne> last failure is not related to the patch
16:06:47 <eharney> this driver has always been an odd case since it has no way to support online snapshots, which means it can't pass all tempest tests AFAIK
16:07:38 <e0ne> eharney: it supports snapshots
16:07:50 <eharney> e0ne: i just looked at the code... it checks for status != 'available'
16:08:14 <e0ne> eharney: oh.. you're right about online snapshots
16:08:27 <e0ne> eharney: but is it minimum features set?
16:08:35 <smcginnis> If we can get the skips in there for the failing tests, at least we would then be testing most of the driver functionality until the tempest changes make it through.
16:08:45 <eharney> i don't know, i'm just pointing out for context that it's not like a quick fix is going to make this pass all tests
16:09:24 <e0ne> here is tempest regexp to run tests for job DEVSTACK_GATE_TEMPEST_REGEX="^(?!.*(volume_in_use|test_volume_boot_pattern)).*volume"
16:09:43 <e0ne> I'll add skip for current failures until tempest patch will be fixed
16:09:44 <jgriffith> eharney: true, and this has been the debate about this driver for a while.  It truly "can't" do a number of things
16:10:00 <e0ne> jgriffith: +1
16:10:23 <smcginnis> And just one project (thinks it) needs it, IIRC.
16:10:29 <jgriffith> I just have to ask....
16:10:36 <eharney> ...for performance reasons which don't have data behind them
16:10:37 <jgriffith> What's our policy now on drivers then?
16:11:12 <jgriffith> If you can't do everything, or pass tests you can just "skip"?
16:11:17 <smcginnis> jgriffith: Must meet the minimum requirements, but this is grandfathered in I guess.
16:11:23 <e0ne> eharney: just ask me if community needs performance tests data
16:11:38 <DuncanT> e0ne: We've asked for performance tests multiple times
16:11:45 <eharney> e0ne: i dunno about "needs", but i've never seen convincing evidence that thin LVM wouldn't work fine for this case
16:11:57 <erlon> smcginnis: Is online snapshots in the minimum requiriments? I don't think so
16:11:59 <smcginnis> eharney: That's my memory of past discussions.
16:12:05 <e0ne> eharney, DuncanT: ok, I'll ask our team to do performance testing for it
16:12:08 <smcginnis> erlon: No, online is not specified.
16:12:25 <eharney> smcginnis: well, not explicitly, but it is by the test suite
16:12:32 <smcginnis> e0ne: And show a difference between bdd and just using the LVM driver.
16:12:43 <e0ne> smcginnis: sure
16:12:54 <DuncanT> Online isn't specified currently. We should possibly make it explicit in future?
16:12:54 <jgriffith> e0ne: I've suggested in the past that a better direction may be to skip the tgt if the volume is local to nova
16:12:57 <hemna> if the drivers were out of tree.........
16:12:59 <hemna> :P
16:13:01 * hemna runs
16:13:11 <e0ne> hemna: :)
16:13:15 <jgriffith> e0ne: so that driver could be made to work in any case.
16:13:18 * smcginnis throws stuff at hemna
16:13:36 <e0ne> jgriffith:are you talking about skipping tgt got LVM?
16:13:44 <jungleboyj> hemna: Really?
16:14:02 <smcginnis> jgriffith: I like that idea.
16:14:03 <hemna> ssh
16:14:09 <jgriffith> e0ne: yes, modify LVM driver, such that if the VG is local to the compute node just export the dev file and do NOT create a target
16:14:33 <e0ne> jgriffith: interesting. we need to test performance for it
16:14:41 <jgriffith> e0ne: there's a good deal of logic needed to coordinate all of that on the Nova side
16:14:43 <e0ne> data I/O performance
16:14:46 <smcginnis> Seems a lot simpler than maintaining a whole separate driver.
16:14:49 <eharney> someone has proposed this and got a lot of grumpy replies, IIRC
16:14:52 <jgriffith> e0ne: the BS excuse I got last time was that LVM wasn't performant enough
16:14:55 <eharney> there's a spec somewhere for it
16:15:13 <jgriffith> eharney: are you thinking of the shared LVM thing maybe?
16:15:31 <bswartz> jgriffith: was the poor performance because it was thick perhaps?
16:15:33 <jgriffith> eharney: there was a proposal to do LVM on a SAN shared to all compute nodes
16:15:50 <e0ne> #link https://blueprints.launchpad.net/cinder/+spec/add-local-tgt-to-lvm
16:15:55 <jgriffith> bswartz: I don't think there was ever any data behind it
16:16:06 <eharney> https://review.openstack.org/#/c/247880/3
16:16:09 <erlon> eharney: FWIW, tempest is not testing online snapshots, the fail in bdd its another problem
16:16:20 <e0ne> #link https://review.openstack.org/#/c/247880/
16:16:20 <jgriffith> e0ne: nice find
16:16:21 <DuncanT> eharney: normally people propose removing the tgt completely from the driver (having an LVM local-only driver) which I give 'grumpy' replies to
16:16:22 <eharney> erlon: pretty sure that test_volume_boot_pattern tests that
16:16:25 <bswartz> sure but any anecdotes about poor performance could easily be because someone tried it with thick LVs
16:16:36 <eharney> bswartz: right
16:17:15 <tbarron> be nice to be able to configure it to do tgt even if local (useful for testing in all-in-one) but that's easy enough
16:17:27 <DuncanT> tbarron: +1
16:17:36 <erlon> eharney: hmm, I looked test_snapshots*, you are righ
16:17:37 <smcginnis> OK, so for the topic at hand, e0ne will skip tests while tempest patch is being worked and see about getting performance comparison to show why LVM can't just be used.
16:17:45 <e0ne> the spec above ^^ was -1'ed because of lack of perf tests result
16:17:54 <smcginnis> We can revisit in a week or two to make sure issues have been addressed.
16:18:03 <jgriffith> I"m sorry but I don't think your Nova Instance is going to be pushing any of this hard enough to make a performance difference but whatever
16:18:06 <e0ne> #actio so for the topic at hand, e0ne will skip tests while tempest patch is being worked and see about getting performance comparison to show why LVM can't just be used.
16:18:11 <smcginnis> But I think if it continues failing we do need to treat it like any other driver with long failing CI.
16:18:12 <e0ne> #action so for the topic at hand, e0ne will skip tests while tempest patch is being worked and see about getting performance comparison to show why LVM can't just be used.
16:18:21 <smcginnis> e0ne: Thanks
16:18:25 <e0ne> smcginnis: +1
16:18:36 <smcginnis> DuncanT: good?
16:18:36 <tommylikehu> +1
16:18:41 <DuncanT> Yup
16:18:49 <smcginnis> Thanks
16:18:55 <e0ne> we implemented CI for bdd last year and will try to get it working
16:18:55 <DuncanT> I was just fed up of seeing the test failing and no apparent progress :-)
16:19:13 <smcginnis> #topic Noncompliant driver removal
16:19:27 <smcginnis> So last week I put up patches to remove four drivers.
16:19:43 <smcginnis> Two of them have resolved their CI issues and are stable(ish) again.
16:20:05 <smcginnis> Of the two remaining, Scality has responded to the ML post and is working on fixing things.
16:20:07 <e0ne> do we verify how many tests are runned by 3rd party CI's?
16:20:18 <jgriffith> e0ne: no we don't
16:20:20 <smcginnis> Tegile has been absent,
16:20:37 <bswartz> e0ne: that would require standardizing the log format for CIs
16:20:37 <jgriffith> e0ne: at least not programmatically or anything
16:20:45 * jungleboyj can see people getting their pitchforks .
16:20:45 <smcginnis> e0ne: I occasionally spot check to make sure they aren't defaulting to LVM and that they are running the right tests.
16:20:54 <jgriffith> bswartz: which we did, and also added req for subunit report
16:21:02 <bswartz> o rly?
16:21:08 <e0ne> smcginnis: yea, I do the same during reviews
16:21:13 <bswartz> then it shouldn't be too difficult to automatically verify
16:21:14 <hemna> It would be nice if the tempest runs would dump out a stats.py at the end
16:21:25 <hemna> that can be consumed programatically
16:21:27 <smcginnis> hemna: Hmm, interesting.
16:21:37 <hemna> we could correlate that to the individual driver runs
16:21:45 <hemna> once we get https://review.openstack.org/#/c/348002/  in
16:21:56 <jgriffith> hemna: http://54.164.167.86/solidfire-ci-logs/refs-changes-44-347944-5/testr_results.html
16:22:04 <smcginnis> ANyway... I'm willing to give Scality a little more time if folks are OK with that.
16:22:08 <jgriffith> hemna: what about scraping that?
16:22:21 <smcginnis> They responded right away and are at least trying to be responsive.
16:22:26 <hemna> yah that might work
16:22:46 <smcginnis> But Tegile is MIA, so I think we've gone the declared grace period and this should be pushed through.
16:22:50 <hemna> that's generated by ostestr right ?
16:22:55 <jungleboyj> smcginnis: If they are responding and active I think that is fine.
16:22:57 <hemna> maybe it can puke out a dict as well ?
16:22:57 <smcginnis> Any objections to either?
16:22:58 <_alastor_> smcginnis: +1
16:23:08 <jungleboyj> smcginnis: +1
16:23:26 <hemna> ok Tegile is to go?
16:23:41 <smcginnis> Tegile to be removed, Scality to get a little more time.
16:23:47 <jgriffith> hemna: yep
16:23:49 <hemna> +A'd
16:23:54 <xyang1> smcginnis: +1
16:23:54 <DuncanT> smcginnis: +1 tegile +1 scality
16:23:59 <tommylikehu> nice
16:24:30 <bswartz> yeah too bad for Tegile, but making an example of somebody will help motivate others to keep their CIs compliant
16:24:35 <smcginnis> OK, good.
16:24:36 <erlon> smcginnis: +1 I think its ok, but I think it would be nice to include the CI maintainer's emails in the next removal warnings
16:24:41 * hemna peeks inside os-testr code...
16:24:47 <smcginnis> bswartz: Yeah, really is unfortunate, but ok well.
16:24:59 <smcginnis> erlon: I didn't include it on purpose.
16:25:10 <smcginnis> If they are not paying attention to their CI...
16:25:19 <erlon> smcginnis: o really?
16:25:20 <smcginnis> And not paying attention to patches proposed against their drivers...
16:25:24 <DuncanT> hemna: Maybe spit out the results as json in a comment in the HTML? That would be easy to parse without causing more files to be produced?
16:25:35 <smcginnis> And not paying attention to mailing list posts with their company name in the title..
16:25:53 <erlon> smcginnis: hmmm got it
16:25:56 <smcginnis> Then they are obviously not participating in the community nor making sure their support is goo.
16:25:59 <smcginnis> *good
16:26:04 <hemna> DuncanT, sure whatevs.  I just want some data that doesn't have to be screen scraped
16:26:06 <hemna> :(
16:26:15 <_alastor_> hemna: The number of tests is already pretty easy to scrape from the tempest output directly...
16:26:20 <smcginnis> So I'm not willing to chase these folks down anymore to try to get them to step up.
16:26:31 <erlon> hemna: have yu seem my comment in you patch?
16:26:53 <jungleboyj> I think this process has been going on for long enough that it shouldn't surprise anyone.
16:26:54 <hemna> erlon, hbsd stuffs ?
16:27:00 <erlon> hemna: yep
16:27:05 <hemna> I'm not sure what to do about that
16:27:07 <hemna> meh
16:27:07 <DuncanT> I, and even more so Mike, went to fairly heroic efforts in the past to chase people down. I don't think it's sane to keep doing so
16:27:30 <smcginnis> DuncanT: +1000
16:27:30 <_alastor_> DuncanT: +1
16:27:34 <hemna> DuncanT, +1
16:27:36 <jungleboyj> DuncanT: +1
16:27:41 <hemna> there really isn't a reason to.
16:27:48 <jungleboyj> Made more sense when it was a new requirement.
16:27:51 <smcginnis> We've spent way too much of our time trying to get companies to be good about this.
16:27:55 <hemna> if a maintainer is participating in the community, there shouldn't be any reason to chase at all
16:28:09 <e0ne> hemna: +1
16:28:11 <smcginnis> The reality is if they are not willing to put in minimal effort, there's no reason we need to work hard to help them out.
16:28:18 <smcginnis> hemna: +1
16:28:19 <DuncanT> It is a fair bit of effort just monitoring CIs and following up to see which ones aren't actually reporting... having to then chase people becomes soul destroying, and certainly isn't a good use of time
16:28:39 <smcginnis> DuncanT: I agree completely.
16:28:58 <tommylikehu> next one?
16:29:17 <smcginnis> Next what?
16:29:27 <tommylikehu> topic
16:29:57 <smcginnis> That's it on the agenda. Anyone else have anything? Or should I open it up to the bikeshedding portion of the program?
16:29:57 <hemna> _alastor_, the html is pretty repeptitive, I just think json would be safer in the long run
16:30:00 <e0ne> tommylikehu: do you have any topic more?
16:30:24 <tommylikehu> actually no~
16:30:26 <tommylikehu> new here
16:30:35 <smcginnis> #topic Discussion
16:30:46 <erlon> o/
16:30:51 <smcginnis> Anything else to discuss? Or should we end early?
16:30:58 * smcginnis is kind of hungry today...
16:30:59 <smcginnis> :)
16:31:01 <erlon> Just one point Id like to mention
16:31:04 <erlon> https://review.openstack.org/#/c/336092/
16:31:04 <tommylikehu> how about the mascot
16:31:10 <flip214> I'd like to post another big "Thank you" to all the core devs that do reviews! It's quite some work, and (I guess) we all appreciate it!
16:31:21 <smcginnis> tommylikehu: Oh, thanks for the reminder!
16:31:24 <_alastor_> hemna: I was talking about the tox output eg: tox -e all -- volume | tee -a console.log.out or whatever the file is normally
16:31:30 <smcginnis> flip214: Thanks!
16:31:39 <erlon> this is changing a lot of drivers, I need the mainteiners to give a a look
16:31:57 <smcginnis> I gave the survey results to the logo team. They will have something for us to look at soon I think.
16:32:04 <DuncanT> _alastor_: Unfortunately the location and format of that info is not consistent between different CIs :-(
16:32:08 <smcginnis> Horse/donkey was the clear winner.
16:32:09 <xyang1> erlon: how can you get all CI's to pass on that one patch?
16:32:10 <hemna> AssBrick !
16:32:36 <DuncanT> smcginnis: I was disappointed to see there wasn't a 'Stay as we are' option on the survey
16:32:39 <smcginnis> I made sure to share xyang1's concerns about having a donkey though, so I've been going with "general equestrian animal". :)
16:32:43 <erlon> xyang1: that actually toook several runs, not sure all CIs run on it tough
16:32:52 <smcginnis> DuncanT: I've certainly shared that as well.
16:32:55 <hemna> smcginnis, lame!
16:32:59 <flip214> erlon: what's the correct way to have a driver restore a snapshot with the correct (new) size, then?
16:33:00 <xyang1> smcginnis: :)
16:33:01 <erlon> xyang1: there are some CIs that are broken
16:33:05 <smcginnis> And they recognized our existing logo as being a very good one.
16:33:26 <hemna> https://goo.gl/pTu3pa
16:33:33 <tommylikehu> great
16:33:36 <hemna> please be that
16:33:51 <smcginnis> hemna: +1
16:33:57 <jungleboyj> hemna: Nice.
16:33:59 <erlon> flip214: well extending as some drivers  are doing is ok, but that is a common solution that should fit for all
16:34:06 <e0ne> hemna: not bad
16:34:06 <jungleboyj> I swear that Donkey is smiling.
16:34:15 <bswartz> hemna: is that more of a comment about the foundation's mascot initiative in general?
16:34:22 <smcginnis> #link https://review.openstack.org/#/c/336092/ Clone/snapshot size patch
16:34:24 <hemna> bswartz, no comment......
16:34:26 <smcginnis> erlon: Thanks!
16:34:44 <smcginnis> jungleboyj: He's winking at you too. :P
16:34:44 <erlon> smcginnis: welcome
16:34:55 <jungleboyj> smcginnis: Hey baby
16:35:05 <smcginnis> erlon: Did you see there's a merge conflict on that now.
16:35:14 <smcginnis> Along with a few dozen other patches...
16:35:19 <erlon> smcginnis: yep, that was from some minutes ago
16:35:29 <xyang1> erlon: it may be better to split that patch because we usually require CI to pass before merging a change on a driver
16:35:33 <erlon> smcginnis: ill respinn it
16:35:34 <smcginnis> erlon: OK good. Just making sure you saw. :)
16:36:37 <smcginnis> OK, I'm going to call it early unless anyone has anything else important.
16:36:44 <erlon> xyang1: that won't guarantee that the CIs will run in the patch, worst, it got I think almost 2 months to get all that CIs runned
16:37:07 <smcginnis> Alright, thanks everyone.
16:37:11 <erlon> and usually is not possible to keep the results after each run
16:37:16 <smcginnis> #endmeeting