17:00:42 <gema> #startmeeting tailgate
17:00:42 <openstack> Meeting started Thu Aug  6 17:00:42 2015 UTC and is due to finish in 60 minutes.  The chair is gema. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:43 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:46 <openstack> The meeting name has been set to 'tailgate'
17:00:48 <jasonsb> hi
17:00:49 <gema> #topic rollcall
17:00:50 <jose-idar> present!
17:00:54 <jasonsb> hi
17:00:58 <gema> o/
17:01:47 <malini> o - I am half here
17:01:52 <gema> ok
17:02:11 <gema> I am going to confess I haven't had time to prepare this  meeting, so we are going to go by ear
17:02:21 <jasonsb> np
17:02:22 <malini> we just had me & jasonsb in the last meeting
17:02:33 <malini> So we conveniently just chatted :)
17:02:33 <gema> were there any actions?
17:02:50 <malini> gema: it'll be good to follow up from the actions on the meeting before
17:02:51 <jasonsb> dont think so
17:02:59 <gema> malini, jasonsb ok
17:03:04 <jasonsb> i reported about the kolla midcycle meetup
17:03:29 <gema> jasonsb: ok, I will read it, I will add the link to it to my summary tomorrow morning so that people have the reference
17:03:42 <gema> #topic actions from previous weeks
17:03:59 <gema> #action spyderdyne to give more updates on HA testing
17:04:12 <gema> I think we are carrying this one forward since he is not around
17:04:31 <gema> did the review of the questionaire start?
17:04:45 <jasonsb> i wanted to ask about the questionaire
17:04:55 <gema> #action gema to sort out through the logs and get the review about the questionaire started
17:04:57 <jasonsb> is it fine to use
17:05:00 <jasonsb> or does it need tweeks
17:05:08 <gema> jasonsb: it needs tweaks imo
17:05:20 <jasonsb> ok, i'll follow up with you
17:05:25 <gema> yep
17:05:44 <gema> there are a bunch of links on the July log
17:06:02 <gema> #link http://eavesdrop.openstack.org/meetings/tailgate/2015/tailgate.2015-07-23-17.00.html (see for questionaire links)
17:06:28 <gema> I am taking the review initiation from spyderdyne and will start it myself with all the links he gave us
17:06:45 <gema> maybe people are on holidays this month
17:06:56 <gema> so I am not sure what would be a good deadline for this review
17:07:07 <gema> first week of September?
17:07:11 <jasonsb> what is the action?
17:07:24 <gema> I will send an email with all the information and instructions on where to put comments
17:07:26 <jasonsb> review the form to finalize?
17:07:34 <gema> yep
17:07:44 <gema> we'll all send comments, then ammend the form, then send it around
17:07:53 <jasonsb> i'll take it
17:07:57 <gema> you'll take what?
17:08:13 <jasonsb> solicit comments and ammend
17:08:24 <gema> coolio, I gave it to myself but I will change that right away :D
17:09:04 <gema> #action jasonb to initiate the review of the questionaire on the ML and handle review comments, review due to end early Sept (in case people are on holidays)
17:09:32 <gema> jasonsb: I will still send you an email with the bunch of links that we gathered the week you were out
17:09:37 <gema> and try to explain what they were for
17:09:38 <gema> :D
17:09:43 <jasonsb> sounds good
17:09:43 <gema> will do that tomorrow morning
17:10:18 * gema malini to get jose-idar on the mailing
17:10:22 <gema> was this done?
17:10:24 <jose-idar> :)
17:10:24 <malini> done
17:10:26 <gema> cool
17:10:40 <gema> #action gema to submit a few test runs of refstack and get familiar with it
17:10:51 <gema> #action gema to talk to matthew about stable tempest kilo and report back
17:11:03 <gema> #action gema to send out the defcore details on actionable gaps
17:11:32 <gema> the two first ones I haven't had time to do (my work has been erratic from one thing to another) but I still want to get to them
17:11:49 <gema> the third one I have been trying to get to the defcore meeting but they haven't had it for the past two weeks
17:11:52 <jasonsb> i still need to attend the defcore meetings
17:11:58 <gema> not sure if it was due to their midcycle
17:12:08 <gema> or if it is because they changed channel and I missed that
17:12:14 <gema> I will get holld of hogepodge  and ask
17:12:44 <gema> #action gema to figure out where the defcore meetings are and send a note around if they have changed
17:13:02 * gema jasonb to give an update on inspector gadget efforts
17:13:15 <jasonsb> http://paste.openstack.org/show/411202/
17:13:28 <jasonsb> i was thinking of extending the guru report
17:13:38 <gema> #link http://paste.openstack.org/show/411202/
17:13:46 <gema> what is the guru report?
17:13:59 <jasonsb> maybe this could be added to api's so that our tools could report it
17:14:08 <jasonsb> #link https://wiki.openstack.org/wiki/GuruMeditationReport
17:14:26 <jasonsb> its already in the code base, and seems like a good start
17:14:36 <hogepodge> gema: meetings are Wednesdays at 1500. We skipped this week, but will be returning next week
17:14:41 <jasonsb> if you send a SIGUSR1 to a api it will dump out that report
17:14:51 <gema> hogepodge: excellent, thanks, I thought I was in the wrong channel or something
17:15:26 <gema> jasonsb: that is awesome, I hadn't heard of it :D
17:15:44 <jasonsb> yes, some of the basic work is alerady there
17:15:59 <jasonsb> we just need to figure out how we are going to use it and incorporate it
17:16:17 <gema> jasonsb: ok, so what is the action this time?>
17:16:49 <gema> jasonsb: as far as I remember, inspector gadget is supposed to tell us what services are available in the cloud so we can run the appropriate tests
17:16:55 <jasonsb> do a demo of it added to nova-api?
17:17:06 <gema> demo of what?
17:17:30 <jasonsb> demo of keystone query to find out what endpoints are registered
17:17:37 <gema> jasonsb: sounds good
17:17:38 <jasonsb> then query the api for the endpoint and dump out the config
17:17:59 <jasonsb> maybe its a so-so idea, but why not
17:18:24 <gema> #action jasonb to figure out the mechanics of getting endpoints and gurumeditationreport and maybe create a simple script that does this
17:18:28 <gema> sounds good?
17:18:47 <jasonsb> sounds gd
17:18:50 <gema> ok
17:19:20 <gema> that's all there is about previous actions
17:19:30 <gema> do you guys have topics you want to discuss?
17:20:07 <gema> jose-idar: you are quiet
17:20:12 <gema> what have you been up to :)
17:20:26 <jose-idar> mostly defcore, tempest and the like
17:20:33 <jose-idar> just a bunch of meetings
17:20:41 <gema> jose-idar: were you in the midcycle?
17:20:44 <gema> for defcore?
17:20:48 <jose-idar> no
17:20:50 <gema> ah, ok
17:21:17 <jasonsb> gema i would encourage you to read the log of last meeting
17:21:27 <gema> #action gema to read last week's meeting
17:21:30 <jasonsb> i think there is some interesting possibilities with kolla
17:21:34 <gema> ok, will do
17:21:48 <jasonsb> they are like-minded people
17:21:59 <gema> are they an o penstack group?
17:22:06 <jasonsb> yes
17:22:12 <gema> ok,will read with great interest
17:22:17 <jasonsb> deploy openstack services in docker
17:22:30 <jasonsb> if you deploy it, does it work?
17:22:44 <gema> but we are interested in the does it work
17:22:44 <jasonsb> what tools should you use to test the production cloud you just deployed?
17:22:50 <gema> ok
17:22:52 <jasonsb> all the same things
17:23:07 <gema> do they have an official project?
17:23:14 <jasonsb> just accepted to big tent
17:23:18 <gema> awesome
17:23:49 <jasonsb> they would like tests
17:23:59 <jasonsb> but dont' want to develop or mananage them
17:24:00 <gema> jasonsb: everybody would like tests, that's the point
17:24:04 <jasonsb> yup
17:24:19 <gema> reliable, repeatable, non-invasive, awesome tests
17:24:25 <jasonsb> yes
17:24:29 <jasonsb> more of that please
17:24:43 <gema> tests that answer: does my cloud work? with a reliable 80% or 100%
17:24:50 <jasonsb> if you have any suggestions of where to start
17:25:02 <jasonsb> :)
17:25:11 <gema> I do, I have been derailed from my strategy and never finished putting it together
17:25:36 <jasonsb> oh ok.  i'll followup with you
17:25:39 <gema> #action gema to continue with the strategy doc
17:25:52 <gema> you can follow up now if you want
17:25:55 <gema> we still have 35 mins
17:26:27 <jasonsb> the people in kolla meetup were more operations oriented than dev oriented
17:26:34 <jasonsb> (at least from experience point of view)
17:26:51 <gema> so they are users
17:26:54 <gema> and sys admins
17:26:57 <jasonsb> so i heard all of the same feedback about tempest and rally that we have discused in past
17:27:10 <jasonsb> dev + heavy admin
17:27:16 <gema> devops, ok
17:27:22 <jasonsb> sure
17:27:50 <jasonsb> so need a tool which can start small and light
17:28:09 <gema> we don't need a tool, jasonsb , we need test cases
17:28:10 <jasonsb> which (i'll cut and paste) reliable, repeatable, non-invasive, awesome tests
17:28:15 <gema> we have plenty of tools
17:28:54 <jasonsb> ok, so lets make some test cases
17:29:01 <jasonsb> you tell me the framework
17:29:05 <gema> tempest
17:29:08 <jasonsb> and we can contribute to kolla
17:29:36 <gema> tempest uses unittest, I believe, which is good enough for us to start with
17:29:45 <gema> and we can always move those scripts somewhere else
17:29:53 <gema> or we could even do standalone unittests
17:30:01 <jasonsb> ok, but we would need to solve some of the operational problems with tempest
17:30:12 <jasonsb> needs to work with multiple versions of openstack
17:30:14 <jasonsb> needs to clean up
17:30:16 <jasonsb> blah blah
17:30:22 <gema> jasonsb: all tests need to clean up
17:30:59 <gema> and tempest offers us a few apis that we can use and common operations
17:31:13 <gema> but it takes longer to ramp up and write tests than unittest
17:31:27 <jasonsb> ok
17:31:31 <malini> sorry..I am only half here
17:31:32 <gema> jasonsb: or we could start by making tempest clean up
17:31:37 <gema> contributing some changes to that
17:31:38 <gema> end
17:31:41 <malini> But is the plan to use tempest-lib ?
17:31:49 <gema> malini: there is no plan
17:31:54 <malini> :D
17:31:57 <gema> malini: we can use whatever we want
17:32:13 <gema> malini: or we could say: our tests are self contained scripts
17:32:20 <gema> if the script returns 0 you are good
17:32:21 <rockyg> o/
17:32:26 <malini> hello rockyg!!
17:32:27 <gema> if it returns something else , not so much
17:32:30 <gema> rockyg: o/
17:32:40 <jasonsb> ok
17:32:47 <malini> gema: hmm…but we shud have better test results than a 0 or 1
17:32:53 <jasonsb> this sounds like defcore effort too
17:33:06 <gema> malini: that's a fail or a pass
17:33:09 <jasonsb> (two birds is good)
17:33:10 <gema> what else do you want? :D
17:33:23 <malini> why/where did it fail?
17:33:24 <gema> malini: we surely can count 0s and 1s
17:33:26 <rockyg> I'm on Defcore.  I can help
17:33:32 <gema> rockyg: go ahead please
17:33:42 <gema> rockyg: trying to decide framework here
17:34:00 <gema> malini: you achieve that by making your tests spit comments
17:34:03 <rockyg> Hey.  Just heard about you guys.  This is *great*  I want to help, but I've got too many irons in the fire.
17:34:17 <gema> rockyg: we are all more or less the same
17:34:18 <rockyg> But, I can do some influencing and advising
17:34:36 <rockyg> I saw Gema
17:34:49 <rockyg> 's post on the Summit session proposals
17:34:55 <gema> rockyg: so if we were to create tests that you guys can use, should we use tempest?
17:35:07 <gema> or tempest-lib
17:35:10 <rockyg> I was gonna try to get a birds of a feather going in Tokyo.
17:35:37 <rockyg> So, it should work *with* tempest_lib, but doesn't have to be in the tempest repo
17:36:04 <gema> so you want the test runner to be tempest lib
17:36:11 <rockyg> Go out to Refstack repo (I think it's under openstack-infra now or will be shortly)
17:36:21 <rockyg> Look at the readme for tempest-client
17:36:28 <gema> rockyg: ok
17:37:03 <rockyg> that's what DefCore is requiring to be the test runner/results collection
17:37:28 <gema> and do you have an idea of where are the biggest gaps in coverage?
17:37:45 <gema> for defcore's purposes
17:37:56 <rockyg> Other thing is that we're entertaining proposals for other test sets within the Refstack server that validate interop.
17:38:14 <rockyg> Oh, yeah, to some extent.
17:38:39 <rockyg> But what we're finding is that there are actual code gaps that make interop hard.
17:38:50 <gema> yeah, testability is an issue for us too
17:39:21 <rockyg> And really, right now DefCore uses api tests for interop where really, we should have tests for user use cases
17:39:40 <gema> yeah, I have a problem with calling refstack tests "interop"
17:39:46 <gema> but I don't want to be picky yet :D
17:39:50 <rockyg> Like  give me a vm with external access and a private management network
17:40:02 <jose-idar> gema: +1!
17:40:38 <rockyg> Monty Taylor and John Garbutt have come out loudly about the fact that there are like three ways to create a vm and none of them are good.
17:41:33 <gema> what's wrong with nova boot?
17:41:40 <rockyg> Sorry for any time gaps, I'm managing the diversity meeting, too right now.
17:41:57 <gema> rockyg: don't worry, thanks for coming, this is very enlightening
17:42:10 <rockyg> I think QA/E should  be considered a minority for OpenStack Foundation's sake ;-)
17:42:27 <gema> :)
17:42:32 <rockyg> I just talked to Sam Danes (rackspace) and he'
17:42:41 <rockyg> s really interested in this group, too.
17:42:54 <gema> sam was with us in vancouver
17:43:02 <gema> at the middle of our early conversations, right malini ?
17:43:02 <rockyg> I have a possible strategy for acceptance -- FMEA test framework
17:43:23 <rockyg> It's not part of tempest charter.  No ERROR messages allowed in tests for passing
17:43:53 <malini> gema: yes
17:43:57 <rockyg> And if we get something like this happening, I think we'd be the first OpenSource project collection with *real* QA
17:44:07 <gema> rockyg: that's the idea
17:44:15 <gema> we were all trying to do it separately
17:44:20 <gema> so we decided to join forces
17:44:24 <gema> it's too big of a task
17:44:30 <rockyg> Yup.  Sam was there.  That's how I heard about you after I saw the proposals
17:45:08 <rockyg> Yes.  Much too big, but once a framework and rules for writing the tests are in place, lots of ops guys will provide tests
17:45:17 <gema> yep, that's my hope
17:45:28 <gema> that's what happened in ubuntu when we started QA few years ago :D
17:45:42 <gema> and that's why I was talking about any binary and using the return code
17:45:43 <rockyg> So, the focus should be on getting some possible frameworks and some sample tests.
17:45:51 <gema> because people like to write tests however they like them
17:45:59 <gema> and if they are good, they can be integrated
17:46:06 <rockyg> Yeah.  Exactly.
17:46:17 <gema> if we are too prescriptive, we won't get far
17:47:09 <gema> ok, I have the feeling I should put an action from our conversation with rockyg , but I don't know where to start
17:47:29 <malini> Identify one scenario from rockyg
17:47:34 <rockyg> So, is this a weekly meeting?  Or just everyone happens to be here?
17:47:45 <malini> 'try' to write a test with tempest-lib
17:47:50 <gema> rockyg: it's a weekly, sometimes there's more people than other times :D
17:48:12 <gema> malini: who should attempt such task?
17:48:14 <rockyg> So, 10am PDT Thursdays?  I'll put it on my calendar
17:48:26 <malini> gema: any of us can. I wouldn't mind doing it
17:48:39 <gema> malini: ok, have a go and let us know, then
17:48:41 <malini> I have always been planning to try out tempest-lib
17:48:46 <gema> yep
17:48:57 <malini> rockyg: is there a scenario for us to start with?
17:48:58 <rockyg> What might also be useful is to get a test or a representative collection of tests that folks are running at their sites
17:49:04 <malini> just as a POC
17:49:11 <rockyg> I know Sam has a number.
17:49:14 <gema> #action malini to try to write a test with tempest-lib that spins a VM and destroys is, as a POC
17:49:33 <rockyg> I think we need a smallish test to start.  give me a vm is too big
17:49:46 <gema> ok, list VMs ?
17:49:49 <gema> nova list
17:49:53 <gema> but from the api?
17:50:17 <gema> and if it doesn't error it is a pass
17:50:36 <rockyg> Ooh, but if you can spin a vm, gather the info from it a user needs (networks, ssh, etc), then delete it, that would be very good.
17:50:52 <gema> rockyg: we were trying to start small, right?
17:51:04 <rockyg> Yeah.  But I can hope?
17:51:10 <gema> that's why I thought spin vm and destroy vm was a good start
17:51:29 <gema> even if it takes a while to get glance serving images and keystone authenticating
17:51:30 <rockyg> Actually, one that is not doable now is to list available images in a cloud
17:51:42 <gema> there's no api for that?
17:51:52 <rockyg> All DefCore tests are user space, no admin.
17:52:01 <gema> that's a good point
17:52:08 <rockyg> There is currently a test that does that, but it requires admin.
17:52:28 <malini> if its admin only, it doesnt make sense to be in defcore -rt?
17:52:31 <jose-idar> rockyg: do you know if the call requires admin, or if the test is requiring that erroneously?
17:52:34 <rockyg> And since glance v2 is not in nova yet, there is no user facing api.
17:53:03 <rockyg> That's why it's not in Defcore as a requirement.  Test is flagge ;-)
17:53:08 <gema> why, I can list the images available to me on my env without admin
17:53:11 <gema> glance image-list
17:53:22 <gema> (on kilo)
17:53:31 <malini> hinking abt it, it shud be user accessible
17:53:38 <rockyg> Is that v1 or v2, and is it an internal api that's exposed, or external
17:53:51 <gema> I use the glance client
17:53:59 <gema> not sure what version of the api it uses
17:54:01 <gema> how do I check that?
17:54:19 <rockyg> Logs, I think, and we all know they suck
17:54:26 <gema> glance logs?
17:54:49 <jasonsb> yuo can specify which api with glance client
17:54:52 <gema> #action gema to look at the glance logs of her test cloud and check what api is being used (we can list images without admin)
17:54:57 <gema> ok, will check and report back
17:55:05 <rockyg> Thanks!
17:55:08 <gema> np
17:55:43 <gema> ok, we are about to hit the end of the meeting, malini are you comfortable with the POC?
17:55:48 <gema> feel free to change the test case
17:55:50 <rockyg> One of the next areas Defcore is expanding into is Neutron for 2016.01
17:55:52 <gema> to whatever is achievable
17:56:12 <malini> "malini to try to write a test with tempest-lib that spins a VM and destroys is, as a POC" is this still the POC test?
17:56:17 <rockyg> They will need user based tests.  What are my networks and which is which
17:56:30 <gema> malini: that's the action, just do whatever yoou think is achievable
17:56:32 <gema> if that is not
17:56:39 <malini> sounds good
17:56:53 <rockyg> I'll let you know that rackspace and helion require different paths to make it happen.
17:57:27 <malini> hmmm..so my test shud theoretically run on any OS cloud
17:57:31 <gema> yep
17:57:33 <rockyg> Monty Taylor also might have a bunch of code checked in under shade that show you how he does it
17:57:39 <gema> malini: but start with yours
17:57:56 <malini> yes..I dont like anybody else's code anyways :D
17:58:02 <gema> no, I mean your cloud xD
17:58:03 <rockyg> lol
17:58:12 <gema> then we can run your script on all the other ones and see xD
17:58:17 <jose-idar> the novaclient manages it via auth plugins, as long as identity is out of scope, it should work on most clouds.
17:58:48 <gema> alright , sounds like we are almost at the end of the day
17:58:58 <gema> any volunteers for chairing next week?
17:59:03 <rockyg> Defcore wants Keystone tests, too, but so far, they are all admin.
17:59:37 <gema> #action gema to check what parts of keystone are accesible without admin on ubuntu's distro
17:59:44 <jose-idar> I'm just advocating that the scopes should be separate.  the nova tests shouldn't fail because of auth if they don't have to, for example.
18:00:02 <gema> jose-idar: it depends on whether the test is end to end or not
18:00:10 <gema> if it is end to end it should fail
18:00:10 <rockyg> +100
18:00:16 <gema> if it is not, it shouldn't
18:00:24 <gema> and for interop reasons, you want end to end
18:00:28 <jose-idar> sure, but there should also be one that isn't end to end
18:00:37 <gema> we need to make auth compatible between clouds
18:00:42 <gema> it shoudln't be that difficult
18:00:48 <jose-idar> because it's just as important to know if the components work as it is to know if the cloud works together.
18:01:15 <gema> jose-idar: true, but the scope of the tests is another long conversation to have in the future I think :D
18:01:16 <rockyg> So far, Defcore is only working on the first part.
18:01:17 <jose-idar> cross-cloud auth compatibility should probably be #1 priority honestly :D
18:01:46 <rockyg> But would accept tests for the second.
18:01:52 <jose-idar> the problem with putting it off is that defcore is going to set the prescedent for end-to-end tests when nothing works together yet.
18:02:12 <rockyg> yeeup
18:02:14 <gema> jose-idar: a failing test is a great starting point, something to fix
18:02:17 <jose-idar> they should have started on per-component first, but that would require them writing actual specs instead of relying on the already exant temptest tests.
18:02:44 <rockyg> jose-idar, welcome to my world :-/
18:02:55 <gema> rockyg: maybe we can help you change that
18:03:06 <rockyg> Yes, please!
18:03:06 <jose-idar> a component test that fails is much clearer than an end to end test that fails, especially considering a user that runs the tests on a cloud who's logs they can't acess
18:03:09 <gema> jose-idar: can we attempt to write one of those specs as an example?
18:03:33 <jose-idar> probably!
18:03:34 <gema> jose-idar: I mean you sound like you know what you want such spec to look like
18:03:43 <gema> can you start one so we can contribute to it
18:03:46 <gema> say .. for nova?
18:03:48 <rockyg> Also, users should have access at least to their vm logs.
18:04:13 <rockyg> jose-idar, please?
18:05:23 <rockyg> The key is, the real test is whether you can have a vm image with your app on it and recreate it or move it to another cloud.  That's interop.
18:05:30 <beisner> o/ hi, apologize for the late pop-in, just reading backscroll
18:05:33 <rockyg> Oh, and manage on both
18:05:50 <rockyg> We ask for so little ;-)
18:05:59 <gema> #action jose-idar to start a spec for component tests for nova (user side, no admin), so we can all contribute cases and maybe implement them at some point , but at least start to get organised
18:06:06 * gema looks shyly at jose-idar
18:06:25 <malini> should be 'stares at jose-idar' ;)
18:06:30 <gema> haha
18:06:32 <jose-idar> sorry, got pulled away
18:06:42 <jose-idar> sure!@
18:06:48 <gema> jose-idar: \o/ thanks!
18:06:55 <rockyg> \o/
18:07:10 <gema> ok, I am going to call it a meeting before I start giving actions to non present people :D
18:07:13 <jose-idar> i'll sync up w/ dary walleck, because I think he's working on a oparallel task
18:07:18 <jose-idar> thansk everyone!@
18:07:24 <gema> #endmeeting