19:03:00 <catherineD> #startmeeting refstack
19:03:01 <openstack> Meeting started Mon Mar 23 19:03:00 2015 UTC and is due to finish in 60 minutes.  The chair is catherineD. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:03:03 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:03:05 <openstack> The meeting name has been set to 'refstack'
19:03:40 <pvaneck> o/
19:03:46 <sslypushenko_> o/
19:04:37 <catherineD> Today's agenda: 1) recap items agreed last week from f2f, 2) continue discuss the remaining items of the Feb f2f action list.
19:04:37 <catherineD> (including OpenID integration item 1.0.1) https://etherpad.openstack.org/p/refstack_f2f_feb_2015 3) pending reviews
19:04:37 <catherineD> 
19:05:14 <catherineD> any other topics?
19:05:37 <sslypushenko_> Lets start with these topics
19:05:43 <catherineD> Last week we agreed on the following:
19:05:54 <catherineD> #agreed Using [refstack] tag for mailing list
19:06:01 <zehicle> I can give a DefCore update since we are moving docs out of refstack
19:06:16 <Rockyg> ++
19:06:32 <catherineD> #agreed For the "uuid issue and refstrack test results json schema" topic, we agree to operate business-as-usual until we     get a new results JSON schema from DefCore
19:06:38 <catherineD> zehicle: ++
19:06:59 <catherineD> #agreed Delay vendor implementation (vendor registration, vendor related UI) until after Vancouver (section 1.0.3 of f2f)
19:07:20 <catherineD> #agreed Only need page 1 and 4, skip page 2 and 3 of the mockup for Vancouver    https://drive.google.com/file/d/0BxvL0emVRKoUVW13TW5BWXg0MXM/view?usp=sharing
19:07:22 <zehicle> I'd be interested in hearing more from hogepodge about UUID thoughts
19:07:29 <catherineD> that is all ...
19:08:48 <Rockyg> do we have mockups for 1&4?
19:09:04 <catherineD> For UUID I opened this bug https://bugs.launchpad.net/tempest/+bug/1433700
19:09:05 <openstack> Launchpad bug 1433700 in tempest "Multiple test cases associated with same test UUID" [Undecided,In progress] - Assigned to Sergey Slipushenko (sslypushenko)
19:09:22 <catherineD> sslypushenko_: is woking on the review ...
19:09:56 <sslypushenko_> catherineD Great thx for that)
19:10:14 <catherineD> Rockyg: same mock up link
19:10:29 <catherineD> #link https://drive.google.com/file/d/0BxvL0emVRKoUVW13TW5BWXg0MXM/view?usp=sharing
19:11:25 <catherineD> hogepodge: your thoughts on UUID ..
19:11:36 <Rockyg> catherineD:  thanks
19:12:02 <hogepodge> #link longer thoughts here http://lists.openstack.org/pipermail/defcore-committee/2015-March/000660.html
19:13:14 <hogepodge> Based on conversations with the openstack-qa team and some others, I think we can work with the idempotent_id and test names to identify functionality testing (the idempotent_id) and the test (the test name)
19:13:43 <hogepodge> sslypushenko_ has a patch in place to generate unique ids for every function, but I'm not sure qa is going to bite on it.
19:14:05 <hogepodge> #link https://review.openstack.org/#/c/165921/
19:14:12 <sslypushenko_> hogepodge Please specify what do you mean on test name?
19:14:19 <catherineD> hogepodge: by test name you means FQN?
19:14:23 <hogepodge> fully qualified test name
19:14:53 <sslypushenko_> full path? or TestCaseName.test_method_name?
19:14:58 <hogepodge> We accomplish the goal of tracking functionality through refactoring with idempoent_id as they stand.
19:15:00 <hogepodge> full path
19:15:06 <hogepodge> which is what we're using right now
19:15:19 <catherineD> hogepodge: using FQN we need to fix to a certain tag or hash ...
19:15:28 <catherineD> and that is what we try to fix with UUID ..
19:15:29 <hogepodge> i.e. tempest.api.compute.images.test_list_image_filters.ListImageFiltersTestJSON.test_list_images_filter_by_server_id
19:16:10 <hogepodge> catherineD: I think that's ok from a defcore standpoint, which is going to be on 6 month cycles.
19:16:39 <hogepodge> Two questions are: can we work with the current state? Will openstack-qa allow us to modify the current state?
19:16:41 <Rockyg> Actually, no need for UUID if you have hash and FQN
19:16:44 <sslypushenko_> FQN + idempotent_id sounds really strange...
19:17:15 <catherineD> Rockyg: correct  That is our business-as-usual
19:17:48 <hogepodge> Rockyg: it's still useful for tracking refactoring
19:18:41 <zehicle> I recall that we had an issue when people refactored tests and it was very hard to update the lists
19:18:44 <hogepodge> My feeling from the qa team is that they're happy with the way things are now, so change would be hard (but not impossible)
19:18:57 <zehicle> QA team use case if different than ours
19:19:00 <Rockyg> Yeah.  But it becomes part of the defcore process and can be ignore in the testing process itself
19:19:23 <catherineD> hogepodge: so we are not sovling what we set out with UUID... but I agree that it might help tracking ...
19:19:26 <hogepodge> zehicle: idempotent_id means you can track where a group of tests went. hash search vs linear search
19:20:12 <hogepodge> I think there's a strong case to be made that even changing the name of a test makes it a different test. It's a pedantic argument though.
19:20:48 <zehicle> and when the tests move to the projects, there will be MUCH pain
19:20:48 <Rockyg> but at least with UUID, you can *find* where the test moved to.
19:21:26 <Rockyg> zehicle: that's why getting the uuids in before the moves start has been so important.
19:21:29 <hogepodge> Rockyg it's not that simple, really, because how do you do it? id on parent class and method? Two things to keep track of. generate from id on parent and method, then you can't search the code directly
19:21:33 <zehicle> test names are human friendly - we just have a habit of wanting them to retain meaning
19:21:47 <zehicle> so, people want to tweak them
19:22:22 <zehicle> ultimately, we'll have to maintain a master test inventory file
19:22:23 <hogepodge> Does tweaking change the meaning? I've been changing a test name in a working tree because I keep finding that without admin I can't to certain things.
19:23:06 <zehicle> tweaking may or may not change the meaning BUT it does make it a new ID
19:23:22 <hogepodge> right now the fallout is we track back to an idempotent_id and you may have to choose from a small subset of tests. i.e. ipv4 vs ipv6
19:23:23 <zehicle> so that will create work for RefStack & DefCore (at least for the tests that we are tracking)
19:23:30 <sslypushenko_> zehicle Test inventory file can easily solve problem with readibility
19:23:41 <zehicle> sslypushenko_, yes, I'
19:23:55 <zehicle> ve been expecting that to be needed - there's a patch for it
19:23:57 <hogepodge> the work is so greatly reduced with what we have. It's not a total failure, we already get a lot.
19:24:30 <catherineD> so where do we stand here ... last week we agreed that o operate business-as-usual until we get a new results JSON schema from DefCore
19:24:53 <hogepodge> My suggestion for fixing this would be to tag classes also. Then the tuple capture all of the information about the class and the function, and is searchable directly in source.
19:25:26 <sslypushenko_> I guess better what we can do - It is to find a usecase where current implementation is not enough.
19:26:04 <sslypushenko_> I may help as to force idea about improvement in qa team
19:26:34 <Rockyg> sslypushenko_ ++ ;-)
19:27:09 <hogepodge> Let's say that in 2015.3 and 2015.[45] we work with what we have and plan for 2015.9 when there's time available.
19:27:16 <hogepodge> (that's just my suggestion)
19:27:21 <catherineD> hogepodge: ++
19:27:24 <Rockyg> So, if the class can change without the test being rev'ed then we aren't tracking the tests properly
19:28:08 <zehicle> it could be that we have to wait to resolve until f2f at the summit
19:28:32 <catherineD> hogepodge: what does work with what we have?
19:28:50 <hogepodge> That's not too far away. I want to restate that what we have is really good, and I think is going to make life easier for everyone as revs start up.
19:29:11 <Rockyg> hogpodge:  ok
19:29:44 <Rockyg> zehicle: at the summit, we have to have concrete examples to get our points across
19:29:46 <hogepodge> I also understand the shortcoming, and that it's not what we originally wanted. That was in part due to my lack of complete understanding on how tempest works.
19:30:04 <zehicle> ok, so we use test names for that
19:30:32 <hogepodge> I think that test names will always have to be part of the capabilities list anyway. There has to be a human readable component.
19:31:02 <hogepodge> You'll always be able to find where a test went to if it moves from what we have, and we didn't have that before.
19:31:17 <zehicle> hogepodge, yes.  I was assuming that inventory file would be the item that handled that
19:32:09 <hogepodge> zehicle: it may be a thing to talk about in the defcore meeting. My feeling is that not having a human readable component in the capabilities.json file would be a crippling mistake
19:32:57 <zehicle> hogepodge, I agree
19:33:05 <zehicle> that's why I was against the long UUIDs
19:33:47 <hogepodge> zehicle: I don't see any conflict between storing tests as tuples. Or using the fully expanded test names (that include tags and ids now)
19:34:02 <catherineD> so from refstack point of view we are still agree as we did last week ...we agree to operate business-as-usual until DefCore decide on schema for test ...
19:34:26 <zehicle> hogepodge, in an inventory file, yes
19:34:43 <zehicle> IMHO it's risky to make the capabilites file too complex
19:34:59 <zehicle> you run the risk of someone missing something and it breaking the file
19:35:17 <zehicle> maybe that's over blown and I'm just a DB admin at heart
19:35:22 <zehicle> and want to normalize everything
19:36:21 <hogepodge> zehicle: I trust your experience.
19:36:42 <hogepodge> zehicle: Maybe I need to make another trip to Austin next month and we can chat about these files f2f. ;-)
19:36:58 <zehicle> always happy to have you in town!
19:37:28 <Rockyg> Jeez, if you were going to do it, you should have done it during SXSW
19:37:46 <zehicle> lol, except I was not here
19:38:35 <catherineD> Time to move to the next topic?
19:38:41 <Rockyg> ++
19:39:18 <catherineD> #topic 2) continue discuss the remaining items of the Feb f2f action list. (including OpenID integration item 1.0.1)
19:39:35 <catherineD> #link  https://etherpad.openstack.org/p/refstack_f2f_feb_2015
19:40:06 <catherineD> section 1.0.1 sslypushenko_: was doing some investigation
19:40:43 <sslypushenko_> Yep... I have disscussion with Vlad and Chris
19:41:42 <catherineD> section 1.0.1 in the "action items for vancouver" at the bottom of https://etherpad.openstack.org/p/refstack_f2f_feb_2015
19:41:49 <sslypushenko_> Final decisions is that in term of auth we should follow common way, which used in openstac gerrit for example
19:42:30 <sslypushenko_> Vlad already have started development
19:42:57 <sslypushenko_> I will post in refstack channel link on workflow diagram
19:43:18 <catherineD> sslypushenko_: thx
19:43:23 <sslypushenko_> I hope it will make things clear
19:43:43 <vladiskuz_> catherineD: It's first step https://review.openstack.org/#/c/166247/
19:44:48 <sslypushenko_> In short, We are going to use OpenstackId for web UI auth and RSA key pairs  for CLI tool
19:45:04 <catherineD> sslypushenko_: vladiskuz_: thx .. Let's move to section 3.2.1.1
19:45:16 <sslypushenko_> Like gerrit or github
19:45:41 <catherineD> we need additional API to support page 4 of the mockup
19:45:53 <catherineD> I have submited a review https://review.openstack.org/#/c/166357/
19:46:11 <catherineD> could you all review ...?
19:46:27 <catherineD> we really need this API ...
19:46:34 <sslypushenko_> Sure thing)
19:47:13 <sslypushenko_> Vlad already pushed on revied basic version of retrival API
19:47:46 <sslypushenko_> It is possible to add some filters to it
19:47:49 <catherineD> currently, after user collects the test results from refstack-client ... they want to look at the results ... and it is very cumbersome now ...
19:48:31 <catherineD> that is the paint that jlk experiences....
19:48:52 <sslypushenko_> https://review.openstack.org/#/c/166247/
19:49:08 <sslypushenko_> It is our first step on that way
19:50:20 <catherineD> sslypushenko_: I need to discuss with vladiskuz_: in detail between https://review.openstack.org/#/c/166247/ and https://review.openstack.org/#/c/166357/ in #refstack after this meeting ..
19:50:52 <vladiskuz_> catherineD: np ;)
19:50:59 <catherineD> that is all the topic I have today ...
19:51:29 <catherineD> zehicle: could you give defcore update?
19:51:36 <zehicle> yy
19:51:45 <zehicle> DefCore's been moving REALLY fast latest
19:52:10 <zehicle> we've got the 2015.03 spec landed with the template for the 2015.next in also
19:52:16 <zehicle> process draft should hit soon too
19:52:21 <zehicle> that means we're on track for the summit
19:52:43 <zehicle> all of that in the openstack/defcore repo.  so we'll start to pull down the refstack/defcore folder
19:52:50 <zehicle> it should be gone by the summit
19:53:16 <hogepodge> in my mind the biggest road block is fixing tests that rely on neutron. Going to nyc next week to try and tie a ribbon on that.
19:53:19 <zehicle> the 2015.03.json file replaces the capabilities/sections files that we used in the past
19:53:27 <zehicle> format is very similar
19:53:51 <zehicle> hogepodge, any idea on that keystone test?
19:53:53 <catherineD> zehicle: will refstack be used to present results to users?
19:53:56 <zehicle> so we can include the capability
19:54:04 <hogepodge> zehicle: in progress
19:54:07 <zehicle> it's not in the requirements
19:54:19 * morganfainberg feels a disturbance in Freenode...
19:54:20 <zehicle> but that is my expectation
19:54:24 <hogepodge> #link https://review.openstack.org/#/c/166327/
19:54:32 <hogepodge> v2 is done, working on v3
19:54:54 <catherineD> what we found that with the capability file in defcore .. how can refstack ensure that refstack will always use the latest capability file?
19:55:21 <zehicle> the process is being written to not create a lot of implementation specifics unless we need to make it so
19:55:29 <hogepodge> catherineD: refstack is going to need versioned codecs I think, at least for this year
19:55:37 <catherineD> do we need to fetch the capabilityf file each time refstack render result ..
19:56:07 <zehicle> good question, would it help if we added an "active guidelines" file w/ the right pointers?
19:56:22 <zehicle> a simple JSON list w/ all the specs and their status
19:56:23 <zehicle> ?
19:56:40 <zehicle> they are designed to sort nicely
19:56:50 <catherineD> I am concern more on the rabpid changing in defcore capability and refstack may miss it ...
19:57:00 <zehicle> so you could also just grab the alpha list max
19:57:33 <zehicle> defcore and "rapid change" should not go together
19:58:05 <catherineD> I will log this for our next week discussion ..since we are about out of time now
19:59:09 <catherineD> sslypushenko_: vladiskuz_: Let's discuss the reviews in #refstack ...
19:59:23 <vladiskuz_> catherineD: Let's go
19:59:27 <catherineD> need to end meeting now ... thank you all!
19:59:42 <catherineD> #endmeeting