19:03:00 <catherineD> #startmeeting refstack 19:03:01 <openstack> Meeting started Mon Mar 23 19:03:00 2015 UTC and is due to finish in 60 minutes. The chair is catherineD. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:03:03 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:03:05 <openstack> The meeting name has been set to 'refstack' 19:03:40 <pvaneck> o/ 19:03:46 <sslypushenko_> o/ 19:04:37 <catherineD> Today's agenda: 1) recap items agreed last week from f2f, 2) continue discuss the remaining items of the Feb f2f action list. 19:04:37 <catherineD> (including OpenID integration item 1.0.1) https://etherpad.openstack.org/p/refstack_f2f_feb_2015 3) pending reviews 19:04:37 <catherineD> 19:05:14 <catherineD> any other topics? 19:05:37 <sslypushenko_> Lets start with these topics 19:05:43 <catherineD> Last week we agreed on the following: 19:05:54 <catherineD> #agreed Using [refstack] tag for mailing list 19:06:01 <zehicle> I can give a DefCore update since we are moving docs out of refstack 19:06:16 <Rockyg> ++ 19:06:32 <catherineD> #agreed For the "uuid issue and refstrack test results json schema" topic, we agree to operate business-as-usual until we get a new results JSON schema from DefCore 19:06:38 <catherineD> zehicle: ++ 19:06:59 <catherineD> #agreed Delay vendor implementation (vendor registration, vendor related UI) until after Vancouver (section 1.0.3 of f2f) 19:07:20 <catherineD> #agreed Only need page 1 and 4, skip page 2 and 3 of the mockup for Vancouver https://drive.google.com/file/d/0BxvL0emVRKoUVW13TW5BWXg0MXM/view?usp=sharing 19:07:22 <zehicle> I'd be interested in hearing more from hogepodge about UUID thoughts 19:07:29 <catherineD> that is all ... 19:08:48 <Rockyg> do we have mockups for 1&4? 19:09:04 <catherineD> For UUID I opened this bug https://bugs.launchpad.net/tempest/+bug/1433700 19:09:05 <openstack> Launchpad bug 1433700 in tempest "Multiple test cases associated with same test UUID" [Undecided,In progress] - Assigned to Sergey Slipushenko (sslypushenko) 19:09:22 <catherineD> sslypushenko_: is woking on the review ... 19:09:56 <sslypushenko_> catherineD Great thx for that) 19:10:14 <catherineD> Rockyg: same mock up link 19:10:29 <catherineD> #link https://drive.google.com/file/d/0BxvL0emVRKoUVW13TW5BWXg0MXM/view?usp=sharing 19:11:25 <catherineD> hogepodge: your thoughts on UUID .. 19:11:36 <Rockyg> catherineD: thanks 19:12:02 <hogepodge> #link longer thoughts here http://lists.openstack.org/pipermail/defcore-committee/2015-March/000660.html 19:13:14 <hogepodge> Based on conversations with the openstack-qa team and some others, I think we can work with the idempotent_id and test names to identify functionality testing (the idempotent_id) and the test (the test name) 19:13:43 <hogepodge> sslypushenko_ has a patch in place to generate unique ids for every function, but I'm not sure qa is going to bite on it. 19:14:05 <hogepodge> #link https://review.openstack.org/#/c/165921/ 19:14:12 <sslypushenko_> hogepodge Please specify what do you mean on test name? 19:14:19 <catherineD> hogepodge: by test name you means FQN? 19:14:23 <hogepodge> fully qualified test name 19:14:53 <sslypushenko_> full path? or TestCaseName.test_method_name? 19:14:58 <hogepodge> We accomplish the goal of tracking functionality through refactoring with idempoent_id as they stand. 19:15:00 <hogepodge> full path 19:15:06 <hogepodge> which is what we're using right now 19:15:19 <catherineD> hogepodge: using FQN we need to fix to a certain tag or hash ... 19:15:28 <catherineD> and that is what we try to fix with UUID .. 19:15:29 <hogepodge> i.e. tempest.api.compute.images.test_list_image_filters.ListImageFiltersTestJSON.test_list_images_filter_by_server_id 19:16:10 <hogepodge> catherineD: I think that's ok from a defcore standpoint, which is going to be on 6 month cycles. 19:16:39 <hogepodge> Two questions are: can we work with the current state? Will openstack-qa allow us to modify the current state? 19:16:41 <Rockyg> Actually, no need for UUID if you have hash and FQN 19:16:44 <sslypushenko_> FQN + idempotent_id sounds really strange... 19:17:15 <catherineD> Rockyg: correct That is our business-as-usual 19:17:48 <hogepodge> Rockyg: it's still useful for tracking refactoring 19:18:41 <zehicle> I recall that we had an issue when people refactored tests and it was very hard to update the lists 19:18:44 <hogepodge> My feeling from the qa team is that they're happy with the way things are now, so change would be hard (but not impossible) 19:18:57 <zehicle> QA team use case if different than ours 19:19:00 <Rockyg> Yeah. But it becomes part of the defcore process and can be ignore in the testing process itself 19:19:23 <catherineD> hogepodge: so we are not sovling what we set out with UUID... but I agree that it might help tracking ... 19:19:26 <hogepodge> zehicle: idempotent_id means you can track where a group of tests went. hash search vs linear search 19:20:12 <hogepodge> I think there's a strong case to be made that even changing the name of a test makes it a different test. It's a pedantic argument though. 19:20:48 <zehicle> and when the tests move to the projects, there will be MUCH pain 19:20:48 <Rockyg> but at least with UUID, you can *find* where the test moved to. 19:21:26 <Rockyg> zehicle: that's why getting the uuids in before the moves start has been so important. 19:21:29 <hogepodge> Rockyg it's not that simple, really, because how do you do it? id on parent class and method? Two things to keep track of. generate from id on parent and method, then you can't search the code directly 19:21:33 <zehicle> test names are human friendly - we just have a habit of wanting them to retain meaning 19:21:47 <zehicle> so, people want to tweak them 19:22:22 <zehicle> ultimately, we'll have to maintain a master test inventory file 19:22:23 <hogepodge> Does tweaking change the meaning? I've been changing a test name in a working tree because I keep finding that without admin I can't to certain things. 19:23:06 <zehicle> tweaking may or may not change the meaning BUT it does make it a new ID 19:23:22 <hogepodge> right now the fallout is we track back to an idempotent_id and you may have to choose from a small subset of tests. i.e. ipv4 vs ipv6 19:23:23 <zehicle> so that will create work for RefStack & DefCore (at least for the tests that we are tracking) 19:23:30 <sslypushenko_> zehicle Test inventory file can easily solve problem with readibility 19:23:41 <zehicle> sslypushenko_, yes, I' 19:23:55 <zehicle> ve been expecting that to be needed - there's a patch for it 19:23:57 <hogepodge> the work is so greatly reduced with what we have. It's not a total failure, we already get a lot. 19:24:30 <catherineD> so where do we stand here ... last week we agreed that o operate business-as-usual until we get a new results JSON schema from DefCore 19:24:53 <hogepodge> My suggestion for fixing this would be to tag classes also. Then the tuple capture all of the information about the class and the function, and is searchable directly in source. 19:25:26 <sslypushenko_> I guess better what we can do - It is to find a usecase where current implementation is not enough. 19:26:04 <sslypushenko_> I may help as to force idea about improvement in qa team 19:26:34 <Rockyg> sslypushenko_ ++ ;-) 19:27:09 <hogepodge> Let's say that in 2015.3 and 2015.[45] we work with what we have and plan for 2015.9 when there's time available. 19:27:16 <hogepodge> (that's just my suggestion) 19:27:21 <catherineD> hogepodge: ++ 19:27:24 <Rockyg> So, if the class can change without the test being rev'ed then we aren't tracking the tests properly 19:28:08 <zehicle> it could be that we have to wait to resolve until f2f at the summit 19:28:32 <catherineD> hogepodge: what does work with what we have? 19:28:50 <hogepodge> That's not too far away. I want to restate that what we have is really good, and I think is going to make life easier for everyone as revs start up. 19:29:11 <Rockyg> hogpodge: ok 19:29:44 <Rockyg> zehicle: at the summit, we have to have concrete examples to get our points across 19:29:46 <hogepodge> I also understand the shortcoming, and that it's not what we originally wanted. That was in part due to my lack of complete understanding on how tempest works. 19:30:04 <zehicle> ok, so we use test names for that 19:30:32 <hogepodge> I think that test names will always have to be part of the capabilities list anyway. There has to be a human readable component. 19:31:02 <hogepodge> You'll always be able to find where a test went to if it moves from what we have, and we didn't have that before. 19:31:17 <zehicle> hogepodge, yes. I was assuming that inventory file would be the item that handled that 19:32:09 <hogepodge> zehicle: it may be a thing to talk about in the defcore meeting. My feeling is that not having a human readable component in the capabilities.json file would be a crippling mistake 19:32:57 <zehicle> hogepodge, I agree 19:33:05 <zehicle> that's why I was against the long UUIDs 19:33:47 <hogepodge> zehicle: I don't see any conflict between storing tests as tuples. Or using the fully expanded test names (that include tags and ids now) 19:34:02 <catherineD> so from refstack point of view we are still agree as we did last week ...we agree to operate business-as-usual until DefCore decide on schema for test ... 19:34:26 <zehicle> hogepodge, in an inventory file, yes 19:34:43 <zehicle> IMHO it's risky to make the capabilites file too complex 19:34:59 <zehicle> you run the risk of someone missing something and it breaking the file 19:35:17 <zehicle> maybe that's over blown and I'm just a DB admin at heart 19:35:22 <zehicle> and want to normalize everything 19:36:21 <hogepodge> zehicle: I trust your experience. 19:36:42 <hogepodge> zehicle: Maybe I need to make another trip to Austin next month and we can chat about these files f2f. ;-) 19:36:58 <zehicle> always happy to have you in town! 19:37:28 <Rockyg> Jeez, if you were going to do it, you should have done it during SXSW 19:37:46 <zehicle> lol, except I was not here 19:38:35 <catherineD> Time to move to the next topic? 19:38:41 <Rockyg> ++ 19:39:18 <catherineD> #topic 2) continue discuss the remaining items of the Feb f2f action list. (including OpenID integration item 1.0.1) 19:39:35 <catherineD> #link https://etherpad.openstack.org/p/refstack_f2f_feb_2015 19:40:06 <catherineD> section 1.0.1 sslypushenko_: was doing some investigation 19:40:43 <sslypushenko_> Yep... I have disscussion with Vlad and Chris 19:41:42 <catherineD> section 1.0.1 in the "action items for vancouver" at the bottom of https://etherpad.openstack.org/p/refstack_f2f_feb_2015 19:41:49 <sslypushenko_> Final decisions is that in term of auth we should follow common way, which used in openstac gerrit for example 19:42:30 <sslypushenko_> Vlad already have started development 19:42:57 <sslypushenko_> I will post in refstack channel link on workflow diagram 19:43:18 <catherineD> sslypushenko_: thx 19:43:23 <sslypushenko_> I hope it will make things clear 19:43:43 <vladiskuz_> catherineD: It's first step https://review.openstack.org/#/c/166247/ 19:44:48 <sslypushenko_> In short, We are going to use OpenstackId for web UI auth and RSA key pairs for CLI tool 19:45:04 <catherineD> sslypushenko_: vladiskuz_: thx .. Let's move to section 3.2.1.1 19:45:16 <sslypushenko_> Like gerrit or github 19:45:41 <catherineD> we need additional API to support page 4 of the mockup 19:45:53 <catherineD> I have submited a review https://review.openstack.org/#/c/166357/ 19:46:11 <catherineD> could you all review ...? 19:46:27 <catherineD> we really need this API ... 19:46:34 <sslypushenko_> Sure thing) 19:47:13 <sslypushenko_> Vlad already pushed on revied basic version of retrival API 19:47:46 <sslypushenko_> It is possible to add some filters to it 19:47:49 <catherineD> currently, after user collects the test results from refstack-client ... they want to look at the results ... and it is very cumbersome now ... 19:48:31 <catherineD> that is the paint that jlk experiences.... 19:48:52 <sslypushenko_> https://review.openstack.org/#/c/166247/ 19:49:08 <sslypushenko_> It is our first step on that way 19:50:20 <catherineD> sslypushenko_: I need to discuss with vladiskuz_: in detail between https://review.openstack.org/#/c/166247/ and https://review.openstack.org/#/c/166357/ in #refstack after this meeting .. 19:50:52 <vladiskuz_> catherineD: np ;) 19:50:59 <catherineD> that is all the topic I have today ... 19:51:29 <catherineD> zehicle: could you give defcore update? 19:51:36 <zehicle> yy 19:51:45 <zehicle> DefCore's been moving REALLY fast latest 19:52:10 <zehicle> we've got the 2015.03 spec landed with the template for the 2015.next in also 19:52:16 <zehicle> process draft should hit soon too 19:52:21 <zehicle> that means we're on track for the summit 19:52:43 <zehicle> all of that in the openstack/defcore repo. so we'll start to pull down the refstack/defcore folder 19:52:50 <zehicle> it should be gone by the summit 19:53:16 <hogepodge> in my mind the biggest road block is fixing tests that rely on neutron. Going to nyc next week to try and tie a ribbon on that. 19:53:19 <zehicle> the 2015.03.json file replaces the capabilities/sections files that we used in the past 19:53:27 <zehicle> format is very similar 19:53:51 <zehicle> hogepodge, any idea on that keystone test? 19:53:53 <catherineD> zehicle: will refstack be used to present results to users? 19:53:56 <zehicle> so we can include the capability 19:54:04 <hogepodge> zehicle: in progress 19:54:07 <zehicle> it's not in the requirements 19:54:19 * morganfainberg feels a disturbance in Freenode... 19:54:20 <zehicle> but that is my expectation 19:54:24 <hogepodge> #link https://review.openstack.org/#/c/166327/ 19:54:32 <hogepodge> v2 is done, working on v3 19:54:54 <catherineD> what we found that with the capability file in defcore .. how can refstack ensure that refstack will always use the latest capability file? 19:55:21 <zehicle> the process is being written to not create a lot of implementation specifics unless we need to make it so 19:55:29 <hogepodge> catherineD: refstack is going to need versioned codecs I think, at least for this year 19:55:37 <catherineD> do we need to fetch the capabilityf file each time refstack render result .. 19:56:07 <zehicle> good question, would it help if we added an "active guidelines" file w/ the right pointers? 19:56:22 <zehicle> a simple JSON list w/ all the specs and their status 19:56:23 <zehicle> ? 19:56:40 <zehicle> they are designed to sort nicely 19:56:50 <catherineD> I am concern more on the rabpid changing in defcore capability and refstack may miss it ... 19:57:00 <zehicle> so you could also just grab the alpha list max 19:57:33 <zehicle> defcore and "rapid change" should not go together 19:58:05 <catherineD> I will log this for our next week discussion ..since we are about out of time now 19:59:09 <catherineD> sslypushenko_: vladiskuz_: Let's discuss the reviews in #refstack ... 19:59:23 <vladiskuz_> catherineD: Let's go 19:59:27 <catherineD> need to end meeting now ... thank you all! 19:59:42 <catherineD> #endmeeting