19:01:29 <catherineD> #startmeeting refstack 19:01:30 <openstack> Meeting started Mon Nov 9 19:01:29 2015 UTC and is due to finish in 60 minutes. The chair is catherineD. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:31 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:01:33 <openstack> The meeting name has been set to 'refstack' 19:01:48 <catherineD> roll call 19:01:49 <pvaneck> o/ 19:03:23 <catherineD> hello pvaneck: let's just wait for a few minutes ... 19:03:31 <pvaneck> sure 19:03:35 <sslypushenko_> Hi, folks! 19:03:49 <pvaneck> hello! 19:03:50 <catherineD> #link meeting agenda and notes, please feel free to add items https://etherpad.openstack.org/p/refstack-meeting-15-11-09 19:04:05 <catherineD> sslypushenko_: hello ... how are you? 19:04:59 <sslypushenko_> Fine! Thx) How are you? 19:05:50 <catherineD> very good ... just come back from one week vacation ... I am sorry that the session during the summit did not work well with you ... 19:06:12 <sslypushenko_> Every thing is ok) 19:06:21 <catherineD> by now we know that remote meeting had not been working for us ... 19:06:28 <sslypushenko_> There were a lot of attendies 19:07:01 <catherineD> yes ... the important thing is we need contributor ... 19:08:21 <sslypushenko_> I think 3 guys from RackSpace it is more than enough 19:08:22 <catherineD> Alexandre Levine from EMC had prototyped on central and EC2 testing 19:08:55 <catherineD> please check this out http://52.8.233.67:8000/#/ 19:09:39 <catherineD> so we try to see how to bring his code into RefStack .. 19:10:05 <catherineD> Let's start with our agenda today ... do you get the agenda link ? 19:10:05 <sslypushenko_> looks cool 19:10:12 <pvaneck> yes go ahead 19:10:26 <sslypushenko_> Yeap, I have the link 19:10:32 <catherineD> #topic Review Tokyo summit action items priority 19:10:56 <catherineD> #link Tokyo summit discussion https://etherpad.openstack.org/p/refstack-tokyo-summit 19:11:47 <catherineD> please scroll to the bottom in the session "RefStack Mitaka cycle priority" 19:12:18 <catherineD> the first item was from Chris 19:13:18 <catherineD> the second item is for Alex to bring in EC2 guideline ... but may also be applicable to others .. 19:13:50 <catherineD> of the 5 items ,.. item #3 will be the one that need a lot of work 19:13:52 <pvaneck> so for the first, do we just generate an arbitrary cpid? 19:14:15 <catherineD> The thinking is let the user enter cpid 19:14:40 <sslypushenko_> yeah) Finally) 19:14:45 <catherineD> but that is the point we should discuss 19:16:02 <catherineD> I do not think that our current data model require that this field has to be unique 19:16:57 <sslypushenko_> cpid should not be uniqu 19:17:50 <catherineD> do we agree that the field should be input from user? 19:18:18 <sslypushenko_> yes from point of view 19:18:49 <catherineD> actually that item should belong to refstack-client ... let me move it 19:18:52 <davidlenwell> I agree with that 19:19:26 <catherineD> davidlenwell: hello 19:19:31 <davidlenwell> hi everyone! 19:19:38 <sslypushenko_> Hi! 19:19:54 <davidlenwell> sorry im late.. had to get to a coffee shop 19:21:23 <catherineD> sslypushenko_: we (davidlenwell: pvaneck: and myself) spent almost a whole day discussing data model at the summit ... very productive f-2-f meeting 19:22:05 <sslypushenko_> That is good 19:22:12 <davidlenwell> thanks catherineD.. I thought it was very productive as well.. 19:22:20 <davidlenwell> I'm fighting to spend more time with ya'll 19:22:44 <catherineD> sslypushenko_: hope we will have a chance to work f-2-f with you some day ... 19:22:53 <catherineD> davidlenwell: that is great ... 19:23:39 <sslypushenko_> Next summit will be in USA, it is much easy than Japan) 19:23:43 <dliu> good evening, sorry I'm late 19:23:56 <catherineD> do you have any input on the priority listed in https://etherpad.openstack.org/p/refstack-tokyo-summit (see the bottom session "RefStack Mitaka cycle priority ") 19:24:11 <catherineD> dliu: Hi .... what time for you now? 19:24:23 <dliu> hi, Catherine 19:24:33 <sslypushenko_> Everything looks good to me 19:24:40 <dliu> It's 3:20~ am 19:24:52 <catherineD> dliu: wow I am so sorry ... 19:25:12 <dliu> It's ok 19:25:24 <dliu> I am glad to join this meeting 19:25:35 <davidlenwell> is someone here from randy's group? alex? 19:25:45 <sslypushenko_> catherineD It is really depends on how much developers will be evolved in process 19:25:49 <davidlenwell> don't know his irc name 19:26:01 <catherineD> davidlenwell: no one from randy's group here ... 19:26:27 <davidlenwell> We should lobby with them offline to start attending 19:26:39 <catherineD> sslypushenko_: that is my concern int the next topic ... 19:26:47 <catherineD> davidlenwell: yep 19:27:21 <catherineD> if everyone is ok with the priority ... we can go on to the next topic 19:27:32 <pvaneck> sure 19:27:32 <catherineD> #topic RefStack contributor situation 19:27:49 <dliu> yes 19:28:45 <catherineD> So we need contributors from other organizations ... right now high percentage of code contribution is from IBM ... 19:29:08 <catherineD> that really concern me... 19:29:45 <catherineD> I will contact Alex (in Randy's group at EMC) ... 19:30:07 <davidlenwell> it seemed like we had plenty of non-ibm folks at the design session 19:31:57 <pvaneck> yea, give it some time 19:31:59 <catherineD> we need code contribution ... I know it is just beginning but I definitely do not want this trend to go on http://stackalytics.com/?release=mitaka&metric=commits&project_type=openstack&module=refstack-group 19:32:56 <davidlenwell> I'll try to get some stuff landed in my off time this cycle.. 19:33:05 <davidlenwell> it will start with better attendance at this meeting 19:33:12 <davidlenwell> then you can assign actions to folks 19:33:59 <davidlenwell> catherineD: if you want to have a seperate recruiting stratigy meeting I can make time to help with that 19:34:41 <catherineD> davidlenwell: thx.. yes let's work on recruiting startegy .... ideally we need to have at least 4 organizations .... 19:34:48 <sslypushenko_> While RefStack is focusing on Foundation related thing we have nothing to do here( 19:35:13 <davidlenwell> what is the status of working with rally? 19:35:30 <davidlenwell> does our api need any changes to accept data from rally? 19:35:58 <catherineD> davidlenwell: per defcore all tests must be tempest or in-tree ... 19:36:00 <sslypushenko_> O_o why do we need rally here? 19:36:19 <davidlenwell> the idea was that rally can feed us data 19:36:28 <davidlenwell> because its running tempest 19:36:32 <davidlenwell> and has result sets 19:36:40 <davidlenwell> why shouldn't it feed our api data 19:36:52 <catherineD> but someone told me at the conference that rally does send data to RefStack 19:37:02 <davidlenwell> no.. 19:37:05 <davidlenwell> it "can" 19:37:09 <catherineD> we need to check that out 19:37:12 <davidlenwell> nobody has patched it to do so yet 19:37:17 <sslypushenko_> As far as I know rally can not run tempest on environment without magic) 19:37:17 <davidlenwell> boris wants to do this 19:37:27 <davidlenwell> boris42 you here? 19:37:38 <davidlenwell> boris-42 19:38:21 <davidlenwell> he isn't in the channel .. but I pinged him 19:38:24 <sslypushenko_> now 10:37 PM at his place 19:38:31 <davidlenwell> he lives in us now 19:38:39 <davidlenwell> its 11:38 am .. 19:38:40 <sslypushenko_> heh) 19:39:37 <davidlenwell> but the marantis office has lots of meeitngs on monday from what I hear.. so he might be busy 19:39:49 <davidlenwell> so lets move on and come back to that.. sorry for derailing 19:39:53 <catherineD> ok there is not reason why anyone running tempest would not be able to send data to refstack with our API 19:40:23 <davidlenwell> sure ... but having an option in rally to just do so at the end of its run .. could give us lots of data from folks who don't want to run the refstack client 19:40:42 <catherineD> davidlenwell: I will ping you on strategy to recruite developers 19:40:45 <boris-42> Hi all 19:40:50 <davidlenwell> hey boris-42 19:40:53 <sslypushenko_> Hi! 19:40:56 <davidlenwell> thanks for popping in 19:40:59 <boris-42> how things are going?) 19:41:16 <davidlenwell> we were just discussing the desire to have rally submit data to the refstack api optionally 19:41:34 <davidlenwell> thought you might have thoughts 19:42:13 <davidlenwell> defcore has asked the community to submit data to refstack.. as much as possible.. to help guide what ends up in defcore 19:42:41 <hogepodge> I will only accept test results in refstack.openstack.org now. 19:42:47 <davidlenwell> with that in mind .. I think it would be low hanging fruit to add a "submit data to refstack" option to rally 19:43:09 <sslypushenko_> Hi! hogepodge 19:43:14 <hogepodge> Previously other methods were ok, but the foundation needs a single repository of results. 19:43:16 <davidlenwell> sure hogepodge.. but for raw data sake.. a tempest run from rally and a tempest run from the refstack client shouldn't be any different 19:43:46 <hogepodge> davidlenwell: I have no opinion as to how it gets there, as long as the results are honest. 19:43:50 <davidlenwell> all we are discussing is possibly letting rally upload its results right into refstack.openstack.org and bypass the need for refstack-client 19:44:10 <davidlenwell> so that we can get data optionally from folks already set up to run rally 19:45:00 <hogepodge> I don't want to undermine the efforts of the refstack team. Collaboration is the best option. 19:45:28 <davidlenwell> sue.. not suggesting to undermine them at all.. just encourage data from as many sources as available 19:45:37 <davidlenwell> the client is still a needed thing.. 19:46:00 <davidlenwell> but there is this other project that runs tempest and has data.. it makes sense to make it easy to collect as much data as possible 19:46:55 <sslypushenko_> Can be rally data considered as meaningful? 19:47:15 <davidlenwell> it should still have raw subunit from its last tempest run 19:47:20 <catherineD> The top priority for refstack-client in Mitaka cycle is "Refstack to provide option with subunit data format as input data" 19:47:32 <davidlenwell> catherineD: agreed 19:47:47 <davidlenwell> but I would suggest that the api needs to be able to take raw sub unit as well 19:48:29 <pvaneck> how large can subunit files get with a full tempest run? 19:48:50 <catherineD> davidlenwell: if we provide an API .. that means that the subunit data will be shipped to the server side? 19:49:03 <davidlenwell> optionally yes 19:49:24 <davidlenwell> I'm suggesting that we make the api friendly to data in lots of formats.. to make it simpler for folks to submit data 19:49:35 <boris-42> uh sorry 19:49:37 <catherineD> That means that RefStack server may be exposed to user confidential data ... 19:49:48 <hogepodge> I'm not sure. It would be nice to have subunit2sql used, but data sanitization is a concern. I'm not sure what the status of it is (mtreinish would know). 19:50:01 <boris-42> so the thing here is next 19:50:07 <catherineD> DefCore guideline is RefStack only expose to pass data not all data ... 19:50:11 <boris-42> rally can manage tempest fully and collect /store results 19:50:27 <boris-42> and now we are working on "export" functionallity 19:50:39 <boris-42> so basically users will write something like 19:50:55 <boris-42> "rally verify export refstack://<connection_string>" 19:51:20 <boris-42> we are working on this spec currently: https://github.com/openstack/rally/blob/master/doc/specs/in-progress/task_and_verification_export.rst 19:51:28 <davidlenwell> boris-42: would the export function have something that scrubs sensitive information? 19:51:33 <mtreinish> hogepodge: data sanitization? 19:52:02 <hogepodge> mtreinish: sensitive information in the subunit files that vendor may not want to send to a public server 19:52:04 <boris-42> davidlenwell: it doesn't contains anything sensitive 19:52:09 <boris-42> davidlenwell: as far as I know 19:52:16 <boris-42> davidlenwell: or you mean exceptions ? 19:52:22 <boris-42> davidlenwell: logs of exceptions* 19:52:22 <davidlenwell> boris-42: sometimes failures contain debug output that can have sensitive info 19:52:28 <davidlenwell> yes 19:52:35 <mtreinish> hogepodge: you just don't store the attachments, by default subunit2sql just drops them on the floor 19:52:40 <boris-42> davidlenwell: so we can add argument maybe 19:52:51 <boris-42> davidlenwell: to rally verify that won't send those logs at all 19:53:11 <mtreinish> hogepodge: that's how it's used by infra for example (not from sec concerns but because it takes too much space and is stored elsewhere) 19:53:13 <davidlenwell> boris-42: that sounds like its worth doing for the export feature in general 19:53:45 <boris-42> davidlenwell: doesn't seem like a hard task 19:53:59 <davidlenwell> yeah .. seems simple to me.. 19:54:43 <davidlenwell> catherineD: I'd suggest that the api method that recieves subunit do the same thing that subunit2sql does and just drop that data .. doesn't need to store it or process it at all.. 19:54:45 <mtreinish> catherineD: for only storing successes I don't think that's the right approach, but you can just use subunit-filter to do that 19:54:55 <mtreinish> catherineD: https://github.com/testing-cabal/subunit/blob/master/filters/subunit-filter 19:55:16 <catherineD> hogepodge: mtreinish: DefCore guideline is RefStack should only be exposed to pass data ... DefCore needs to give new guildeline that we can accept the data but not to store it ... 19:55:27 <boris-42> davidlenwell: so I will ping you guys when we finish export functionallity 19:55:35 <davidlenwell> thanks boris-42 19:55:36 <boris-42> davidlenwell: so we will be able to do some kind of demo 19:55:39 <catherineD> with that then using subunit2sql is possible 19:55:42 <hogepodge> catherineD: ok, we can make that an agenda item for Wednesday 19:55:54 <catherineD> hogepodge: yup .. 19:56:29 <davidlenwell> I like the idea of using subunit2sql.. because it lets us focus energy on the website ui rather than maintaining redunand subunit tools 19:56:57 <sslypushenko_> +1 19:58:08 <catherineD> as hogepodge: said we can make that as an agenda item at DefCore meeting that RefStack server can accept subunit data but won't store them . 19:58:29 <pvaneck> as a side note while time is dwindling down: https://review.openstack.org/#/c/242919/ this patch fixes https://bugs.launchpad.net/refstack/+bug/1514290 19:58:29 <openstack> Launchpad bug 1514290 in refstack "Occasional Internal Server error when trying to log in" [High,Confirmed] - Assigned to Paul Van Eck (pvaneck-z) 19:58:52 <pvaneck> i would frequently hit this when trying to login to the refstack site 19:59:07 <catherineD> pvaneck: thx ... that is a high priority bug 19:59:18 <catherineD> need to end the meeting now .... 19:59:21 <catherineD> thank you all 19:59:48 <catherineD> #endmeeting