19:01:29 <catherineD> #startmeeting refstack
19:01:30 <openstack> Meeting started Mon Nov  9 19:01:29 2015 UTC and is due to finish in 60 minutes.  The chair is catherineD. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:31 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:01:33 <openstack> The meeting name has been set to 'refstack'
19:01:48 <catherineD> roll call
19:01:49 <pvaneck> o/
19:03:23 <catherineD> hello pvaneck: let's just wait for a few minutes ...
19:03:31 <pvaneck> sure
19:03:35 <sslypushenko_> Hi, folks!
19:03:49 <pvaneck> hello!
19:03:50 <catherineD> #link meeting agenda and notes, please feel free to add items https://etherpad.openstack.org/p/refstack-meeting-15-11-09
19:04:05 <catherineD> sslypushenko_: hello ... how are you?
19:04:59 <sslypushenko_> Fine! Thx) How are you?
19:05:50 <catherineD> very good ... just come back from one week vacation ... I am sorry that the session during the summit did not work well with you ...
19:06:12 <sslypushenko_> Every thing is ok)
19:06:21 <catherineD> by now we know that remote meeting had not been working for us ...
19:06:28 <sslypushenko_> There were a lot of attendies
19:07:01 <catherineD> yes ... the important thing is we need contributor ...
19:08:21 <sslypushenko_> I think 3 guys from RackSpace it is more than enough
19:08:22 <catherineD> Alexandre Levine from EMC had prototyped on central and EC2 testing
19:08:55 <catherineD> please check this out http://52.8.233.67:8000/#/
19:09:39 <catherineD> so we try to see how to bring his code into RefStack ..
19:10:05 <catherineD> Let's start with our agenda today ... do you get the agenda link ?
19:10:05 <sslypushenko_> looks cool
19:10:12 <pvaneck> yes go ahead
19:10:26 <sslypushenko_> Yeap, I have the link
19:10:32 <catherineD> #topic Review Tokyo summit action items priority
19:10:56 <catherineD> #link Tokyo summit discussion  https://etherpad.openstack.org/p/refstack-tokyo-summit
19:11:47 <catherineD> please scroll to the bottom in the session "RefStack Mitaka cycle priority"
19:12:18 <catherineD> the first item was from Chris
19:13:18 <catherineD> the second item is for Alex to bring in EC2 guideline ... but may also be applicable to others ..
19:13:50 <catherineD> of the 5 items ,.. item #3 will be the one that need a lot of work
19:13:52 <pvaneck> so for the first, do we just generate an arbitrary cpid?
19:14:15 <catherineD> The thinking is let the user enter cpid
19:14:40 <sslypushenko_> yeah) Finally)
19:14:45 <catherineD> but that is the point  we should discuss
19:16:02 <catherineD> I do not think that our current data model require that this field has to be unique
19:16:57 <sslypushenko_> cpid should not be uniqu
19:17:50 <catherineD> do we agree that the field should be input from user?
19:18:18 <sslypushenko_> yes from point of view
19:18:49 <catherineD> actually that item should belong to refstack-client ... let me move it
19:18:52 <davidlenwell> I agree with that
19:19:26 <catherineD> davidlenwell: hello
19:19:31 <davidlenwell> hi everyone!
19:19:38 <sslypushenko_> Hi!
19:19:54 <davidlenwell> sorry im late.. had to get to a coffee shop
19:21:23 <catherineD> sslypushenko_: we (davidlenwell: pvaneck: and myself) spent almost a whole day discussing data model at  the summit ... very productive f-2-f meeting
19:22:05 <sslypushenko_> That is good
19:22:12 <davidlenwell> thanks catherineD.. I thought it was very productive as well..
19:22:20 <davidlenwell> I'm fighting to spend more time with ya'll
19:22:44 <catherineD> sslypushenko_: hope we will have a chance to work f-2-f with you some day ...
19:22:53 <catherineD> davidlenwell: that is great ...
19:23:39 <sslypushenko_> Next summit will be in USA, it is much easy than Japan)
19:23:43 <dliu> good evening, sorry I'm late
19:23:56 <catherineD> do you have any input on the priority listed in https://etherpad.openstack.org/p/refstack-tokyo-summit   (see the bottom session "RefStack Mitaka cycle priority ")
19:24:11 <catherineD> dliu: Hi .... what time for you now?
19:24:23 <dliu> hi, Catherine
19:24:33 <sslypushenko_> Everything looks good to me
19:24:40 <dliu> It's 3:20~ am
19:24:52 <catherineD> dliu: wow I am so sorry ...
19:25:12 <dliu> It's ok
19:25:24 <dliu> I am glad to join this meeting
19:25:35 <davidlenwell> is someone here from randy's group? alex?
19:25:45 <sslypushenko_> catherineD It is really depends on how much developers will be evolved in process
19:25:49 <davidlenwell> don't know his irc name
19:26:01 <catherineD> davidlenwell: no one from randy's group here ...
19:26:27 <davidlenwell> We should lobby with them offline to start attending
19:26:39 <catherineD> sslypushenko_: that is my concern int the next topic ...
19:26:47 <catherineD> davidlenwell: yep
19:27:21 <catherineD> if  everyone is ok with the priority ... we can go on to the next topic
19:27:32 <pvaneck> sure
19:27:32 <catherineD> #topic RefStack contributor situation
19:27:49 <dliu> yes
19:28:45 <catherineD> So we need contributors from other organizations ... right now high percentage of code contribution is from IBM ...
19:29:08 <catherineD> that really concern me...
19:29:45 <catherineD> I will contact Alex (in Randy's group at EMC) ...
19:30:07 <davidlenwell> it seemed like we had plenty of non-ibm folks at the design session
19:31:57 <pvaneck> yea, give it some time
19:31:59 <catherineD> we need code contribution ... I know it is just beginning but I definitely do not want this trend to go on http://stackalytics.com/?release=mitaka&metric=commits&project_type=openstack&module=refstack-group
19:32:56 <davidlenwell> I'll try to get some stuff landed in my off time this cycle..
19:33:05 <davidlenwell> it will start with better attendance at this meeting
19:33:12 <davidlenwell> then you can assign actions to folks
19:33:59 <davidlenwell> catherineD: if you want to have a seperate recruiting stratigy meeting I can make time to help with that
19:34:41 <catherineD> davidlenwell: thx.. yes let's work on recruiting startegy ....  ideally we need to have at least 4 organizations ....
19:34:48 <sslypushenko_> While RefStack is focusing on Foundation related thing we have nothing to do here(
19:35:13 <davidlenwell> what is the status of working with rally?
19:35:30 <davidlenwell> does our api need any changes to accept data from rally?
19:35:58 <catherineD> davidlenwell: per defcore all tests must be tempest or in-tree ...
19:36:00 <sslypushenko_> O_o why do we need rally here?
19:36:19 <davidlenwell> the idea was that rally can feed us data
19:36:28 <davidlenwell> because its running tempest
19:36:32 <davidlenwell> and has result sets
19:36:40 <davidlenwell> why shouldn't it feed our api data
19:36:52 <catherineD> but someone told me at the conference that rally does send data to RefStack
19:37:02 <davidlenwell> no..
19:37:05 <davidlenwell> it "can"
19:37:09 <catherineD> we need to check that out
19:37:12 <davidlenwell> nobody has patched it to do so yet
19:37:17 <sslypushenko_> As far as I know rally can not run tempest on environment without magic)
19:37:17 <davidlenwell> boris wants to do this
19:37:27 <davidlenwell> boris42 you here?
19:37:38 <davidlenwell> boris-42
19:38:21 <davidlenwell> he isn't in the channel .. but I pinged him
19:38:24 <sslypushenko_> now 10:37 PM at his place
19:38:31 <davidlenwell> he lives in us now
19:38:39 <davidlenwell> its 11:38 am ..
19:38:40 <sslypushenko_> heh)
19:39:37 <davidlenwell> but the marantis office has lots of meeitngs on monday from what I hear.. so he might be busy
19:39:49 <davidlenwell> so lets move on and come back to that.. sorry for derailing
19:39:53 <catherineD> ok there is not reason why anyone running tempest would not be able to send data to refstack with our API
19:40:23 <davidlenwell> sure ... but having an option in rally to just do so at the end of its run .. could give us lots of data from folks who don't want to run the refstack client
19:40:42 <catherineD> davidlenwell: I will ping you on strategy to recruite developers
19:40:45 <boris-42> Hi all
19:40:50 <davidlenwell> hey boris-42
19:40:53 <sslypushenko_> Hi!
19:40:56 <davidlenwell> thanks for popping in
19:40:59 <boris-42> how things are going?)
19:41:16 <davidlenwell> we were just discussing the desire to have rally submit data to the refstack api optionally
19:41:34 <davidlenwell> thought you might have thoughts
19:42:13 <davidlenwell> defcore has asked the community to submit data to refstack.. as much as possible.. to help guide what ends up in defcore
19:42:41 <hogepodge> I will only accept test results in refstack.openstack.org now.
19:42:47 <davidlenwell> with that in mind .. I think it would be low hanging fruit to add a "submit data to refstack" option to rally
19:43:09 <sslypushenko_> Hi! hogepodge
19:43:14 <hogepodge> Previously other methods were ok, but the foundation needs a single repository of results.
19:43:16 <davidlenwell> sure hogepodge.. but for raw data sake.. a tempest run from rally and a tempest run from the refstack client shouldn't be any different
19:43:46 <hogepodge> davidlenwell: I have no opinion as to how it gets there, as long as the results are honest.
19:43:50 <davidlenwell> all we are discussing is possibly letting rally upload its results right into refstack.openstack.org and bypass the need for refstack-client
19:44:10 <davidlenwell> so that we can get data optionally from folks already set up to run rally
19:45:00 <hogepodge> I don't want to undermine the efforts of the refstack team. Collaboration is the best option.
19:45:28 <davidlenwell> sue.. not suggesting to undermine them at all.. just encourage data from as many sources as available
19:45:37 <davidlenwell> the client is still a needed thing..
19:46:00 <davidlenwell> but there is this other project that runs tempest and has data.. it makes sense to make it easy to collect as much data as possible
19:46:55 <sslypushenko_> Can be rally data considered as meaningful?
19:47:15 <davidlenwell> it should still have raw subunit from its last tempest run
19:47:20 <catherineD> The top priority for refstack-client in Mitaka cycle is "Refstack to provide option with subunit data format as input data"
19:47:32 <davidlenwell> catherineD: agreed
19:47:47 <davidlenwell> but I would suggest that the api needs to be able to take raw sub unit as well
19:48:29 <pvaneck> how large can subunit files get with a full tempest run?
19:48:50 <catherineD> davidlenwell:  if we provide an API .. that means that the subunit data will be shipped to the server side?
19:49:03 <davidlenwell> optionally yes
19:49:24 <davidlenwell> I'm suggesting that we make the api friendly to data in lots of formats.. to make it simpler for folks to submit data
19:49:35 <boris-42> uh sorry
19:49:37 <catherineD> That means that RefStack server may be exposed to user confidential data ...
19:49:48 <hogepodge> I'm not sure. It would be nice to have subunit2sql used, but data sanitization is a concern. I'm not sure what the status of it is (mtreinish would know).
19:50:01 <boris-42> so the thing here is next
19:50:07 <catherineD> DefCore guideline is RefStack only expose to pass data not all data ...
19:50:11 <boris-42> rally can manage tempest fully and collect /store results
19:50:27 <boris-42> and now we are working on "export" functionallity
19:50:39 <boris-42> so basically users will write something like
19:50:55 <boris-42> "rally verify export refstack://<connection_string>"
19:51:20 <boris-42> we are working on this spec currently: https://github.com/openstack/rally/blob/master/doc/specs/in-progress/task_and_verification_export.rst
19:51:28 <davidlenwell> boris-42:  would the export function have something that scrubs sensitive information?
19:51:33 <mtreinish> hogepodge: data sanitization?
19:52:02 <hogepodge> mtreinish: sensitive information in the subunit files that  vendor may not want to send to a public server
19:52:04 <boris-42> davidlenwell: it doesn't contains anything sensitive
19:52:09 <boris-42> davidlenwell: as far as I know
19:52:16 <boris-42> davidlenwell: or you mean exceptions ?
19:52:22 <boris-42> davidlenwell: logs of exceptions*
19:52:22 <davidlenwell> boris-42: sometimes failures contain debug output that can have sensitive info
19:52:28 <davidlenwell> yes
19:52:35 <mtreinish> hogepodge: you just don't store the attachments, by default subunit2sql just drops them on the floor
19:52:40 <boris-42> davidlenwell: so we can add argument maybe
19:52:51 <boris-42> davidlenwell: to rally verify that won't send those logs at all
19:53:11 <mtreinish> hogepodge: that's how it's used by infra for example (not from sec concerns but because it takes too much space and is stored elsewhere)
19:53:13 <davidlenwell> boris-42: that sounds like its worth doing for the export feature in general
19:53:45 <boris-42> davidlenwell: doesn't seem like a hard task
19:53:59 <davidlenwell> yeah .. seems simple to me..
19:54:43 <davidlenwell> catherineD: I'd suggest that the api method that recieves subunit do the same thing that subunit2sql does and just drop that data .. doesn't need to store it or process it at all..
19:54:45 <mtreinish> catherineD: for only storing successes I don't think that's the right approach, but you can just use subunit-filter to do that
19:54:55 <mtreinish> catherineD: https://github.com/testing-cabal/subunit/blob/master/filters/subunit-filter
19:55:16 <catherineD> hogepodge: mtreinish: DefCore guideline is RefStack should only be exposed to pass data ... DefCore needs to give new guildeline that we can accept the data but not to store it ...
19:55:27 <boris-42> davidlenwell: so I will ping you guys when we finish export functionallity
19:55:35 <davidlenwell> thanks boris-42
19:55:36 <boris-42> davidlenwell: so we will be able to do some kind of demo
19:55:39 <catherineD> with that then using subunit2sql is possible
19:55:42 <hogepodge> catherineD: ok, we can make that an agenda item for Wednesday
19:55:54 <catherineD> hogepodge: yup ..
19:56:29 <davidlenwell> I like the idea of using subunit2sql.. because it lets us focus energy on the website ui rather than maintaining redunand subunit tools
19:56:57 <sslypushenko_> +1
19:58:08 <catherineD> as hogepodge: said we can make that as an agenda item at DefCore meeting that RefStack server can accept subunit data but won't store them .
19:58:29 <pvaneck> as a side note while time is dwindling down: https://review.openstack.org/#/c/242919/ this patch fixes https://bugs.launchpad.net/refstack/+bug/1514290
19:58:29 <openstack> Launchpad bug 1514290 in refstack "Occasional Internal Server error when trying to log in" [High,Confirmed] - Assigned to Paul Van Eck (pvaneck-z)
19:58:52 <pvaneck> i would frequently hit this when trying to login to the refstack site
19:59:07 <catherineD> pvaneck: thx ... that is a high priority bug
19:59:18 <catherineD> need to end the meeting now ....
19:59:21 <catherineD> thank you all
19:59:48 <catherineD> #endmeeting