18:01:37 <krtaylor> #startmeeting third-party
18:01:37 <openstack> Meeting started Mon Jun 30 18:01:37 2014 UTC and is due to finish in 60 minutes.  The chair is krtaylor. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:01:38 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:01:40 <openstack> The meeting name has been set to 'third_party'
18:01:47 <krtaylor> #chair anteaya
18:01:48 <openstack> Current chairs: anteaya krtaylor
18:02:03 <krtaylor> Hi everyone
18:02:06 <mestery> o/
18:02:12 <krtaylor> Who is here for third-party?
18:02:13 <anteaya> o/
18:02:15 <fawadkhaliq> hello everyone
18:02:18 <lyxus> +1
18:02:57 <ilyashakhat__> +1
18:03:06 <krtaylor> so let's get started then
18:03:12 <krtaylor> here is the agenda:
18:03:30 * mestery pings lukego as well.
18:03:34 <lukego> hoi
18:03:36 <krtaylor> #link https://wiki.openstack.org/wiki/Meetings/ThirdParty#Agenda_for_next_meeting
18:03:48 <mestery> lukego: Nice timing. ;_P
18:04:05 <krtaylor> hi all
18:04:10 <krtaylor> #topic Welcome & Reminder of OpenStack Mission
18:04:27 <krtaylor> #info The OpenStack Open Source Cloud Mission: to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable.
18:04:44 <asselin> hi
18:04:47 <krtaylor> #topic Review of previous week's open action items
18:05:23 <krtaylor> ilyashakhat__, are you here?
18:05:32 <ilyashakhat__> krtaylor: yes
18:05:40 <anteaya> he doen'st have an item from last week
18:05:47 <anteaya> that is from the prior weeks agenda
18:05:58 <anteaya> the open items from last week are the two etherpads
18:06:07 <anteaya> one to discuss log suggestions
18:06:21 <anteaya> one to discuss length of time to report test results
18:06:32 <krtaylor> ah, yes, it helps to refresh the page
18:06:36 <anteaya> it does
18:06:55 <SlickNik> (sorry a tad late) o/
18:07:09 <anteaya> glad your here SlickNik
18:07:12 <luqas> me too, hi
18:07:15 <anteaya> you're
18:07:22 <krtaylor> #topic etherpad log suggestions
18:07:30 <krtaylor> here is the etherpad
18:08:06 <krtaylor> #link https://etherpad.openstack.org/p/LogServerGeneralRecommendations
18:08:15 <krtaylor> anteaya, this was the right one?
18:08:21 <anteaya> yes
18:08:29 <anteaya> you're doing great
18:08:46 <anteaya> so I'll jump in here
18:08:57 <anteaya> so right now we have some suggestions for logging
18:09:00 <lyxus> Can we comment on it ? or you prefer the etherpad ?
18:09:03 <krtaylor> so, this was the discussion around the doc patch for logs
18:09:14 <anteaya> lyxus: either
18:09:21 <anteaya> krtaylor: was it?
18:09:25 <krtaylor> and where they should be kept?
18:09:30 <krtaylor> that was a question :)
18:09:33 <anteaya> I felt this was motivated by it
18:09:35 <anteaya> ah
18:09:37 <krtaylor> yes
18:09:47 <anteaya> I thought this was our suggestions for others
18:10:01 <anteaya> not requirements, but suggestions to help people, if they want
18:10:25 <lyxus> I think it's a good thing to specify that the logs should be freely availble and not altered in anyways (no dropbox or box, etc..)
18:10:34 <anteaya> so given the current suggestions, I'm wondering if anyone is willing to put together a patch to offer to third_party.rst
18:10:49 <lyxus> but the apache etc... should be a suggestions, some people would use nginx, etc...
18:10:57 <anteaya> lyxus: can you expand what you mean by not alterned
18:11:10 <anteaya> how does dropbox alter them?
18:11:14 <lyxus> * altered
18:11:27 <anteaya> yeah altered
18:12:11 <lyxus> You want an example ?
18:12:15 <anteaya> sure
18:12:25 <lyxus> https://www.dropbox.com/sh/11r10ij1gbejw8b/AABSFSAXduSlQUsIktiruNYHa
18:12:43 <lyxus> I want to see what is run, i have to download the file and search in it
18:12:50 <anteaya> ah
18:12:57 <anteaya> you have me at download
18:13:05 <anteaya> files must be browsable
18:13:13 <anteaya> that is in a patch for requirements
18:13:18 <lyxus> another vendor http://119.15.112.63/ci_midokura_logs_2014-06-30-171123/logs/
18:13:31 <anteaya> we haven't gotten there yet in the agenda
18:13:33 <krtaylor> anteaya, do we need a new patch? or add to your existing one?
18:13:40 <anteaya> let's have a new one
18:13:41 <krtaylor> #link https://review.openstack.org/#/c/101227/
18:13:48 <anteaya> mine is a requirement
18:13:59 <anteaya> these are suggestions for how to meet the reuqirement
18:14:21 <krtaylor> anteaya, understood, a how-to type discussion, gotcha
18:14:37 <anteaya> lyxus: and thanks for the link on the midokura logs, I will send an email after the meeting
18:14:43 <anteaya> krtaylor: right
18:14:50 <krtaylor> ok, since you are out, I'll take that one
18:14:58 <anteaya> krtaylor: awesome thanks
18:15:00 <lyxus> anteaya, there are not the only one. It's just an example
18:15:38 <anteaya> lyxus: any other examples you find where folks are expecting openstack devs to download logs please bring them to the attention of me, krtaylor or anyone in infra
18:15:50 <anteaya> so we can address this and get browsable logs for devs
18:16:00 <lyxus> anteaya, ok will do
18:16:03 <anteaya> shall we move to the next item, krtaylor?
18:16:03 <luqas> sorry
18:16:09 <krtaylor> #action krtaylor to document how-to for hosting requirements in third_party.rst
18:16:13 <luqas> whatis wrong with midokura logs?
18:16:21 <krtaylor> anteaya, typing fast as I can :)
18:16:44 <anteaya> luqas: in firefox you are forced to download them
18:16:51 <anteaya> krtaylor: you are doing great
18:16:53 <krtaylor> #topic time to report test results
18:16:58 <luqas> oh, I see
18:17:15 <luqas> I didn't know
18:17:16 <krtaylor> #link https://etherpad.openstack.org/p/ThirdPartyTimeLimits
18:17:25 <anteaya> luqas: hence the need for this meeting :D
18:17:40 <luqas> what do I need to be browsable in firefox?
18:17:45 <krtaylor> ok, new etherpad for time limits discussion, this is a general requirement
18:17:50 <luqas> anteaya: yep :)
18:17:55 <anteaya> this etherpad was to be sweston's email to the ml about how long it takes for tests to report
18:18:05 <anteaya> sweston is on a plane right now
18:18:16 <lyxus> I have put a comment for that too on the etherpad :)
18:18:29 <anteaya> this is an important point we should discuss, can anyone take this and put together an email for the ml
18:18:46 <krtaylor> well we can roll that into a patch discussion for third_party.rst
18:18:52 <anteaya> we need to get some consensus around length of time to wait for results
18:18:54 <krtaylor> but it should be discussed on ml
18:19:06 <anteaya> yes let's hash it out on the ml first
18:19:08 <krtaylor> we may need by-project times too
18:19:14 <anteaya> too hard
18:19:26 <anteaya> if a ci has mulitple projects it is tracking
18:19:29 <krtaylor> because one may need faster results than another
18:19:34 <anteaya> and how to we enforce?
18:19:41 <krtaylor> thats the question
18:19:48 <anteaya> same as log retention = 1 month
18:19:52 <krtaylor> else, slow = missed
18:19:52 <kevinbenton> anteaya: i think 12 hours is fair (e.g. we only have 3 nodes that often get backed up during crunch times)
18:19:57 <anteaya> we should aim to find one timeframe
18:20:11 <anteaya> kevinbenton: I look forward to hearing responses to that on the ml
18:20:11 <lyxus> kevinbenton, +1
18:20:21 <krtaylor> previous discussions have been 4 hours
18:20:21 <anteaya> kevinbenton: would you be willing to start the ml thread?
18:20:27 <krtaylor> in nova at least
18:20:35 <fawadkhaliq> kevinbenton: +1
18:20:36 <anteaya> we need both ci admins and devs to weigh in
18:20:44 <krtaylor> absolutely
18:20:55 <anteaya> can someone start this discussion on the ml?
18:21:01 <kevinbenton> anteaya: sure. what tag should i put in the subject? just  [openstack-dev]?
18:21:17 <anteaya> if you post to the -dev ml don't use that tag
18:21:21 <anteaya> it is the default
18:21:30 <anteaya> use [3rd party]
18:21:33 <krtaylor> kevinbenton, thanks, I'll jump in on that one too
18:21:40 <anteaya> then the post will have both tags
18:21:46 <kevinbenton> anteaya: ok
18:21:48 <anteaya> thanks
18:22:09 <krtaylor> anteaya, if you dont mind, I'd rather keep it consisstent with the meeting
18:22:18 <anteaya> #action kevinbenton to start ml discussion on -dev regard time to post test results
18:22:22 <anteaya> krtaylor: good point
18:22:23 <krtaylor> third-party
18:22:27 <anteaya> yes
18:22:35 <krtaylor> that is one of my peeves, but it is a nit
18:22:40 <anteaya> can you use tag [third-party] kevinbenton?
18:22:44 <anteaya> it is a good nit
18:22:54 <kevinbenton> anteaya: sure
18:22:59 <anteaya> great
18:23:33 <krtaylor> ok that was the discussion for last weeks items, lets move on
18:23:45 <krtaylor> #topic Announcements
18:24:01 <anteaya> I have nothing here
18:24:06 <krtaylor> anyone have any ?
18:24:34 <krtaylor> I'll take that as a no
18:24:36 <krtaylor> next
18:24:45 <krtaylor> #topic OpenStack Program Items
18:25:13 <krtaylor> So, there are the existing patches for third_party.rst
18:25:26 <anteaya> please do review them
18:25:43 <anteaya> and please use the topic third-party for your patches
18:25:57 <anteaya> so we can all track them easily
18:26:13 <krtaylor> ++
18:26:19 <krtaylor> #link https://review.openstack.org/#/c/101013/
18:26:21 <krtaylor> and
18:26:32 <krtaylor> #link https://review.openstack.org/#/c/101227/
18:26:44 <anteaya> oh there are 5
18:26:48 <anteaya> #link https://review.openstack.org/#/q/status:open+project:openstack-infra/config+branch:master+topic:third-party,n,z
18:26:52 <anteaya> is probably easier
18:27:09 <krtaylor> yes, good point
18:27:22 <krtaylor> any discussion on these for today?
18:27:36 <anteaya> let's move on, we can discuss in review
18:27:55 <anteaya> I'd rather get to new business
18:28:25 <krtaylor> yep. next
18:28:30 <anteaya> thanks
18:28:32 <krtaylor> #topic Deadlines & Deprecations
18:28:58 <krtaylor> anything to communicate here?
18:29:05 <krtaylor> any upcoming deadlines?
18:29:08 <anteaya> anyone from cinder here?
18:29:31 <anteaya> I was talking to someone who felt there was no third party requirement from cinder
18:29:46 <asselin> hi, i'm from cinder
18:29:48 <anteaya> and I believe there is
18:29:56 <krtaylor> hi asselin
18:29:56 <anteaya> asselin: hi there
18:30:00 <anteaya> glad you are here
18:30:12 <anteaya> was hoping a cinder core was here
18:30:27 <krtaylor> well, there is a requirement, the problem is what and when :)
18:30:28 <anteaya> asselin: what do you understand about third party deadlines for cinder
18:30:36 <anteaya> krtaylor: true
18:30:39 <asselin> we're supposed to have it setup by J2
18:30:44 <anteaya> okay great
18:30:50 <asselin> for all icehouse drivers
18:30:56 <anteaya> great
18:31:01 <asselin> i.e. new drivers are optional
18:31:11 <anteaya> interesting
18:31:33 <anteaya> I will need to follow up with jgriffith and ensure I understand what that means
18:31:37 <anteaya> because I don't right now
18:31:46 <anteaya> but thanks for sharing your understanding asselin
18:31:51 <krtaylor> yes, and what will be tested
18:31:59 <anteaya> mestery: what is current for neutron?
18:32:09 <asselin> that means is that jgriffith did not want to add additional load to new vendors submitting drivers in Juno.
18:32:22 <mestery> anteaya: The requirement here is that all in-tree drivers need to have functioning CI up and running 2 weeks prior to Juno-2.
18:32:33 * mestery can dig the email up he sent to the list a month ago.
18:32:40 <anteaya> mestery: and how far away is 2 weeks prior to juno
18:32:41 <asselin> so he's giving them more time to setup 3rd party ci
18:32:48 <anteaya> and think that is next week
18:32:59 <mestery> anteaya: July 10 I think.
18:33:01 <mestery> anteaya: Correct.
18:33:04 <anteaya> asselin: okay, would be good to get the timeframes so we can communicate them
18:33:29 <anteaya> mestery: okay and what systems need to show something different than they are doing today by July 10th?
18:33:33 <asselin> mestery, I didn't know it was 2 weeks before J2. we should have jgriffith clarify
18:33:44 <anteaya> asselin: mestery is neutron
18:33:52 <anteaya> asselin: we haev switched programs
18:34:11 <mestery> asselin: Yes, we're on neutron now. :)
18:34:18 <asselin> anteaya, ok, great, so cinder's still J2 :)
18:34:31 <mestery> anteaya: I am just going back through the emails and looking at data now, I should reply to my thread with the timelines post this meeting.
18:34:38 <anteaya> asselin: I'm not setting policy for cinder, I'm asking what cinder's policy is so I know
18:34:39 <krtaylor> I can ping jungleboyj too
18:34:48 <anteaya> mestery: yes please
18:35:02 <mestery> anteaya: Will do.
18:35:05 <anteaya> mestery: if they only have a week to demonstrate compliance, they should know it
18:35:08 <anteaya> mestery: thanks
18:35:09 <asselin> I can look it up in the cinder meeting minutes
18:35:17 <mestery> anteaya: Yes, good point. :)
18:35:23 <anteaya> asselin: let's try for clarity for next week
18:35:28 <anteaya> anyone here from nova?
18:35:41 <krtaylor> #action communicate cinder and neutron timeline
18:35:50 <anteaya> thanks krtaylor
18:35:51 <krtaylor> I have not heard anything new for nova, there was some discussion about raising the bar at summit
18:35:57 <mestery> anteaya: Actually, per my original email, I just indicated Juno-2, not 2 weeks prior, so I guess people have 3 weeks left :)
18:36:03 <mestery> anteaya: I need to stick to what I emailed out.
18:36:09 <anteaya> let's check in with nova and see what current expectations are, krtaylor
18:36:13 <mestery> anteaya: If people do not comply, we will remove drivers in Juno-3.
18:36:14 <krtaylor> but I have not heard anything else about it
18:36:23 <anteaya> mestery: yes, don't change the rules during the game
18:36:27 <mestery> anteaya: ++
18:36:40 <anteaya> mestery: let's still give those folks not in compliance that information
18:36:45 <anteaya> mestery: so they know their status
18:36:59 <mestery> anteaya: Yes, I will reply to the thread and indicate timeframes, etc.
18:37:06 <krtaylor> anteaya, I'll make sure we have current state for next week
18:37:06 <anteaya> mestery: great, thank you
18:37:11 <mestery> anteaya: np
18:37:14 <anteaya> krtaylor: awesome thanks
18:37:23 <anteaya> I've been lax in this topic and that is my fault
18:37:25 <asselin> anteaya, 16:24:41 <jgriffith> navneet1: J2 was our goal. http://eavesdrop.openstack.org/meetings/cinder/2014/cinder.2014-06-25-16.00.log.html
18:37:38 <anteaya> we need to be more assertive here, and get information accurate and spread around
18:37:47 <anteaya> asselin: thanks for finding that
18:38:05 <asselin> 16:24:44 <jgriffith> July 27
18:38:29 <krtaylor> ok, any other deadlines to talk about?
18:38:58 <krtaylor> next item then
18:39:05 <krtaylor> #topic Highlighting a Program or Gerrit Account
18:39:51 <krtaylor> mestery, lukego - discussion on simple ci setup?
18:39:57 <lukego> I have found running a 3rd party CI surprisingly hard in practice. This has been due to diverse operational issues, and never actually related to the code under test. I'm going to try setting up a really minimalist CI and see if that works better. The idea and draft code (written over the past hours) is here: https://github.com/SnabbCo/shellci. I plan to use this for a new mech driver that I am offering for Juno. I welcome comment
18:39:57 <lukego> and ideas :).
18:39:58 <lukego> The initial inspiration is the script that I got from the OpenDaylight developers which is short and effective. I hope the new script will work for my driver and then be useful at least as a reference to others too.
18:39:59 <lukego> See also mailing list thread: http://lists.openstack.org/pipermail/openstack-dev/2014-June/038951.html
18:40:38 <anteaya> mestery: your thoughts?
18:40:39 * mestery reads lukego's comments here.
18:41:16 <asselin> lukego, if that works for you, please share as I'm also interested in looking at it.
18:41:24 <krtaylor> lukego, I share your pain, we have looked at ways to simplify too
18:41:29 <mestery> anteaya: In general, I like this idea. As long as what we're mostly concerned about is the result, this looks slick!
18:41:40 <mestery> lukego: Do you accept pull requests? ;)
18:41:50 <lukego> yep. but code budget is strictly 100 lines :)
18:41:50 <anteaya> mestery: are we going to see actual test results this time?
18:42:11 <mestery> anteaya: I think that's lukego's goal indeed.
18:42:26 <krtaylor> I think the actual difficulty of 3rd party actually surprised a lot of people
18:42:27 <anteaya> mestery: you are the ptl and you are the one with untested code in master
18:42:33 <anteaya> mestery: so it is your decision
18:42:42 <mestery> anteaya: Got it.
18:42:52 <anteaya> krtaylor: yes
18:42:52 <mestery> anteaya: I'd like to see what comes of this and get some infra input as well.
18:43:04 <lukego> the goal is definitely to fulfil the 3rd party requirements as written in the wiki
18:43:06 <anteaya> mestery: do add an item to the infra agenda
18:43:13 <anteaya> mestery: so that you don't get lost in the crowd
18:43:42 <mestery> anteaya: Good idea, will do that once lukego has this fleshed out a bit more.
18:43:45 <kevinbenton> lukego: does this handle cases where the system stalls during package retrieval or code checkout?
18:43:54 <anteaya> the third party requirements aren't written in the wiki, they are written here: http://ci.openstack.org/third_party.html
18:44:12 <anteaya> and that document has a number of patches to it, as linked above
18:44:18 * krtaylor is lost in reading about shellci and must come back to meeting
18:44:25 <lukego> kevinbenton: not yet, but yes it’s a requirement to differentiate those “CI is stuck” issues from actual test failures, to a reasonable extent. (don’t vote -1 unless Tempest say something is wrong.)
18:44:27 * mestery was busy posting review comments on those patches during this meeting.
18:44:39 <anteaya> mestery: thank you
18:44:51 <mestery> anteaya: np
18:45:08 <kevinbenton> lukego: yes, the other case to watch for that might be caused by the patch is neutron fails to start in devstack
18:46:03 <lukego> kevinbenton: please post these ideas to the ML thread! I have had about 4 different weird operational issues and I would love to know what else people have seen — as geneal themes for which parts of the script can go wrong and how to avoid misvotings that could screw up workflows
18:46:12 * krtaylor will have to look into log publishing and storage
18:46:51 <lyxus> kevinbenton, I actually did this test too. I fix the package retrieval issue by mirroring locally the server ubuntu/pypi
18:47:01 <lyxus> *fixed
18:47:06 <anteaya> krtaylor: I'd like to ensure we have some time for stackatlytics discussion too today, though it isn't on the agenda
18:47:31 <krtaylor> anteaya, agreed, this topic is rather open-ended
18:47:40 <ilyashakhat__> anteaya: i'm ready :)
18:47:46 <krtaylor> lukego, thanks for sharing this, I will read more about it
18:48:04 <krtaylor> ok, lets move on then
18:48:21 <krtaylor> #topic Open Discussion
18:48:37 <anteaya> so actually I want to talk about stackalytics
18:48:43 <anteaya> and thanks ilyashakhat__ for being here
18:48:46 <Sukhdev> anteaya: me too
18:48:50 <krtaylor> ilyashakhat__, yes, thanks
18:48:59 <Sukhdev> anteaya: about http://stackalytics.com/report/ci/neutron/7
18:49:01 <anteaya> I'd like to being by saying thank you for making that change to diverlog
18:49:12 <anteaya> that you agreed to do, I appreciate that
18:49:35 <anteaya> and at some point I would like to get back to discussing driverlog, since I think there are more ways to improve that page
18:49:41 <ilyashakhat__> it was easy change
18:49:52 <anteaya> but I think the topic at hand is what Sukhdev has posted
18:50:02 <anteaya> #link http://stackalytics.com/report/ci/neutron/7
18:50:08 <anteaya> so let's start here
18:50:17 <anteaya> ilyashakhat__: it was nice of you to make that change, thank you
18:50:26 <ilyashakhat__> yes, Sukhdev already found bug with this report (with missing CI)
18:50:33 <anteaya> good
18:50:51 <anteaya> so my concern with this is the concern I mentioned on the ml
18:51:06 <krtaylor> Sukhdev, there was ml discussion on this, on what was being asked, it is slightly different than this
18:51:06 <anteaya> we don't have a way of assessing fitness of a given ci system
18:51:07 <krtaylor> yes
18:51:31 <anteaya> 100% success to me indicates a broken system
18:51:45 <anteaya> since there are failures which it is missing
18:51:45 <krtaylor> it was mentioned that there needs to be a list of who is expected to post and if they actually did
18:51:51 <anteaya> right
18:51:53 <ilyashakhat__> 100% for merged patches is ok, but for all looks suspicious
18:52:00 <anteaya> and that is a different conversation, krtaylor
18:52:20 <anteaya> krtaylor: that conversation is how devs need to access information to decide patch flow
18:52:23 <krtaylor> anteaya, but that is the one Sukhdev posted the link for
18:52:29 <Sukhdev> krtaylor: not sure which ML discussion you referrering to - I saw ML discussion with ilyashakhat__ and anteaya
18:52:31 <kevinbenton> ilyashakhat__: how is success determined?
18:52:41 <anteaya> ilyashakhat__: right
18:52:42 <kevinbenton> right now BSN doesn’t downvote on failure
18:52:54 <anteaya> we don't have a criteria for success
18:53:10 <krtaylor> well, that is the question we must document
18:53:14 <ilyashakhat__> 'success' = ci voted +1 or left comment with positive result
18:53:16 <Sukhdev> ilyashakhat__: Even Arista does not vote -1 on failures
18:53:30 <anteaya> ilyashakhat__: but that is not success
18:53:40 <ilyashakhat__> for that cases it is possible to extract result from comments
18:53:42 <krtaylor> a -1 could be a success too
18:53:42 <anteaya> since that gives false results
18:53:53 <anteaya> we need to determine accuracy of a system
18:54:04 <anteaya> and we don't have a way for doing that in the community
18:54:07 <ilyashakhat__> krtaylor: what do you mean "-1 is success"?
18:54:22 <kevinbenton> ilyashakhat__: successfully identified a broken patch
18:54:24 <krtaylor> ilyashakhat__, if a patch is bad
18:54:27 <anteaya> and until we do, every time someone makes up a definition, it disrupts our ability to have that conversation
18:54:32 <ilyashakhat__> krtaylor: got it :)
18:54:33 <anteaya> ilyashakhat__: can you see that?
18:54:40 <fawadkhaliq> means CI which only comment on failure will be counted as a success
18:54:40 <krtaylor> then ci did what it was supposed to do
18:55:05 <anteaya> ilyashakhat__: can you see that?
18:55:18 <krtaylor> no, we need to separate voting and testing
18:55:25 <lukego> It’s scary to enable -1 voting because the effect of casting a -1 is a bit unpredictable. (does it cause a hundred people’s work to stall by screwing up a workflow somewhere? etc)
18:55:30 <anteaya> can you see that making up definitions prevents us from arriving at our definition?
18:55:38 <anteaya> krtaylor: I agree
18:55:47 <lyxus> lukego, FULLY agree
18:55:59 <ilyashakhat__> fawadkhaliq: not exactly, it is possible to set pattern for positive result and for negative, however the CI must report about success results too
18:56:09 <lukego> especially when the majority of failures are caused by “other environmental issues” and should not be -1’s
18:56:30 <anteaya> ilyashakhat__: can you see that making up definitions prevents us from arriving at our definition?
18:56:35 <lukego> hard to keep a CI running reliably even before you worry about false negatives.
18:56:39 <lyxus> lukego, that's why i do manual -1
18:56:43 <krtaylor> testing a patchset and publishing a result, without CI system failure is a successful run
18:56:54 <krtaylor> regardless of vote
18:57:06 <anteaya> ilyashakhat__: can you anwser my question, please
18:57:17 <ilyashakhat__> anteaya: sorry, what was the question?
18:57:21 <anteaya> ilyashakhat__: can you see that making up definitions prevents us from arriving at our definition?
18:57:24 <lukego> lyxus: How do you get notified that there is a failure to review, and how do you post the -1 ?
18:58:00 <fawadkhaliq> ilyashakhat__: let's talk about this offline. I see some CIs failed with comments and they are counted as a success.
18:58:15 <anteaya> fawadkhaliq: let's talk about this ONLINE
18:58:16 <ilyashakhat__> anteaya: hmmm, don't quite understand such difficult question
18:58:30 <anteaya> ilyashakhat__: what do you not understand
18:58:41 <fawadkhaliq> anteaya: sure, even better :)
18:58:42 <ilyashakhat__> what "definition" you mean
18:59:05 <anteaya> ilyashakhat__: I am asking if you are willing to stop making up stackalytic definitions of success
18:59:10 <anteaya> for third party ci systems
18:59:23 <anteaya> and allow the openstack community to arrive at those defintions
18:59:29 <lyxus> lukego, I use Jenkins;  I do a test if neutron is up , if it's not I dod a ' exit 1' which make the jenkins build fail. In jenkins, i have  post build which is to send me an email when it fails
18:59:29 <anteaya> are you willing to do that
18:59:52 <anteaya> ilyashakhat__: plese respond before we are out of time
18:59:58 <Sukhdev> anteaya: What if we had a status as - CI operational or not operational
18:59:59 <fawadkhaliq> ilyashakhat__: anteaya: https://review.openstack.org/#/c/102101/ is an example
19:00:15 <krtaylor> 1 minute warning  :)
19:00:18 <anteaya> ilyashakhat__: please respond before we are out of time
19:00:30 <ilyashakhat__> anteaya: please give an example
19:00:40 <anteaya> I've given you many on the ml
19:00:45 <anteaya> and we are out of time
19:00:46 <fawadkhaliq> PLUMgrid CI failed and http://stackalytics.com/report/ci/neutron/7 reports 100% success
19:00:47 <anteaya> fine
19:00:54 <krtaylor> yes, time to wrap this up
19:00:56 <anteaya> I will continue to chase you down on teh ml
19:01:03 <ilyashakhat__> fawadkhaliq: I'll look at it
19:01:10 <fawadkhaliq> ilyashakhat__:  thanks a lot
19:01:10 <krtaylor> thanks everyone
19:01:16 <luqas> thanks
19:01:20 <krtaylor> #endmeeting