16:00:14 <xarses> #startmeeting fuel
16:00:14 <xarses> #chair xarses
16:00:15 <xarses> Todays Agenda:
16:00:15 <xarses> #link https://etherpad.openstack.org/p/fuel-weekly-meeting-agenda
16:00:15 <xarses> Who's here?
16:00:16 <openstack> Meeting started Thu Mar  3 16:00:14 2016 UTC and is due to finish in 60 minutes.  The chair is xarses. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:17 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:20 <openstack> The meeting name has been set to 'fuel'
16:00:20 <warpc> hi!
16:00:21 <openstack> Current chairs: xarses
16:00:23 <Tatyanka_Leontov> hi
16:00:26 <mwhahaha> hi
16:00:29 <IvanBerezovskiy1> hi
16:00:34 <degorenko> hi
16:00:37 <ikalnitsky> o/
16:00:44 <igorbelikov> o/
16:00:47 <skolekonov> hi
16:00:51 <mattymo> hi
16:00:55 <ashtokolov> o/
16:00:56 <pigmej> hello
16:00:59 <akislitsky_> hi
16:01:01 <aglarendil> \o/
16:01:02 <salmon_> hi
16:01:08 <dklenov> o/
16:01:20 <xarses> A bit of house keeping, we have a large agenda and have to get through the FFE, if we run over, we will switch to #fuel-dev at the top of the hour
16:01:22 <yottatsa> o/
16:01:33 <angdraug> \o
16:01:36 <ogelbukh> hello there
16:01:48 <bgaifullin> hello
16:01:52 <xarses> #topic action items from last week
16:02:05 <xarses> aspiers will create ML to find out who is interested in RA convergance and probably set up dedicated meeting for such
16:02:32 <xarses> A saw a bug from aspiers that he was going to follow this up more
16:02:56 <rvyalov_> hi
16:03:05 <xarses> #action xarses to follow up with aspiers on RA convergence
16:03:08 <xarses> pigmej will post on the ML with regards to solar packages merge
16:03:25 <pigmej> so, I posted mail regarding this problem, I also described it in weekly
16:03:31 <pigmej> that's pretty much all.
16:03:44 <xarses> thanks
16:04:02 <xarses> on to the fun.
16:04:34 <xarses> ok, lets get into the FFE requests
16:04:46 <xarses> #topic [FFE] FF exception request for SR-IOV - 2 weeks (dklenov)
16:04:50 <angdraug> can I assume that everyone who still needs an FFE has added their request to the meeting agenda?
16:05:17 <angdraug> dklenov: can you comment based on the template I posted yesterday?
16:05:20 <azvyagintsev> guys, check-echo ?
16:05:28 <angdraug> azvyagintsev: we see you
16:05:41 <dklenov> oh, I didn't prepared it according to template
16:05:42 <dklenov> sorry
16:05:46 <dklenov> have in free form
16:05:54 <angdraug> lets see what you have
16:05:59 <dklenov> https://www.irccloud.com/pastebin/O646svPq/
16:06:29 <yottatsa> there are 3 patches on review #link https://review.openstack.org/#/q/status:open+branch:master+topic:bp/support-sriov
16:06:30 <yottatsa> We are expecting one patch for fuel-ui.
16:06:47 <angdraug> ok, lets talk about risks
16:06:59 <angdraug> which components are impacted by the remaining patches?
16:07:08 <angdraug> are all patches already on review or will there be more?
16:07:11 <dklenov> it is mostly nailgun
16:07:18 <dklenov> + a couple of library patches
16:07:21 <ikalnitsky> no patches are expected?
16:07:32 <ikalnitsky> except the one in ui ?
16:07:40 <dklenov> for nailgun this is it
16:07:49 <angdraug> how easy is it to isolate potential impact? can we mark it experimental?
16:07:50 <dklenov> one for UI and one for library to create
16:07:57 <dklenov> all nailgun patches are on review
16:08:22 <ikalnitsky> that's good
16:08:30 <ikalnitsky> seems ok to me, they ain't huge
16:08:38 <ikalnitsky> and already reviewed by some folks
16:08:43 <angdraug> if it's disabled, how big is the risk of it introducing regressions?
16:09:05 <dklenov> I would say risk is low
16:09:14 <dklenov> core changes are already merged
16:09:21 <angdraug> what's your ETA for completion?
16:09:32 <angdraug> 3/16?
16:10:32 <dklenov> it is 3/16 for HP, Numa and SR-iOV
16:10:40 <dklenov> 3/24 for DPDK
16:10:58 <angdraug> no objections about the risk from anyone?
16:11:27 <angdraug> if not, I propose to grant the exception but mark all these features experimental in this release
16:12:16 <yottatsa> angdraug why experimental?
16:12:52 <angdraug> because the length of exception is large and there is a risk we won't have time to QA it properly
16:13:05 <angdraug> and because it's easily disabled and disabling it mitigates that risk
16:13:33 <kozhukalov> but making them experimental is going to make exception even longer, right?
16:13:38 <dklenov> I think for QA we should have enough time
16:13:42 <angdraug> kozhukalov: why so?
16:13:59 <dklenov> and work on QA side is in progress
16:14:01 <alex_didenko> I'm afraid that hiding them under experimental may lead to even more work
16:14:02 <dnovakovskyi> customer won't be happy for experimental status. i suggest to do it only if we indeed face QA problem
16:14:06 <angdraug> dklenov: it's going to be 3 weeks less than for features that have already landed
16:14:09 <alex_didenko> please note that a bunch of code is already merged
16:14:15 <kozhukalov> because it is not only about documentation
16:14:16 <alex_didenko> we'll have to refactor that
16:14:42 <angdraug> alex_didenko: are you saying it's not easily disabled?
16:14:42 <kozhukalov> some additional configuration is required to allow user to disable this features
16:15:04 <yottatsa> angdraug core fuctions of the feature has delivered and is under QA for a week already
16:15:05 <kozhukalov> for example in nailgun-agent it is a part of agent code
16:15:13 <alex_didenko> can't say right now, needs to be checked in the code
16:15:18 <dklenov> for QA activities we should be good
16:15:36 <dklenov> we are handing parts of the features to QA once these parts are ready
16:16:05 <ikalnitsky> hiding nfv features from nailgun serialization shouldn't be a problem. however, i'm not sure about ui part
16:16:31 <angdraug> if these features can't be easily disabled, I'm not comfortable granting the exception at all
16:17:11 <alex_didenko> they can't be easily enabled I'd say :)
16:17:15 <alex_didenko> you need to have a proper HW
16:17:25 <angdraug> "should be" is not good enough, no matter how you slice it there's 2-3 weeks until this can be tested in its entirety, and we will start finding new bugs 3 weeks later than now
16:17:27 <xarses> angdraug: should we differ this discussion to the ML and move on?
16:17:29 <mihgen> vkramskikh: can you comment on how easy to disable these features in UI?
16:18:12 <angdraug> xarses: no, we only have today to decide FFEs
16:18:30 <angdraug> if we defer to ML the discussion will drag on, and we're not granting exceptions a week after FF :/
16:19:29 <kozhukalov> are all those feature of the same size and of the same level of potential regression?
16:19:34 <yottatsa> angdraug like I said, core feature has been passed to QA already, we're waiting for UI/serialization now
16:19:35 <alex_didenko> we need feedback from UI team on how easy it would be to hide those parts
16:19:56 <kozhukalov> afaik numa is quite small
16:20:14 <alex_didenko> yottatsa, you won't need to change anything in library code, that's a matter of serialization only
16:20:15 <angdraug> if vkramskikh is not around we'll have to grant a conditional exception
16:20:46 <angdraug> kozhukalov: see pastebin above, numa has 5 patches on review. that's not small
16:20:51 <alex_didenko> and we also don't need to change nailgun-agent, it can report the info, it's does not affect anything
16:21:00 <ikalnitsky> alex_didenko: sure
16:21:15 <ikalnitsky> all changes on nailgun and ui side - doesn't provide that info if it's disabled
16:21:15 <vkramskikh> it would be easy to disable them
16:21:17 <angdraug> nurla: can you comment on how much testing nfv features from dklenov have seen so far?
16:22:03 <nurla-> from fuel side we are only on test design state
16:22:29 <angdraug> nurla-: is it possible that they will be fully tested by 3/24?
16:22:32 <alex_didenko> yep, but dklenov was talking about different QA engineers :)
16:22:50 <angdraug> alex_didenko: are those QA engineers here in this meeting?
16:22:54 <nurla-> I think - yes, why not :)
16:22:56 <dklenov> I was talking about Telco QA who started validation of HugePages and Numa
16:23:21 <angdraug> ok, a half-way solution:
16:23:32 <nurla-> I was talking about fuel parts
16:23:43 <angdraug> mark it experimental now, if we have QA signoff by SCF that these are solid we un-experimentalize it
16:23:54 <alex_didenko> lgtm
16:23:58 <dklenov> +1
16:23:59 <ikalnitsky> +1
16:24:01 <nurla-> +1
16:24:27 <angdraug> #info SR-IOV FFE granted, feature marked experimental until QA signoff that it's stable
16:24:47 <angdraug> #info HugePages FFE granted, feature marked experimental until QA signoff that it's stable
16:24:54 <angdraug> #info Numa FFE granted, feature marked experimental until QA signoff that it's stable
16:24:59 <angdraug> #info DPDK FFE granted, feature marked experimental until QA signoff that it's stable
16:25:15 <angdraug> #info merge deadline 3/16, except DPDK 3/24
16:25:17 * zigo reads the backlog
16:25:21 <angdraug> moving on?
16:25:29 <dklenov> thanks!
16:25:35 <xarses> #topic [FFE] FF exception request for ConfigDB API and clients (ogelbukh)
16:25:46 <ogelbukh> hi again
16:26:13 <ogelbukh> current status of configdb extension
16:26:16 <ogelbukh> 1. API extension for Nailgun is designed, spec in review :https://review.openstack.org/284109/
16:26:18 <ogelbukh> source code is developed in https://github.com/Mirantis/tuning-box
16:26:20 <ogelbukh> request for repo in openstack/ namespace in review https://review.openstack.org/286137
16:26:22 <ogelbukh> 2. Upload of serialized data to the API is designed, spec in review: https://review.openstack.org/286012
16:26:24 <ogelbukh> 3. Hiera backend is in development, source code is in WIP status in fuel-library: https://review.openstack.org/285236/                         we requested FFE for this feature till Mar 24, since it is not intrusive, partially developed in external repos and covers significant use casenamely integration of 3rd-party components with out deployment flow
16:26:57 <ogelbukh> *our deployment flow
16:26:59 <angdraug> ogelbukh: same questions:
16:27:05 <ogelbukh> sorry for formatting mess
16:27:08 <angdraug> what components are impacted, what are the risks?
16:27:24 <xarses> ogelbukh: what is open that requires changes in fuel? just the fuel-lib?
16:27:25 <ogelbukh> actually, in the current version of design the impact is minimal
16:27:42 <angdraug> define minimal
16:27:49 <ogelbukh> nailgun extension will require no changes to core code
16:28:01 <angdraug> core code = nailgun? what changes?
16:28:05 <ogelbukh> deployment task and hiera backend will be part of plugin
16:28:12 <angdraug> ikalnitsky: can you confirm the impact there?
16:28:19 <ikalnitsky> angdraug: yep
16:28:26 <ogelbukh> since we already have stevedore support in master
16:28:39 <ogelbukh> we just need to plug it in properly on extension's side
16:28:42 <ikalnitsky> only one question, ogelbukh: have you decided to move with deployment task in favor of changes in astute ?
16:28:59 <ogelbukh> yes
16:29:13 <angdraug> are nailgun changes on review yet?
16:29:14 <ogelbukh> the only limitation is that is has to run on master node most likely
16:29:32 <ikalnitsky> angdraug: there's no nailgun changes
16:29:33 <ogelbukh> the extension is developed in dedicated repo
16:29:35 <ikalnitsky> it's an extension
16:29:41 <angdraug> it's *all* pluggable?
16:29:47 <ogelbukh> we are adding it to openstack namespace right now
16:29:49 <ikalnitsky> angdraug: looks so
16:29:53 <angdraug> in that case, I see no reason not to grant exception
16:29:56 <angdraug> objections anyone?
16:30:25 <angdraug> ok then
16:30:52 <angdraug> #info ConfigDB FFE granted, deadline 3/24, no core codebase impact expected
16:31:01 <angdraug> moving on
16:31:11 <xarses> #topic [FFE] FF exception request for Unlock "Setting" tab - 3 weeks (ashtokolov)
16:31:24 <ashtokolov> https://www.irccloud.com/pastebin/VCI2Lybt/Unlock%20settings%20tab
16:31:45 <angdraug> define "low risk"
16:31:48 <ogelbukh> thanks everyone, I appreciate that
16:32:10 <ashtokolov> low risk to be on time
16:32:17 <angdraug> ashtokolov: what are the impacted components?
16:32:42 <ashtokolov> nailgun
16:32:48 <angdraug> ashtokolov: are all your patches on review already?
16:33:11 <ashtokolov> https://www.irccloud.com/pastebin/jyTyD5EW/
16:33:28 <ashtokolov> not all
16:34:14 <angdraug> of 4 patches you listed in pastebin, 1 is merged and the other 3 are failing CI
16:34:15 <ashtokolov> we should discuss the implementation with ikalnitsky
16:34:19 <angdraug> how many more you expect to push?
16:34:51 <ashtokolov> I expect 4 patches for custom graph +
16:35:04 <ashtokolov> tiny ones^^
16:35:12 <ashtokolov> 2 for store-deployment-tasks-history
16:35:13 <angdraug> all nailgun?
16:35:20 <ikalnitsky> ok, angdraug, since the changes are limited by nailgun, there will be no affect on other projects
16:35:24 <ashtokolov> and 2 or 3 for computable-task-fields-yaql
16:35:33 <ikalnitsky> and the profit we have with that features - are huge
16:35:43 <angdraug> nailgun is a core component, it can impact almost anything
16:35:53 <ikalnitsky> thoug, I'm not sure about that one
16:35:54 <ikutukov> There will be 6 patches, one is relatively big ~200-300 LOC with graph merging
16:35:55 <ikalnitsky> #link https://blueprints.launchpad.net/fuel/+spec/store-deployment-tasks-history
16:36:03 <ashtokolov> number depend on how we can split it to make review easier
16:36:08 <aglarendil> the changes themselves are essentially to '/task' and '/orchestrator' pieces of nailgun and to some data models and do not affect regular deployment flow
16:36:28 <angdraug> aglarendil: thanks, that helps a lot!
16:36:46 <angdraug> ikalnitsky: can you confirm ^?
16:36:52 <ikalnitsky> angdraug: confirm
16:36:57 <ashtokolov> an ability to view the deployment tasks history makes UX not so poor
16:37:14 <angdraug> I don't feel that making this experimental is going to help with mitigating the risks :(
16:37:37 <ikalnitsky> ashtokolov: yes, but we already have a lot to do.
16:37:51 <aglarendil> changes such as store deployment data or tasks history is just a dump to the database instead of throwing messages to/from astute away
16:37:57 <ikalnitsky> besides, there will be no support from UI, afaik
16:37:59 <ikalnitsky> so why then ?
16:38:02 <angdraug> what's the plan if some of these changes are not ready by 3/24?
16:38:20 <ashtokolov> afaik you and vsharshov had decision about how to do it
16:38:48 <ikalnitsky> ashtokolov: yes, but it was few monthes ago
16:38:55 <ikalnitsky> why it wasn't done till ff ?
16:38:58 <angdraug> will it regress UX? or will we be able to safely postpone the remaining patches to Newton?
16:39:30 <angdraug> ikalnitsky: "why" is not the right question to ask
16:39:31 <ashtokolov> angdraug we prioritise changes by impotency
16:39:40 <ashtokolov> *importency
16:39:47 <angdraug> by importance? :P
16:39:51 <ashtokolov> =)
16:39:55 <zigo> :)
16:39:55 <aglarendil> idemportance
16:39:55 <nurla-> -)
16:40:11 <ashtokolov> by both cases
16:40:17 <zigo> 囧
16:40:18 <ikalnitsky> angdraug: well, the last word is your's.
16:40:20 <mihgen> :D
16:40:39 <angdraug> I'd rather see a consensus between you than have to resolve a dispute
16:40:47 <ikalnitsky> angdraug: i just warned you that we're trying to do more than we need, and we won't make it in full scope
16:41:01 <ashtokolov> angdraug, we'll be able to safely postpone the remaining patches to Newton for task history
16:41:04 <angdraug> so far, I see a long chain of small patches, many of which are at risk of being ready even by 3/24
16:41:08 <angdraug> is that a correct summary?
16:41:13 <aglarendil> angdraug: nope
16:41:14 <zigo> If the full of it wont make it, then what's the point of having *part* of it merged?
16:41:24 <zigo> Will it still be useful?
16:41:46 <aglarendil> the other changes like computable field do simple YAQL evaluation which has already been used in Murano. and enabling it is really a piece of cake while benefits are huge
16:41:47 <angdraug> zigo: yup, that's the question I've been working towards
16:41:50 <xarses> what is at risk of not landing at 3/24?
16:41:55 <xarses> task history?
16:42:05 <xarses> or the rest of custom graph?
16:42:27 <aglarendil> I am not aware of the things that have risks to not land until 3/24
16:42:35 <angdraug> aglarendil: I wasn't arguing that the value is small, small patches means low risk
16:42:36 <aglarendil> major part of custom graph has already been landed
16:42:37 <ashtokolov> xarses IMO it's low risk, but ikalnitsky has another opinion
16:42:59 <ashtokolov> xarses the rest of custom graph will be landed next week
16:43:03 <angdraug> ikalnitsky: same question, which parts do you see as high risk of not landing
16:43:15 <angdraug> mind that risk of not landing != risk of regression
16:43:23 <ikalnitsky> i told nothing about "risks" not to landing
16:43:31 <angdraug> if any of this can introduce a regression that CI won't catch, it's a no-go from the start
16:43:32 <ikalnitsky> it fully depends on development team
16:43:49 <aglarendil> store stuff in DB parts are also a piece of cake - get a message to/from astute and dump it into the database instead of throwing it out
16:44:04 <angdraug> so far, the fact that outstanding patches fail CI is a good sign, it means that affected codepaths are covered by our tests
16:44:33 <angdraug> ikalnitsky: what about risk of regression then?
16:44:45 <ikalnitsky> the risks are always high
16:44:47 <ashtokolov> now we have deployment test in fuel-web CI
16:44:54 <ikalnitsky> i just suggest to do less work
16:44:55 <angdraug> can it break things in a way that CI won't catch?
16:45:04 <ikalnitsky> because task history won't be implemented on UI side anyway
16:45:22 <ikalnitsky> and chants to astute and nailgun may block master if something goes wrong
16:45:24 <ikalnitsky> that's the risk
16:45:25 <aglarendil> the regressions will be easily caught by current deployment tests at Fuel-Web CI
16:45:32 <ikalnitsky> if you think it's really needed in 9.0
16:45:36 <angdraug> I'm not sure not having it in UI is a strong argument
16:45:37 <ikalnitsky> so be it
16:45:49 <aglarendil> as all of these changes are about orchestration and business logic of which tasks to execute on which nodes
16:45:49 <bgaifullin> ikalnitsky, task history is useful without support from UI side, for example telko team asks about this feature
16:45:50 <angdraug> if you need task history for investigations, just having it in DB would be useful
16:45:50 <ikalnitsky> and why then at all?
16:46:00 <dnovakovskyi> UI is not critical
16:46:02 <ikalnitsky> i have logs for investigation
16:46:08 <ikalnitsky> it doesn't look like a critical feature
16:46:12 <dnovakovskyi> engineers are relaying on CLI often
16:46:15 <xarses> UI isn't important for this yet
16:46:21 <dnovakovskyi> especially for smart stuff like task history :)
16:46:23 <ikalnitsky> well, stop it, guys
16:46:27 <ikalnitsky> i told you what i think
16:46:32 <xarses> having it available in the DB is very good improvement over logs
16:46:33 <ikalnitsky> the decision is ptl's
16:46:48 <angdraug> ikalnitsky: and everyone is trying to convince you so that I don't have to override you
16:47:00 <dnovakovskyi> :)
16:47:01 <angdraug> you're the component lead, I really don't want to override your recommendation
16:47:12 <angdraug> no pressure :)
16:47:13 <xarses> angdraug: +
16:47:23 <aglarendil> angdraug: I want to mention that the whole feature has been very fairly granularized and distributed among the developers
16:47:32 <aglarendil> so we do not have any lack of resources right now
16:47:33 <angdraug> aglarendil: yup, I've noticed
16:48:08 <xarses> ikalnitsky: is your position because you think the feature has no value in its current state or because of technical risks
16:48:08 <angdraug> can we impose additional testing requirements on this feature to mitigate the risks?
16:48:21 <ikalnitsky> xarses: both
16:48:24 <angdraug> aglarendil: resources question is a double-edged sword
16:48:34 <angdraug> everyone who's still working on features past FF isn't fixing bugs
16:48:53 <ikalnitsky> changing communication protocol may block master for day, and hence - other features
16:48:54 <ashtokolov> ikalnitsky angdraug can we get a FFE and make decision based on time track to push it on not?
16:49:06 <ashtokolov> or not
16:49:11 <ikalnitsky> yes, or not
16:49:15 <ikalnitsky> it's hard to predict
16:49:32 <ikalnitsky> rememer recent blocker ? week ago? when we change it?
16:49:50 <aglarendil> you mean that one with hanging zuul with depends-on?
16:49:55 <ikalnitsky> yes
16:50:03 <ikalnitsky> shit are happens
16:50:12 <aglarendil> well, that was almost infra issue
16:50:13 <ikalnitsky> and not only because of our bad
16:50:23 <aglarendil> openstack-infra issue
16:50:25 <ikalnitsky> however, a lot of patches were blocked
16:50:44 <angdraug> ikalnitsky: I mostly care about risks introduced by feature itself
16:50:57 <angdraug> ok, time for another halfway solution
16:51:07 <xarses> REMINDER: we will continue FFE discussion in #fuel-dev at the top of the hour
16:51:10 <aglarendil> no risk. it is new functionality. regressions will be caught by CI deployment tests easily
16:51:21 <ashtokolov> angdraug, this feature is about to store existing information in DB
16:51:33 <aglarendil> all the code pieces have been designed in full compliance with retaining of old functionality
16:51:34 <angdraug> how about: ffe granted, task history to be discussed next week. really hate to do it this way
16:52:06 <angdraug> depending on the outcome of that discussion, task history might get excluded from the exception
16:52:17 <angdraug> good enough to move on?
16:52:43 <ashtokolov> lgtm
16:53:09 <angdraug> #info Unlock "Settings" Tab FFE granted, task history feature to be discussed until 3/10 and may get excluded from the exception
16:53:23 <angdraug> lets move
16:53:23 <xarses> #topic [FFE] FF exception request for Multi-release plugin packages - 3 weeks (ashtokolov)
16:53:33 <dnovakovskyi> IMO given the granularity explained by aglarendil
16:53:38 <ashtokolov> https://www.irccloud.com/pastebin/THErNBiA/
16:53:54 <dnovakovskyi> nevermind
16:54:12 <angdraug> what are the risks and impacts?
16:54:29 <ikalnitsky> impact in nailgun and fpb side
16:54:38 <ikalnitsky> however, nailgun database must be restructured
16:54:38 <ashtokolov> nailgun +FPB
16:55:01 <angdraug> hm
16:55:07 <xarses> what's the impact on nailgun? FPB is decoupled already
16:55:08 <angdraug> sounds too risky to me
16:55:24 <ikalnitsky> the bad thing here is that it's affected by db changes made by custom graph
16:55:25 <angdraug> xarses: nailgundb schema change is a no-go
16:56:16 <ikutukov> additional table storing per-release configuration for every plugin (networg groups, graph reference e.t.c.) is planned
16:56:16 <ikalnitsky> and i didn't review the design spec :(
16:56:31 <ashtokolov> ikalnitsky do you think we need extra db changes  for custom graph?
16:56:40 <zigo> fpb?
16:56:44 <ashtokolov> or you mean we have to redesign plugins?
16:56:47 <zigo> What's that? (sorry to ask...)
16:56:50 <ikalnitsky> ashtokolov: perhaps, perhaps not
16:56:56 <ikutukov> spec https://review.openstack.org/#/c/271417/9/specs/9.0/plugins-v5.rst
16:57:02 <ashtokolov> fpb=Fuel Plugin Builder
16:57:05 <xarses> zigo: fuel-plugin-builder, it lives in openstack/fuel-plugins
16:57:08 <zigo> Cheers.
16:57:10 <angdraug> looks like it's too raw for FFE
16:57:13 <ikalnitsky> i'd prefer to take a good decision where to go, instead trying to fix it quicly
16:57:29 <ikalnitsky> angdraug: +1. let's postpone it till 10.0
16:58:02 <angdraug> #info Multi-release packages (plugins-v5) FFE denied
16:58:13 <angdraug> lets wrap up here and move to #fuel-dev, we're out of time
16:58:22 <xarses> we have to min left on the channel, we need to move over to #fuel-dev
16:58:55 <xarses> #endmeeting