10:00:57 <ekcs> #startmeeting congressteammeeting
10:00:58 <openstack> Meeting started Thu Nov 15 10:00:57 2018 UTC and is due to finish in 60 minutes.  The chair is ekcs. Information about MeetBot at http://wiki.debian.org/MeetBot.
10:00:59 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
10:01:01 <openstack> The meeting name has been set to 'congressteammeeting'
10:01:10 <ramineni> ekcs: hi
10:01:19 <akhil_jain> ekcs: Hi!
10:01:38 <ekcs> hello all. thanks for making this meeting time. typically we’d just skip a meeting during summit week, but with biweekly meetings that messes things up a bit.
10:01:42 <ekcs> hi ramineni
10:01:44 <ekcs> hi akhil_jain
10:01:58 <ekcs> topics kept here as usual. please add any topics you have: https://etherpad.openstack.org/p/congress-meeting-topics
10:05:16 <ekcs> ok let’s get started then.
10:05:40 <ramineni> ekcs: have few questions/concerns regarding the spec
10:05:51 <ekcs> ramineni: great!
10:06:01 <ekcs> #topic json spec
10:06:13 <ramineni> ekcs: will there be mapping b/w congress and new repo cloudstate?
10:06:22 <ramineni> or its totally independent
10:07:20 <ekcs> at this point it’s proposed as a totally independent experiment. if it proves useful we can make a decision to integrate.
10:08:02 <ramineni> ekcs: so separate api will be there in new repo
10:08:08 <ramineni> to poll data ?
10:09:32 <ekcs> I think to start out, data sources can be configured in yaml files rather than by API.
10:09:47 <ekcs> so basically it won’t need an API except to accept webhooks.
10:09:49 <ramineni> if I understand correctly, user needs to add a yaml file with all the details to poll, instead of datasource drivers and all the polling data would be stored in database as opposed to temporary strucrture
10:10:01 <ekcs> yes.
10:10:32 <ramineni> ekcs: but polling needs to started, or will it read yaml from time to time
10:11:31 <ekcs> as a starting point, I’m thinking just read the yaml files on start. if user adds a new one, they can just restart service to trigger reread.
10:11:42 <ramineni> ekcs: if I remember , congress also supports persistent option to store data permanently right
10:12:18 <ekcs> yes congress already has some support for that. at least on push drivers.
10:13:04 <ekcs> the main idea of the experiment is to see if a model of congress where every datasource driver is effectively generic, using queries over json data, is useful to ops.
10:13:14 <ramineni> ekcs: I like the part of using yaml instead of adding datasource drivers , but im not sure hw much it would benefit end users to write policies on database
10:14:01 <ramineni> ekcs: and also schema would be confusing for end users , in tabular its more consistent and easy to write policies I feel
10:15:23 <ekcs> ramineni: that makes sense. i’m not sure either. that’s why the thinking is to make it a experiment that’s small and quick to build. by relying on a database directly rather than rebuilding one, this can be done very quickly. and can be tried out to see whether the model works well. we may find that direct interaction with database is not good. or find it’s great.
10:15:28 <ramineni> ekcs: and also for the same, direct connectivity to database should be exposed , won't be security issue? how much feasible in production environs
10:16:39 <ekcs> ramineni: yes that’s a good point to consider. So far in my research postgres can be secured with very fine grained access control. but if it turns out that’s bad for production, then we can add a redirect layer.
10:17:30 <ekcs> on the issue of schema, it’s definitely a trade off. table data is simplest for the user, but requires complex translation and is inflexible.
10:17:47 <ramineni> ekcs: adding redirection would be again huge code to add , eventually new repo will become huge project
10:19:16 <ramineni> ekcs: what do you mean by complex translation?
10:19:35 <ekcs> my hope is no redirection will be needed. but if it turns out to be needed, then maybe the experiment has served its purpose and we don’t keep adding to it.
10:21:44 <ekcs> so if we want to allow ops to add sources via config, then either we don’t do translation (keep json) or the config basically need to have all[ the translation code currently in datasource drivers.
10:22:23 <ekcs> I think it’d serve a different class of users well to do json directly. but if i turn out to be wrong then the experiment has served its purpose and we can just stop it.
10:22:45 <ekcs> I think we all know it’s hard to write the translator code.
10:23:30 <ekcs> I still have to fiddle around and trial and error sometimes even after doing it so long. unrealistic to expect a user to do that.
10:25:04 <ramineni> ekcs: For developer, writing translation is not much hardwork , as they know language and have examples of other translations , but endusers who need not have to know language , would find hard to write database queries everytime
10:25:20 <ramineni> ekcs: I meant , for endusers ulmitely we have to make it easy to use
10:26:05 <ramineni> as they won't have much knowledge of internal databse queries , or need not to know database schema
10:26:14 <ramineni> ekcs: isn't it much to ask from endusers
10:27:43 <ramineni> and also writing poliies for complex schema would be difficult in database ? where we have many nested structures
10:28:04 <ekcs> I think that’s the case where congress already works great. the user stays within the data made available by congress developers, then the current way is best. the problem is when the user want to use new data as they write policy, then it’s terrible. I want to experiment to see whether this is a way to address that case.
10:28:40 <ekcs> writing the queries is certainly harder than pure table, but I think it’s actually easier than we may think.
10:28:50 <ekcs> json queries is becoming very standard
10:28:54 <ekcs> with the nosql movement.
10:29:11 <ekcs> nesting is not a big issue.
10:29:17 <ekcs> just -> operator to access.
10:29:58 <ekcs> I think my main thinking is that this is quick to build. and we don’t even have to support it because it’s experiment.
10:30:12 <ekcs> instead of us thinking about whether it’s too hard. we can just put it out there.
10:30:20 <ramineni> ekcs: for new data being accssible in congress , can we find a way for adding dynamic schema instead
10:30:50 <ramineni> ekcs: im just thinking , its hard for endusers to use, then why is it needed , we should look for other alternatives
10:30:58 <ramineni> if its hard**
10:33:16 <ramineni> ekcs:  just a new repo would be hardwork , thinking if we have simpler alternative to address the issue , but if you are convinced that is the best way , please go ahead
10:34:43 <ekcs> my thinking is we don’t know whether it’ll be a little bit hard or way too hard. I’m thinking it’ll only be a little bit hard because the language and concept of querying json is widespread. operators generally have at least general familiarity with sql syntax. also it may be used by openstack distributors / NFV vendors to specify policy.
10:35:00 <ekcs> I’ve presented it briefly to ops at a summit forum and so far the feedback is positive.
10:35:09 <ekcs> but really we won’t know until it’s gets used.
10:36:11 <ekcs> I appreciate your concerns! do you have something in mind for dynamic schema?
10:37:28 <ramineni> ekcs: just thought , using the yaml , you have specified , we can do something integrated in congress as well to address issue, but not concrete
10:38:15 <ramineni> ekcs: but if ops interested , we can go ahead with you are approach
10:38:21 <ramineni> your**
10:39:50 <ekcs> I see what you mean. I’ve been thinking about the same thing for a while too. the problem is the data comes in in json. so either it needs to be translated to tables (the current translator code), or a language is needed to query the json. We could develop our own language and engine to query json, but for the moment at least it seems way too much investment for experimenting.
10:40:31 <ekcs> akhil_jain: do yo have any thoughts?
10:41:45 <ramineni> ekcs: right
10:42:19 <akhil_jain> ekcs: just to avoid translator making whole repo should not be good. i would prefer conventional way we are following. but anyways ready to support your idea and contribute
10:43:18 <ramineni> ekcs: but for yaml part , I would like that to be integrated as alternative in congress as well sometime:)
10:45:07 <ekcs> ramineni: that makes sense.
10:51:13 <ekcs> akhil_jain: I see. I think the conventional way is good too when the data needed is already available. just proposing to explore a new model of unlimited data, without complicating existing code.
10:53:34 <akhil_jain> ramineni: yes that will be good, but not sure if we can add another layer in congress itself(for yaml) without complicating things. we can also reuse many things by this
10:53:57 <akhil_jain> ekcs: *
10:57:48 <ekcs> akhil_jain: I see.
10:58:33 <ekcs> well i think we’re coming up on time.
10:58:53 <ekcs> maybe comment on gerrit for further comments?
10:58:55 <ekcs> any other quick topics now?
11:00:00 <akhil_jain> none from my side
11:00:29 <ramineni> ekcs: no, ill comment on gerrit if any other questions on spec
11:00:40 <ekcs> ok well thanks for all the discusion and feedback!
11:01:09 <ekcs> let’s wrap up then. have a great day!
11:01:15 <ekcs> #endmeeting