17:02:22 #startmeeting CongressTeamMeeting 17:02:23 Meeting started Tue Oct 28 17:02:22 2014 UTC and is due to finish in 60 minutes. The chair is thinrichs1. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:02:24 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:02:28 The meeting name has been set to 'congressteammeeting' 17:03:04 Let's get started with status updates. 17:03:08 #topic status 17:03:15 arosen: want to start? 17:03:32 sure 17:03:53 So, The congress CI is now voting on upsteam patches. The CI system ran into a couple of issues though over the last week (quick summery ): 17:04:02 - openstack-infra/config was split into two repos openstack-infra/project-config and openstack-infra/system-config so some files on the CI side needed to change in order to account for that change. 17:04:24 - The pypi mirror pypi.openstack.org is no longer a public mirror which the puppet scripts were using. This patch (https://review.openstack.org/#/c/131474/) just merged upsteam which fixes the mirror 17:04:29 Right now nodepool is receating the image and we should be back in business shortly :) 17:04:37 On the congress development side the db patch integration patches should be ready to go in: https://review.openstack.org/#/c/127045/ , https://review.openstack.org/#/c/127046/ 17:04:56 though I think we should wait till the CI is back up before we approve these just to ensure everythign is working functionally 17:05:21 also, I should be close to having the glance datasource driver ready for review: https://review.openstack.org/#/c/126377/ just trying to sort though the tags table and how it should be layed out. 17:05:42 that's it from me unless anyone would like to talk about any of the above..,,. 17:06:26 arosen: when do you expect the CI to be back up? need to recheck my review 17:06:49 jwy: should be up within the hour.. Just waiting on the image to be built and pushed out to glance. 17:07:07 jwy: you're patch on horizon should be fine. The failure on the ci you were hitting was related to the pypi mirror issue. 17:07:24 I'll go a head an recheck all the congress patches this "recheck congress" once it's back up. 17:07:32 arosen: ok, seemed like that, thanks 17:07:45 a digression for a quick introduction. I'm david wood from IBM. I've worked on policy on/off for the past few years. I've even implemented a datalog parser/engine in java ;). Just coming up to speed on openstack though. 17:08:11 dawood: Great to have you here! 17:08:11 awesome, welcome David! 17:08:42 thanks 17:08:45 David has wide experience in policies and engines 17:09:25 That sounds great! Typically we ask new people to tell us about their interests in the project at the end of the meeting. 17:09:40 But if you'd rather go now, that's fine too. 17:11:00 alexsyip: want to give your status report? 17:11:41 Tim and I discussed how the policy engine will make requests to drivers like ceilometer which data to request. 17:12:23 lo m8s 17:12:36 I’m also learning about tempest tests. 17:12:51 And debugging some bugs related to congress crashing and drivers. 17:13:24 alexsyip: good to have an update on that discussion. Particularly interested in how we might go about a simple scenario: error(host) :- average_mem_util(x > y, t), i.e., memory utilization is greater than y for agiven period t 17:14:05 error(host) :- average_mem_util(x > y, t, host) to be precise 17:14:54 The idea is for the engine to derive a list of constraints related to the data in ceilometer. 17:15:01 is there support for 'over time' metrics 17:15:29 alexsyip: there was the question of whether should a function should be implemented as a ceilometer alarm (in which case how does the alarm communicate to Congress) versus bringing data to ceilometer, and ceilometer checking the datalog body over time. 17:15:30 So, the driver may have a table named ceilometer:stats 17:16:03 kudva: We’ve not made any decisions about alarms. 17:16:09 alxesyip: but how is that table interpreted? As dawood asked, 'over a period of time'? 17:16:11 But rather how does the runtime talk to the driver. 17:17:20 kudva: would have to be, e.g. ave over last 30 sec, 10 sec, 1 sec, 1 min 17:18:03 glebo: two questions. 1. can we express that in datalog (are there semantics for it) 2. How is this translated to communicated between runtime and tbales 17:18:15 tables 17:18:16 kudva: could take a snapshot every n sec (n can be very small fraction) and put them all the in the table, then query over that data, but seems like overkill 17:18:47 glebo: so the table is a snapshot over a time t specified in the datalog clause correct? 17:19:05 kudva: not a datalog guy, so don't know. Sorry. But I think that is the right question to ask. Others? 17:19:24 thinrichs: thoughts on time in datalog? 17:19:27 kudva: that's what I was thinking. Just one idea 17:19:46 glebo: definitely, at this point good to put all options 17:19:56 kudva: optional setting for the "n" time period over which ave is taken, right? 17:20:04 Except for the period, it seems we can extract the info we need from the policy. 17:20:08 if the rule runtime provides the current time, then the rule can decide what 'over time' means and keep track of that itself? 17:20:27 thinrichs1: period would be one of the policy elements, 17:20:43 generally there is a default setting and then "advanced" setting where op can tune it 17:20:48 There are different things we could do to embed the period within the policy (or in a config file as a baby-step toward) that. 17:21:09 I think we should discuss this at Paris and work out what seems to be the right thing. 17:21:20 dawood: systems we've done this in for past projects normally don't use clock, but cycles 17:21:39 While we could always add special-purpose policy fragments to handle Ceilometer, it would be nice if we had a general way of dealing with this kind of thing. 17:22:01 … for the function that calcs the ave, then when enter it into table, time stamp using clock, if I'm remembering correctly 17:22:02 Something to handle time-series data. 17:22:54 cycles is fine i guess, but the rule writer will want to know how long a cycle is. 17:22:58 thinrichs1: anything you want to take an average over to represent out will need this dial / option. Ex: 17:23:03 sorry, example 17:23:04 : 17:23:13 For example, we've talked about introducing aggregates into the policy proper (like SQL). We might be able to piggy-back on such a thing to extract the info we want and hand it off to ceilometer. 17:23:44 Remember that we're talking about someone giving us an arbitrary policy, and the policy engine needs to figure out what data from ceilometer is relevant for that policy so that it can pull it in. 17:23:57 bps, cpu util per sec (or per n), memory util per n, smoke derived from flames shooting out the back of the box per n, etc. 17:24:19 Lots of good ideas here about how to proceed. I think this might take some face-to-face whiteboarding. 17:24:51 thinrichs1: or a virtual whiteboard if possible 17:25:03 kudva: etherpad would be a good option. 17:25:08 kudva: could it be a glass board then? 17:25:11 ;-) 17:25:19 thinrichs1: thanks :-) 17:25:39 Perhaps this is a good thing to go through at the design session. 17:25:41 * glebo btw, apologises for being late 17:25:57 thinrichs1: its tuesday afternoon? 17:25:57 * glebo and for spelling "apologises" incorrectly 17:26:13 #link https://etherpad.openstack.org/p/par-kilo-congress-design-session 17:26:29 alexsyip: thanks! 17:27:50 I added a star to that work item on the etherpad so we make sure to allocate enough time for it. 17:28:13 Let's keep moving with status updates and return to this at the end if there's time. 17:28:24 kudva: want to go next? 17:29:27 thinrichs1: been thinking about integration with Ceilometer. Had an offline phone call with alexsyip, very useful. Looks like this will be a group effort at a more basic level (such as time series etc), so will join the effort. Will pick another problem for myself by next week to code. 17:30:24 Sounds good. 17:30:46 jwy: have any news to share? 17:31:28 i have a patch out for review, which adds a Data Sources panel in Horizon 17:31:30 https://review.openstack.org/#/c/130922 17:31:48 i moved the policy data tables that are currently in the Policies panel into the new one 17:31:57 * glebo brb 17:32:06 * glebo in 5 or less 17:32:07 once it gets merged, Rajdeep will merge in his code to add the data sources data tables 17:32:27 then we'll be able to see all the policy and data source info 17:33:08 That sounds great! 17:33:39 That'll give us the rules, the datasource tables, and the policy tables, I believe. 17:33:40 i also emailed jason-bishop, who had expressed interest in contributing in last week's IRC meeting but haven't heard back from him 17:33:44 yep 17:34:05 Do we have plans to enable rule creation in Horizon? 17:34:13 jwy: i just mailed you 17:34:18 jwy: sorry so late! 17:34:35 jasonb: no problem! 17:34:56 thinrichs1: nothing concrete yet, would be another good thing to discuss 17:35:35 * glebo back 17:35:41 Agreed. The GUI is near the top of the list in terms of making Congress accessible to the target audience. 17:35:41 could start off with just having a big text box to enter the rule, like from the CLI 17:35:54 jwy: agreed that's a good way to start. 17:36:13 Many people I've talked to are skeptical of our target audience writing policy in Datalog. 17:36:21 So making that as easy as possible is a high priority. 17:36:54 i need a better understanding of datalog myself, haha 17:37:28 Last week I talked to some folks at Stanford who are working on something similar. 17:37:53 the Dexter tool? 17:37:56 People are still writing datalog directly, but they have some ideas about helping people understanding the datalog they've written. 17:37:59 #link http://dexter.stanford.edu/main/ 17:38:03 jwy: yep. 17:38:47 we have a policy authoring tool that allows people to fillin the blanks of an natural language phrase. the fill-ins are then mapped into the underlying policy. 17:39:10 dawood: I'd love to see that in action sometime. 17:39:15 allows for non-technical policy authors, but perhaps that is not the audience 17:39:27 jwy: +1 better understanding of datalog. /me was just pinging my eng buddies here to see if anyone had clue. ha ha 17:39:52 dawood: we've always talked about having different GUIs for different kinds of users, raw Datalog being the interface for super-users. 17:40:38 dawood: sort of like drop-down-menus, but fill in, rather than visual list choice? 17:40:59 jwy, glebo: we do have a few docs on a Datalog intro. 17:41:11 A more thorough intro has been on the list for a while. 17:41:16 Maybe it's time to move that up the list. 17:41:18 "allow users with role (admin|reader|...) access database tables (a|b|c)." 17:42:03 thinrichs1: Who is "we", OS or your employeer? 17:42:19 I suppose "we" == "me". 17:42:39 thinrichs1: got to b one on internet somewhere already, right? Not like we r the first ones to try to use it in a project 17:43:12 * glebo looking at dexter 17:43:27 dawood: so it's a visual representation of the underlying tables then? 17:43:36 does the GUI authoring tool do any validation of the rule? syntax, tables names, etc.? 17:43:45 Sure, but it would be nice to have one that is tailored to our language, that talks about the datasource tables we're using, has examples from our particular domain (instead of the typical parent-child-ancestor examples that you'll see). 17:44:01 currently it doesn't 17:44:26 glebo: its a visual representation of the underlying rule 17:44:39 dawood: Currently there's no way to add rules via the GUI. Just view rules. 17:45:13 dawood: are you asking about dexter or horizon? 17:45:20 ok, then adding them from the command line. are they checked for errors before installing them in the system where they might not work/throw and error 17:45:36 dawood: yes. Errors are reported via the API. 17:45:51 Though for some reason those errors aren't reported by the CLI. 17:46:20 Time check 15 min remaining. 17:46:31 Let's make sure we give everyone a chance to report on the work they've been doing. 17:46:35 rajdeep: want to go next? 17:46:58 sure though my update is very brief 17:47:49 i completed my implementation of data source panels in the meantime jwy has done some reorg of the panels separating datasource and policies 17:48:03 once her CL goes in will do my integration again with the new code 17:48:29 that completes my update .. 17:48:55 Great. Thanks. 17:49:01 * glebo snaps for rajdeep, for completed work, for short report 17:49:31 sarob: any news we should all know about for the summit? 17:50:05 Nope 17:50:17 If you guys can 17:50:37 Join in as many cross project and ops sessions as possible 17:51:05 Likely a lot of policy related issues and discussions 17:51:33 Neutron GBP is one that continues to be top of mind 17:51:58 and our interactions with policy specific modules like FWaaS, VPNaaS and LBaaS 17:52:02 The congress design session Tuesday rooms 124/125 16:40-18:10 17:53:06 And the congress etherpad #link https://etherpad.openstack.org/p/par-kilo-congress-design-session 17:53:16 Are the most important 17:53:35 Making sure everyone has those details 17:54:21 The gbp session is tues in Duffy 12:05-12:45 17:54:40 sarob: Thanks! 17:54:56 Did I miss anyone for a status report? 17:55:04 glebo: see https://review.openstack.org/#/c/129350/ 17:55:05 * glebo peep 17:55:23 :) no conflicts for my talks 17:55:38 Nice 17:55:45 LouisF: yup, was just marketing it to ensure everyone else had it on mind 17:56:31 thinrichs1: want me to go? 17:56:37 glebo: yes 17:56:51 re: write up from last weeks discussion on declarative protocol to carry policy declarations from Congress to policy description points for enforcement systems 17:57:01 had committed to get that done by CoB last night, but 17:57:07 dog ate my homework, so 17:57:18 have blocked off right after this meeting to do it, and post to ML 17:57:21 done 17:57:46 Sounds good. Looking forward to it! 17:57:55 * glebo slightly embarassed look on face, but not much 17:57:59 LouisF: thanks, I’ll create a blueprint to match that spec. 17:58:10 alexsyip: thx 17:58:34 2 minutes left. 17:58:53 dawood: want to tell us anything more about why you're interested in Congress? 17:58:59 i'm part of a cloud networking group at IBM (research). We are looking at a wide array of things, including network function virtualization, particularly in the context of open stack. using policy to manage across and between data centers is of interest. a focus on networking would be nice, but not required. Not sure what role i'm looking to play here yet. 17:59:51 dawood: welcome! 18:00:00 Interesting stuff. Sounds like similar background to many of us. 18:00:00 thanks 18:00:12 And we're out of time. 18:00:18 This has been a lively discussion today! 18:00:26 dawood: is your team an IT team for the rest of IBM, or a cloud offering to IBM customers, or both, or neither? 18:00:27 I'm looking forward to more of the same next week in Paris. 18:00:35 thinrichs1: meeting next week? 18:00:38 cheers all 18:00:41 neither. research for next gen cloud stuff 18:00:49 Let's cancel next week's IRC meeting. 18:00:55 we try to flow stuff into products 18:00:58 won't be in paris, unless mom cancels her 70th bday party ;-) 18:00:59 We'll have enough going on to keep track of. 18:01:07 have a great time. I'll be online, when possible 18:01:27 enjoy paris everyone! 18:01:28 We'll be good about using the etherpad for real-time as well as summaries. 18:01:36 Thanks all! 18:01:38 cheers 18:01:39 #endmeeting