15:00:28 #startmeeting monasca 15:00:29 Meeting started Wed Sep 14 15:00:28 2016 UTC and is due to finish in 60 minutes. The chair is rhochmuth. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:30 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:33 The meeting name has been set to 'monasca' 15:00:33 o/ 15:00:39 o/ 15:00:42 o/ 15:00:49 https://etherpad.openstack.org/p/monasca-team-meeting-agenda 15:01:05 o/ 15:01:10 o/ 15:01:11 Agenda for Wednesday September 14, 2016 (15:00 UTC) 15:01:12 1. MT for logging with Kibana plugin 15:01:12 1. blueprint => https://blueprints.launchpad.net/monasca/+spec/mt-in-kibana 15:01:12 2. Reviews 15:01:12 1. https://review.openstack.org/#/c/350470/ 15:01:12 o/ 15:01:12 2. https://review.openstack.org/#/c/286782/ - bug fix ? 15:01:24 Fujitsu's agenda... 15:01:30 :P 15:01:35 After last weeks killer agenda, this week is looking rather light 15:01:37 hello 15:01:49 hello everyone 15:02:05 so, does anyone have any open commentary 15:02:09 o/ 15:02:16 logistical issues 15:02:21 … 15:02:23 o/ 15:02:26 o/ 15:02:46 road next to my apartment ceased to exist if that account for logistical issue... 15:02:54 One thing I should mention is that the PTL candidacy/elections are coming up 15:03:24 tomasztrebski: hopefully it will be put back 15:03:33 rhochmuth: do you candidate? ;) 15:03:41 noob question - who's PTL ? 15:03:55 o/ 15:03:58 witek: i'll put my name in again 15:04:06 someone important ? very important ? or just representative like queen of England ? 15:04:16 project technical lead 15:04:25 tomasztrebski: queen of England 15:04:30 :-) 15:05:05 what's the procedure? 15:05:40 in the openstack/elections repo add your candidacy 15:05:48 if there is more than one, there is a vote 15:06:08 there is an official doc in the openstack wiki somewhere 15:06:14 but i dont' have it right now 15:06:35 i can send to you later 15:06:57 shall we start then? 15:07:05 yes, please 15:07:20 #topic MT for logging with Kibana plugin 15:07:35 ok, so it's on us 15:07:38 blueprint => https://blueprints.launchpad.net/monasca/+spec/mt-in-kibana 15:07:46 we would like to propose multi-tenancy solution for logging 15:08:14 we save tenant-id in elasticsearch index already 15:08:17 so, what is involved? 15:08:25 correct, tenant-id is saved 15:08:34 along with dimensions and the log message 15:08:47 well 15:08:57 index name has the tennant-id 15:09:09 that's part of the data of single project - so it's implicitly involved here 15:09:20 so you even don't have to read the fields 15:09:56 the proposal is to make it with kibana plugin 15:10:12 we already have authorisation plugin for kibana 15:10:43 now we extend the plugin to filter indices and save project specific data 15:11:31 some questions to that? 15:11:33 so, this goes back to the original arch diagrams showing the kibana plugin, correct? 15:12:11 yes, kibana sends query requests to ES 15:12:47 so, scoping the queries to return only the relevant data for a tenant 15:13:17 I think it allows quering only specific indices, am I right tomasztrebski ? 15:13:26 as well as being able to login to Kibana via Keystone authentication, similar to maybe what was done with Grafana 15:13:40 queries are scoped to the single project retrieved from token 15:13:45 similar to Grafana1 15:14:52 that scope goes to entities like actual data, dashboards and visualizations 15:14:59 would this plugin be hosted as part of the monasca repos in openstack 15:15:10 because kibana stores those too in elasticsearch 15:15:12 or part of the Kibana repos 15:15:20 rhochmuth: that is one question we wanted to ask 15:15:23 we would like to have this as openstack project 15:15:39 is that how the other kibana plugins are done 15:15:47 there are no official Kibana plugin repos 15:15:52 we cannot have this as kibana plugin because officially kibana does not support plugins as it is beatufilly presented in one of the readme files 15:15:55 AFAIK 15:15:56 we've already done this for the monasca-grafana-plugin repo 15:16:21 so basically a monasca-kibana-plugin repo 15:16:29 i thoguht kibana was adding the plugin capability 15:16:38 is that still in progress? 15:16:39 it would be great to have it @openstack 15:17:06 rhochmuth: yes, kibana has plugin capability, but does not support any plugins officially 15:17:13 they have framework to do this, but do not recommend writing custom plugins because framework is likely to changef from version to version 15:17:27 I mean, that's their statement I read somewhere 15:17:34 so, if the kibana way is similar to the grafana way, where a plugin lives outside of the project, then i think hosting it in openstack makes sense 15:17:59 so, in that case we would have to keep up with Kibana's changes 15:18:12 i'm assuming they would lock down on the plugin interface/api at some point 15:18:29 check this => https://github.com/elastic/kibana/blob/v4.5.1/src/plugins/README.txt :P 15:19:15 well, i don't see any alternative, other than re-writing plugins as they change 15:19:23 hopefully, they don't do something too drastic 15:20:08 even though it isn't official from Kibana, it still seems like a monasca-kibana-plugin repo, would be the correct approach 15:20:20 Kibana isn't going to allow us to merge the plugin in their repo 15:20:29 and we have to host it somewhere 15:20:32 tomasztrebski: do you have other opinion on that? 15:21:15 well even timelion, a plugin from ELK itself, is not part of kibana repo 15:21:20 i think the hpe team that works on centralized logging would be interested in developing too 15:21:39 so timelion is in a separate github repo? 15:22:04 https://github.com/elastic/timelion 15:22:22 heh, it was apparently Kibana while movign forward had timelion merged 15:22:29 It says, HEADS UP: This repo is deprecated. Timelion has been merged into Kibana. File issues over at https://www.github.com/elastic/kibana 15:22:33 maybe they've done this for Kibana 5.x 15:22:41 it was external, installable plugin for Kibana 4.x 15:23:00 that way I've been playing around with that 15:23:04 Are you interested in Kibana 4.X or 5.X? 15:23:09 tomasztrebski: I meant, is plugin the right aproach or would you prefer to fork kibana? 15:24:32 rhochmuth: tomasztrebski started with initial implementation as plugin 15:24:36 no opinion on that, though Kibana 5.x comes with several UI improvements, improved builtin applications (old elasticsearch site plugins would be now kibana apps) 15:25:03 does 5.X still support plugins? 15:25:12 but it is still in alpha stage, so we did not look into this 15:25:41 but it should support this because of the kibana apps and plugins are mainly designed to support kibana apps AFAIK 15:26:37 maybe we should discuss with ES? 15:26:47 just to make sure that plugins will be OK 15:27:07 my guess is that they would be interested in what we are doing 15:27:19 tomasztrebski: could it be an action item for you? 15:27:32 and if we let them know, they might provide us additional info 15:28:17 and help to not make the wrong decision 15:28:22 we can try that, I could write an issue (question) @ github informing about our intention and asking about some feedback 15:28:50 sure, that is one way, they might have an email list or irc channel that would be better 15:29:11 i can check here to see if we've every contacted them 15:29:13 or do they want to keep that functionality proprietary in Shield? 15:29:47 i don't know, but i still think it is worth a discussion with them 15:29:51 Shield is bound to basic http + maybe some plugins, but I don't recall keystone as one of them 15:30:03 if there are any plugins I mean 15:30:38 should we wrap-up then 15:30:56 action itmes are to learn a little more about plugins, hopefully get some feedback from ES 15:31:15 if all is OK, then we would create a monasca-kibana-plugin repo 15:31:24 step 3, write code 15:31:28 step 4 drink beer 15:31:31 sounds fine 15:31:46 rhochmuth: you forgot barbecue 15:31:55 lol 15:32:03 ok, I will write this issue @ github asking about some stuff + add your guys (roland, witek) to this 15:32:08 cool with that ? 15:32:12 sure 15:32:19 please send email, so i can follow 15:32:43 i can't beleive we are discussion yet another monasca repo 15:32:47 I think you will receive notification from github ( monasca-notification plugin -> lol ) but I will wrap this up in email as well 15:32:48 you guys are killing me 15:32:52 :) 15:33:24 I think you can take 5 additional repos ;-) 15:33:30 before collapsing or sth :) 15:33:37 Instead of project PTL, we need Monasca Chief of Repos 15:34:15 shield: " Login and Session Management in Kibana Shield contains a Kibana plugin that provides user authentication and session support, making it easier than ever to fully protect Kibana." 15:34:15 #topic reviews 15:34:16 https://review.openstack.org/#/c/350470/ 15:34:41 so, i see craig's been working on this with you 15:35:04 but, he hasnt' commented recently 15:35:12 should i ping him 15:35:28 mhm, we already have newer kafka, looks great btw, and that agent's stuff from this review would help us a bit 15:35:42 if you'd be so kind, that'd be great 15:35:47 ok, i'll do that 15:35:52 we have a planning session today 15:35:54 in hpe 15:36:00 i'll mention that review too 15:36:28 perhaps Joe Keen too 15:36:30 + think about upgrading kafka in devstack, there's one feature in newer kafka that might be interesting from MT POV, or limiting access to kafka in general 15:36:46 upgrade to 10.X 15:36:48 ? 15:37:03 right now to 0.9, kafka-python won't work with kafka 0.10 15:37:27 at least it says that client currently available in global requirements works only with kafka 0.9.x AFAIR 15:37:51 if openstack goes up with this I suspect we will need to upgrade kafka + monasca-common 15:37:56 up to kafka 0.10.x 15:38:28 well, unfortuatley, we are the one's blocking the upgrade becayse so far the latest python kafka client has got some problems 15:39:05 assuming that upgrade occurs, then we can move to kafka 10 15:39:13 if there are issues, I don't think nobody wants to break stuff up, me in the first place 15:39:30 right 15:39:57 i don't think we've looked at the latest python kafka client 15:40:07 as I said upgrading kafka/zookeeper in devstack is optional item I could do if you all agree to that, but not something that needs to be done 15:40:09 but joe has been following pretty closely all the developments 15:40:32 i think we should ping joe 15:40:40 and get this thoughts on that 15:40:48 Joe is in this review as well 15:41:05 I'd appreciate their (Joe, Craig) insight on this matter 15:41:22 ok, i'l; bring it up with them 15:42:01 that's all from my side btw 15:42:07 https://review.openstack.org/#/c/286782/ 15:42:26 ok, this one is old (I know) but recently we reported/fixed bug about logging in log-api here => https://review.openstack.org/#/c/365932/ 15:42:31 and noticed that review in the process 15:43:14 basically that would fix issue where files are not rolled after reaching certain size/age + enable granular control over logging different pieces of information at different severities 15:43:41 So, you want to get this change merged too 15:43:42 however I couldn't receive any feedback from creator of this change, so I am posting this here to ask if we could take it over and finish this review 15:44:02 + match up to other OS projects 15:44:09 in terms of handling the logging 15:44:32 sure, i don't see any problems taking it over 15:45:01 tomasztrebski: well, you've sent email today :) 15:45:33 i don't think he'll mind 15:45:45 checking with him via email is a good step 15:46:00 I did send an email, you were in CC ;-) 15:46:04 i'm assuming he'll be thrilled to see it progressed after 6 months 15:46:15 :) 15:46:28 but last monascan (I think that was the phrase) got bunch of emails :D so it might be overlooked :P 15:47:56 ok, so I will wait two extra days and if there's no progress me or someone else from Fujitsu will match this code to the log-api and it will be ready for final review/merge 15:47:57 ok ? 15:48:07 yes 15:48:17 sounds good 15:48:25 yupi :D 15:48:45 #topic summit 15:49:09 i thought we could spend a few minutes discussing the summit 15:49:19 right now we have 6 design summit slots 15:49:30 one more than austin 15:49:35 on which day? 15:49:54 i don't know 15:50:01 i beleive thursday and friday 15:50:04 spance multiple days 15:50:07 spans 15:50:32 i was hoping that several slots would be to discuss monasca analytics, like 3 15:50:55 i expecting several folks to show up 15:51:12 that leaves 3 slots or possibly 4 for more general monasca disucssions 15:51:13 from hp labs? 15:51:29 correct, as well as fujitsu 15:52:01 hosanai and his team 15:52:27 rhochmuth: yeah, i and daisuke will go there 15:52:43 hi hosanai 15:53:11 rhochmuth: hello 15:53:11 on friday, i would like to spend all my time on monasca-analytics if possible 15:53:33 and possibly part of thursday too 15:53:52 is there quick way to install analytics (like devstack plugin for monasca-transform) we could use to set this up quickly and play around ? 15:53:54 anyway, i think we have lot's of opportunities to do more monasca planning 15:54:07 cassandra would need 1 session or 2 15:54:10 there is a devstack plugin, but it isn't fast to setup 15:54:22 yes, i think cassandra would be 1 or 2 sessions 15:54:48 what other topics would be valuable to discuss in barcelona, with the restricted set of folks we have available 15:54:56 possibly the kibana discussion we were just having 15:55:03 yes, could be 15:55:15 and logging in general 15:55:40 tomasztrebski: i'm assuming you are going as well a kamil 15:56:07 yes... i will join 15:56:17 great! 15:56:26 tomasztrebski is sleeping... :) 15:56:53 one the bootcamp, what i did for austin is at, https://github.com/roland-hochmuth/monasca-bootcamp 15:56:59 no I am not, just responded privately...a little bit of intimacy please ;-) 15:57:00 that is an ipython notebook 15:57:23 great. i will take a look into 15:57:37 i don't know if you want to keep that format 15:57:44 i think the material shoudl be different though 15:57:58 covering different areas of monasca, including logging 15:58:24 if you have any questions about the ipython, please let me know 15:58:35 well, i guess that is a wrap for today 15:58:50 sounds like we've got some ideas on good topics for barcelona too 15:59:02 wish i was gonna be there 15:59:05 i'll follow up on those reviews that were mentioned with my team 15:59:12 me to 15:59:23 it is very difficult to so design summits without everyone there 15:59:31 bklei: yes, it's pitty 15:59:37 sad panda 15:59:42 the follow-on summit in february in boston, is the one i'm, hoping everyone can attend 15:59:51 panda wanna cry 16:00:10 kung-fu panda don't cry o.o 16:00:17 bklei, the dimensions names/values and names reivews are ready 16:00:20 there's no crying in monasca 16:00:26 ok -- will pull in and test 16:00:30 ooops got to go and close the meeting 16:00:34 thanks everyone 16:00:35 thanks 16:00:36 bye 16:00:37 bye 16:00:39 bye 16:00:52 #endmeeting