15:00:29 #startmeeting monasca 15:00:30 Meeting started Wed Nov 21 15:00:29 2018 UTC and is due to finish in 60 minutes. The chair is witek. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:31 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:33 The meeting name has been set to 'monasca' 15:00:45 Hello everyone 15:00:49 hi 15:01:05 hi koji_n 15:01:24 hello 15:01:29 Hey all 15:01:37 nice to see you all 15:01:57 the agenda in the usual place: 15:02:01 https://etherpad.openstack.org/p/monasca-team-meeting-agenda 15:02:20 let's start with the quick Summit recap 15:02:32 #topic Berlin Summit recap 15:03:00 we had couple presentations in Berlin 15:03:21 Project Update 15:03:24 https://www.openstack.org/summit/berlin-2018/summit-schedule/events/22869/monasca-openstack-monitoring 15:03:40 unfortunately still no videos available 15:04:12 I have listed our work around events and logs as the focus in Stein release 15:04:47 the interest was quite big 15:05:09 2. Project Onboarding 15:05:26 http://btw23.de/johannes/talks/monasca-onboarding.tar.bz2 15:05:52 then we had a presentation from StackHPC 15:06:02 3. Monitoring as a Service in the HPC Cloud 15:06:09 https://www.openstack.org/summit/berlin-2018/summit-schedule/events/22446/monitoring-as-a-service-in-the-hpc-cloud 15:06:23 very good informative presentation 15:07:04 presenting Monasca deployed with Kolla Ansible 15:07:43 good to hear the interest in monasca was high 15:07:58 4. Monitoring Hands-On Workshop 15:08:03 https://github.com/martinchacon/monasca-bootcamp 15:08:13 that one was really packed! 15:08:39 we had almost 100 people doing hands-on excercises with Monasca 15:08:51 wow 15:08:55 Great job from Martin on that one 15:09:53 after both, Project Update and Hands-On there were questions about Ceilometer 15:10:08 if Monasca can/will replace it? 15:10:34 also, there were questions about deployment methods for the agent 15:11:08 I have pointed to the monasca-ceilometer plugin 15:11:46 another very interesting presentation was from Red Hat 15:11:58 https://www.openstack.org/assets/presentation-media/OpenStack-Summit-2018-Prometheus-Operator-to-monitor-OpenStack.pdf 15:13:00 they have admitted that Gnocchi is not a viable solution for monitoring at scale 15:13:26 facinating. Scale was one of the main points of Gnocchi 15:13:27 and proposed an architecture similar to Monasa's 15:14:05 with Collectd, message bus and Prometheus 15:14:22 everything deployed on OpenShift 15:15:35 Quite the stack 15:16:43 in general, my conclusion from the summit is that we should invest some more effort in the agent, to make it more accessible and useful for users 15:17:11 we should define `core` plugins and support them better, add some testing 15:17:36 what has come up in the PTG already, we should adevertise our support for Prometheus 15:17:55 I think there are also some issues with some of the detection plugins not working well from inside a container 15:18:31 yes, we have seen some as well, some fixed, some not 15:19:01 we should also think about making official packages for the agent 15:19:10 for Debian/Ubuntu and CentOS 15:20:17 Official Ansible playbook could be nice as well 15:20:39 have to check if there is one in OSA 15:21:03 #action check for official Ansible playbook 15:21:21 There is https://docs.openstack.org/openstack-ansible-os_monasca/latest/ but I'm not sure if it is up to date/usable 15:21:34 We have forked one and got it working with Ubuntu/Centos/SuSE 15:22:06 As you say it would be nice to have one under the Monasca namespace 15:22:10 joadavis: that's for all Monasca, should be functional 15:22:19 long ago we talked internally about contributing changes from Ardana to OSA, but haven't put time in to it 15:23:16 dougsz: good to hear, perhaps we should move it to OSA? 15:23:26 can you share the repo? 15:23:56 https://github.com/stackhpc/ansible-role-monasca-agent 15:24:03 thanks 15:24:35 another topic which pulled some attention was auto-scaling 15:24:43 https://github.com/ArdanaCLM/monasca-ansible is part of it. There are also repos for monasca-transform-ansible and spark-ansible 15:25:26 👍 15:26:02 from what I heard Monasca role in OSA should be working fine 15:26:13 but does not has logging and monasca-transform 15:26:26 or cassandra support from what I see 15:26:31 correct 15:27:12 we could ping the guys if they're interested adding support for these 15:27:51 anyway, we've side-tracked a little 15:27:59 and add monasca-ceilometer to the list. :) 15:28:20 I think Ansible is not providing that kind of comfort as distribution packages 15:29:12 and I think most operators expect to install it from the package 15:29:40 I think we have a slightly different perspective. Galaxy roles have worked well for us. 15:29:56 agree for your case 15:30:22 Whatever official solution there is, it would be nice to import it to Kolla Ansible. Packages would work, or Galaxy role. Not sure about importing OSA components. 15:30:40 Anyway, sorry for sidetracking! 15:31:40 another think I wanted to mention is auto-scaling discussions in Forum 15:32:00 they think about creating a new SIG for auto-scaling 15:32:20 to unify approaches, identify problems, collect documentation 15:32:38 so a separate SIG from self-healing? 15:32:43 yes 15:32:59 ricolin from Heat drives it 15:33:14 https://etherpad.openstack.org/p/autoscaling-integration-and-feedback 15:33:46 self-healing had a meeting of course too: 15:33:55 https://etherpad.openstack.org/p/berlin-self-healing-sig-brainstorm 15:35:07 another thought after Summit, we should put more attention to upgrading the middleware regularly 15:35:21 in particular our ELK stack is very outdated 15:35:50 dougsz: any comments about the summit from you? 15:36:13 Was great to see so many people turn up to the Monasca meetings 15:36:35 yes, that was indeed very positive 15:36:59 I think you covered it well 15:37:12 we also had a chat with Fdaisuke_ about plans of monasca-analytics 15:37:47 yes 15:38:28 I asked for more work on monitoring related use-cases, like anomaly prediction or forecasting 15:39:21 We talked about monasca-analytics as follows 15:39:34 - Update top page of monasca-analytics 15:40:00 - Submit sample with monasca 15:41:32 And now, we update existing functions, and add new functions 15:41:41 for monasca-analytics 15:42:08 thanks, yes, I would like to have some simple example use case, how the repo can be used with the rest of Monasca 15:42:24 with easy step-by-step procedure 15:43:13 oops, I wanted it to be a *short* recap 15:43:26 :) 15:43:35 I'm looking forward to the videos 15:43:40 if there are no other comments, we can move on 15:44:03 #topic specs 15:44:50 https://review.openstack.org/583803 15:45:11 I have added some small changes and comments yesterday 15:45:45 thanks 15:45:54 I'll take a look 15:46:07 one question is, should we store all events in one ES index rotated daily? 15:46:17 for admin project 15:47:34 https://review.openstack.org/609055 15:47:39 I haven't looked at that side of events yet. And I'm not an ES expert. Anyone else have an opinion? 15:47:45 What are the other options? Classify events and save those to separate indices? 15:47:58 I suppose we still need to rotate them in that case. 15:48:03 for logs we store them per tenant 15:48:37 Kibana plugin filters the indices for a given project 15:48:53 so you cannot see other projects' data 15:49:29 for events, we could do similarly and spread events across projects 15:49:40 there are some problems with that approach 15:50:11 Yes - are all events associated with a project id? 15:50:11 operator(admin) would have to change the index for every project 15:50:42 dougsz: that's another problem, not all events include project information 15:50:43 some events, like keystone creating a user, are not associated with a project 15:51:59 Hmm. It does seem easier to make them all admin only, at least to begin with. 15:52:36 We could always have something which selectively copies out events to a tenants project if required in the future. 15:52:54 folks, please think through and leave the comment in review 15:53:16 👍 15:53:24 for merging APIs https://review.openstack.org/609055 15:53:37 someone from our team will polish it this week 15:53:50 and I think is ready to merge 15:54:15 there are two others: 15:54:28 Metrics retention policy enhancement 15:54:33 https://review.openstack.org/547233 15:54:45 Spec for Performance metrics of Python Persister 15:54:46 https://review.openstack.org/605782 15:55:44 I think both of those are ready to merge. We may have more details to add to metrics retention policy, but can add those in a subsequent commit 15:56:16 OK, I'll try to find time to review 15:56:29 thanks 15:56:34 #topic self-healing meeting 15:56:43 one more announcement 15:56:57 Self-healing SIG has now a regular meeting 15:57:03 every second Wed 15:57:09 http://eavesdrop.openstack.org/#Self-healing_SIG_Meeting 15:57:29 at 9am and 5pm UTC 15:57:39 the second one in one hour 15:57:53 I have attended the morning one 15:58:27 and advertised that we're looking for developers willing to work on Monasca - Vitrage integration 15:58:40 cool 15:59:10 that's all from me 15:59:18 the time is also over 15:59:24 thanks all, bye 15:59:30 thank you everyone for joining 15:59:37 thx! 15:59:38 see you next time 15:59:47 #endmeeting