15:00:07 <rhochmuth> #startmeeting monasca
15:00:12 <openstack> Meeting started Wed Oct  7 15:00:07 2015 UTC and is due to finish in 60 minutes.  The chair is rhochmuth. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:13 <ddieterly> o/
15:00:13 <rhochmuth> role call
15:00:14 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:17 <openstack> The meeting name has been set to 'monasca'
15:00:20 <rhochmuth> o/
15:00:21 <bklei_> o/
15:00:22 <jimbaker> o/
15:00:25 <ddieterly> o/
15:00:34 <rhochmuth> hi everyone
15:00:35 <witek> o/
15:00:39 <bogdan_> 0/
15:00:43 <rhochmuth> we have a lot of items to cover today
15:00:44 <Kamil__> Hello
15:00:44 <bklei_> good morning
15:00:51 <mroderus> o/
15:01:00 <fabiog> hi
15:01:01 <rhochmuth> let's just try and go in order
15:01:08 <rhochmuth> and see how far we make it through
15:01:19 <rhochmuth> bklei_ you are up first
15:01:21 <tomasztrebski> let's roll out
15:01:24 <bklei_> cool
15:01:32 <rhochmuth> #topic twc
15:01:35 <bklei_> performance update on vertica
15:01:47 <bklei_> the bug is part of the issue, but not the whole thing
15:02:26 <bklei_> i put details in the agenda, but highlights include: cache where possible, try pre-join projections, and keep vertica on it's own box
15:02:57 <bklei_> and avoid that gnarly multiple inner join query -- requires alot more of the planner
15:03:11 <bklei_> and some minor encoding changes in projections
15:03:13 <rhochmuth> i thought you had tried removing the inner join, and didn't see much of an improvement
15:03:24 <rhochmuth> i realize it helps, but not nearly enough
15:03:31 <bklei_> minor improvement -- something like 20 to 30 %
15:03:43 <bklei_> but it adds up during concurrent queries
15:03:55 <rhochmuth> ok, so if we remove the inner joins, it would be nice to address deklan's comments related to converting over everywhere too
15:04:02 <bklei_> (and 'queries' includes the copy/merge stuff)
15:04:36 <bklei_> ok -- can do that -- not sure where else that needs to change, haven't seen any other ones in the profiler
15:04:39 <bklei_> but can look at the code
15:04:48 <rhochmuth> i'm not sure what it all includes, but we use the inner joins in multiple locations
15:05:05 <bklei_> ok -- will look and amend the patch i have up
15:05:23 <rhochmuth> ok, thanks, let us know if you have any questions and we can help
15:05:32 <bklei_> but -- we've talked about caching definition_dimension_id before -- would like to ask for help there
15:05:36 <rhochmuth> so, what else should we try
15:05:41 <bklei_> any resource avail at HP to help there?
15:05:50 <bklei_> i know you had some ideas there
15:05:54 <rhochmuth> probably not this week
15:05:55 <bklei_> (roland/deklan)
15:06:01 <rhochmuth> we are wrapping up a release
15:06:09 <rhochmuth> but, maybe next week
15:06:13 <rhochmuth> will have to check
15:06:32 <rhochmuth> would you recommend the caching as the next alternative to try?
15:07:03 <bklei_> i'm thinking in parallel -- if hp picks up caching, i'll work on projections?
15:07:15 <bklei_> and inner join removal
15:07:32 <rhochmuth> ok, i'll discuss with deklan and see what we can come up with on this side to help
15:07:37 <bklei_> bueno
15:07:50 <rhochmuth> ok, moving along
15:07:53 <bklei_> i think that covers performance
15:08:11 <rhochmuth> #topic libvirt
15:08:37 <bklei_> would like to see this merge if possible https://review.openstack.org/#/c/225337/
15:08:52 <bklei_> we've hit it several times at twc -- any blockers/objections to getting in?
15:09:05 <rhochmuth> ok, i see a bungh of +1, so it looks like it is ready
15:09:08 <rhochmuth> no objections
15:09:13 <rhochmuth> assuming you need a tag too?
15:09:14 <bklei_> cool
15:09:37 <bklei_> por favor
15:09:42 <rhochmuth> ok, just +2'd
15:09:52 <bklei_> awesome -- other libvirt topic
15:09:54 <rhochmuth> ping me if i forget to tag
15:09:58 <bklei_> https://bugs.launchpad.net/monasca/+bug/1491903
15:09:58 <openstack> Launchpad bug 1491903 in Monasca "libvirt plugin shows 100% CPU for an instance running at 6%" [Undecided,Triaged] - Assigned to David Schroeder (david-schroeder)
15:10:15 <bklei_> wondering what we can do to help get more inof
15:10:24 <bklei_> sort of stuck on this one
15:10:28 <rhochmuth> not exactly sure
15:10:44 <rhochmuth> strange that this is isolated to load balancer instances
15:10:59 <bklei_> talking to david m -- he thought i could provide a libvirt dumpxml
15:11:11 <bklei_> that may have other info for david s to digest
15:11:29 <bklei_> i can add that to the ticket today, beyond that i don't know what else to provide
15:11:48 <rhochmuth> so, i think we are waiting on david to respond to this
15:11:55 <bklei_> yeah
15:12:10 <rhochmuth> i can ping him
15:12:20 <bklei_> thx -- and i'll add the dumpxml
15:12:32 <rhochmuth> but this sounds like a pretty nasty area and difficult to diagnose
15:12:49 <bklei_> yeah, a bit of a black hole
15:13:04 <bklei_> keeping moving
15:13:08 <bklei_> #topic monasca-ui
15:13:09 <rhochmuth> sure
15:13:27 <bklei_> i have no power i guess
15:13:42 <rhochmuth> #topic client versions
15:14:01 <rhochmuth> monasca-ui test fragility (https://review.openstack.org/#/c/231071/ as an example).  We should pin to client versions?
15:14:04 <bklei_> monasca-ui seems fragile, most times i push a patch, tests fail because we don't pin clients/oslo
15:14:18 <bklei_> it's fixed today, just want to try to pin more in the future
15:14:28 <bklei_> specifically stuff in test_requirements.txt
15:14:28 <rhochmuth> sounds like a good idea
15:14:34 <rhochmuth> we've been doing that in other parts of monasca
15:14:42 <bklei_> i'll try to do that in my patches, just a public service announcement
15:14:47 <bklei_> good pip practice
15:14:52 <rhochmuth> i agree
15:14:56 <bklei_> pipin' ain't easy :)
15:15:08 <rhochmuth> we've been running into issues lately with oslo
15:15:18 <bklei_> yeah, it's messy
15:15:22 <rhochmuth> so, i think we need to lock down versions
15:15:33 <bklei_> but i'm not blocked now, someone merged a change that fixed it
15:15:38 <bklei_> i rebased, good now
15:15:46 <bklei_> moving on
15:16:03 <rhochmuth> #topic libvirt
15:16:16 <rhochmuth> libvirt over-reporting bug (https://bugs.launchpad.net/monasca/+bug/1491903) -- how to move along?
15:16:16 <openstack> Launchpad bug 1491903 in Monasca "libvirt plugin shows 100% CPU for an instance running at 6%" [Undecided,Triaged] - Assigned to David Schroeder (david-schroeder)
15:16:23 <bklei_> done with libvirt -- memory bug
15:16:26 <bklei_> https://bugs.launchpad.net/monasca/+bug/1498188
15:16:27 <openstack> Launchpad bug 1498188 in Monasca "Memory/swap metrics recorded in wrong size" [Undecided,Invalid]
15:16:49 <bklei_> this got closed along with another as invalid -- would be good to have an explanation as to why invalid
15:17:24 <rhochmuth> if i recall correctly, we report memory utilization is mb
15:17:42 <rhochmuth> that is just the convention we've used so far
15:17:57 <rhochmuth> if you wnat to fave another unit reported that isn't a bug
15:18:01 <rhochmuth> right?
15:18:08 <bklei_> yeah, and grafana in it's current incantation doesn't supprot that as an axis option
15:18:21 <bklei_> so we just don't put units on those graphs
15:18:47 <bklei_> not a biggie -- but seems like good practice to say why the bug(s) are invalid when closing
15:18:55 <rhochmuth> so, i think we need someway to change the default reporting convention
15:19:05 <rhochmuth> i'll talk to alan
15:19:13 <rhochmuth> i agree, he could have provided more description
15:19:15 <bklei_> it could be grafana 2... will fix this
15:19:20 <rhochmuth> i thought he was going to do that
15:19:26 <bklei_> cool
15:19:45 <rhochmuth> so, if grafana 2 resolves, do we need to address in the agent
15:19:53 <rhochmuth> seems like the presentation layer should be able to handle this
15:20:02 <bklei_> no -- that one would take care of itself i think
15:20:03 <bklei_> agreed
15:20:19 <rhochmuth> if not, then a new switch in the agent to change the units that are reported would be one way
15:20:32 <rhochmuth> i dont' think we can/should change the convention
15:20:46 <rhochmuth> in other words need backwards compaibility
15:20:50 <bklei_> agree
15:21:27 <bklei_> i'm not finding the other memory bug that b. stilwell opened, but it was more about memory used -- and it got closed in the same way w/out explanation
15:21:31 <rbak> It looks like grafana 2 does offer a MB axis option
15:21:31 <rhochmuth> ok, so sounds like action is check on grafana 2 and if still doesn't address, then add a new switch to agent
15:21:44 <rhochmuth> #rbak cool
15:21:51 <bklei_> sounds good
15:21:52 <rhochmuth> so, are we good then
15:21:55 <bklei_> twc is done i think
15:22:09 <rhochmuth> i think we need a weekly meeting just for twc
15:22:11 <rhochmuth> :-)
15:22:13 <bklei_> sorry
15:22:19 <rhochmuth> no problem
15:22:20 <bklei_> we love and use monasca man!
15:22:40 <rhochmuth> #topic summit
15:22:50 <rhochmuth> #fabiog u r up
15:22:57 <fabiog> ok
15:22:58 <rhochmuth> how that for some irc foo
15:23:06 <fabiog> I will be quick
15:23:27 <rhochmuth> yes
15:23:39 <fabiog> Congress team gracefully agreed to meet with us and they are going to use one of their slot to hold a joint meeting
15:23:48 <fabiog> they only have 3 slots all on Wed
15:23:53 <rhochmuth> awesome, thanks for coordinating that
15:24:04 <fabiog> as soon as I know which one is assigned to us, I will notify it to the team here
15:24:08 <fabiog> rhochmuth: no problem
15:24:17 <rhochmuth> ok
15:24:20 <bogdan_> Is anyone wellcome to join?
15:24:25 <fabiog> yes
15:24:27 <bogdan_> :)
15:24:29 <fabiog> bogdan_: sure
15:24:32 <bogdan_> I'd love to
15:24:48 <rhochmuth> all sessions are open to everyone
15:24:50 <fabiog> on top of that I managed to get Cisco to give us a room for a Monasca meeting
15:24:52 <bogdan_> cool
15:25:11 <fabiog> so we can have a meeting and discuss few topics
15:25:19 <rhochmuth> on a related topic, i still need to check with Searchlight
15:25:24 <fabiog> is going to be on Thu 4:30pm to 6pm
15:25:39 <fabiog> I will send the details as soon as I get the room number
15:25:53 <rhochmuth> there is also a session being proposed by Ceilometer on Aodh
15:26:02 <bogdan_> when?
15:26:04 <fabiog> it is really tough to get space this time, I guess the Tokyo venue is even smaller than Paris
15:26:31 <fabiog> bogdan_: Thursday 4:30pm to 6pm
15:26:58 <fabiog> bogdan_: Thu 10/29
15:27:12 <rhochmuth> so, in summary, looks like we'll have some good sessions
15:27:20 <fabiog> at least 3
15:27:33 <rhochmuth> i'll check with travis tripp on searchlight
15:27:35 <witek> #info Session at the Design Summit. Thursday 4:30pm to 6pm local time
15:27:38 <fabiog> Ceilometer, Congress and one for us or overflow of the other two
15:27:54 <fabiog> witek: thanks!!
15:28:13 <fabiog> rhochmuth: I am done
15:28:17 <rhochmuth> i'm sure after the presentations there will be lot's of requests for more meetings
15:28:22 <rhochmuth> this happened to me in Paris
15:28:24 <fabiog> rhochmuth: yup
15:28:46 <rhochmuth> so, with all the monasca sessions, expect lot's of impromtu requests for meetings
15:29:16 <rhochmuth> we'll need to share phone numbers and contact info to coordinate
15:29:57 <rhochmuth> #topic InfluxDB
15:30:09 <rhochmuth> fabio take it away on InfluxDB
15:30:14 <fabiog> let's get 0.9.4.2 !!!
15:30:31 <rhochmuth> is srini working on this?
15:30:36 <fabiog> we have been playing with the newer versions of InfluxSV
15:30:38 <fabiog> DB
15:30:39 <fabiog> yes
15:30:59 <rhochmuth> so, what have you measure for the latest influxdb performance
15:31:07 <rhochmuth> as in raw performance?
15:31:12 <fabiog> and we noticed that there is a huge performance difference between 0.9.1 and 0.9.4.x
15:31:17 <fabiog> yes
15:31:22 <fabiog> especially in writing
15:31:28 <fabiog> it is at least a factor of 10
15:31:32 <bogdan_> difference means increase?
15:31:46 <fabiog> bogdan_: I mean roughly 10 times faster
15:31:52 <bogdan_> thx
15:31:58 <fabiog> we use it in the Ceilosca context and it worked fine
15:32:17 <mroderus> fabiog: was performance the only reason to try out 0.9.4.2?
15:32:20 <fabiog> but we really haven't done any extensive testing for Monasca outside the part that interests Ceilosca
15:32:27 <fabiog> mroderus: yes
15:32:32 <bklei_> 0.9.5 sounds promising with the time structured merge tree stuff too fabiog
15:32:45 <fabiog> bklei_: we will try that one when it comes out
15:32:59 <bklei_> hope it's not a step back in terms of cluster stability...
15:33:05 <fabiog> for now Srini is going to patch the Ansible script to use 0.9.4.2
15:33:17 <fabiog> then we need to see it all the Monasca tests pass or not
15:33:32 <fabiog> is Cray in the meeting?
15:33:48 <fabiog> they were doing some tests with the latest version of Influx ...
15:33:50 <rhochmuth> is was hoping bmotz was here
15:33:57 <rhochmuth> but i don't see listed
15:34:08 <fabiog> rhochmuth: nevermind
15:34:20 <rhochmuth> anyway, he had mentioned that somethign had changed between some versino of influxdb
15:34:31 <fabiog> I think the stability and performance of 0.9.4.x is a big improvement, so we should do it anyway
15:34:39 <rhochmuth> yes, and they were running with the latest inclusdb
15:34:54 <fabiog> rhochmuth: inclusdb?
15:34:57 <fabiog> :-)
15:35:13 <fabiog> so, please watch this space
15:35:13 <rhochmuth> if srini updates and we are pretty sure there aren't any issues, then we'll gladly take it
15:35:23 <fabiog> rhochmuth: that is the idea
15:35:43 <fabiog> we are going to change it in the "mini-mon" Ansible script and see if it works
15:35:47 <rhochmuth> will twc be able to do some testing?
15:35:57 <tomasztrebski> we 'd be interested in that as well 'cause we were about to start with the same thing, so it's to hear that is in progress as we speak
15:36:05 <fabiog> rhochmuth: also I think it would be good to have a parameter for the devstack where you can specify the version
15:36:05 <tomasztrebski> *good to hear
15:36:20 <rhochmuth> the other issue around influxdb is support for binary protocol
15:36:32 <fabiog> so in the future you can decide which version to run setting it in the local.conf
15:37:07 <ddieterly> we don't have a local.conf in mini-mon
15:38:21 <fabiog> ddieterly: no I am talking about devstack
15:38:49 <ddieterly> ok
15:39:06 <fabiog> ddieterly: it would be cool to have a shell variable where you can express the version of Influx so then when devstack runs it will pick and install the one you want
15:39:12 <bklei_> twc probably won't do much influxdb testing for a couple of months rhochmuth
15:39:27 <rhochmuth> blkei_: ok
15:40:04 <rhochmuth> is it time for next topic?
15:40:04 <witek> fabiog: did you test clustering and/or HA?
15:40:16 <fabiog> witek: not yet
15:40:40 <fabiog> witek: as I mentioned I just tested in the context of Ceilosca, which is a devstack implementation of Ceilometer and Monasca
15:40:58 <fabiog> witek: but after the summit we will deploy a more complex version
15:41:12 <witek> thanks
15:41:26 <tomasztrebski> fabiog: but the change in ansible role will be soon available ?  I just checked and it's not at github at the moment
15:41:27 <fabiog> witek: I also want to try Monasca in Containers
15:41:28 <tomasztrebski> ?
15:41:51 <fabiog> tomasztrebski: I have to check with Srini
15:42:07 <tomasztrebski> fabiog: thx
15:42:16 <fabiog> tomasztrebski: np
15:42:39 <Kamil__> monasca in containers +1
15:42:52 <tomasztrebski> yeah, that would nice to see how that goes :)
15:42:56 <rhochmuth> are you aware of monasca-docker
15:43:09 <Kamil__> not really
15:43:19 <rhochmuth> https://github.com/hpcloud-mon/monasca-docker
15:43:23 <Kamil__> cool.. thx
15:43:28 <rhochmuth> it is a demo environment primarily
15:43:30 <fabiog> rhochmuth: is that a single instance of all the services, like kafka and influx?
15:43:33 <tomasztrebski> i've taken a quick peek there, but did not run it actually, but I assume that's a demo env /
15:43:37 <fabiog> rhochmuth: can it be scaled out?
15:43:37 <tomasztrebski> of some sort ?
15:43:58 <rhochmuth> it is not clustered and really for demo purposes
15:44:06 <bogdan_> tomasztrebski: are you asking Fabio about ansible because you're working on the same thing and you want to avoid working redundantly?
15:44:34 <Kamil__> rhochmuth: But at the end, ansible is still provisioned in container, or not?
15:45:14 <rhochmuth> it uses ansible to provision in the container i beleive
15:45:28 <rhochmuth> i haven't spent time looking into how it works
15:45:35 <rhochmuth> i used it a few times
15:46:19 <rhochmuth> next topic?
15:46:27 <fabiog> sure
15:46:29 <rhochmuth> #topic security
15:46:35 <mroderus> that's mine
15:46:37 <rhochmuth> 5. Security: clear-text passwords in agent config
15:46:46 <tomasztrebski> bogdan_: I wrote you on priv to clear the ground here
15:47:01 <mroderus> our sec department mentioned that in the agent config files, passwords are written in clear text
15:47:08 <rhochmuth> correct
15:47:17 <mroderus> has this issue been discussed before?
15:47:18 <rhochmuth> they are clear text everywhere in Monasca
15:47:27 <mroderus> do you see it as a security issue as well?
15:47:29 <rhochmuth> we've done our own internal security reviews
15:47:37 <rhochmuth> but we haven't discussed with community
15:47:57 <mroderus> has it been addressed as an issue in your internal reviews?
15:48:07 <rhochmuth> yes
15:48:20 <rhochmuth> we don't see it as an issue
15:48:49 <mroderus> TWC: do you see this as a problem?
15:49:06 <bklei_> no -- file perms are enough for us
15:49:19 <rhochmuth> but, if your security team at Fujitsu sees this as an issue, then it could be addressed
15:49:29 <rhochmuth> bklei_ same here
15:49:46 <mroderus> yes, let's see first what comes out in our internal discussions
15:50:00 <mroderus> thanks rhochmuth and bklei_
15:50:08 <rhochmuth> mroderus sounds good
15:50:12 <rhochmuth> i understand the concerns
15:50:12 <bklei_> +1
15:50:15 <tomasztrebski> as for the file permission, mroderus I think we might want to adress that we are also taking look at it
15:50:25 <mroderus> ok#
15:50:41 <rhochmuth> if your security team doesn't agree then we could look into encrypting the password
15:50:48 <rhochmuth> as a new feature
15:51:07 <tomasztrebski> the same sec team spotted that as well that file permissions are not so consistent, so we've taken a look here and try to modify them as much as possible
15:51:30 <rhochmuth> ok, let us know if you find anything
15:51:46 <rhochmuth> we are definitely interested in addressing issues there
15:52:09 <mroderus> ok.. if security becomes a greater issue, we may start a separate meeting on this
15:52:20 <rhochmuth> #topic log api
15:52:35 <rhochmuth> so i see the python log api has been posted
15:52:45 <rhochmuth> i left some comments
15:53:05 <rhochmuth> i'm wondering if we should just get this merged and then start improving it
15:53:12 <rhochmuth> working on it ...
15:53:37 <tomasztrebski> I am all for it, just want to address current issues you written comments for.
15:54:08 <rhochmuth> well, i think it is a work in progress, so I don't want to be too critical at this point and hold up progress
15:54:17 <tomasztrebski> + it looks like problematic services.py (the one from monasca-common) can be abandoned
15:54:27 <rhochmuth> yes
15:54:32 <tomasztrebski> according to constant you pointed out as obsolete
15:54:37 <rhochmuth> correct
15:55:16 <tomasztrebski> ok, thx for the feedback, feeling awesome with that :), I will apply changes and post new PS as soon as I can
15:55:17 <bogdan_> tomasztrebski: by when you plan to finish what's been commented?
15:56:01 <tomasztrebski> bogdan_: tommorow, that seems like light task to handle
15:56:07 <witek> tomasztrebski: so i think we can wait for your PS and merge
15:56:22 <tomasztrebski> witek: that's fine with me
15:56:31 <bogdan_> rhochmuth: fair to you?
15:56:36 <rhochmuth> sounds good
15:56:40 <bogdan_> great
15:56:41 <tomasztrebski> cool
15:57:06 <rhochmuth> #topic paging issues
15:57:17 <tomasztrebski> yeah, that's one for me
15:57:57 <rhochmuth> not sure if we can continue at top of the hour
15:58:15 <tomasztrebski> that's not so easy to explain
15:58:25 <witek> lets move it to the next meeting
15:58:30 <rhochmuth> ok
15:58:31 <bklei_> before we part
15:58:32 <bklei_> would like some reviews/love on https://review.openstack.org/#/c/231587/ and https://review.openstack.org/#/c/231071/
15:58:34 <tomasztrebski> +1
15:58:39 <rhochmuth> #topic devstack
15:58:45 <rhochmuth> how are you doing deklan
15:58:50 <rhochmuth> on devstack
15:59:01 <ddieterly> basic monasca working in devstack
15:59:06 <rhochmuth> awesome
15:59:09 <ddieterly> java api and persister
15:59:15 <ddieterly> working on python api and persister now
15:59:24 <ddieterly> please +1 and +2 https://review.openstack.org/#/c/229523/
15:59:33 <rhochmuth> ok, will start to review
15:59:43 <rhochmuth> also, please checkout out the Tempest tests
15:59:56 <rhochmuth> still a wrok in progress, but i would like to get first commit
16:00:01 <rhochmuth> approved soon
16:00:08 <rhochmuth> so, i think we're done
16:00:16 <rhochmuth> i'll need to close the meeting out
16:00:20 <rhochmuth> we got rushed at the end
16:00:28 <rhochmuth> so follow-up if you need to
16:00:35 <bklei_> thx guys
16:00:37 <rhochmuth> bye everyone
16:00:37 <Kamil__> have a nice day
16:00:38 <mroderus> thanks
16:00:40 <mroderus> bye
16:00:41 <tomasztrebski> thx
16:00:42 <bogdan_> cheers!
16:00:45 <ddieterly> ciao
16:00:52 <rhochmuth> #endmeeting