15:00:58 <rhochmuth> #startmeeting monasca
15:00:59 <openstack> Meeting started Wed Jun  1 15:00:58 2016 UTC and is due to finish in 60 minutes.  The chair is rhochmuth. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:00 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:01 <rhochmuth> o/
15:01:03 <openstack> The meeting name has been set to 'monasca'
15:01:16 <hosanai> o/
15:01:17 <flinthpe> o/
15:01:18 <iurygregory> o/
15:01:19 <rbak> o/
15:01:20 <koji> o/
15:01:21 <arturbasiak> o/
15:01:22 <witek> hi
15:01:22 <shinya_kwbt> o/
15:01:29 <tsv_> <tsv> 0/
15:01:30 <ericksonsantos_> Hi o/
15:01:33 <rhochmuth> https://etherpad.openstack.org/p/monasca-team-meeting-agenda
15:01:45 <rhochmuth> 1.	Horizon pagination style https://blueprints.launchpad.net/monasca/+spec/horizon-pagination-style (shinya)
15:01:45 <rhochmuth> 2.	Log-API
15:01:45 <rhochmuth> 3.	Deterministic alarms
15:01:45 <rhochmuth> 4.	Storm upgrade status ?
15:01:45 <rhochmuth> 5.	Reviews
15:01:45 <rhochmuth> 1.	https://review.openstack.org/#/c/321864/
15:01:45 <rhochmuth> 2.	https://review.openstack.org/#/c/286281/
15:01:46 <rhochmuth> 3.	https://review.openstack.org/#/c/319887/
15:01:46 <rhochmuth> 4.	https://review.openstack.org/#/c/323277/
15:01:47 <rhochmuth> 5.	https://review.openstack.org/#/c/322750/
15:01:59 <tomasztrebski> o/
15:02:16 <rhochmuth> Looks like a reasonable amount of stuff to get through
15:02:37 <rhochmuth> #topic Horizon pagination style
15:02:46 <shinya_kwbt> Hi.
15:02:50 <rhochmuth> hi
15:03:00 <shinya_kwbt> Can you guys understand my proposal?
15:03:17 <shinya_kwbt> This is my first blue print :-)
15:03:23 <claudiub> o/
15:03:27 <shinya_kwbt> So it may be not good document.
15:03:34 <tomasztrebski> I think I do understand :)
15:03:48 <shinya_kwbt> Thanks I'm relieved.
15:04:17 <fabiog> o/
15:04:34 <shinya_kwbt> Nova and Glance and may be other API has marker option.
15:04:41 <shinya_kwbt> This is similar offset.
15:05:04 <shinya_kwbt> I think marker option is almost same as offset.
15:05:17 <shinya_kwbt> But there is a difference between marker and offset.
15:05:30 <shinya_kwbt> On Nova, if sort_dir=desc and marker are given.
15:05:47 <shinya_kwbt> API will return elements decsending from marker.
15:06:01 <shinya_kwbt> On Monasca, if "sort_by=id desc" and offset are given.
15:06:16 <shinya_kwbt> API will return elements descending from end to offset.
15:06:41 <shinya_kwbt> Horizon pagination style expects elements decending from marker(offset).
15:07:01 <shinya_kwbt> When prev page button pressed.
15:07:03 <rhochmuth> i think i understand
15:07:13 <shinya_kwbt> So we need to change sort spec or add marker option to apply Horizon style.
15:07:29 <rhochmuth> so, in your example, the offset is 11
15:07:52 <rhochmuth> i would expect 10..1 being returned in descending order
15:08:27 <shinya_kwbt> Yes, when id desc and offset are given.
15:08:31 <rhochmuth> the offset should be the starting location - 1 in the sorted (whether ascending or descending)
15:08:49 <rhochmuth> in the sorted lest
15:09:20 <rhochmuth> if 20..12 is being returned and the offset is 11, then that seems incorrect
15:10:27 <rhochmuth> the offset should apply to the sorted list
15:10:34 <rhochmuth> does that make sense
15:10:41 <rhochmuth> i'm working off of your example
15:11:13 <rhochmuth> so, given list [1, …, 20]
15:11:54 <rhochmuth> if query with sort descending and offset 11, i would expect [10, …, 1] as the result set
15:12:08 <shinya_kwbt> When ascending, api returns 12..20 right?
15:12:22 <rhochmuth> correct
15:12:52 <shinya_kwbt> Yes > if query with sort descending and offset 11, i would expect [10, ???, 1] as the result set
15:13:02 <rhochmuth> to me this seems like a bug.
15:13:22 <shinya_kwbt> I see.
15:13:24 <rhochmuth> did se actually specify something in the spec that works the other way
15:13:37 <rhochmuth> i don't think the spec went to this level of detail if i recall
15:14:26 <rhochmuth> but the behaviour that you specify seems to be the correct way, and what is being returned is not as expected
15:15:10 <shinya_kwbt> OK so I don't add new marker option. I will fix sort spec.
15:15:45 <rhochmuth> isn't there a bug that needs to be fixed
15:16:23 <shinya_kwbt> I think this is all.
15:16:48 <rhochmuth> hmmm, maybe i was confused
15:17:10 <rhochmuth> in your example, you have offset 11
15:17:16 <rhochmuth> sort by descending
15:17:41 <rhochmuth> i would expect [10..1] being returned
15:17:51 <rhochmuth> but you are saying 20..12 is returned
15:18:01 <rhochmuth> so, i consider that a bug, not a spec change
15:18:23 <rhochmuth> an horizon expect 10..1 to be returned
15:18:36 <rhochmuth> so, don't we need to change the code
15:18:44 <rhochmuth> and possibly update the API spec
15:18:48 <rhochmuth> and update the blueprint
15:20:09 <shinya_kwbt> Yes this is a bug, not spec change. So I will fix a bug.
15:20:16 <rhochmuth> thanks
15:20:24 <rhochmuth> ok, so let's move on
15:20:41 <rhochmuth> #topic Log-API
15:21:12 <witek> I have sent a comment to the blueprint TSV published last week
15:21:22 <witek> http://lists.openstack.org/pipermail/openstack-dev/2016-May/096245.html
15:21:27 <rhochmuth> thanks witek
15:21:37 <tsv_> witek, thanks. i replied to your mail
15:21:46 <tsv_> just now
15:22:10 <tsv_> i am ok with your proposal, if others don't have any issues
15:22:53 <rhochmuth> not sure about using the kafka_topic_name in the POST
15:23:05 <rhochmuth> that seems to expose internals
15:23:23 <witek> it does not have to be kafka topic
15:23:55 <tsv_> that's what i understood it as too, topic could be a generic name, no Roland ?
15:23:55 <witek> just some name which can be identified by API and redirected to the given topic
15:25:34 <tomasztrebski> I think the main question here do we want API to be fully flexible or force clients to organize payloads per target topic [routing]
15:26:03 <tomasztrebski> with request or attributes specified globally client sending logs would have to ensure that entire bulk goes into single topic
15:26:03 <rhochmuth> i was thinking general
15:26:10 <rhochmuth> attributes could be provided
15:26:21 <rhochmuth> and then it is up to internals of the service to route accordingly
15:26:54 <witek> agree
15:27:36 <tomasztrebski> by attributes you mean that it can either specified globally and locally (per entry in bulk, same as it was for dimensions) or just globally or just locally ?
15:27:46 <rhochmuth> correct
15:27:59 <tomasztrebski> but which one :D
15:28:00 <rhochmuth> i think same levles as dimensions
15:28:11 <tomasztrebski> ah, ok, so we have the same thing in mind
15:28:54 <tsv_> +1
15:29:18 <rhochmuth> if you have "attributes" that have all the same levels as "dimensions" then i think you can do everything required
15:29:30 <tomasztrebski> another question from me at least, because I did not fully understand how that might work, is "what is TTL of attributes", do they cease to exist at log-api level ? or do they follow log inside envelope or even log itself ?
15:29:39 <rhochmuth> but, i don't think the POST /v3.0/logs/topics/{kafka_topic_name}
15:29:40 <rhochmuth> is required
15:30:28 <rhochmuth> i haven't thought that far into implementation
15:30:32 <witek> if we want to implement it with attributes we have to reserve some name to identify it as routing target
15:30:56 <rhochmuth> tsv just wanted to use attributes to route to a specific topoic
15:31:09 <rhochmuth> in that case, they aren't required after they are published to a topic
15:31:45 <tomasztrebski> for me that is sufficient
15:31:53 <witek> but in general you may want to send some additional info with logs which could even be stored in elastic
15:32:11 <rhochmuth> i think that is good for now too, as that is all that tsv was attempting to address
15:32:19 <rhochmuth> but i agree with you witek too
15:32:31 <rhochmuth> but, we don't have a need for that at the moment
15:32:45 <tsv_> rhochmuth: i see value in the API name change. With the topic name in the API endpoint, it may be easy to enforce RBAC using a policy file later - if we want to restrict access to specify topics. But i agree, not an immediate requirement
15:34:21 <rhochmuth> so, can we close for now on adding "attributues" and just use them to route to a configurable topic
15:34:27 <tomasztrebski> TSV: yet another question from me here, please keep in mind the problem I've been working on recently - limits that are put on topics in kafka regarding maximum size of the message that can be received ? I highly value the idea of ensuring that logs are not lost because envelope exceeded that maximum size, but right now we have only single topic and log-api can be configured to know that limit, but what about multi
15:34:50 <tomasztrebski> ups...ok...that's just a thing to think about, let's not dwell on this right now
15:35:58 <witek> yes, we can close the topic I think
15:36:09 <rhochmuth> thanks witek
15:36:11 <tsv_> tomasz, good point, will think through. thanks
15:36:14 <tsv_> thanks witek
15:36:21 <rhochmuth> #topic Deterministic alarms
15:37:04 <rhochmuth> tomasztrebski: i guess that is u
15:37:16 <tomasztrebski> so...that's me...again...and I simply lost faith here. Thing that bothers me is that I managed to create those alarm definitions and they were set to OK right away for deterministic case
15:37:27 <tomasztrebski> old behaviour worked for non-deterministic
15:37:36 <rhochmuth> that is what i expected based on the code reviews
15:37:37 <tomasztrebski> but 3 people told me it does not work for them
15:38:01 <rhochmuth> is there anything we are missing in your environment
15:38:11 <tomasztrebski> that's what I expect, imagine my surprise :(
15:38:11 <rhochmuth> i altered the mysql table to add the required column
15:38:25 <rhochmuth> by hand after install
15:38:47 <rhochmuth> it is jsut monasca-common, monasca-thresh and monasca-api
15:39:02 <rhochmuth> and i used POSTman to simulate requests
15:39:10 <rhochmuth> i haven't tried debugging yet
15:39:30 <tomasztrebski> well I've posted some questions to thresh change, the most relevant thing is if ORM was used or not, because by default I have it enabled for thresh and api [java]
15:39:39 <tomasztrebski> and I thought about it quite recently
15:40:00 <rhochmuth> ryban and i are using mysql
15:40:03 <tomasztrebski> however python api was used without ORM, just copied devstack configuration and adjusted IP addresses + keystone roles
15:40:20 <rhochmuth> i didn't use python, just java
15:40:39 <tomasztrebski> ok, I will try tomorrow without ORM
15:40:49 <tomasztrebski> same steps as in gist I posted
15:40:51 <witek> I tested java + mysql - orm
15:41:25 <tomasztrebski> so that might be a place to look for a bug
15:42:24 <rhochmuth> so to wrap up, tomasztrebski will test with mysql and try and replicate our problem
15:42:32 <tomasztrebski> ok, I think that's all from my side, I know the difference I was looking for
15:42:48 <rhochmuth> and the rest of us will be in stand-by
15:42:57 <rhochmuth> sound good
15:43:40 <rhochmuth> #topic Storm upgrade status ?
15:44:24 <rhochmuth> this is still in progress, but not sure of the exact state right now
15:44:40 <rhochmuth> there are several reviews in flight
15:44:41 <tomasztrebski> Are there any news regarding that ? I am partially interested in that because changes I've made there and if that's one is merged that will add some effort
15:45:09 <rhochmuth> i think we will wait for your review to merge first
15:45:18 <rhochmuth> we wanted to get that one in
15:45:22 <rhochmuth> too
15:45:42 <tomasztrebski> that's very nice :), I will do my best to get this into shape
15:45:45 <rhochmuth> assuming it won't take too much longer to get deterministic alarms in
15:45:51 <tomasztrebski> kk
15:45:59 <rhochmuth> and you have just a minor issue
15:46:14 <tomasztrebski> well I am loosing 3:1, that's not very good :)
15:46:21 <rhochmuth> i'll mention to craig and ryan
15:46:27 <tomasztrebski> thx
15:46:31 <rhochmuth> :-)
15:46:50 <rhochmuth> #topic reviews
15:47:05 <rhochmuth> https://review.openstack.org/#/c/321864/
15:47:31 <rhochmuth> looks like that one is ready, i'll merge it, both ryan and hoppal reviewed
15:48:01 <rhochmuth> https://review.openstack.org/#/c/286281/
15:48:27 <rbak> This is the kv hint.  I just wanted to bring it up again since Brad isn't around this week.
15:48:43 <rhochmuth> sorry, i didn't get to that one last week
15:49:04 <rhochmuth> i wanted to review again, but assuming everything is addressed, i dont see a problem merging
15:49:11 <rhochmuth> will try and get to it today
15:49:25 <rbak> sounds good.  that's all I had on that.
15:49:27 <rhochmuth> https://review.openstack.org/#/c/319887/
15:49:32 <tomasztrebski> sorry to interrupt but since mysql change in agent was brought in - I think this change here https://review.openstack.org/#/c/322764/ is also worth merging
15:49:42 <tomasztrebski> I forgot to add this to agenda I guess
15:50:28 <rhochmuth> tomasztrebskiL i just merged that one too
15:50:52 <rhochmuth> for the next two minutes i'll merge anythign
15:51:19 <rhochmuth> https://review.openstack.org/#/c/319887/
15:51:47 <tomasztrebski> yes, truncation I mentioned to TSV, Artur was kind to test this for me
15:52:01 <rhochmuth> i haven't looked at the change
15:52:22 <rhochmuth> if you truncate the client won't know there is lost data
15:52:28 <rhochmuth> right?
15:52:32 <rhochmuth> maybe i shoudl look at the code
15:53:14 <tomasztrebski> if I won't truncate I will loose entire batch of logs :(...anyway we find it better right now to truncate log message instead of loosing it
15:53:46 <tomasztrebski> and the problem of safe bulk processing is another topic, but I don't want to discuss it now
15:53:47 <witek> the field 'truncated'=true is added, right?
15:53:57 <tomasztrebski> only for logs that were actually truncated
15:54:29 <rhochmuth> i'll review
15:54:35 <tomasztrebski> thx
15:55:07 <rhochmuth> looks like that one should be merged
15:55:08 <witek> you didn't make it in 2 minutes, Tomasz :)
15:55:20 <tomasztrebski> I did...didn't want to enforce anything :D
15:55:52 <rhochmuth> https://review.openstack.org/#/c/323277/
15:56:25 <rhochmuth> looks like shoudl be merged too
15:57:08 <tomasztrebski> thx
15:57:14 <rhochmuth> https://review.openstack.org/#/c/322750/
15:57:35 <rhochmuth> looks ready to me
15:58:00 <tomasztrebski> I feel like I won teddy bear today
15:58:08 <rhochmuth> lol
15:58:21 <tomasztrebski> ;-)
15:58:30 <witek> international children day
15:59:05 <rhochmuth> so, i think that is a wrap for today
15:59:08 <tomasztrebski> Roland is wonderful father for monasca project
15:59:23 <rhochmuth> oh gosh
15:59:35 <tomasztrebski> and we've managed to end before time up :D
15:59:54 <rhochmuth> we need to still decide mid-cycle week, but we'll being doign remotely
16:00:03 <rhochmuth> but, we've run out of time again
16:00:13 <rhochmuth> so, first item up next week is week for mid-cycle
16:00:16 <rhochmuth> and how to do remote
16:00:24 <rhochmuth> have to shut it down
16:00:41 <rhochmuth> #endmeeting